Re: jgit performance update

From: Shawn Pearce <spearce@spearce.org>
Date: Dec 3, 2006, 22:47 UTC
Message-ID: <20061203224726.GC15965@spearce.org>
In-Reply-To: <Pine.LNX.4.64.0612030938140.3476@woody.osdl.org>

Linus Torvalds <torvalds@osdl.org> wrote:

> Very good. Are we any closer to actually having an eclipse plugin then?

We have some parts in place.  But nothing that's user-ready.

Show 5 quoted lines

> > Walking the 50,000 most recent commits from the Mozilla trunk[1]:
> 
> Now, the _interesting_ case in many ways is not "--max-count", but the 
> revision limiter. It _should_ be equally fast, but if you've done 
> something wrong, it won't be.

We haven't implemented a rev-list equivalent yet. That's a major feature which is missing. The --max-count test was simple to put together. I really need to start thinking about replicating some of the features of the revision functions in core Git.

Show 8 quoted lines

> > One of the biggest annoyances has been the fact that although Java
> > 1.4 offers a way to mmap a file into the process, the overhead to
> > access that data seems to be far higher than just reading the file
> > content into a very large byte array, especially if we are going
> > to access that file content multiple times.
> 
> That must suck for big packed repositories. What JVM and other environment 
> are you using?

Mac OS 10.4.8 / Java 1.4.2. It appears as though Sun isn't going to fix the mmap performance problems as they can't do it "securely". So its likely to be an issue anywhere jgit gets used...

Show 5 quoted lines

> Also, I have to say, one of the reasons I'm interested in your project is 
> that I've never done any Java programming, because quite frankly, I've 
> never had any reason what-so-ever to do so. But if there is some simple 
> setup, and you have jgit exposed somewhere as a git archive, I'd love to 
> take a look, if only to finally learn more about Java.

Of course its in Git.  :-)

There's no webpage here, but you can clone it:

  http://www.spearce.org/projects/scm/egit.git

The code you would recognize most is in org.spearce.jgit/.../lib. E.g. BinaryDelta.java came from patch-delta.c in core Git. Repository has some of the functionality of sha1_file.c, the pack reading code is in PackFile.java.