Re: jgit performance update

From: Linus Torvalds <torvalds@osdl.org>
Date: Dec 3, 2006, 17:45 UTC
Message-ID: <Pine.LNX.4.64.0612030938140.3476@woody.osdl.org>
In-Reply-To: <20061203045953.GE26668@spearce.org>

On Sat, 2 Dec 2006, Shawn Pearce wrote:

>
> With the help of Robin Rosenberg I've been able to make jgit's log
> operation run (on average) within a few milliseconds of core Git.

Very good. Are we any closer to actually having an eclipse plugin then?

Not that I've ever actually used eclipse, but maybe I should try it, just to see what those strange user-land people actually do. I'll be a veritable Jane Goodall..

Show 8 quoted lines

> Walking the 50,000 most recent commits from the Mozilla trunk[1]:
> 
>   $ time git rev-list --max-count=50000 HEAD >/dev/null
> 
>   core Git:  1.882s (average)
>   jgit:      1.932s (average)
> 
>   (times are with hot cache and from repeated executions)

Now, the _interesting_ case in many ways is not "--max-count", but the revision limiter. It _should_ be equally fast, but if you've done something wrong, it won't be.

IOW, try to find a point far enough back in time to get about the same number of commits, and then do

	time git rev-list <thatpoint>..HEAD >/dev/null

because one of the things you want to handle is ranges, more so than simple counts. And that is not only the much more common case, it also triggers a few cases that you probably didn't trigger with the regular "list the first 50 thousand commits" case.

Show 5 quoted lines

> One of the biggest annoyances has been the fact that although Java
> 1.4 offers a way to mmap a file into the process, the overhead to
> access that data seems to be far higher than just reading the file
> content into a very large byte array, especially if we are going
> to access that file content multiple times.

That must suck for big packed repositories. What JVM and other environment are you using?

Also, I have to say, one of the reasons I'm interested in your project is that I've never done any Java programming, because quite frankly, I've never had any reason what-so-ever to do so. But if there is some simple setup, and you have jgit exposed somewhere as a git archive, I'd love to take a look, if only to finally learn more about Java.