Re: jgit performance update

From: Robin Rosenberg <robin.rosenberg.lists@dewire.com>
Date: Dec 3, 2006, 13:55 UTC
Message-ID: <200612031455.48032.robin.rosenberg.lists@dewire.com>
In-Reply-To: <20061203045953.GE26668@spearce.org>

söndag 03 december 2006 05:59 skrev Shawn Pearce:

Show 11 quoted lines

> With the help of Robin Rosenberg I've been able to make jgit's log
> operation run (on average) within a few milliseconds of core Git.
>
> Walking the 50,000 most recent commits from the Mozilla trunk[1]:
>
>   $ time git rev-list --max-count=50000 HEAD >/dev/null
>
>   core Git:  1.882s (average)
>   jgit:      1.932s (average)
>
>   (times are with hot cache and from repeated executions)

Nice indeed. That was a ten-fold improvement for getting my full history.

So, just go on to the next case. I added filtering on filenames (yes, CVS-induced brain damage, I should track the content. next version. filenames are so much handier to work with). That gives me 4.5s to retrieve a filtered history (from 10800 commits).Half of the time is spent in re-sorting tree entries. Is that really necessary?

> I think that is actually pretty good given that jgit is written
> in Java using a fairly object-oriented design and has to deal with
> some of the limitations of the language.

Most of java's slowness comes from the programmers using it. (Lutz Prechelt. Technical opinion: comparing Java vs. C/C++ efficiency differences to interpersonal differences. ACM, Vol 42,#10, 1999)

Show 10 quoted lines

> One of the biggest annoyances has been the fact that although Java 
> 1.4 offers a way to mmap a file into the process, the overhead to
> access that data seems to be far higher than just reading the file
> content into a very large byte array, especially if we are going
> to access that file content multiple times.  So jgit performs worse
> than core Git early on while it copies everything from the OS buffer
> cache into the Java process, but then performs reasonably well once
> the internal cache is hot.  On the other hand using the mmap call
> reduces early latency but hurts the access times so much that we're
> talking closer to 3s average read times for the same log operation.

Have you tried that with difference JVM's?