Re: [PATCH 1/2] blame: large-scale performance rewrite

From: David Kastrup <dak@gnu.org>
Date: Apr 26, 2014, 21:39 UTC
Message-ID: <87vbtv7ou0.fsf@fencepost.gnu.org>
In-Reply-To: <CAJo=hJs-Nn=o=aGS_3bO9mnxb+urst6JTZf29_qAejBipz_ZHg@mail.gmail.com>

Shawn Pearce <spearce@spearce.org> writes:

Show 23 quoted lines

> On Sat, Apr 26, 2014 at 10:30 AM, David Kastrup <dak@gnu.org> wrote:
>> David Kastrup <dak@gnu.org> writes:
>>
>> Here's some example:
>>
>> dak@lola:/usr/local/tmp/wortliste$ time git blame -n -s wortliste >/tmp/wl1
>>
>> real    15m47.118s
>> user    14m39.928s
>> sys     1m1.872s
>
> Hah, this is quite the torture test. git before your patch is taking
> 22m11s on my laptop to compute this. (This was with default options, I
> noticed you passed -s to suppress the author formatting.)
>
>> dak@lola:/usr/local/tmp/wortliste$ time ../git/git blame -n -s wortliste >/tmp/wl2
>>
>> real    3m40.947s
>> user    2m40.296s
>> sys     0m59.440s
>
> Meanwhile JGit computed in 4m30s on the same hardware. So I guess we
> are "fine".

At least the stuff I fixed with regard to performance would seem to be done right in JGit to start with.

> Its still not as fast as I want it to be. :-)

Most of the diff data/CRC is computed over and over because of the blackbox use of xdiff. And then the delta-chain storage is packing stuff based on CRCs as well (not sure whether it keeps them around for unpacking). So there is a lot that could likely be improved while keeping the same basic algorithms, by cracking open the black boxes of the xdiff engine and the delta-chain coding.

-- 
David Kastrup