Re: git annotate runs out of memory

From: Junio C Hamano <gitster@pobox.com>
Date: Dec 12, 2007, 00:56 UTC
Message-ID: <7vwsrkbuje.fsf@gitster.siamese.dyndns.org>
In-Reply-To: <alpine.LFD.0.9999.0712111548200.25032@woody.linux-foundation.org>

Linus Torvalds <torvalds@linux-foundation.org> writes:

Show 21 quoted lines

> On Tue, 11 Dec 2007, Linus Torvalds wrote:
>> 
>> and while I suspect xdiff could be optimized a bit more for the cases 
>> where we have no changes at the end, that's beyond my skills.
>
> Ok, I lied.
>
> Nothing is beyond my skills. My mad k0der skillz are unbeatable.
>
> This speeds up git-blame on ChangeLog-style files by a big amount, by just 
> ignoring the common end that we don't care about, since we don't want any 
> context anyway at that point. So I now get:
>
> 	[torvalds@woody gcc]$ time git blame gcc/ChangeLog > /dev/null
>
> 	real    0m7.031s
> 	user    0m6.852s
> 	sys     0m0.180s
>
> which seems quite reasonable, and is about three times faster than trying 
> to diff those big files.

Funny. I did not understand what you were talking about "no changes at the end" when I read it ('cause I am at work and do not have the data you are looking at handy), but now I see what you meant. It is a cute hack that optimizes for a very special case of "prepend only" files (aka "ChangeLog").

I suspect that this optimization has an interesting corner case, though. What happens if you chomp at the middle of the last line that is different between the two files? xdiff will report the line number but wouldn't its (now artificial) "No newline at the end of the file" affect the blame logic?

Besides, "prepend only" (or "append only") files would be good candidates for the original -S"pickaxe" search, I would imagine, and unless you are looking at that ChangeLog-2000 consolidated log, isn't blame way overkill?