Re: git annotate runs out of memory

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Dec 12, 2007, 02:20 UTC
Message-ID: <alpine.LFD.0.9999.0712111816220.25032@woody.linux-foundation.org>
In-Reply-To: <7vwsrkbuje.fsf@gitster.siamese.dyndns.org>

On Tue, 11 Dec 2007, Junio C Hamano wrote:

Show 6 quoted lines

> 
> I suspect that this optimization has an interesting corner case, though.
> What happens if you chomp at the middle of the last line that is
> different between the two files?  xdiff will report the line number but
> wouldn't its (now artificial) "No newline at the end of the file" affect
> the blame logic?

It shouldn't. I thought about it, but there doesn't seem to be any reason why blame could possibly care - the message can come at the end of a _real_ file, of course, so if the extra message confuses the blame logic, there's already a bug there.

But no, I didn't create a test-case.

> Besides, "prepend only" (or "append only") files would be good
> candidates for the original -S"pickaxe" search, I would imagine, and
> unless you are looking at that ChangeLog-2000 consolidated log, isn't
> blame way overkill?

Actually, I suspect that this makes a difference for totally normal files too. I bet it cuts the size of the files to be tested for the common case (ie just a few small changes) down by 30-50% even on average. The fact that it cuts it down by 99.9% on ChangeLog files is just an added bonus.

As Davide mentioned, xdiff actually does something like that hack for the beginning and end of files internally _anyway_, the problem with that is that it does it so late that it's already done a fairly expensive hash for the file (and allocated space for it based on guesses that are in turn based on the original size) that it doesn't actually get the full effect of the optimization.

			Linus