Re: [RFC] Packing large repositories

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mar 31, 2007, 21:56 UTC
Message-ID: <Pine.LNX.4.64.0703311450030.6730@woody.linux-foundation.org>
In-Reply-To: <Pine.LNX.4.64.0703311408560.6730@woody.linux-foundation.org>

On Sat, 31 Mar 2007, Linus Torvalds wrote:

> 
> Ok, here's a slightly updated patch that does a few more iterations..

Don't use this. There's something wrong with it, and unlike the first patch, it doesn't even pass all tests.

I think it's because I decided to try to use rounding in the newton thing (which got things closer), but I suspect that rounding is broken - I think it can round to "outside" the valid range. I didn't notice that with the eclipse performance testing, probably because when you have enough objects you'll never see it, but with smaller packs, being off-by-one means that you can easily fall off the map ;)

So I'd like to re-iterate the note I had in that email:

> Not really meant to be applied, but it was interesting to play with 
> this. NOTE! I don't think the math is really strictly correct (ie the 
> scaling inside the loop), but it's "close enough" to not matter, and this 
> really was meant to be an request-for-discussion.

IOW, it's very much a RFD patch. I suspect it is trivial to fix, but I also suspect that unless somebody else decides that this is interesting, I'll just leave the patch behind. None of the loads I tested really seemed to be very sensitive to object lookup performance - the binary search simply seems ot be "good enough", and usually the costs are in generating patches ("git blame") or just diffing trees ("git log <pathspec>"), and the object lookup doesn't seem to be critical enough to worry about.

If somebody can point to a load where we spend a noticeable amount of time on doing index lookups, that would obviously change my opinion, but in the meantime, I think this is better considered a case of "we _could_ do something like this if it's ever an issue.."

		Linus