Re: Figured out how to get Mozilla into git

From: Nicolas Pitre <nico@cam.org>
Date: Jun 11, 2006, 22:00 UTC
Message-ID: <Pine.LNX.4.64.0606111747110.2703@localhost.localdomain>
In-Reply-To: <Pine.LNX.4.64.0606091825080.5498@g5.osdl.org>

On Fri, 9 Jun 2006, Linus Torvalds wrote:

Show 15 quoted lines

> 
> 
> On Sat, 10 Jun 2006, Martin Langhoff wrote:
> > 
> > Now I don't know how much memory or time this took, but it clearly
> > completed ok. And, it's now a single pack, weighting a grand total of
> > 617MB
> 
> Ok, that's more than reasonable. That should be fairly easily mapped on a 
> 32-bit architecture without any huge problems, even with some VM 
> fragmentation going on. It might be borderline (and you definitely want a 
> 3:1 VM user:kernel split), but considering that the original CVS archive 
> was apparently 3GB, having a single 617M pack-file is still pretty damn 
> good.  That's like 20% of the original, with all the obvious distribution 
> advantages.

I played a bit with git-repack on that repo. the git-pack-objects memory usage grew to around 760MB (git-rev-list was less than that). So LRU of partial pack mappings might bring that down significantly.

Then I used git-repack -a -f --window=20 --depth=20 which produced a nice 468MB pack file along with the invariant 45MB index file for a grand total of 535MB for the whole repo (the .git/refs/ directory alone still occupies 17MB on disk).

So it is probably worth having deeper delta chains for large historic repositories as the deep revisions are unlikely to be referenced that often while the saving is quite significant.

Nicolas