Re: Errors cloning large repo

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mar 10, 2007, 05:54 UTC
Message-ID: <Pine.LNX.4.64.0703092145550.10832@woody.linux-foundation.org>
In-Reply-To: <20070310030718.GA2927@spearce.org>

On Fri, 9 Mar 2007, Shawn O. Pearce wrote:

> 
> I'm shocked you were able to repack an 8.5 GiB repository.

Side note - it would be nice to hear just how big the repository *really* is.

For example, if "du -sh" says 8.5GB, it doesn't necessarily mean that there really is 8.5GB of data there.

With a normal 4kB blocksize filesystem, and ~150.000 filesystem objects, you'd have an average of 300MB of just padding (roughly 2kB per file). Depending on the file statistics, it could be even more.

And if it's compressible, it's entirely possible that even without much delta compression, it could fit in a pack-file smaller than 4GB. At which point a 32-bit index file should work fine, just not with a 32-bit off_t.

So this really could be a situation where just small tweaks makes it work out for now. We'll need the full 64-bit index eventually for sure, but..

		Linus