Re: git-index-pack really does suck..

From: David Lang <david.lang@digitalinsight.com>
Date: Apr 3, 2007, 22:31 UTC
Message-ID: <Pine.LNX.4.63.0704031530040.21680@qynat.qvtvafvgr.pbz>
In-Reply-To: <Pine.LNX.4.64.0704031544110.6730@woody.linux-foundation.org>

On Tue, 3 Apr 2007, Linus Torvalds wrote:

Show 19 quoted lines

> On Tue, 3 Apr 2007, Dana How wrote:
>>
>> Larger and larger pack files make me nervous.
>> They are expensive to manipulate,
>> and >2GB requires a file format change.
>
> It sometimes also requires a new filesystem. There are a lot of
> filesystems that can handle more than 4GB total, but not necessarily in a
> single file.
>
> The only really useful such filesystem is probably FAT, which is still
> quite useful for things like USB memory sticks. But that is probably
> already worth supporting.
>
> So I think we want to support 64-bit (or at least something like 40+ bit)
> pack-files, but yes, I think that even if/when we support it, we still
> want to support the "multiple smaller pack-files" schenario exactly
> because for some uses it's much *better* to have ten 2GB packfiles rather
> than one 20GB pack-file.

however, for historical archives you may end up with wanting to do a 2GB packfile of 'recent' stuff, and a 14GB packfile of 'ancient' stuff (with the large one built with all space-saving options turned up all the way, no matter how much time it takes to build the pack)

David Lang