Re: Cygwin can't handle huge packfiles?

From: Junio C Hamano <junkio@cox.net>
Date: Apr 7, 2006, 18:31 UTC
Message-ID: <7vhd55jkz0.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <Pine.LNX.4.64.0604071002530.2215@localhost.localdomain>

Nicolas Pitre <nico@cam.org> writes:

Show 24 quoted lines

> On Fri, 7 Apr 2006, Junio C Hamano wrote:
>
>> Linus Torvalds <torvalds@osdl.org> writes:
>> 
>> > On Mon, 3 Apr 2006, Linus Torvalds wrote:
>> >> 
>> >> That said, I think git _does_ have problems with large pack-files. We have 
>> >> some 32-bit issues etc
>> >
>> > I should clarify that. git _itself_ shouldn't have any 32-bit issues, but 
>> > the packfile data structure does. The index has 32-bit offsets into 
>> > individual pack-files. 
>> >
>> > That's not hugely fundamental,...
>> 
>> Linus _does_ understand what he means, but let me clarify and
>> outline a possible future direction.
>
> For the record, the delta code also has 32-bit limitations of its own 
> presently.  It cannot encode a delta against a buffer which is larger 
> than 4GB.
>
> I however made sure the byte 0 could be used as a prefix for future 
> encoding extensions, like 64-bit file offsets for example.

True the delta data representation, not just the "delta code", has that limitation, but I do not think you issue "insert 0-byte literal data" command from the deltifier side right now, so we should be OK.

Maybe we would want to check (cmd == 0) case to detect delta extension that we do not handle right now?