Re: [PATCH] Support 64-bit indexes for pack files.

From: Nicolas Pitre <nico@cam.org>
Date: Feb 27, 2007, 00:24 UTC
Message-ID: <alpine.LRH.0.82.0702261916560.29426@xanadu.home>
In-Reply-To: <20070226235510.GF1639@spearce.org>

On Mon, 26 Feb 2007, Shawn O. Pearce wrote:

Show 12 quoted lines

> Our topic also requires us to change the index file format, and
> in doing so we have decided to extend the index records to look
> something like the following[*1*]:
> 
> 	object SHA-1
> 	64-bit offset within packfile
> 	32-bit index of next object in packfile
> 
> The latter field is to help pack-objects reuse existing packfile
> data, as today it needs to sort everything on its own on the fly.
> Having that last field of data will help avoid that, and will keep
> the index nicely aligned for 64-bit accesses to the offset.

Wouldn't that later field help the sliding mmap code as well, knowing in advance what storage size a given object has? (I didn't look at the sliding mmap code so I don't know).

Actually I've been thinking about another format already.

What about keeping the pack offset as 32 bits like it is today, but for index v2 if the top bit is set then this become an index into another table containing 64-bit offsets as needed. This way there is no waste of space for most projects where the pack has yet to reach the 2GB limit for many years to come.

Nicolas