Re: [PATCH] Support 64-bit indexes for pack files.

From: Troy Telford <ttelford.groups@gmail.com>
Date: Feb 27, 2007, 01:16 UTC
Message-ID: <200702261816.20258.ttelford.groups@gmail.com>
In-Reply-To: <20070226235510.GF1639@spearce.org>

On Monday 26 February 2007, Shawn O. Pearce wrote:

> Nico and I are neck deep in our pack version 4 topic.  That topic
> hits all of the same code you touched with your patch.

It looked like that may have been the case; but I figured it wouldn't hurt. I've been coping with the 'too-small' index issue for about a year now, and have been on IRC and kept at least a lazy eye on it hoping that it would be attended to.

I've decided I need to start doing some C coding again, and it seemed to be as good an itch as any to scratch...

Show 12 quoted lines

> Our topic also requires us to change the index file format, and
> in doing so we have decided to extend the index records to look
> something like the following[*1*]:
>
> 	object SHA-1
> 	64-bit offset within packfile
> 	32-bit index of next object in packfile
>
> The latter field is to help pack-objects reuse existing packfile
> data, as today it needs to sort everything on its own on the fly.
> Having that last field of data will help avoid that, and will keep
> the index nicely aligned for 64-bit accesses to the offset.

Exactly why I left the packfile itself alone. Well, that and laziness. I may have a huge repository, but I don't have single files so large that it needs a 32-bit index for the next object in the packfile.

Most of the things that large I've seen are binary anyway.

> I want to say your patch shouldn't be merged without even bothering
> to review it.

No hard feelings on my part, it was as much a learning thing for me as anything else.

Show 5 quoted lines

> The last time I was in this part of the git code 
> (implementing sliding mmap window) Nico and Junio also both went in
> here and rewrote huge chunks.  Their changes prevented sliding mmap
> window from applying.  It was 6 months before I got back around to
> rewriting the patch.

Ouch.

Show 7 quoted lines

> Right now I'm neck deep in pack v4.  I hope to have the topic in
> pu-ready state by some time mid-next week, hopefully in time for
> Junio's git day.  I'm very unlikely to have the time to rewrite the
> topic again until late June/July if something like your patch goes
> in now.
>
> So would you mind waiting a couple of weeks for 64 bit indexes?

All I'm concerned with is being able to handle my (huge) source repository with git. I wrote the patch in the hopes that it would accelerate the process, and that I'd learn something. If all I have to do is wait that's fine. I figured I'd at least be able to bring an idea or two to the table. If the code doesn't get accepted, but still get the desired functionality, I still met 1/2 of my goals in doing it.

-- 
Troy Telford