Re: [RFC] Add --create-cache to repack

From: Nicolas Pitre <nico@fluxnic.net>
Date: Jan 30, 2011, 22:26 UTC
Message-ID: <alpine.LFD.2.00.1101301716400.8580@xanadu.home>
In-Reply-To: <AANLkTi=mbeBsR5tr4J7kQCL6YqiGfttK01VUN016aapC@mail.gmail.com>

On Sun, 30 Jan 2011, Shawn Pearce wrote:

Show 21 quoted lines

> On Sun, Jan 30, 2011 at 00:05, Junio C Hamano <gitster@pobox.com> wrote:
> > Shawn Pearce <spearce@spearce.org> writes:
> >
> >> Using this for object enumeration shaves almost 1 minute off server
> >> packing time; the clone dropped from 3m28s to 2m29s.  That is close to
> >> what I was getting with the cached pack idea, but the network transfer
> >> stayed the small 376 MiB.
> >
> > I like this result.
> 
> I'm really leaning towards putting this cached object list into JGit.
> 
> I need to shave that 1 minute off server CPU time. I can afford the 41
> MiB disk (and kernel buffer cache), but I cannot really continue to
> pay the 1 minute of CPU on each clone request for large repositories.
> The object list of what is reachable from commit X isn't ever going to
> change, and the path hash function is reasonably stable.  With a
> version code in the file we can desupport old files if the path hash
> function changes.  10% more disk/kernel memory is cheap for some of my
> servers compared to 1 minute of CPU, and some explicit cache
> management by the server administrator to construct the file.

Yep, I think this is probably the best short term solution. Just walk the commit graph as usual, and whenever the commit tip from the cache is matched then just shove the entire cache content in the object list.

And let's hope that eventually some future developments will make this cache redundant and obsolete.

Nicolas