Re: [RFC] super indexes to span multiple packfiles

From: Nicolas Pitre <nico@cam.org>
Date: May 29, 2007, 16:31 UTC
Message-ID: <alpine.LFD.0.99.0705291227010.11491@xanadu.home>
In-Reply-To: <465C52D3.3010605@qumranet.com>

On Tue, 29 May 2007, Avi Kivity wrote:

Show 13 quoted lines

> Jon Smirl wrote:
> > 
> > My work with databases leads me to believe that figuring out how to
> > pack everything into a smaller space always beats efforts put into
> > incrementally improving the indexing scheme. Packing into a smaller
> > space reduces the total IO needs and that's always a winner.
> > 
> 
> Another way to achieve that is to place objects that are accessed together
> nearby, and issue a larger read so as to bring them into cache.  I imagine
> that placing commit objects and associated tree and blobs in history order
> should help here (but maybe git already does that, I'm not familiar with the
> internals).

GIT already does that indeed, except for commit objects which are all together for better performances on history traversal operations.

After a fresh repack, the checkout of the latest revision should produce a nearly perfect linear and contigous access into the early portion of the same pack. Things will get more random with access to objects further back in history of course, but those objects are less likely to be accessed as often.

Nicolas