Re: [PATCH 0/5] Suggested for PU: revision caching system to significantly speed up packing/walking

From: Sam Vilain <sam@vilain.net>
Date: Aug 8, 2009, 23:54 UTC
Message-ID: <1249775658.7114.66.camel@maia.lan>
In-Reply-To: <alpine.DEB.1.00.0908081709380.8306@pacific.mpi-cbg.de>

On Sat, 2009-08-08 at 17:18 +0200, Johannes Schindelin wrote:

Show 18 quoted lines

> > Speeding up rev-list with a rev cache is completely orthogonal to 
> > whether the repository is packed or not.
> 
> No, it is not.
> 
> For both technical and practical reasons, caching revision walker data is
> very closely related to packing.
> [...]
> ... the rev cache has a certain target audience, 
> and that the regular user is not part of that audience, and that it just 
> so happens that the _technical_ similarities with the pack index can be 
> exploited in those scenarios?
> 
> IOW we can be pretty certain that a heavy-load server has a fully (or 
> next-to-fully) packed object database.  The pack indices already contain a 
> SHA-1 table that we can simply reuse.  And it should not be hard (or 
> fragile) at all to put the "cached" information about parents, 
> referenced tree and blob objects into that file, into a different section.

I think your argument would work better if packs and bundles were the same thing, and we always stored bundles in the objects/packs directory, but they're not and we don't. You can't assume that a pack has any particular properties, such as representing the objects returned from a single rev-cache walk. And I will say that *especially* on a busy git server, serving active projects you can't expect people to repack their repository for every single update. Repacking daily or so by a batch job, sure. Expecting the repository to always be fully packed? No. Too much churn, or inefficient packing. You can't just pretend that the mixed packed/loose case doesn't exist.

The 10% size seems a very good bang for your buck to me and a good start.

Sam