Re: [PATCH 0/5] Suggested for PU: revision caching system to significantly speed up packing/walking

From: Sam Vilain <sam@vilain.net>
Date: Aug 7, 2009, 02:47 UTC
Message-ID: <4A7B95A8.2010000@vilain.net>
In-Reply-To: <alpine.DEB.1.00.0908062030340.8306@pacific.mpi-cbg.de>

Johannes Schindelin wrote:

Show 10 quoted lines

>> the short answer is that cache slices are totally independant of pack 
>> files.
>>     
>
> My idea with that was that you already have a SHA-1 map in the pack index, 
> and if all you want to be able to accelerate the revision walker, you'd 
> probably need something that adds yet another mapping, from commit to 
> parents and tree, and from tree to sub-tree and blob (so you can avoid 
> unpacking commit and tree objects).
>

Tying indexes together like that is not a good idea in the database world. Especially as in this case as Nick mentions, the domain is subtly different (ie pack vs dag). Unfortunately you just can't try to pretend that they will always be the same; you can't force a full repack on every ref change!

> Still, there is some redundancy between the pack index and your cache, as 
> you have to write out the whole list of SHA-1s all over again.  I guess it 
> is time to look at the code instead of asking stupid questions.
>

"Disk is cheap" :-) It should be a welcome trade-off; perhaps it's worth including numbers about how big the indexes are with the time statistics. It sounds though like it should be a significant win as a single index can be used to accelerate a wide range of rev-list operations, and store indexes for many different questions that can be asked.

Sam