Re: [RFC] Cache negative delta pairs

From: Jeff King <peff@peff.net>
Date: Jun 29, 2006, 19:52 UTC
Message-ID: <20060629195201.GA10786@coredump.intra.peff.net>
In-Reply-To: <Pine.LNX.4.64.0606291458110.1213@localhost.localdomain>

On Thu, Jun 29, 2006 at 03:04:15PM -0400, Nicolas Pitre wrote:

> Right.  Your use pattern is a special case that doesn't work well with 
> the whole window hash approach.  I'd expect it to work beautifully with 
> the kernel repository though.

I don't necessarily care about the kernel repository. It packs fine as it is, and you only waste 30 seconds on a repack checking deltas that could be avoided. I do care on my special repository where packing is virtually unusable and I can achieve a 45x speedup. Maybe my original caching is not worth it for the kernel and should be configurable, but obviously this window caching cannot REPLACE mine since it fails utterly for the one thing I wanted it for.

That being said, I'm not sure that window caching is all that great for "normal" repos.

Same test as before, but instead of simulating the commits, I merely
looked at the window hashes produced by 
  git-rev-list --objects master~$x

For the git repo: x=0 tries 6698 windows x=0 and x=50 contain 5197 identical windows x=0 and x=100 contain 2484 identical windows x=0 and x=500 contain 455 identical windows

For linux-2.6 repo: x=0 tries 57208 windows x=0 and x=50 contain 53677 identical windows x=0 and x=100 contain 52886 identical windows x=0 and x=500 contain 41196 identical windows

Obviously the kernel repo is doing better, but x=500 is only 4 days ago. Trying with --before=2.weeks.ago yields only 31505 matches.

So the windows do clearly experience a fair bit of churn, but whether or not this is worth it depends on how long you think is reasonable before something gets "churned out" .

-Peff