Re: pack-object's try_delta fast path for v2 trees?

From: Jeff King <peff@peff.net>
Date: Oct 15, 2013, 01:51 UTC
Message-ID: <20131015015113.GA17486@sigill.intra.peff.net>
In-Reply-To: <alpine.LFD.2.03.1310142141200.1873@syhkavp.arg>

On Mon, Oct 14, 2013 at 09:45:12PM -0400, Nicolas Pitre wrote:

Show 17 quoted lines

> > No, as far as I know, it is a novel idea. When we were discussing commit
> > caching a while back, Shawn suggested slicing trees on boundaries and
> > store delta instructions that were pure "change this entry", "add this
> > entry", and "delete this entry" chunks. The deltas might end up a little
> > bigger, but if the reader knew the writer had sliced in this way, it
> > could get a packv4-style cheap tree-diff, while remaining backwards
> > compatible with implementations that just blindly reassemble the buffer
> > from delta instructions.
> > 
> > I didn't get far enough to try it, but doing what you propose would be
> > the first step. Now that packv4 is more of a reality, it may not be
> > worth pursuing, though.
> 
> The "easy" way to produce pack v2 tree objects from a pack v4 would be 
> exactly that: take the pack v4 tree encoding and do a straight 
> translation into delta encoding using the base from which the most 
> entries are copied from.

Yeah, that makes a lot of sense.

By the way, I'm sorry I haven't looked more carefully at the packv4 patches yet. I am excited about it, but I've just got a long queue of other things (and because it's big and challenging, it's easy to put off).

One of the things that makes me most nervous about switching to it on the server side is that we'll have packv2-only clients for a while, and I worry that converting to v2 on the fly is going to end up costing a lot (even with clever tricks like this, you still have to pay the cost to zlib deflate each item). But even if it is slow, the sooner we have packv4 readers, the sooner the clocks start ticking for it being a reasonable decision for a big server provider to switch.

-Peff