Re: fast-import deltas

From: Jeff King <peff@peff.net>
Date: Apr 1, 2014, 11:45 UTC
Message-ID: <20140401114502.GA15549@sigill.intra.peff.net>
In-Reply-To: <20140401102554.GA32231@glandium.org>

On Tue, Apr 01, 2014 at 07:25:54PM +0900, Mike Hommey wrote:

> I am currently prototyping a "native" mercurial remote handler for git,

For my own curiosity, how does this differ from what is in contrib/remote-helpers/git-remote-hg?

> Would adding a fast-import command to handle deltas be considered useful
> for git? If so, what kind of format would be suitable?

It breaks fast-import's "lowest common denominator" data model that is just passing commits and their contents over the stream. But we already do that in other cases for the sake of performance. I think the important thing is that the alternate formats are optional and enabled by the caller with command-line options.

That being said, I foresee a few complications:

  1. Git needs to know the sha1 of the full object. So unless the
     generating script knows that ahead of time, git has to expand the
     delta immediately anyway (this could still be a win if we end up
     using a good delta from elsewhere rather than doing our own delta
     search, but I suspect it's not so big a win as if we can just blit
     the delta straight to disk).

  2. Git does not store on-disk deltas between objects that are not in
     the same packfile. So you'd only be able to delta against an object
     that came in the same stream (or you'd have to "fix" the packfile
     on disk by adding an extra copy of the delta base, but that
     probably eliminates any savings).

As for format, I believe that git is basically xdelta under the hood, so you'd want either that or something that can be trivially converted to it.

-Peff