Re: Are binary xdeltas only used if you use git-gc?

From: Nicolas Pitre <nico@cam.org>
Date: Nov 4, 2008, 01:18 UTC
Message-ID: <alpine.LFD.2.00.0811031959070.13034@xanadu.home>
In-Reply-To: <f1d2d9ca0811031453p49390911p956149ca76b9b80d@mail.gmail.com>

On Tue, 4 Nov 2008, Thanassis Tsiodras wrote:

Show 25 quoted lines

> RESOLVED!!!
> 
> Finally...
> What happened was actually quite reasonable, in hindsight...
> As I said in the original mail, this was what I did:
> 
> cp version7.1.tar version7.2.tar
> git add version7.2.tar
> git commit -m "same data as old, so git will use old blob"
> echo MAGICPLACE read below...
> cp /path/to/work/realNewVersion7.2.tar version7.2.tar
> git add version7.2.tar
> git commit -m "and now, commit the really new version, so git can xdelta"
> git push --thin
> 
> The problem was solved (that is, the "git push" became optimal,
> when I added a "git push" right after the MAGICPLACE mark above...
> In that way, the remote repo learned about the "dummy" commit that
> referenced the old blob... and when I did the subsequent "git push"
> at the end, the remote side could see that it already had this "dummy"
> commit to "xdelta on", and that it only needed the delta...
> 
> Originally, when I used only one "git push --thin" at the end, the remote
> side didn't have the "dummy" commit, so it probably said: "I can't
> apply a delta, give me the full object".

Oh! But of course...

In fact, the way thin packs work is to store delta against a base object which is not included in the pack. Those objects which are not included but used as delta base are currently only the previous version of a file which is part of the update to be pushed/fetched. In other words, there must be a previous version under the same name for this to work. Doing otherwise wouldn't scale if the previous commit had thousands of files to test against.

But this particularity had escaped my mind somehow.

Show 19 quoted lines

> Phew.
> 
> So it seems that if you must introduce a new file that is
> very similar to an existing one (in my case, a new version
> of software kept in an uncompressed .tar file),
> you have to do what I did above to allow for optimal "git push"es:
> that is:
> 
> 1. Create the new filename by just copying the old
>  (so the old blob is used)
> 2. commit
> 3. PUSH
> 4. copy the real new file
> 5. commit
> 6. PUSH.
> 
> If you omit the middle PUSH in step 3, neither "git push", nor "git push --thin"
> can realize that this new file can be "incrementally built" on the remote side
> (even though git-gc totally squashes it in the pack).

Right. Those thin packs were designed for different versions of the same file in mind, not different files with almost the same content. This could possibly be improved at some point...

Nicolas