threads / discuss / 48097

Understanding Binary Deltas within Packfile

Subject: Understanding Binary Deltas within Packfile

## tl;dr

4 messages between Mar 20, 2018 and Mar 20, 2018.

replies: 3people: 2as markdown or json

Luke Robison· Mar 20, 2018, 15:43 UTC · lore

Is there any documentation of the contents of the binary delta datain a packfile, and how to interpret them? I found https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt documenting the packfile itself, but the "compressed delta data" seems largely undocumented. The source code of https://github.com/git/git/blob/master/diff-delta.c is pretty dense. This is what I've got so for:

+-------------+
| Varint src_size
| Varint trg_size
| 1-byte inscnt
| N-bytes up to 16 bytes of trg_buf
|
| A series of Operations
|
+-------------
Any details on how to interpret the opcodes would be greatly appreciated.

Thanks, Luke

Duy Nguyen· Mar 20, 2018, 16:03 UTC · re: Luke Robison · lore

Re: Understanding Binary Deltas within Packfile

On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
Show 6 quoted lines
> Is there any documentation of the contents of the binary delta datain a
> packfile, and how to interpret them?  I found
> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
> documenting the packfile itself, but the "compressed delta data" seems
> largely undocumented.  The source code of
> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.

The output is consumed by patch_delta() in patch-delta.c if I'm not mistaken. This function is less than 100 lines, probably much easier to see the delta format.

Show 19 quoted lines
>  This
> is what I've got so for:
>
> +-------------+
> | Varint src_size
> | Varint trg_size
> | 1-byte inscnt
> | N-bytes up to 16 bytes of trg_buf
> |
> | A series of Operations
> |
> +-------------
>
>
> Any details on how to interpret the opcodes would be greatly appreciated.
>
> Thanks,
> Luke
>
-- 
Duy
Luke Robison· Mar 20, 2018, 18:34 UTC · re: Duy Nguyen · lore

Re: Understanding Binary Deltas within Packfile

On 3/20/2018 11:03 AM, Duy Nguyen wrote:
Show 10 quoted lines
> On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
>> Is there any documentation of the contents of the binary delta datain a
>> packfile, and how to interpret them?  I found
>> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
>> documenting the packfile itself, but the "compressed delta data" seems
>> largely undocumented.  The source code of
>> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.
> The output is consumed by patch_delta()  in patch-delta.c if I'm not
> mistaken. This function is less than 100 lines, probably much easier
> to see the delta format.

Thank you, that was much easier to read, and I've got my prototype working now. I also found this site to be quite helpful: http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding

Duy Nguyen· Mar 20, 2018, 19:12 UTC · re: Luke Robison · lore

Re: Understanding Binary Deltas within Packfile

On Tue, Mar 20, 2018 at 7:34 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
Show 20 quoted lines
> On 3/20/2018 11:03 AM, Duy Nguyen wrote:
>>
>> On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu>
>> wrote:
>>>
>>> Is there any documentation of the contents of the binary delta datain a
>>> packfile, and how to interpret them?  I found
>>>
>>> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
>>> documenting the packfile itself, but the "compressed delta data" seems
>>> largely undocumented.  The source code of
>>> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.
>>
>> The output is consumed by patch_delta()  in patch-delta.c if I'm not
>> mistaken. This function is less than 100 lines, probably much easier
>> to see the delta format.
>
> Thank you, that was much easier to read, and I've got my prototype working
> now.  I also found this site to be quite helpful:
> http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding

By the way, I forgot to add, if you want to improve pack-format.txt (since you study it anyway), patches are always welcome.

-- 
Duy

← back to recent threads