# Understanding Binary Deltas within Packfile 4 messages from 2018-03-20 to 2018-03-20. Participants: Luke Robison, Duy Nguyen. Thread: https://gitlist.dev/t/48097 ## Luke Robison, 2018-03-20 15:43 Subject: Understanding Binary Deltas within Packfile Message-ID: URL: https://gitlist.dev/e/f301d093-af93-016b-79b9-3102475260cf%40arlut.utexas.edu ``` Is there any documentation of the contents of the binary delta datain a packfile, and how to interpret them? I found https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt documenting the packfile itself, but the "compressed delta data" seems largely undocumented. The source code of https://github.com/git/git/blob/master/diff-delta.c is pretty dense. This is what I've got so for: +-------------+ | Varint src_size | Varint trg_size | 1-byte inscnt | N-bytes up to 16 bytes of trg_buf | | A series of Operations | +------------- Any details on how to interpret the opcodes would be greatly appreciated. Thanks, Luke ``` ## Duy Nguyen, 2018-03-20 16:03 Subject: Re: Understanding Binary Deltas within Packfile Message-ID: URL: https://gitlist.dev/e/CACsJy8ABgZy%3DeJ7niUysb2XZ3qUr3J%2Bjmh_5YWZ1ZDiFCrb0tA%40mail.gmail.com In-Reply-To: ``` On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison wrote: > Is there any documentation of the contents of the binary delta datain a > packfile, and how to interpret them? I found > https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt > documenting the packfile itself, but the "compressed delta data" seems > largely undocumented. The source code of > https://github.com/git/git/blob/master/diff-delta.c is pretty dense. The output is consumed by patch_delta() in patch-delta.c if I'm not mistaken. This function is less than 100 lines, probably much easier to see the delta format. > This > is what I've got so for: > > +-------------+ > | Varint src_size > | Varint trg_size > | 1-byte inscnt > | N-bytes up to 16 bytes of trg_buf > | > | A series of Operations > | > +------------- > > > Any details on how to interpret the opcodes would be greatly appreciated. > > Thanks, > Luke > -- Duy ``` ## Luke Robison, 2018-03-20 18:34 Subject: Re: Understanding Binary Deltas within Packfile Message-ID: <751c0493-435c-4a98-4d1c-e3d6677c62d8@arlut.utexas.edu> URL: https://gitlist.dev/e/751c0493-435c-4a98-4d1c-e3d6677c62d8%40arlut.utexas.edu In-Reply-To: ``` On 3/20/2018 11:03 AM, Duy Nguyen wrote: > On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison wrote: >> Is there any documentation of the contents of the binary delta datain a >> packfile, and how to interpret them? I found >> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt >> documenting the packfile itself, but the "compressed delta data" seems >> largely undocumented. The source code of >> https://github.com/git/git/blob/master/diff-delta.c is pretty dense. > The output is consumed by patch_delta() in patch-delta.c if I'm not > mistaken. This function is less than 100 lines, probably much easier > to see the delta format. Thank you, that was much easier to read, and I've got my prototype working now. I also found this site to be quite helpful: http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding ``` ## Duy Nguyen, 2018-03-20 19:12 Subject: Re: Understanding Binary Deltas within Packfile Message-ID: URL: https://gitlist.dev/e/CACsJy8ChV5sXV7N0%2BN0y9ehO56UCJ8fuqjFbHbbnez35P5af4g%40mail.gmail.com In-Reply-To: <751c0493-435c-4a98-4d1c-e3d6677c62d8@arlut.utexas.edu> ``` On Tue, Mar 20, 2018 at 7:34 PM, Luke Robison wrote: > On 3/20/2018 11:03 AM, Duy Nguyen wrote: >> >> On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison >> wrote: >>> >>> Is there any documentation of the contents of the binary delta datain a >>> packfile, and how to interpret them? I found >>> >>> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt >>> documenting the packfile itself, but the "compressed delta data" seems >>> largely undocumented. The source code of >>> https://github.com/git/git/blob/master/diff-delta.c is pretty dense. >> >> The output is consumed by patch_delta() in patch-delta.c if I'm not >> mistaken. This function is less than 100 lines, probably much easier >> to see the delta format. > > Thank you, that was much easier to read, and I've got my prototype working > now. I also found this site to be quite helpful: > http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding By the way, I forgot to add, if you want to improve pack-format.txt (since you study it anyway), patches are always welcome. -- Duy ```