git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RFC: adding xdelta compression to git

From
AZAlon Ziv <alonz@nolaviz.org>
Date
May 3, 2005, 03:57 UTC
Message-ID
<200505030657.38309.alonz@nolaviz.org>

Looking for novel methods of wasting my time :), I am considering adding xdelta to git.

I have two concrete proposals, both of which (IMO) are consistent with the git philosophy:

1. Add a git-deltify command, which will take two trees and replace the second 
tree's blobs with delta-blobs referring to the first tree. Each delta-blob is 
self-contained; from the outside it looks like any other blob, but internally 
it contains another blob reference + an xdelta. The only function which would 
need to understand the new format would be unpack_sha1_file.
The scripting level will be in charge of deciding which trees to deltify (or 
undeltify--we could also have a "git-undeltify" command). A sane 
deltification schedule, for example, could be to always keep tagged versions 
as stand-alone objects, and deltify intermediate versions against the latest 
tag. It would also do its best to avoid delta chains (i.e. a delta referring 
to another delta).
Pros:
* Interoperates with the existing structure (including pull/push) with almost 
no changes to existing infrastructure.
Cons:
* Changes the repository format.
* Some performance impact (probably quite small).
* Same blob may have different representation in two repositories (one 
compressed, on deltified). [I am not sure this is really a bad thing...]
2. Add a completely external framework which manages a "deltas repository" of 
deltas. The shadow repository will contain delta objects between selected 
trees; again the scripts will need to populate it.
Pros:
* No changes at all to existing code.
Cons:
* Push/pull tools will need to be taught to talk with the new "deltas  
repository".
* Synchronization between the deltas repository and the real one may be lost, 
leading to odd failures.

Personally I'm rooting for #1 above... I would like to begin implementation in a few days, so any discussion will be useful.

	-az
Next: Nicolas Pitre
Message 1 of 32 in “RFC: adding xdelta compression to git”
  1. Alon ZivMay 3, 2005
  2. Nicolas PitreMay 3, 2005
  3. Linus TorvaldsMay 3, 2005
  4. Davide LibenziMay 3, 2005
  5. C. Scott AnanianMay 3, 2005
  6. Linus TorvaldsMay 3, 2005
  7. Davide LibenziMay 3, 2005
  8. add the ability to create and retrieve delta objectsNicolas Pitre, May 3, 2005
  9. Chris MasonMay 3, 2005
  10. Nicolas PitreMay 3, 2005
  11. Linus TorvaldsMay 3, 2005
  12. Chris MasonMay 3, 2005
  13. C. Scott AnanianMay 3, 2005
  14. Chris MasonMay 3, 2005
  15. Chris MasonMay 3, 2005
  16. Nicolas PitreMay 3, 2005
  17. Chris MasonMay 3, 2005
  18. Nicolas PitreMay 3, 2005
  19. Chris MasonMay 3, 2005
  20. Linus TorvaldsMay 3, 2005
  21. add the ability to create and retrieve delta objectsNicolas Pitre, May 3, 2005
  22. Chris MasonMay 4, 2005
  23. C. Scott AnanianMay 4, 2005
  24. Chris MasonMay 4, 2005
  25. Linus TorvaldsMay 4, 2005
  26. Chris MasonMay 4, 2005
  27. Nicolas PitreMay 5, 2005
  28. Geert BoschMay 4, 2005
  29. Chris MasonMay 4, 2005
  30. Nicolas PitreMay 5, 2005
  31. Dan HolmsandMay 3, 2005
  32. C. Scott AnanianMay 3, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.