RFC: adding xdelta compression to git
- From
- Alon Ziv <alonz@nolaviz.org>
- Date
- May 3, 2005, 03:57 UTC
- Message-ID
- <200505030657.38309.alonz@nolaviz.org>
Looking for novel methods of wasting my time :), I am considering adding xdelta to git.
I have two concrete proposals, both of which (IMO) are consistent with the git philosophy:
1. Add a git-deltify command, which will take two trees and replace the second tree's blobs with delta-blobs referring to the first tree. Each delta-blob is self-contained; from the outside it looks like any other blob, but internally it contains another blob reference + an xdelta. The only function which would need to understand the new format would be unpack_sha1_file. The scripting level will be in charge of deciding which trees to deltify (or undeltify--we could also have a "git-undeltify" command). A sane deltification schedule, for example, could be to always keep tagged versions as stand-alone objects, and deltify intermediate versions against the latest tag. It would also do its best to avoid delta chains (i.e. a delta referring to another delta). Pros: * Interoperates with the existing structure (including pull/push) with almost no changes to existing infrastructure. Cons: * Changes the repository format. * Some performance impact (probably quite small). * Same blob may have different representation in two repositories (one compressed, on deltified). [I am not sure this is really a bad thing...]
2. Add a completely external framework which manages a "deltas repository" of deltas. The shadow repository will contain delta objects between selected trees; again the scripts will need to populate it. Pros: * No changes at all to existing code. Cons: * Push/pull tools will need to be taught to talk with the new "deltas repository". * Synchronization between the deltas repository and the real one may be lost, leading to odd failures.
Personally I'm rooting for #1 above... I would like to begin implementation in a few days, so any discussion will be useful.
-az