Re: Resumable clone/Gittorrent (again) - stable packs?

On Wed, Jan 5, 2011 at 18:29, Zenaan Harkness <zen@freedbms.net> wrote:

No. We have been trying to avoid doing that, because it ties us into one particular compression scheme. We can't tune the algorithm and get better compression later, because it would generate a different pack. We also rely on the system's libz to generate the compressed data. A version change to libz may generate a different encoding for the same uncompressed data, simply because they made a tweak to how the compression was performed. Likewise our own delta compression code can be tweaked to produce a different (but logically identical) delta between the same two objects.

Right now packs aren't deterministic because they use multiple threads to generate the deltas, the thread scheduling impacts which base objects deltas are tried against because threads can steal work from each other if one finishes before the other one. Disabling threading entirely slows down delta compression considerably on multi-core machines, but does remove this work-stealing, making the pack deterministic... but only for this exact Git binary, with this same shared libz. If the system libz or Git changes, all bets are off.

We've been down this road before; we don't want to box ourselves into a tight corner by setting for all time these tunable portions of the compression algorithms.