Re: git pack/unpack over bittorrent - works!

From: Luke Kenneth Casson Leighton <luke.leighton@gmail.com>
Date: Sep 2, 2010, 21:10 UTC
Message-ID: <AANLkTikSHXivniUk-1KU30Ws23ebnbDhOmjKmpmVH-Y9@mail.gmail.com>
In-Reply-To: <m3y6bjnadu.fsf@localhost.localdomain>

On Thu, Sep 2, 2010 at 9:45 PM, Jakub Narebski <jnareb@gmail.com> wrote:

> If I remember the discussion stalled (i.e. no working implementation),
> and one of the latest proposals was to have some way of recovering
> objects from partially downloaded file, and a way to request packfile
> without objects that got already downloaded.

 oo.  ouch.  i can understand why things stalled, then.  you're
effectively adding an extra layer in, and even if you could add a
unique naming scheme on those objects (if one doesn't already exist?),
those object might (or might not!) come up the second time round (for
reasons mentioned already - threads resulting in different deltas
being picked etc.) ... and if they weren't picked for the re-generated
pack, you'd have to _delete_ them from the receiving end so as to
avoid polluting the recipient's object store haaarrgh *spit*, *cough*.

ok.

 what _might_ work however iiiiIiis... to split the pack-object into
two parts.  or, to add an "extra part", to be more precise:

a) complete list of all objects. _just_ the list of objects. b) existing pack-object format/structure.

in this way, the sender having done all the hard work already of determining what objects are to go into a pack-object, transfers that *first*. _theeen_ you begin transferring the pack-object. theeeen, if the pack-object transfer is ever interrupted, you simply send back that list of objects, and ask "uhh, you know that list of objects we were talking about? well, here it is *splat* - are you able to recreate the pack-object from that, for me, and if so please gimme again"

and, 10^N-1 times out of 10^N, for reasons that shawn kindly explained, i bet you the answer would be "yes".

... um... in fact... um... i believe i'm merely talking about the .idx index file, aren't i? because... um... the index file contains the list of object refs in the pack, yes?

sooo.... taking a wild guess, here: if you were to parse the .idx file and extract the list of object-refs, and then pass that to "git pack-objects --window=0 --delta=0", would you end up with the exact same pack file, because you'd forced git pack-objects to only return that specific list of object-refs?

l.