Re: RFC: Resumable clone based on hybrid "smart" and "dumb" HTTP

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 10, 2016, 20:57 UTC
Message-ID: <xmqqh9hgz3km.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <CAGZ79kZMvxa5Np4GbShv_A6NZwVAqff94+d8MFTZwrZS+2CqeQ@mail.gmail.com>

Stefan Beller <sbeller@google.com> writes:

Show 6 quoted lines

>> 6. Once stored and indexed with .idx, clients run `git fsck
>> --lost-found` to discover the roots of the pack it downloaded. These
>> are saved as temporary references.
>
> jrn:
>> I suspect we can do even faster by making index-pack do the work

I somehow doubt it. Both index-pack and lost-found need to trace "object A depends on object B", but the similarity ends there. index-pack traces the delta dependency, which does not have any relation with the reachability, which is what lost-found needs to compute.

Show 7 quoted lines

>> 7. Client runs incremental fetch, and then deletes the temporary
>> references from 6.
>>
>>
>> An advantage to this process is its much more flexible for the server.
>> There is no additional pack-*.info file required. GC can organize
>> packs anyway it wants, etc.

I am not quite sure if that is an advantage, though. The second message proposes that the lost-found computation to be done by the client using *.pack, but any client, given the same *.pack, will compute the same result, so if the result is computed on the server side just once when the *.pack is prepared and downloaded to the client, it would give us a better overall resource utilization. And in essence, that was what the *.info file in the first message was.

The original proposal explicitly specified that the initial bulk transfer pack must cover all heads and no tags. For the purpose of offloading the clone traffic and making it resumable, there is no need for such a restriction, and I do like the fact that the second one loosened the requirement.