Re: [ANNOUNCE] Git Merge Contributor Summit topic planning

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 1, 2017, 18:06 UTC
Message-ID: <xmqqh94dpyzs.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20170201145300.4pn3faodhdb72jly@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

Show 13 quoted lines

> If you _can_ do that latter part, and you take "I only care about
> resumability" to the simplest extreme, you'd probably end up with a
> protocol more like:
>
>   Client: I need a packfile with this want/have
>   Server: OK, here it is; its opaque id is XYZ.
>   ... connection interrupted ...
>   Client: It's me again. I have up to byte N of pack XYZ
>   Server: OK, resuming
>           [or: I don't have XYZ anymore; start from scratch]
>
> Then generating XYZ and generating that bundle are basically the same
> task.

The above allows a simple and naive implementation of generating a packstream and "tee"ing it to a spool file to be kept while sending to the first client that asks XYZ.

The story I heard from folks who run git servers at work for Android and other projects, however, is that they rarely see two requests with want/have that result in an identical XYZ, unless "have" is an empty set (aka "clone"). In a busy repository, between two clone requests relatively close together, somebody would be pushing, so you'd need many XYZs in your spool even if you want to support only the "clone" case.

So in the real life, I think that the exchange needs to be more like this:

    C: I need a packfile with this want/have
    ... C/S negotiate what "have"s are common ...
    S: Sorry, but our negitiation indicates that you are way too
       behind.  I'll send you a packfile that brings you up to a
       slightly older set of "want", so pretend that you asked for
       these slightly older "want"s instead.  The opaque id of that
       packfile is XYZ.  After getting XYZ, come back to me with
       your original set of "want"s.  You would give me more recent
       "have" in that request.  
    ... connection interrupted ...
    C: It's me again.  I have up to byte N of pack XYZ
    S: OK, resuming (or: I do not have it anymore, start from scratch)
    ... after 0 or more iterations C fully receives and digests XYZ ...

and then the above will iterate until the server does not have to say "Sorry but you are way too behind" and returns a packfile without having to tweak the "want".

That way, you can limit the number of XYZ you would need to keep to a reasonable number.

The recent proposal by Jonathan Tan also allows the server side to tweak the final tips the client receives after the protocol exchange started. I suspect the above two will become related.