Re: [RFC 02/14] upload-pack: allow ref name and glob requests

> Jonathan Tan <jonathantanmy@google.com> writes:
>
>> Currently, while performing packfile negotiation [1], upload-pack allows
>> clients to specify their desired objects only as SHA-1s. This causes:
>> (a) vulnerability to failure when an object turns non-existent during
>>     negotiation, which may happen if, for example, upload-pack is
>>     provided by multiple Git servers in a load-balancing arrangement,
>>     and
>> (b) dependence on the server first publishing a list of refs with
>>     associated objects.
>>
>> To eliminate (a) and take a step towards eliminating (b), teach
>> upload-pack to support requests in the form of ref names and globs (in
>> addition to the existing support for SHA-1s) through a new line of the
>> form "want-ref <ref>" where ref is the full name of a ref, a glob
>> pattern, or a SHA-1. At the conclusion of negotiation, the server will
>> write "wanted-ref <SHA-1> <name>" for all requests that have been
>> specified this way.
>
> I am not sure if this "at the conclusion of" is sensible.  It is OK
> to assume that what the client side has is fixed, and it is probably
> OK to desire that what the server side has can change, but at the
> same time, it feels quite fragile to move the goalpost in between.

> Stepping back a bit, in an environment that involves multiple server
> instances that have inconsistent set of refs, can the negotiation
> even be sensibly and safely implemented?  The first server the
> client contacts may, in response to a "have", say "I do have that
> commit so you do not have to send its ancestors to me.  We found one
> cut-off point.  Please do explore other lines of histories."  The
> next server that concludes the negotiation exchange may not have
> that commit and will be unable to produce a pack that excludes the
> objects reachable from that commit---wouldn't that become a problem?

> One way to prevent such a problem from hurting clients may be for
> these multiple server instances to coordinate and make sure they
> have a shared perception of the common history among them.  Some
> pushes may have come to one instance but may not have propagated to
> other instances, and such a commit cannot be accepted as usable
> "have" if the servers anticipate that the final client request would
> go to any of the servers.  Otherwise the multiple server arrangement
> would not work safely, methinks.
>
> And if the servers are ensuring the safety using such a mechanism,
> they can use the same mechanism to restrain "faster" instances from
> sending too fresh state of refs that other instances haven't caught
> up to, which would mean they can present a consistent set of refs to
> the client in the first place, no?
>
> So I am not sure if the mechanism to request history by refname
> instead of the tip commit would help the multi-server environment as
> advertised.  It may help solving other problems, though (e.g. like
> "somebody pushed to update after the initial advertisement was sent
> out" which can happen even in a single server environment).

>> To be flexible with respect to client needs, the server does not
>> indicate an error if a "want-ref" line corresponds to no refs, but
>> instead relies on the client to ensure that what the user needs has been
>> fetched. For example, a client could reasonably expand an abbreviated
>> name "foo" to "want-ref foo", "want-ref refs/heads/foo", "want-ref
>> refs/tags/foo", among others, and ensure that at least one such ref has
>> been fetched.
>
> Cute.  This may be one way to implement the DWIM thing within the
> constraint of eventually wanting to go to "client speaks first, the
> server does not advertise things the client is not interested in"
> world.
>
> But at the same time it may end up bloating the set of refs the
> client asks instead.  Instead of receiving the advertisement and
> then sending one request after picking the matching one from it,
> the client needs to send "refs/{heads,tags,whatever}/foo".