Re: Is anyone working on a next-gen Git protocol?

From: Jeff King <peff@peff.net>
Date: Oct 7, 2012, 22:08 UTC
Message-ID: <20121007220833.GD1743@sigill.intra.peff.net>
In-Reply-To: <CACBZZX6b+3P8M+z+X13k9Pq3tvVUfs_k1=foQVreX8K801=efQ@mail.gmail.com>

On Sun, Oct 07, 2012 at 09:57:56PM +0200, Ævar Arnfjörð Bjarmason wrote:

> Has anyone started working on a next-gen Git protocol as a result of
> this discussion? If not I thought I'd give it a shot if/when I have
> time.

I haven't, and don't really plan on it soon (I have a few smaller things I'm working on, then I'd like to look into the EWAH bitmap stuff from Shawn next).

Show 7 quoted lines

> The current protocol is basically (S = Server, C = Client)
> 
>  S: Spew out first ref
>  S: Advertisement of capabilities
>  S: Dump of all our refs
>  C/S: Declare wanted refs, negotiate with server
>  S: Send pack to client, if needed

In the "C" portion there, there is also "client acknowledges a subset of capabilities shown by server" while it is declaring wanted refs.

Show 5 quoted lines

> And I thought I'd basically turn it into:
> 
>  C: Connect to server, declare what protocol we understand
>  C: Advertisement of capabilities
>  S: Advertisement of capabilities

The capability negotiation right now is that the server offers and the client accepts. Are you swapping that so that the client offers and the server accepts? Or are you thinking that they would be sent simultaneously here? That could drop one round-trip (it's probably not that important for git-over-tcp, but smart-http cares a lot about round trips). But it also introduces a complexity with future additions (one side may not know how to present its capabilities until understanding what the other side can do).

>  C/S: Negotiate what we want

Refs we want, or capabilities we want?

Show 5 quoted lines

>  C/S: Same as v1, without the advertisement of capabilities, and maybe
> don't dump refs at all
> 
> Basically future-proofing it by having the client say what it supports
> to begin with along with what it can handle (like in HTTP).

I feel like this "maybe..." bit needs more fleshed out before designing the first part. I like the idea of future-proofing first and then adding new features second, but what does the "don't advertise all refs" protocol look like? Presumably the client is going to say "I'm interested in refs/heads/* and refs/tags/*" or something. Does that come with the capabilities? Or is it a new protocol phase?

I think we need to know what the second half of the two-step process will look like to be sure the first half will accommodate it (and the answer may be as simple as saying "they're not sending capabilities, they're sending arbitrary key/value items, with the knowledge that the other side may not understand particular keys, and we have to be prepared to handle both cases).

Show 15 quoted lines

> Then in the negotiation phase the client & server would go back &
> forth about what they want & how they want it. I'd planned to
> implement something like:
> 
>     C: want_refs refs/heads/*
>     S: OK to that
>     C: want_refs refs/tags/*
>     S: OK to that
> 
> Or:
> 
>     C: want_refs refs/heads/master
>     S: OK to that
>     C: want_refs refs/tags/v*
>     S: OK to that

That seems simple. But how will it work over smart-http? Are we adding a round-trip to do want_refs negotiation?

-Peff