Re: [RFC/PATCH 0/3] protocol v2

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 27, 2015, 23:05 UTC
Message-ID: <xmqqioenhs4p.fsf@gitster.dls.corp.google.com>
In-Reply-To: <xmqqzj80l9c7.fsf@gitster.dls.corp.google.com>

Junio C Hamano <gitster@pobox.com> writes:

Show 6 quoted lines

> I do not think v1 can be fixed by "send one ref with capability,
> newer client may respond immediately so we can stop enumerating
> remaining refs and older one will get stuck so we can have a timeout
> to see if the connection is from the newer one, and send the rest
> for the older client", because anything that involves such a timeout
> would not reliably work over WAN.

Just for fun, I was trying to see if there is a hole in the current protocol that allows a new client to talk a valid v1 protocol exchange with existing, deployed servers without breaking, while letting it to know a new server that it is a new client and it does not want to get blasted by megabytes of ref advertisement.

The idea is to find a request that can be sent as the first utterance by the client to an old server that is interpreted as a no-op and can be recognised by a new server as such a "no-op probe". If there is such a request, then the exchange can go like this with (new client, old server) pair:

    - new client connects and sends that no-op.

    - old server starts blasting the ref advertisement

    - new client monitors and notices that the other side
      started speaking, and the ref advertisement lacks the
      capability bit for new protocol.

    - new client accepts the ref advertisement and does the v1
      protocol thing as a follow-up to what it already sent.

As long as the first one turns out to be no-op for old server, we would be OK. On the other hand, (new client, new server) pair would go like this:

    - new client connects and sends that no-op.

    - new server notices that there is already a data from the
      client, and recognises the "no-op probe".

    - new server gives the first v2 protocol message with
      capability.

    - new client notices thqat the other side started speaking, and
      it is the first v2 protocol message.

    - both sides happily speak v2.

and (old client, new server) pair would go like this:

    - old client connects and waits.

    - new server notices that there is *no* data sent from the
      client and decides that the other side is a v1 client.  It
      starts blasting the ref advertisement.

    - both sides happily speak v1 from here on.

A misdetected case between (new client, new server) pair might go like this:

    - new client connects and sends that no-op.

    - new server accepts the connection, but that no-op probe has
      not arrived yet.".  It misdetects the other side as a v1
      client and it starts blasting the ref advertisement.

    - new client notices that the ref advertisement has the
      capability bit and the server is capable of v2 protocol.  it
      waits until the server sends "sorry, I misdetected" message.

    - new server eventually notices the "no-op probe" while blasting
      the ref advertisement and it can stop in the middle.
      hopefully this can happen after only sending a few kilobytes
      among megabytes of ref advertisement data ;-).  The server
      sends "sorry, I misdetected" message to synchronise.

    - both sides happily speak v2 from here on.

So the topic of this exercise ("just for fun") is to see if there is such a no-op request the client side can send as the first thing for probing.

On the fetch side, the first response upload-pack expects are one of:

  - "want " followed by an object name.
  - "shallow " followed by an object name.
  - "deepen " followed by a positive integer.

And there _is_ a hole ;-). The parsing of "shallow " object name is done in such a way that an object name that passes get_sha1_hex() that results in a NULL return from parse_object() is _ignored_. So a new client can use "shallow 0{40}" as a no-op probe.

It appears that on the push side, there is a similar hole that can be used. receive-pack expects either "shallow ", "push-cert" or the refname updates (i.e. two "[0-9a-f]{40}" followed by a refname); the parsing of "shallow " is not as loose as the fetch side in that using a "shallow 0{40}" as a no-op probe will end up causing prepare_shallow_info() sift the "0{40}" object name into "theirs", but I think it will be ignored at the end as "unreachable cruft" without causing harm.

I am _not_ proposing that we should go this route, at least not yet. I am merely pointing out that an in-place sidegrade from v1 to a protocol that avoids the megabyte-advertisement-at-the-beginning seems to be possible, as a food for thought.