Re: [RFC/PATCH 0/3] protocol v2

From: Shawn Pearce <spearce@spearce.org>
Date: Mar 4, 2015, 04:27 UTC
Message-ID: <CAJo=hJtgda4ATtPeLWbhgDPiTm9h-gzBkpknxsOdRddueUgTbw@mail.gmail.com>
In-Reply-To: <CACsJy8B_r_0nP9NyKFBnr9bXgwjx8dJkSVkHbZw+Mxin_YpZZw@mail.gmail.com>

On Tue, Mar 3, 2015 at 5:54 PM, Duy Nguyen <pclouds@gmail.com> wrote:

Show 10 quoted lines

> On Wed, Mar 4, 2015 at 12:13 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> My recollection is that the consensus from the last time we
>> discussed protocol revamping was to list one capability per packet
>> so that packet length limit does not matter, but you may want to
>> check with the list archive yourself.
>
> I couldn't find that consensus mail, but this one [1] is good enough
> evidence that we can hit packet length limit in capability line
> easily.
> With an escape hatch to allow maximum packet length up to  uint_max, I

The symbolic ref thing was done badly. There isn't an escape hatch in current v1 protocol sufficient to allow this but each ref should be its own pkt-line, or should be a small batch of refs per pkt-line, or the ref advertisement should be a data stream in a side-band-64k sort of format inside the pkt-line framing.

At 64k per frame of side-band there is plenty of data to header ratio that we don't need to escape to uint_max.

> Looks like one cap per pkt-line is winning..

Yes.

> [1] http://thread.gmane.org/gmane.comp.version-control.git/237929

Let me go on a different tangent a bit from the current protocol.

http://www.grpc.io/ was recently released and is built on the HTTP/2 standard. It uses protobuf as a proven extensibility mechanism. Including a full C based grpc stack just to speak the Git wire protocol is quite likely overkill, but I think the embedding of a proven extensible format inside of a bi-directional framed streaming system like HTTP/2 offers some good guidance.

Network protocol parsing is hard. Especially in languages like C where buffer overflows are possible. Or where a client could trivially DoS a server by sending a packet of size uint_max and the server naively trying to malloc() that buffer. Defining the network protocol in an IDL like protobuf 3 and being machine generated from stable well maintained code has its advantages.