Re: Git-aware HTTP transport

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 28, 2008, 04:42 UTC
Message-ID: <7vhc95iwcs.fsf@gitster.siamese.dyndns.org>
In-Reply-To: <20080828035018.GA10010@spearce.org>

"Shawn O. Pearce" <spearce@spearce.org> writes:

Show 7 quoted lines

> HTTP Redirects
> --------------
>
> If a POST request results in an HTTP 302 or 303 redirect response
> clients should retry the request by updating the URL and POSTing
> the same request to the new location.  Subsequent requests should
> still be sent to the original URL.

At the first reading I was confused because this seemed to contradict with the server pinning that is done by the payload level redirect.

Show 42 quoted lines

> Service upload-pack
> -------------------
>
> Prepares an estimated minimal pack to transfer new objects to the
> client.
>
> URL: $url/backend.git-http/upload-pack
> Content-Type: application/x-git; service=upload-pack
>
> The computation to select the minimal pack proceeds as follows
> (c = client, s = server):
>
>  init step:
>  (c) Use show-ref to obtain the advertised refs.
>  (c) Place any object seen in show-ref into set ADVERTISED.
>
>  (c) Build a set, WANT, of the objects from ADVERTISED the client
>      wants to fetch, based on what it saw from show-ref.
>
>  (c) Start a queue, C_PENDING, ordered by commit time (popping newest
>      first).  Add all client refs.  When a commit is popped from the
>      queue its parents should be automatically inserted back.  Commits
>      should only enter the queue once.
>
>  one compute step:
>  (c) Send an upload-pack request:
>
> 	C: 0011capabilities
> 	C: 0024thin-pack include-tag ofs-delta
> 	C: 0009want
> 	C: 0xxx<WANT list>
> 	C: 000bcommon
> 	C: 0xxx<COMMON list>
> 	C: 0009have
> 	C: 0xxx<HAVE list>
> 	C: 0000
>
>      The stream is organized into "sections", where each section is
>      composed of two git pkt-lines.  The first pkt-line provides the
>      name of the section ("capabilities", "want", "have", "common").
>      The second pkt-line has the binary SHA-1 ids which compose that
>      section.

It appears that you really meant "Binary", as opposed to "Hexadecimal" that show-ref example illustrate, judging from the later 3,276 number. I'd prefer hexadecimal here.

As a protocol specification, you'd eventually need to describe the pkt-line format, namely, (1) four hexadecimal digits that represents the length of the line (including that four bytes), followed by that many number of bytes as the line's payload, or (2) "0000" which is "flush". Also typically the text based line payload is LF terminated (hence the four-hexdigit length counts the terminating LF). Also "capabilities" need to be defined.

Show 7 quoted lines

>   (s) Parse the upload-pack request:
>
>       Verify all objects in WANT are reachable from refs.  As
>       this may require walking backwards through history to
>       the very beginning on invalid requests the server may
>       use a reasonable limit of commits (e.g. 1000) walked
>       beyond any ref tip before giving up.

I suspect moving as much work to the client side by erroring out and having the client restart from show-ref might be a better tradeoff (also this has been advertised as a security feature on the native protocol side).

Show 9 quoted lines

>       If any WANT object is not reachable, send an error:
>
> 	S: 001estatus error invalid want
>
>      Create an empty list, S_COMMON.
>
>      If 'common' was sent:
>
>      Load all objects into S_COMMON.

Security? Error out if some of them do not exist on the server end, at least.

Show 6 quoted lines

>   (s) Send the upload-pack response:
>
>      If the server has found a closed set of objects to pack, it
>      replies with the pack and the enabled capabilities.  The set
>      of enabled capabilities is limited to the intersection of
>      what the client requested and what the server supports.

Define "closed set".

>      The stream formatting rules are the same as the request.
>
>      The section "common" details the contents of S_COMMON,
>      that is all objects from HAVE that the server also has.

An object in HAVE that exists on the server end can be a descendant of many other HAVEs. Answering with that youngest one alone is enough, without the other HAVEs the server end also has as its ancestors, as they are redundant information.

Show 10 quoted lines

>   (c) Parse the upload-pack response:
>
>       If the status pkt-line is "status pack:"
>
>       Process the pack stream and update the local refs.
>
>       If the status pkt-line is "status continue":
>
>       Reset COMMON to the items in S_COMMON.  The new S_COMMON
>       should be a superset of the existing COMMON set.

Is there a way to detect bad clients that does not obey this rule without server side states?