git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] archive: allow archive over HTTP(S) with proto v2

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Sep 14, 2018, 05:57 UTC
Message-ID
<20180914055703.GC219147@aiede.svl.corp.google.com>
In-Reply-To
<20180912053519.31085-4-steadmon@google.com>
Hi,
Josh Steadmon wrote:
> Subject: archive: allow archive over HTTP(S) with proto v2
It's interesting how little this has to touch the client.
Show 6 quoted lines
> Signed-off-by: Josh Steadmon <steadmon@google.com>
> ---
>  builtin/archive.c  |  8 +++++++-
>  http-backend.c     | 10 +++++++++-
>  transport-helper.c |  5 +++--
>  3 files changed, 19 insertions(+), 4 deletions(-)
[....]
Show 9 quoted lines
> --- a/builtin/archive.c
> +++ b/builtin/archive.c
> @@ -87,7 +87,13 @@ static int run_remote_archiver(int argc, const char **argv,
>  		status = packet_reader_read(&reader);
>  		if (status != PACKET_READ_FLUSH)
>  			die(_("git archive: expected a flush"));
> -	}
> +	} else if (version == protocol_v2 &&
> +		   starts_with(transport->url, "http"))

As Stefan noticed, this starts_with test seems a bit too loose. For example, what happens if I try an scp-style SSH URL like http.example.com:path/to/repo, a local path like http/foo/bar, or a custom protocol like httplikebutbetter://path/to/repo (honest question: I haven't tried)?

Show 5 quoted lines
> +		/*
> +		 * Commands over HTTP require two requests, so there's an
> +		 * additional server response to parse.
> +		 */
> +		discover_version(&reader);

Can this be made consistent with the non-http case? The original capabilities (/info/refs) response told us what protocol version the server wants to use, which means that a hypothetical protocol v3 could use a completely different request format for the followup commands: so could the server omit the protocol version in the v2 /git-upload-archive response? Alternatively, if we want to include the protocol version again, could we do that in stateful protocols as well?

Related question: what should happen if the two responses declare different protocol versions? Should we diagnose that as a protocol error?

[...]
Show 7 quoted lines
> --- a/http-backend.c
> +++ b/http-backend.c
> @@ -32,6 +32,7 @@ struct rpc_service {
>  static struct rpc_service rpc_service[] = {
>  	{ "upload-pack", "uploadpack", 1, 1 },
>  	{ "receive-pack", "receivepack", 0, -1 },
> +	{ "upload-archive", "uploadarchive", 1, 1 },

shell.c orders these in almost-alphabetical order (receive-pack, upload-pack, upload-archive). I guess they should both use actual alphabetical order? (If you agree, then please feel free to do that in a separate patch.)

[...]
Show 6 quoted lines
> @@ -637,6 +638,12 @@ static void service_rpc(struct strbuf *hdr, char *service_name)
>  	struct rpc_service *svc = select_service(hdr, service_name);
>  	struct strbuf buf = STRBUF_INIT;
>  
> +	if (!strcmp(service_name, "git-upload-archive")) {
> +		/* git-upload-archive doesn't need --stateless-rpc */

This comment doesn't seem actionable. Can it say why? E.g. "[...] because an upload-archive command always involves a single round-trip". Or alternatively, I think it's fine to omit the comment.

> +		argv[1] = ".";
> +		argv[2] = NULL;
> +	}
[...]
Show 7 quoted lines
> @@ -713,7 +720,8 @@ static struct service_cmd {
>  	{"GET", "/objects/pack/pack-[0-9a-f]{40}\\.idx$", get_idx_file},
>  
>  	{"POST", "/git-upload-pack$", service_rpc},
> -	{"POST", "/git-receive-pack$", service_rpc}
> +	{"POST", "/git-receive-pack$", service_rpc},
> +	{"POST", "/git-upload-archive$", service_rpc},
Same comment about services seeming to be in a randomish order.
[...]
Show 6 quoted lines
> --- a/transport-helper.c
> +++ b/transport-helper.c
> @@ -605,7 +605,8 @@ static int process_connect_service(struct transport *transport,
>  		ret = run_connect(transport, &cmdbuf);
>  	} else if (data->stateless_connect &&
>  		   (get_protocol_version_config() == protocol_v2) &&

(not about this patch) These parens don't help --- they make it harder for me to read, especially with the new parens to try to match them up with.

> -		   !strcmp("git-upload-pack", name)) {
> +		   (!strcmp("git-upload-pack", name) ||
> +		    !strcmp("git-upload-archive", name))) {

A part of me wonders about the wasted cycles comparing to "git-upload-" twice, but (1) it is tiny relative to actually serving the request and (2) if we're lucky, the compiler (or a compiler of the future) inlines the strcmp call and could optimize it out.

[...]
Show 7 quoted lines
> @@ -639,7 +640,7 @@ static int connect_helper(struct transport *transport, const char *name,
>  
>  	/* Get_helper so connect is inited. */
>  	get_helper(transport);
> -	if (!data->connect)
> +	if (!data->connect && !data->stateless_connect)
>  		die(_("operation not supported by protocol"));

I don't understand this part. Can you explain it further (possibly by putting it in its own patch)?

Thanks for a pleasant read, Jonathan

Previous: Josh SteadmonNext: Jonathan Nieder
Message 19 of 33 in “Add proto v2 archive command with HTTP support”
  1. Josh SteadmonSep 12, 2018
  2. 1/3 archive: use packet_reader for communicationsJosh Steadmon, Sep 12, 2018
  3. Stefan BellerSep 12, 2018
  4. Junio C HamanoSep 13, 2018
  5. Junio C HamanoSep 13, 2018
  6. 2/3 archive: implement protocol v2 archive commandJosh Steadmon, Sep 12, 2018
  7. Stefan BellerSep 12, 2018
  8. Ævar Arnfjörð BjarmasonSep 13, 2018
  9. Jonathan NiederSep 14, 2018
  10. Ævar Arnfjörð BjarmasonSep 14, 2018
  11. Junio C HamanoSep 14, 2018
  12. Jonathan NiederSep 14, 2018
  13. Junio C HamanoSep 13, 2018
  14. Jonathan NiederSep 14, 2018
  15. 3/3 archive: allow archive over HTTP(S) with proto v2Josh Steadmon, Sep 12, 2018
  16. Stefan BellerSep 12, 2018
  17. Junio C HamanoSep 13, 2018
  18. Josh SteadmonSep 27, 2018
  19. Jonathan NiederSep 14, 2018
  20. Jonathan NiederSep 14, 2018
  21. 0/4 Add proto v2 archive command with HTTP supportJosh Steadmon, Sep 27, 2018
  22. 1/4 archive: follow test standards around assertionsJosh Steadmon, Sep 27, 2018
  23. Stefan BellerSep 27, 2018
  24. 2/4 archive: use packet_reader for communicationsJosh Steadmon, Sep 27, 2018
  25. Stefan BellerSep 27, 2018
  26. 3/4 archive: implement protocol v2 archive commandJosh Steadmon, Sep 27, 2018
  27. 4/4 archive: allow archive over HTTP(S) with proto v2Josh Steadmon, Sep 27, 2018
  28. Stefan BellerSep 27, 2018
  29. Jonathan NiederSep 27, 2018
  30. Junio C HamanoSep 27, 2018
  31. Josh SteadmonSep 27, 2018
  32. Junio C HamanoSep 28, 2018
  33. Josh SteadmonSep 27, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.