git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/5] remote-curl: add 'get' capability

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Aug 2, 2022, 12:29 UTC
Message-ID
<4df4a1d679a1003ccd3a4dd049bc05141831970c.1659443384.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1300.v2.git.1659443384.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

A future change will want a way to download a file over HTTP(S) using the simplest of download mechanisms. We do not want to assume that the server on the other side understands anything about the Git protocol but could be a simple static web server.

Create the new 'get' capability for the remote helpers which advertises that the 'get' command is avalable. A caller can send a line containing 'get <url> <path>' to download the file at <url> into the file at <path>.

Reviewed-by: Josh Steadmon <steadmon@google.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 Documentation/gitremote-helpers.txt |  9 +++++++
 remote-curl.c                       | 28 ++++++++++++++++++++++
 t/t5557-http-get.sh                 | 37 +++++++++++++++++++++++++++++
 3 files changed, 74 insertions(+)
 create mode 100755 t/t5557-http-get.sh
diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt
index 6f1e269ae43..ed8da428c98 100644
--- a/Documentation/gitremote-helpers.txt
+++ b/Documentation/gitremote-helpers.txt
@@ -168,6 +168,9 @@ Supported commands: 'list', 'import'.
 	Can guarantee that when a clone is requested, the received
 	pack is self contained and is connected.
 
+'get'::
+	Can use the 'get' command to download a file from a given URI.
+
 If a helper advertises 'connect', Git will use it if possible and
 fall back to another capability if the helper requests so when
 connecting (see the 'connect' command under COMMANDS).
@@ -418,6 +421,12 @@ Supported if the helper has the "connect" capability.
 +
 Supported if the helper has the "stateless-connect" capability.
 
+'get' <uri> <path>::
+	Downloads the file from the given `<uri>` to the given `<path>`. If
+	`<path>.temp` exists, then Git assumes that the `.temp` file is a
+	partial download from a previous attempt and will resume the
+	download from that position.
+
 If a fatal error occurs, the program writes the error message to
 stderr and exits. The caller should expect that a suitable error
 message has been printed if the child closes the connection without
diff --git a/remote-curl.c b/remote-curl.c
index b8758757ece..72dfb8fb86a 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -1286,6 +1286,29 @@ static void parse_fetch(struct strbuf *buf)
 	strbuf_reset(buf);
 }
 
+static void parse_get(const char *arg)
+{
+	struct strbuf url = STRBUF_INIT;
+	struct strbuf path = STRBUF_INIT;
+	const char *space;
+
+	space = strchr(arg, ' ');
+
+	if (!space)
+		die(_("protocol error: expected '<url> <path>', missing space"));
+
+	strbuf_add(&url, arg, space - arg);
+	strbuf_addstr(&path, space + 1);
+
+	if (http_get_file(url.buf, path.buf, NULL))
+		die(_("failed to download file at URL '%s'"), url.buf);
+
+	strbuf_release(&url);
+	strbuf_release(&path);
+	printf("\n");
+	fflush(stdout);
+}
+
 static int push_dav(int nr_spec, const char **specs)
 {
 	struct child_process child = CHILD_PROCESS_INIT;
@@ -1564,9 +1587,14 @@ int cmd_main(int argc, const char **argv)
 				printf("unsupported\n");
 			fflush(stdout);
 
+		} else if (skip_prefix(buf.buf, "get ", &arg)) {
+			parse_get(arg);
+			fflush(stdout);
+
 		} else if (!strcmp(buf.buf, "capabilities")) {
 			printf("stateless-connect\n");
 			printf("fetch\n");
+			printf("get\n");
 			printf("option\n");
 			printf("push\n");
 			printf("check-connectivity\n");
diff --git a/t/t5557-http-get.sh b/t/t5557-http-get.sh
new file mode 100755
index 00000000000..09fb0bd9a8a
--- /dev/null
+++ b/t/t5557-http-get.sh
@@ -0,0 +1,37 @@
+#!/bin/sh
+
+test_description='test downloading a file by URL'
+
+. ./test-lib.sh
+
+. "$TEST_DIRECTORY"/lib-httpd.sh
+start_httpd
+
+test_expect_success 'get by URL: 404' '
+	test_when_finished "rm -f file.temp" &&
+	url="$HTTPD_URL/none.txt" &&
+	cat >input <<-EOF &&
+	capabilities
+	get $url file1
+	EOF
+
+	test_must_fail git remote-http $url <input 2>err &&
+	test_path_is_missing file1 &&
+	grep "failed to download file at URL" err
+'
+
+test_expect_success 'get by URL: 200' '
+	echo data >"$HTTPD_DOCUMENT_ROOT_PATH/exists.txt" &&
+
+	url="$HTTPD_URL/exists.txt" &&
+	cat >input <<-EOF &&
+	capabilities
+	get $url file2
+
+	EOF
+
+	git remote-http $url <input &&
+	test_cmp "$HTTPD_DOCUMENT_ROOT_PATH/exists.txt" file2
+'
+
+test_done
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 18 of 36 in “Bundle URIs II: git clone --bundle-uri”
  1. 0/5 Bundle URIs II: git clone --bundle-uriDerrick Stolee via GitGitGadget, Jul 25, 2022
  2. 1/5 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Jul 25, 2022
  3. Josh SteadmonJul 27, 2022
  4. Ævar Arnfjörð BjarmasonJul 27, 2022
  5. Derrick StoleeAug 1, 2022
  6. Ævar Arnfjörð BjarmasonAug 1, 2022
  7. 2/5 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, Jul 25, 2022
  8. Josh SteadmonJul 27, 2022
  9. Derrick StoleeAug 1, 2022
  10. 3/5 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, Jul 25, 2022
  11. 5/5 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, Jul 25, 2022
  12. 4/5 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, Jul 25, 2022
  13. Josh SteadmonJul 27, 2022
  14. Derrick StoleeAug 1, 2022
  15. Josh SteadmonJul 27, 2022
  16. Derrick StoleeAug 1, 2022
  17. 0/5 Bundle URIs II: git clone --bundle-uriDerrick Stolee via GitGitGadget, Aug 2, 2022
  18. 1/5 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Aug 2, 2022
  19. 2/5 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, Aug 2, 2022
  20. 3/5 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, Aug 2, 2022
  21. 4/5 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, Aug 2, 2022
  22. Junio C HamanoAug 2, 2022
  23. Derrick StoleeAug 4, 2022
  24. Junio C HamanoAug 4, 2022
  25. 5/5 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, Aug 2, 2022
  26. 0/5 Bundle URIs II: git clone --bundle-uriDerrick Stolee via GitGitGadget, Aug 9, 2022
  27. 1/5 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Aug 9, 2022
  28. 2/5 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, Aug 9, 2022
  29. 3/5 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, Aug 9, 2022
  30. Junio C HamanoAug 22, 2022
  31. Derrick StoleeAug 23, 2022
  32. Junio C HamanoAug 24, 2022
  33. 4/5 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, Aug 9, 2022
  34. 4/5 bundle-uri: add support for http(s):// and file://Teng Long, Aug 29, 2022
  35. Derrick StoleeAug 30, 2022
  36. 5/5 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, Aug 9, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.