git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 16/24] protocol v2: add server-side "bundle-uri" skeleton

From
Ævar Arnfjörð Bjarmason via GitGitGadget <gitgitgadget@gmail.com>
Date
May 20, 2022, 18:40 UTC
Message-ID
<94f552e771644aef60c23e5d9379723951542031.1653072042.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1234.git.1653072042.gitgitgadget@gmail.com>
From: =?UTF-8?q?=C3=86var=20Arnfj=C3=B6r=C3=B0=20Bjarmason?=
 <avarab@gmail.com>

Add a skeleton server-side implementation of a new "bundle-uri" command to protocol v2. This will allow conforming clients to optionally seed their initial clones or incremental fetches from URLs containing "*.bundle" files created with "git bundle create".

This change only performs the basic boilerplate of advertising a new protocol v2 capability. The new 'bundle-uri' capability allows a client to request a list of bundles. Right now, the server only returns a flush packet, which corresponds to an empty advertisement.

The critical bit right now is that the new boolean uploadPack.adverstiseBundleURIs config value signals whether or not this capability should be advertised at all.

RFC TODO: Write documentation.
RFC TODO: Compare to original implementation.
Co-authored-by: Derrick Stolee <derrickstolee@github.com>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 bundle-uri.c         | 36 ++++++++++++++++++++++++++++++++++++
 bundle-uri.h         |  7 +++++++
 serve.c              |  6 ++++++
 t/t5701-git-serve.sh | 40 +++++++++++++++++++++++++++++++++++++++-
 4 files changed, 88 insertions(+), 1 deletion(-)
diff --git a/bundle-uri.c b/bundle-uri.c
index 601e6e87822..79a6b7ed8b8 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -557,6 +557,42 @@ cleanup:
 	return result;
 }
 
+/**
+ * API for serve.c.
+ */
+
+static int advertise_bundle_uri = -1;
+
+int bundle_uri_advertise(struct repository *r, struct strbuf *value)
+{
+	if (advertise_bundle_uri != -1)
+		goto cached;
+
+	advertise_bundle_uri = 0;
+	git_config_get_maybe_bool("uploadpack.advertisebundleuris", &advertise_bundle_uri);
+
+cached:
+	return advertise_bundle_uri;
+}
+
+int bundle_uri_command(struct repository *r,
+		       struct packet_reader *request)
+{
+	struct packet_writer writer;
+	packet_writer_init(&writer, 1);
+
+	while (packet_reader_read(request) == PACKET_READ_NORMAL)
+		die(_("bundle-uri: unexpected argument: '%s'"), request->line);
+	if (request->status != PACKET_READ_FLUSH)
+		die(_("bundle-uri: expected flush after arguments"));
+
+	/* TODO: Implement the communication */
+
+	packet_writer_flush(&writer);
+
+	return 0;
+}
+
 /**
  * General API for {transport,connect}.c etc.
  */
diff --git a/bundle-uri.h b/bundle-uri.h
index adce5dc07e2..9f6fc2d75f9 100644
--- a/bundle-uri.h
+++ b/bundle-uri.h
@@ -4,6 +4,7 @@
 #include "hashmap.h"
 #include "strbuf.h"
 
+struct packet_reader;
 struct repository;
 struct strbuf;
 struct string_list;
@@ -97,6 +98,12 @@ int parse_bundle_list_in_config_format(const char *uri,
  */
 int fetch_bundle_uri(struct repository *r, const char *uri);
 
+/**
+ * API for serve.c.
+ */
+int bundle_uri_advertise(struct repository *r, struct strbuf *value);
+int bundle_uri_command(struct repository *r, struct packet_reader *request);
+
 /**
  * General API for {transport,connect}.c etc.
  */
diff --git a/serve.c b/serve.c
index b3fe9b5126a..f3e0203d2c6 100644
--- a/serve.c
+++ b/serve.c
@@ -8,6 +8,7 @@
 #include "protocol-caps.h"
 #include "serve.h"
 #include "upload-pack.h"
+#include "bundle-uri.h"
 
 static int advertise_sid = -1;
 static int client_hash_algo = GIT_HASH_SHA1;
@@ -136,6 +137,11 @@ static struct protocol_capability capabilities[] = {
 		.advertise = always_advertise,
 		.command = cap_object_info,
 	},
+	{
+		.name = "bundle-uri",
+		.advertise = bundle_uri_advertise,
+		.command = bundle_uri_command,
+	},
 };
 
 void protocol_v2_advertise_capabilities(void)
diff --git a/t/t5701-git-serve.sh b/t/t5701-git-serve.sh
index 1896f671cb3..f21e5e9d33d 100755
--- a/t/t5701-git-serve.sh
+++ b/t/t5701-git-serve.sh
@@ -13,7 +13,7 @@ test_expect_success 'test capability advertisement' '
 	wrong_algo sha1:sha256
 	wrong_algo sha256:sha1
 	EOF
-	cat >expect <<-EOF &&
+	cat >expect.base <<-EOF &&
 	version 2
 	agent=git/$(git version | cut -d" " -f3)
 	ls-refs=unborn
@@ -21,8 +21,11 @@ test_expect_success 'test capability advertisement' '
 	server-option
 	object-format=$(test_oid algo)
 	object-info
+	EOF
+	cat >expect.trailer <<-EOF &&
 	0000
 	EOF
+	cat expect.base expect.trailer >expect &&
 
 	GIT_TEST_SIDEBAND_ALL=0 test-tool serve-v2 \
 		--advertise-capabilities >out &&
@@ -342,4 +345,39 @@ test_expect_success 'basics of object-info' '
 	test_cmp expect actual
 '
 
+test_expect_success 'test capability advertisement with uploadpack.advertiseBundleURIs' '
+	test_config uploadpack.advertiseBundleURIs true &&
+
+	cat >expect.extra <<-EOF &&
+	bundle-uri
+	EOF
+	cat expect.base \
+	    expect.extra \
+	    expect.trailer >expect &&
+
+	GIT_TEST_SIDEBAND_ALL=0 test-tool serve-v2 \
+		--advertise-capabilities >out &&
+	test-tool pkt-line unpack <out >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'basics of bundle-uri: dies if not enabled' '
+	test-tool pkt-line pack >in <<-EOF &&
+	command=bundle-uri
+	0000
+	EOF
+
+	cat >err.expect <<-\EOF &&
+	fatal: invalid command '"'"'bundle-uri'"'"'
+	EOF
+
+	cat >expect <<-\EOF &&
+	ERR serve: invalid command '"'"'bundle-uri'"'"'
+	EOF
+
+	test_must_fail test-tool serve-v2 --stateless-rpc <in >out 2>err.actual &&
+	test_cmp err.expect err.actual &&
+	test_must_be_empty out
+'
+
 test_done

-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 15 of 25 in “[RFC] Bundle URIs Combined RFC”
  1. 00/24 [RFC] Bundle URIs Combined RFCDerrick Stolee via GitGitGadget, May 20, 2022
  2. 01/24 docs: document bundle URI standardDerrick Stolee via GitGitGadget, May 20, 2022
  3. 02/24 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, May 20, 2022
  4. 06/24 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, May 20, 2022
  5. 05/24 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  6. 04/24 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, May 20, 2022
  7. 03/24 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, May 20, 2022
  8. 12/24 bundle-uri: unit test "key=value" parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  9. 10/24 bundle-uri: create base key-value pair parsingDerrick Stolee via GitGitGadget, May 20, 2022
  10. 07/24 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  11. 11/24 bundle-uri: create "key=value" line parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  12. 14/24 bundle-uri: parse bundle list in config formatDerrick Stolee via GitGitGadget, May 20, 2022
  13. 09/24 bundle-uri: create bundle_list struct and helpersDerrick Stolee via GitGitGadget, May 20, 2022
  14. 15/24 bundle-uri: fetch a list of bundlesDerrick Stolee via GitGitGadget, May 20, 2022
  15. 16/24 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  16. 08/24 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, May 20, 2022
  17. 13/24 bundle-uri: limit recursion depth for bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  18. 18/24 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  19. 17/24 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  20. 19/24 bundle-uri: serve URI advertisement from bundle.* configDerrick Stolee via GitGitGadget, May 20, 2022
  21. 22/24 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, May 20, 2022
  22. 24/24 t5601: basic bundle URI testsDerrick Stolee via GitGitGadget, May 20, 2022
  23. 20/24 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  24. 21/24 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  25. 23/24 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, May 20, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.