git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 06/13] bundle-uri client: add transfer.injectBundleURI support

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Aug 5, 2021, 15:07 UTC
Message-ID
<RFC-patch-06.13-3427cb4de7-20210805T150534Z-avarab@gmail.com>
In-Reply-To
<RFC-cover-00.13-0000000000-20210805T150534Z-avarab@gmail.com>

Add the ability to inject "fake" bundle URIs into the newly supported bundle-uri dialog. As discussed in the added documentation this allows us to pretend as though the remote supports bundle URIs.

This will be useful both for ad-hoc testing, and for the real use-case of retrofitting bundle URI support on-the-fly, i.e. to have:

	git -c transfer.injectBundleURI "file://$(pwd)/local.bdl" \
	clone https://example.com/git.git"
Be similar in spirit to:
	git clone --reference local-clone.git --disassociate \
	https://example.com/git.git"
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config/transfer.txt     | 20 ++++++++++++
 t/lib-t5730-protocol-v2-bundle-uri.sh | 46 +++++++++++++++++++++++++++
 transport.c                           | 33 +++++++++++++++++++
 3 files changed, 99 insertions(+)
diff --git a/Documentation/config/transfer.txt b/Documentation/config/transfer.txt
index 505126a780..a9aff7f982 100644
--- a/Documentation/config/transfer.txt
+++ b/Documentation/config/transfer.txt
@@ -73,3 +73,23 @@ transfer.unpackLimit::
 transfer.advertiseSID::
 	Boolean. When true, client and server processes will advertise their
 	unique session IDs to their remote counterpart. Defaults to false.
+
+transfer.injectBundleURI::
+	Allows for the injection of `bundle-uri` lines into the
+	protocol v2 transport dialog (see `protocol.version` in
+	linkgit:git-config[1]). See `bundle-uri` in
+	link:technical/protocol-v2.html[the Git Wire Protocol, Version
+	2] documentation for what the format looks like.
++
+Can be given more than once, each key being injected as one line into
+the dialog.
++
+This is useful for testing the `bundle-uri` facility, and to e.g. use
+linkgit:git-clone[1] to clone from a server which does not support
+`bundle-uri`, but where the clone can benefit from getting some or
+most of the data from a static bundle retrieved from elsewhere.
++
+Impacts any command that uses the transport to communicate with remote
+linkgit:git-upload-pack[1] processes, e.g. linkgit:git-clone[1],
+linkgit:git-fetch[1] and the linkgit:git-ls-remote-bundle-uri[1]
+inspection command, this includes the `file://` protocol.
diff --git a/t/lib-t5730-protocol-v2-bundle-uri.sh b/t/lib-t5730-protocol-v2-bundle-uri.sh
index fccf7ccaa2..e5d17f6a81 100644
--- a/t/lib-t5730-protocol-v2-bundle-uri.sh
+++ b/t/lib-t5730-protocol-v2-bundle-uri.sh
@@ -270,3 +270,49 @@ test_expect_success "ls-remote-bundle-uri --[no-]quiet with $T5730_PROTOCOL:// u
 	test_must_be_empty err &&
 	test_cmp out.expect out.actual
 '
+
+test_expect_success "ls-remote-bundle-uri with -c transfer.injectBundleURI using with $T5730_PROTOCOL:// using protocol v2" '
+	test_when_finished "rm -f log" &&
+
+	test_config -C "$T5730_PARENT" uploadpack.bundleURI \
+		"$T5730_BUNDLE_URI_ESCAPED" &&
+
+	cat >expect <<-\EOF &&
+	https://injected.example.com/fake-1.bdl
+	https://injected.example.com/fake-2.bdl
+	EOF
+	GIT_TRACE_PACKET="$PWD/log" \
+	git \
+		-c protocol.version=2 \
+		-c transfer.injectBundleURI="https://injected.example.com/fake-1.bdl" \
+		-c transfer.injectBundleURI="https://injected.example.com/fake-2.bdl" \
+		ls-remote-bundle-uri \
+		"$T5730_URI" \
+		>actual 2>err &&
+	test_cmp expect actual &&
+	test_path_is_missing log
+'
+
+test_expect_success "ls-remote-bundle-uri with bad -c transfer.injectBundleURI protocol v2 with $T5730_PROTOCOL://" '
+	test_when_finished "rm -f log" &&
+
+	test_config -C "$T5730_PARENT" uploadpack.bundleURI \
+		"$T5730_BUNDLE_URI_ESCAPED" &&
+
+	cat >err.expect <<-\EOF &&
+	error: bad (empty) transfer.injectBundleURI
+	error: could not get the bundle-uri list
+	EOF
+
+	test_must_fail env \
+		GIT_TRACE_PACKET="$PWD/log" \
+		git \
+		-c protocol.version=2 \
+		-c transfer.injectBundleURI \
+		ls-remote-bundle-uri \
+		"$T5730_URI" \
+		>out 2>err.actual &&
+	test_must_be_empty out &&
+	test_cmp err.expect err.actual &&
+	test_path_is_missing log
+'
diff --git a/transport.c b/transport.c
index 286c96269d..bd8a914652 100644
--- a/transport.c
+++ b/transport.c
@@ -1501,14 +1501,47 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs)
 	return rc;
 }
 
+struct config_cb {
+	struct transport *transport;
+	int configured;
+	int ret;
+};
+
+static int bundle_uri_config(const char *var, const char *value, void *data)
+{
+	struct config_cb *cb = data;
+	struct transport *transport = cb->transport;
+	struct string_list *uri = &transport->bundle_uri;
+
+	if (!strcmp(var, "transfer.injectbundleuri")) {
+		cb->configured = 1;
+		if (!value)
+			cb->ret = error(_("bad (empty) transfer.injectBundleURI"));
+		else if (bundle_uri_parse_line(uri, value) < 0)
+			cb->ret = error(_("bad transfer.injectBundleURI: '%s'"),
+					value);
+		return 0;
+	}
+	return 0;
+}
+
 int transport_get_remote_bundle_uri(struct transport *transport, int quiet)
 {
 	const struct transport_vtable *vtable = transport->vtable;
+	struct config_cb cb = {
+		.transport = transport,
+	};
 
 	/* Lazily configured */
 	if (transport->got_remote_bundle_uri++)
 		return 0;
 
+	git_config(bundle_uri_config, &cb);
+
+	/* Our own config can fake it up with transport.injectBundleURI */
+	if (cb.configured)
+		return cb.ret;
+
 	/*
 	 * This is intentionally below the transport.injectBundleURI,
 	 * we want to be able to inject into protocol v0, or into the
-- 
2.33.0.rc0.646.g585563e77f
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 11 of 26 in “Add bundle-uri: resumably clones, static "dumb" CDN etc.”
  1. 00/13 Add bundle-uri: resumably clones, static "dumb" CDN etc.Ævar Arnfjörð Bjarmason, Aug 5, 2021
  2. 01/13 serve: add command to advertise bundle URIsÆvar Arnfjörð Bjarmason, Aug 5, 2021
  3. Derrick StoleeAug 10, 2021
  4. Ævar Arnfjörð BjarmasonAug 23, 2021
  5. 02/13 bundle-uri client: add "bundle-uri" parsing + testsÆvar Arnfjörð Bjarmason, Aug 5, 2021
  6. 04/13 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason, Aug 5, 2021
  7. 03/13 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason, Aug 5, 2021
  8. 05/13 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason, Aug 5, 2021
  9. 07/13 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason, Aug 5, 2021
  10. 09/13 fetch-pack: add a deref_without_lazy_fetch_extended()Ævar Arnfjörð Bjarmason, Aug 5, 2021
  11. 06/13 bundle-uri client: add transfer.injectBundleURI supportÆvar Arnfjörð Bjarmason, Aug 5, 2021
  12. 08/13 bundle.h: make "fd" version of read_bundle_header() publicÆvar Arnfjörð Bjarmason, Aug 5, 2021
  13. 10/13 fetch-pack: move --keep=* option filling to a functionÆvar Arnfjörð Bjarmason, Aug 5, 2021
  14. 11/13 index-pack: add --progress-title optionÆvar Arnfjörð Bjarmason, Aug 5, 2021
  15. 12/13 bundle-uri client: support for bundle-uri with "clone"Ævar Arnfjörð Bjarmason, Aug 5, 2021
  16. 13/13 bundle-uri docs: add design notesÆvar Arnfjörð Bjarmason, Aug 5, 2021
  17. brian m. carlsonAug 24, 2021
  18. Ævar Arnfjörð BjarmasonAug 24, 2021
  19. Jonathan NiederAug 6, 2021
  20. Ævar Arnfjörð BjarmasonAug 6, 2021
  21. Jonathan NiederAug 6, 2021
  22. Ævar Arnfjörð BjarmasonAug 7, 2021
  23. Derrick StoleeAug 10, 2021
  24. Ævar Arnfjörð BjarmasonAug 23, 2021
  25. Derrick StoleeAug 24, 2021
  26. Ævar Arnfjörð BjarmasonAug 24, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.