git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH v2 13/36] bundle-uri client: add transfer.injectBundleURI support

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Apr 18, 2022, 17:23 UTC
Message-ID
<RFC-patch-v2-13.36-5c50daa92bb-20220418T165545Z-avarab@gmail.com>
In-Reply-To
<RFC-cover-v2-00.36-00000000000-20220418T165545Z-avarab@gmail.com>

Add the ability to inject "fake" bundle URIs into the newly supported bundle-uri dialog. As discussed in the added documentation this allows us to pretend as though the remote supports bundle URIs.

This will be useful both for ad-hoc testing, and for the real use-case of retrofitting bundle URI support on-the-fly, i.e. to have:

	git -c transfer.injectBundleURI "file://$(pwd)/local.bdl" \
	clone https://example.com/git.git"
Be similar in spirit to:
	git clone --reference local-clone.git --disassociate \
	https://example.com/git.git"
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config/transfer.txt     | 20 ++++++++++++
 t/lib-t5730-protocol-v2-bundle-uri.sh | 46 +++++++++++++++++++++++++++
 transport.c                           | 33 +++++++++++++++++++
 3 files changed, 99 insertions(+)
diff --git a/Documentation/config/transfer.txt b/Documentation/config/transfer.txt
index b49429eb4db..71b9b8f29e6 100644
--- a/Documentation/config/transfer.txt
+++ b/Documentation/config/transfer.txt
@@ -77,3 +77,23 @@ transfer.unpackLimit::
 transfer.advertiseSID::
 	Boolean. When true, client and server processes will advertise their
 	unique session IDs to their remote counterpart. Defaults to false.
+
+transfer.injectBundleURI::
+	Allows for the injection of `bundle-uri` lines into the
+	protocol v2 transport dialog (see `protocol.version` in
+	linkgit:git-config[1]). See `bundle-uri` in
+	link:technical/protocol-v2.html[the Git Wire Protocol, Version
+	2] documentation for what the format looks like.
++
+Can be given more than once, each key being injected as one line into
+the dialog.
++
+This is useful for testing the `bundle-uri` facility, and to e.g. use
+linkgit:git-clone[1] to clone from a server which does not support
+`bundle-uri`, but where the clone can benefit from getting some or
+most of the data from a static bundle retrieved from elsewhere.
++
+Impacts any command that uses the transport to communicate with remote
+linkgit:git-upload-pack[1] processes, e.g. linkgit:git-clone[1],
+linkgit:git-fetch[1] and the linkgit:git-ls-remote-bundle-uri[1]
+inspection command, this includes the `file://` protocol.
diff --git a/t/lib-t5730-protocol-v2-bundle-uri.sh b/t/lib-t5730-protocol-v2-bundle-uri.sh
index d0b15a47ec2..28c095c1224 100644
--- a/t/lib-t5730-protocol-v2-bundle-uri.sh
+++ b/t/lib-t5730-protocol-v2-bundle-uri.sh
@@ -267,3 +267,49 @@ test_expect_success "ls-remote-bundle-uri --[no-]quiet with $T5730_PROTOCOL:// u
 	test_must_be_empty err &&
 	test_cmp out.expect out.actual
 '
+
+test_expect_success "ls-remote-bundle-uri with -c transfer.injectBundleURI using with $T5730_PROTOCOL:// using protocol v2" '
+	test_when_finished "rm -f log" &&
+
+	test_config -C "$T5730_PARENT" uploadpack.bundleURI \
+		"$T5730_BUNDLE_URI_ESCAPED" &&
+
+	cat >expect <<-\EOF &&
+	https://injected.example.com/fake-1.bdl
+	https://injected.example.com/fake-2.bdl
+	EOF
+	GIT_TRACE_PACKET="$PWD/log" \
+	git \
+		-c protocol.version=2 \
+		-c transfer.injectBundleURI="https://injected.example.com/fake-1.bdl" \
+		-c transfer.injectBundleURI="https://injected.example.com/fake-2.bdl" \
+		ls-remote-bundle-uri \
+		"$T5730_URI" \
+		>actual 2>err &&
+	test_cmp expect actual &&
+	test_path_is_missing log
+'
+
+test_expect_success "ls-remote-bundle-uri with bad -c transfer.injectBundleURI protocol v2 with $T5730_PROTOCOL://" '
+	test_when_finished "rm -f log" &&
+
+	test_config -C "$T5730_PARENT" uploadpack.bundleURI \
+		"$T5730_BUNDLE_URI_ESCAPED" &&
+
+	cat >err.expect <<-\EOF &&
+	error: bad (empty) transfer.injectBundleURI
+	error: could not get the bundle-uri list
+	EOF
+
+	test_must_fail env \
+		GIT_TRACE_PACKET="$PWD/log" \
+		git \
+		-c protocol.version=2 \
+		-c transfer.injectBundleURI \
+		ls-remote-bundle-uri \
+		"$T5730_URI" \
+		>out 2>err.actual &&
+	test_must_be_empty out &&
+	test_cmp err.expect err.actual &&
+	test_path_is_missing log
+'
diff --git a/transport.c b/transport.c
index e648d3110bb..342e39d81f3 100644
--- a/transport.c
+++ b/transport.c
@@ -1521,14 +1521,47 @@ int transport_fetch_refs(struct transport *transport, struct ref *refs)
 	return rc;
 }
 
+struct config_cb {
+	struct transport *transport;
+	int configured;
+	int ret;
+};
+
+static int bundle_uri_config(const char *var, const char *value, void *data)
+{
+	struct config_cb *cb = data;
+	struct transport *transport = cb->transport;
+	struct string_list *uri = &transport->bundle_uri;
+
+	if (!strcmp(var, "transfer.injectbundleuri")) {
+		cb->configured = 1;
+		if (!value)
+			cb->ret = error(_("bad (empty) transfer.injectBundleURI"));
+		else if (bundle_uri_parse_line(uri, value) < 0)
+			cb->ret = error(_("bad transfer.injectBundleURI: '%s'"),
+					value);
+		return 0;
+	}
+	return 0;
+}
+
 int transport_get_remote_bundle_uri(struct transport *transport, int quiet)
 {
 	const struct transport_vtable *vtable = transport->vtable;
+	struct config_cb cb = {
+		.transport = transport,
+	};
 
 	/* Lazily configured */
 	if (transport->got_remote_bundle_uri++)
 		return 0;
 
+	git_config(bundle_uri_config, &cb);
+
+	/* Our own config can fake it up with transport.injectBundleURI */
+	if (cb.configured)
+		return cb.ret;
+
 	/*
 	 * This is intentionally below the transport.injectBundleURI,
 	 * we want to be able to inject into protocol v0, or into the
-- 
2.36.0.rc2.902.g60576bbc845
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 54 of 77 in “bundle-uri: "dumb" static CDN offloading, spec & server implementation”
  1. 0/3 bundle-uri: "dumb" static CDN offloading, spec & server implementationÆvar Arnfjörð Bjarmason, Oct 25, 2021
  2. 1/3 leak tests: mark t5701-git-serve.sh as passing SANITIZE=leakÆvar Arnfjörð Bjarmason, Oct 25, 2021
  3. 2/3 protocol v2: specify static seeding of clone/fetch via "bundle-uri"Ævar Arnfjörð Bjarmason, Oct 25, 2021
  4. Derrick StoleeOct 26, 2021
  5. Ævar Arnfjörð BjarmasonOct 26, 2021
  6. Derrick StoleeOct 27, 2021
  7. Ævar Arnfjörð BjarmasonOct 27, 2021
  8. Derrick StoleeOct 27, 2021
  9. Ævar Arnfjörð BjarmasonOct 27, 2021
  10. Derrick StoleeOct 27, 2021
  11. Ævar Arnfjörð BjarmasonOct 27, 2021
  12. Derrick StoleeOct 27, 2021
  13. Ævar Arnfjörð BjarmasonOct 27, 2021
  14. Derrick StoleeOct 29, 2021
  15. Philip OakleyOct 30, 2021
  16. 3/3 bundle-uri client: add "bundle-uri" parsing + testsÆvar Arnfjörð Bjarmason, Oct 25, 2021
  17. Derrick StoleeOct 26, 2021
  18. Derrick StoleeOct 29, 2021
  19. Ævar Arnfjörð BjarmasonOct 30, 2021
  20. Derrick StoleeNov 1, 2021
  21. Ævar Arnfjörð BjarmasonNov 1, 2021
  22. 00/13 bundle-uri: a "dumb CDN" for gitÆvar Arnfjörð Bjarmason, Mar 11, 2022
  23. 02/13 bundle-uri docs: add design notesÆvar Arnfjörð Bjarmason, Mar 11, 2022
  24. 01/13 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason, Mar 11, 2022
  25. 04/13 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason, Mar 11, 2022
  26. 06/13 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason, Mar 11, 2022
  27. 05/13 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason, Mar 11, 2022
  28. 07/13 bundle-uri client: add transfer.injectBundleURI supportÆvar Arnfjörð Bjarmason, Mar 11, 2022
  29. 03/13 bundle-uri client: add "bundle-uri" parsing + testsÆvar Arnfjörð Bjarmason, Mar 11, 2022
  30. 08/13 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason, Mar 11, 2022
  31. 09/13 fetch-pack: add a deref_without_lazy_fetch_extended()Ævar Arnfjörð Bjarmason, Mar 11, 2022
  32. 10/13 fetch-pack: move --keep=* option filling to a functionÆvar Arnfjörð Bjarmason, Mar 11, 2022
  33. 13/13 bundle-uri: make the download program configurableÆvar Arnfjörð Bjarmason, Mar 11, 2022
  34. 11/13 bundle.h: make "fd" version of read_bundle_header() publicÆvar Arnfjörð Bjarmason, Mar 11, 2022
  35. 12/13 bundle-uri client: support for bundle-uri with "clone"Ævar Arnfjörð Bjarmason, Mar 11, 2022
  36. Derrick StoleeMar 11, 2022
  37. 00/36 bundle-uri: a "dumb CDN" for git + TOC formatÆvar Arnfjörð Bjarmason, Apr 18, 2022
  38. 01/36 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason, Apr 18, 2022
  39. 02/36 dir API: add a generalized path_match_flags() functionÆvar Arnfjörð Bjarmason, Apr 18, 2022
  40. Derrick StoleeApr 21, 2022
  41. 03/36 fetch-pack: add a deref_without_lazy_fetch_extended()Ævar Arnfjörð Bjarmason, Apr 18, 2022
  42. Derrick StoleeApr 21, 2022
  43. 04/36 fetch-pack: move --keep=* option filling to a functionÆvar Arnfjörð Bjarmason, Apr 18, 2022
  44. 05/36 http: make http_get_file() externalÆvar Arnfjörð Bjarmason, Apr 18, 2022
  45. 07/36 remote: allow relative_url() to return an absolute urlÆvar Arnfjörð Bjarmason, Apr 18, 2022
  46. 06/36 remote: move relative_url()Ævar Arnfjörð Bjarmason, Apr 18, 2022
  47. 08/36 bundle.h: make "fd" version of read_bundle_header() publicÆvar Arnfjörð Bjarmason, Apr 18, 2022
  48. 09/36 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason, Apr 18, 2022
  49. 10/36 bundle-uri client: add "bundle-uri" parsing + testsÆvar Arnfjörð Bjarmason, Apr 18, 2022
  50. 11/36 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason, Apr 18, 2022
  51. 12/36 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason, Apr 18, 2022
  52. 14/36 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason, Apr 18, 2022
  53. 15/36 bundle-uri client: support for bundle-uri with "clone"Ævar Arnfjörð Bjarmason, Apr 18, 2022
  54. 13/36 bundle-uri client: add transfer.injectBundleURI supportÆvar Arnfjörð Bjarmason, Apr 18, 2022
  55. 16/36 bundle-uri: make the download program configurableÆvar Arnfjörð Bjarmason, Apr 18, 2022
  56. 18/36 bundle: implement 'fetch' command for direct bundlesÆvar Arnfjörð Bjarmason, Apr 18, 2022
  57. 17/36 remote-curl: add 'get' capabilityÆvar Arnfjörð Bjarmason, Apr 18, 2022
  58. 19/36 bundle: parse table of contents during 'fetch'Ævar Arnfjörð Bjarmason, Apr 18, 2022
  59. 20/36 bundle: add --filter option to 'fetch'Ævar Arnfjörð Bjarmason, Apr 18, 2022
  60. 21/36 bundle: allow relative URLs in table of contentsÆvar Arnfjörð Bjarmason, Apr 18, 2022
  61. 22/36 bundle: make it easy to call 'git bundle fetch'Ævar Arnfjörð Bjarmason, Apr 18, 2022
  62. 23/36 clone: add --bundle-uri optionÆvar Arnfjörð Bjarmason, Apr 18, 2022
  63. 26/36 fetch: fetch bundles before fetching original dataÆvar Arnfjörð Bjarmason, Apr 18, 2022
  64. 25/36 bundle: only fetch bundles if timestamp is newÆvar Arnfjörð Bjarmason, Apr 18, 2022
  65. 24/36 clone: --bundle-uri cannot be combined with --depthÆvar Arnfjörð Bjarmason, Apr 18, 2022
  66. 28/36 serve: understand but do not advertise 'features' capabilityÆvar Arnfjörð Bjarmason, Apr 18, 2022
  67. 27/36 protocol-caps: implement cap_features()Ævar Arnfjörð Bjarmason, Apr 18, 2022
  68. 30/36 connect: implement get_recommended_features()Ævar Arnfjörð Bjarmason, Apr 18, 2022
  69. 29/36 serve: advertise 'features' when config existsÆvar Arnfjörð Bjarmason, Apr 18, 2022
  70. 33/36 t5601: basic bundle URI testÆvar Arnfjörð Bjarmason, Apr 18, 2022
  71. 32/36 clone: use server-recommended bundle URIÆvar Arnfjörð Bjarmason, Apr 18, 2022
  72. 31/36 transport: add connections for 'features' capabilityÆvar Arnfjörð Bjarmason, Apr 18, 2022
  73. 34/36 protocol v2: add server-side "bundle-uri" skeleton (docs)Ævar Arnfjörð Bjarmason, Apr 18, 2022
  74. 35/36 bundle-uri docs: add design notesÆvar Arnfjörð Bjarmason, Apr 18, 2022
  75. 36/36 docs: document bundle URI standardÆvar Arnfjörð Bjarmason, Apr 18, 2022
  76. Derrick StoleeApr 21, 2022
  77. Ævar Arnfjörð BjarmasonApr 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.