git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 22/24] bundle-uri: download bundles from an advertised list

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 20, 2022, 18:40 UTC
Message-ID
<c0d6354388168c85fb7bb42824cdc8fd2879e898.1653072042.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1234.git.1653072042.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

The logic in fetch_bundle_uri() is useful for the --bundle-uri option of 'git clone', but is not helpful when the clone operation discovers a list of URIs from the bundle-uri protocol v2 verb. To actually download and unbundle the advertised bundles, we need a different mechanism.

Create the new fetch_bundle_list() method which is very similar to fetch_bundle_uri() except that it relies on download_bundle_list() instead of fetch_bundle_uri_internal(). The download_bundle_list() method will recursively call fetch_bundle_uri_internal() if any of the advertised URIs serve a bundle list instead of a bundle. This will also follow the bundle.list.mode setting from the input list: "any" will download only one such URI while "all" will download data from all of the URIs.

In an identical way to the fetch_bundle_uri(), the bundles are unbundled after all of the bundle lists have been expanded and all necessary URIs.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 bundle-uri.c | 21 +++++++++++++++++++++
 bundle-uri.h | 11 +++++++++++
 2 files changed, 32 insertions(+)
diff --git a/bundle-uri.c b/bundle-uri.c
index 9e7dc0fb4ca..43815b3c5fd 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -561,6 +561,27 @@ cleanup:
 	return result;
 }
 
+int fetch_bundle_list(struct repository *r, const char *uri, struct bundle_list *list)
+{
+	int result;
+	struct bundle_list global_list;
+
+	init_bundle_list(&global_list);
+
+	/* If a bundle is added to this global list, then it is required. */
+	global_list.mode = BUNDLE_MODE_ALL;
+
+	if ((result = download_bundle_list(r, list, &global_list, 0)))
+		goto cleanup;
+
+	result = unbundle_all_bundles(r, &global_list);
+
+cleanup:
+	for_all_bundles_in_list(&global_list, unlink_bundle, NULL);
+	clear_bundle_list(&global_list);
+	return result;
+}
+
 /**
  * API for serve.c.
  */
diff --git a/bundle-uri.h b/bundle-uri.h
index fdcaea048a8..1bcbd43320f 100644
--- a/bundle-uri.h
+++ b/bundle-uri.h
@@ -107,6 +107,17 @@ int parse_bundle_list_in_config_format(const char *uri,
  */
 int fetch_bundle_uri(struct repository *r, const char *uri);
 
+/**
+ * Given a bundle list that was already advertised (likely by the
+ * bundle-uri protocol v2 verb) at the given uri, fetch and unbundle the
+ * bundles according to the bundle strategy of that list.
+ *
+ * Returns non-zero if no bundle information is found at the given 'uri'.
+ */
+int fetch_bundle_list(struct repository *r,
+		      const char *uri,
+		      struct bundle_list *list);
+
 /**
  * API for serve.c.
  */
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 21 of 25 in “[RFC] Bundle URIs Combined RFC”
  1. 00/24 [RFC] Bundle URIs Combined RFCDerrick Stolee via GitGitGadget, May 20, 2022
  2. 01/24 docs: document bundle URI standardDerrick Stolee via GitGitGadget, May 20, 2022
  3. 02/24 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, May 20, 2022
  4. 06/24 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, May 20, 2022
  5. 05/24 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  6. 04/24 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, May 20, 2022
  7. 03/24 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, May 20, 2022
  8. 12/24 bundle-uri: unit test "key=value" parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  9. 10/24 bundle-uri: create base key-value pair parsingDerrick Stolee via GitGitGadget, May 20, 2022
  10. 07/24 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  11. 11/24 bundle-uri: create "key=value" line parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  12. 14/24 bundle-uri: parse bundle list in config formatDerrick Stolee via GitGitGadget, May 20, 2022
  13. 09/24 bundle-uri: create bundle_list struct and helpersDerrick Stolee via GitGitGadget, May 20, 2022
  14. 15/24 bundle-uri: fetch a list of bundlesDerrick Stolee via GitGitGadget, May 20, 2022
  15. 16/24 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  16. 08/24 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, May 20, 2022
  17. 13/24 bundle-uri: limit recursion depth for bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  18. 18/24 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  19. 17/24 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  20. 19/24 bundle-uri: serve URI advertisement from bundle.* configDerrick Stolee via GitGitGadget, May 20, 2022
  21. 22/24 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, May 20, 2022
  22. 24/24 t5601: basic bundle URI testsDerrick Stolee via GitGitGadget, May 20, 2022
  23. 20/24 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  24. 21/24 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  25. 23/24 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, May 20, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.