git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 10/24] bundle-uri: create base key-value pair parsing

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 20, 2022, 18:40 UTC
Message-ID
<2c94a55553bffdf90df7f80c8e971bfe2887fa1d.1653072042.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1234.git.1653072042.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

There will be two primary ways to advertise a bundle list: as a list of packet lines in Git's protocol v2 and as a config file served from a bundle URI. Both of these fundamentally use a list of key-value pairs. We will use the same set of key-value pairs across these formats.

Create a new bundle_list_update() method that is currently unusued, but will be used in the next change. It inspects each key to see if it is understood and then applies it to the given bundle_list. Here are the keys that we teach Git to understand:

* bundle.list.version: This value should be an integer. Git currently
  understands only version 1 and will ignore the list if the version is
  any other value. This version can be increased in the future if we
  need to add new keys that Git should not ignore. We can add new
  "heuristic" keys without incrementing the version.
* bundle.list.mode: This value should be one of "all" or "any". If this
  mode is not understood, then Git will ignore the list. This mode
  indicates whether Git needs all of the bundle list items to make a
  complete view of the content or if any single item is sufficient.

The rest of the keys use a bundle identifier "<id>" as part of the key name, and this cannot equal "list". Keys using the same "<id>" describe a common bundle list item.

* bundle.<id>.uri: This stores the URI of the bundle item. This
  currently is expected to be an absolute URI, but will be relaxed to be
  a relative URI in the future.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 bundle-uri.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
diff --git a/bundle-uri.c b/bundle-uri.c
index b9a219d3202..f18bba7071d 100644
--- a/bundle-uri.c
+++ b/bundle-uri.c
@@ -67,6 +67,86 @@ int for_all_bundles_in_list(struct bundle_list *list,
 	return 0;
 }
 
+/**
+ * Given a key-value pair, update the state of the given bundle list.
+ * Returns 0 if the key-value pair is understood. Returns 1 if the key
+ * is not understood or the value is malformed.
+ */
+MAYBE_UNUSED
+static int bundle_list_update(const char *key, const char *value,
+			      struct bundle_list *list)
+{
+	const char *pkey, *dot;
+	struct strbuf id = STRBUF_INIT;
+	struct remote_bundle_info lookup = REMOTE_BUNDLE_INFO_INIT;
+	struct remote_bundle_info *bundle;
+
+	if (!skip_prefix(key, "bundle.", &pkey))
+		return 1;
+
+	if (!strcmp(pkey, "list.version")) {
+		int version = atoi(value);
+		if (version != 1)
+			return 1;
+
+		list->version = version;
+		return 0;
+	}
+
+	if (!strcmp(pkey, "list.mode")) {
+		if (!strcmp(value, "all"))
+			list->mode = BUNDLE_MODE_ALL;
+		else if (!strcmp(value, "any"))
+			list->mode = BUNDLE_MODE_ANY;
+		else
+			return 1;
+		return 0;
+	}
+
+	/*
+	 * All remaining keys must be of the form "bundle.<id>.*" where
+	 * <id> != "list"
+	 */
+
+	dot = strchr(pkey, '.');
+	if (!dot)
+		return 1;
+	if (dot - pkey == 4 &&
+	    !strncmp(pkey, "list", 4))
+		return 1;
+
+	strbuf_add(&id, pkey, dot - pkey);
+	dot++;
+
+	/*
+	 * Check for an existing bundle with this <id>, or create one
+	 * if necessary.
+	 */
+	lookup.id = id.buf;
+	hashmap_entry_init(&lookup.ent, strhash(lookup.id));
+	if (!(bundle = hashmap_get_entry(&list->bundles, &lookup, ent, NULL))) {
+		CALLOC_ARRAY(bundle, 1);
+		bundle->id = strbuf_detach(&id, NULL);
+		strbuf_init(&bundle->file, 0);
+		hashmap_entry_init(&bundle->ent, strhash(bundle->id));
+		hashmap_add(&list->bundles, &bundle->ent);
+	}
+	strbuf_release(&id);
+
+	if (!strcmp(dot, "uri")) {
+		free(bundle->uri);
+		bundle->uri = xstrdup(value);
+		return 0;
+	}
+
+	/*
+	 * At this point, we ignore any information that we don't
+	 * understand, assuming it to be hints for a heuristic the client
+	 * does not currently understand.
+	 */
+	return 0;
+}
+
 static void find_temp_filename(struct strbuf *name)
 {
 	int fd;
-- 
gitgitgadget
Previous: Ævar Arnfjörð Bjarmason via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 9 of 25 in “[RFC] Bundle URIs Combined RFC”
  1. 00/24 [RFC] Bundle URIs Combined RFCDerrick Stolee via GitGitGadget, May 20, 2022
  2. 01/24 docs: document bundle URI standardDerrick Stolee via GitGitGadget, May 20, 2022
  3. 02/24 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, May 20, 2022
  4. 06/24 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, May 20, 2022
  5. 05/24 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  6. 04/24 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, May 20, 2022
  7. 03/24 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, May 20, 2022
  8. 12/24 bundle-uri: unit test "key=value" parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  9. 10/24 bundle-uri: create base key-value pair parsingDerrick Stolee via GitGitGadget, May 20, 2022
  10. 07/24 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, May 20, 2022
  11. 11/24 bundle-uri: create "key=value" line parsingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  12. 14/24 bundle-uri: parse bundle list in config formatDerrick Stolee via GitGitGadget, May 20, 2022
  13. 09/24 bundle-uri: create bundle_list struct and helpersDerrick Stolee via GitGitGadget, May 20, 2022
  14. 15/24 bundle-uri: fetch a list of bundlesDerrick Stolee via GitGitGadget, May 20, 2022
  15. 16/24 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  16. 08/24 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, May 20, 2022
  17. 13/24 bundle-uri: limit recursion depth for bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  18. 18/24 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  19. 17/24 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  20. 19/24 bundle-uri: serve URI advertisement from bundle.* configDerrick Stolee via GitGitGadget, May 20, 2022
  21. 22/24 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, May 20, 2022
  22. 24/24 t5601: basic bundle URI testsDerrick Stolee via GitGitGadget, May 20, 2022
  23. 20/24 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, May 20, 2022
  24. 21/24 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, May 20, 2022
  25. 23/24 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, May 20, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.