git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 10/25] bundle: add --filter option to 'fetch'

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 23, 2022, 18:30 UTC
Message-ID
<fa0e83b39dd3c3bb16e932c39aff080a1fdf8fe4.1645641063.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1160.git.1645641063.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

When a repository uses an object filter for partial clone, the 'git bundle fetch' command should try to download bundles that match that filter.

Teach 'git bundle fetch' to take a '--filter' option and then only consider bundles that match that filter (or lack thereof). This allows the bundle server to advertise different sets of bundles for different filters.

Add some verification to be sure that the bundle we downloaded actually uses that filter. This is especially important when no filter is requested but the downloaded bundle _does_ have a filter.

RFC-TODO: add tests for the happy path.
RFC-TODO: add tests for these validations.
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 builtin/bundle.c | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
diff --git a/builtin/bundle.c b/builtin/bundle.c
index 66f3b3c9376..27da5e3737f 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -9,6 +9,7 @@
 #include "refs.h"
 #include "config.h"
 #include "packfile.h"
+#include "list-objects-filter-options.h"
 
 /*
  * Basic handler for bundle files to connect repositories via sneakernet.
@@ -406,10 +407,13 @@ static int cmd_bundle_fetch(int argc, const char **argv, const char *prefix)
 	};
 	struct remote_bundle_info *stack = NULL;
 	struct hashmap toc = { 0 };
+	const char *filter = NULL;
 
 	struct option options[] = {
 		OPT_BOOL(0, "progress", &progress,
 			 N_("show progress meter")),
+		OPT_STRING(0, "filter", &filter,
+			   N_("filter-spec"), N_("only install bundles matching this filter")),
 		OPT_END()
 	};
 
@@ -449,6 +453,17 @@ static int cmd_bundle_fetch(int argc, const char **argv, const char *prefix)
 
 		/* initialize stack using timestamp heuristic. */
 		hashmap_for_each_entry(&toc, &iter, info, ent) {
+			/* Skip if filter does not match. */
+			if (!filter && info->filter_str)
+				continue;
+			if (filter &&
+			    (!info->filter_str || strcasecmp(filter, info->filter_str)))
+				continue;
+
+			/*
+			 * Now that the filter matches, start with the
+			 * bundle with largest timestamp.
+			 */
 			if (info->timestamp > max_time || !stack) {
 				stack = info;
 				max_time = info->timestamp;
@@ -468,6 +483,7 @@ static int cmd_bundle_fetch(int argc, const char **argv, const char *prefix)
 	while (stack) {
 		int valid = 1;
 		int bundle_fd;
+		const char *filter_str = NULL;
 		struct string_list_item *prereq;
 		struct bundle_header header = BUNDLE_HEADER_INIT;
 
@@ -483,6 +499,16 @@ static int cmd_bundle_fetch(int argc, const char **argv, const char *prefix)
 		if (bundle_fd < 0)
 			die(_("failed to read bundle from '%s'"), stack->uri);
 
+		if (header.filter)
+			filter_str = list_objects_filter_spec(header.filter);
+
+		if (filter && (!filter_str || strcasecmp(filter, filter_str)))
+			die(_("bundle from '%s' does not match expected filter"),
+			    stack->uri);
+		if (!filter && filter_str)
+			die(_("bundle from '%s' has an unexpected filter"),
+			    stack->uri);
+
 		reprepare_packed_git(the_repository);
 		for_each_string_list_item(prereq, &header.prerequisites) {
 			struct object_info info = OBJECT_INFO_INIT;
-- 
gitgitgadget
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 18 of 46 in “[RFC] Bundle URIs”
  1. 00/25 [RFC] Bundle URIsDerrick Stolee via GitGitGadget, Feb 23, 2022
  2. 02/25 bundle: alphabetize subcommands betterDerrick Stolee via GitGitGadget, Feb 23, 2022
  3. Ævar Arnfjörð BjarmasonMar 11, 2022
  4. 05/25 remote: allow relative_url() to return an absolute urlDerrick Stolee via GitGitGadget, Feb 23, 2022
  5. 03/25 dir: extract starts_with_dot[_dot]_slash()Derrick Stolee via GitGitGadget, Feb 23, 2022
  6. 04/25 remote: move relative_url()Derrick Stolee via GitGitGadget, Feb 23, 2022
  7. 01/25 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Feb 23, 2022
  8. Elijah NewrenMar 2, 2022
  9. Derrick StoleeMar 2, 2022
  10. Elijah NewrenMar 3, 2022
  11. Derrick StoleeMar 3, 2022
  12. 06/25 http: make http_get_file() externalDerrick Stolee via GitGitGadget, Feb 23, 2022
  13. 07/25 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  14. 08/25 bundle: implement 'fetch' command for direct bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  15. 09/25 bundle: parse table of contents during 'fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  16. 11/25 bundle: allow relative URLs in table of contentsDerrick Stolee via GitGitGadget, Feb 23, 2022
  17. Ævar Arnfjörð BjarmasonMar 11, 2022
  18. 10/25 bundle: add --filter option to 'fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  19. Ævar Arnfjörð BjarmasonMar 11, 2022
  20. 12/25 bundle: make it easy to call 'git bundle fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  21. 13/25 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  22. 14/25 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, Feb 23, 2022
  23. 15/25 config: add git_config_get_timestamp()Derrick Stolee via GitGitGadget, Feb 23, 2022
  24. Ævar Arnfjörð BjarmasonMar 11, 2022
  25. 18/25 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason via GitGitGadget, Feb 23, 2022
  26. 17/25 fetch: fetch bundles before fetching original dataDerrick Stolee via GitGitGadget, Feb 23, 2022
  27. 16/25 bundle: only fetch bundles if timestamp is newDerrick Stolee via GitGitGadget, Feb 23, 2022
  28. 19/25 protocol-caps: implement cap_features()Derrick Stolee via GitGitGadget, Feb 23, 2022
  29. 21/25 serve: advertise 'features' when config existsDerrick Stolee via GitGitGadget, Feb 23, 2022
  30. 20/25 serve: understand but do not advertise 'features' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  31. 22/25 connect: implement get_recommended_features()Derrick Stolee via GitGitGadget, Feb 23, 2022
  32. 23/25 transport: add connections for 'features' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  33. 24/25 clone: use server-recommended bundle URIDerrick Stolee via GitGitGadget, Feb 23, 2022
  34. 25/25 t5601: basic bundle URI testDerrick Stolee via GitGitGadget, Feb 23, 2022
  35. Ævar Arnfjörð BjarmasonFeb 23, 2022
  36. Derrick StoleeFeb 24, 2022
  37. Derrick StoleeMar 4, 2022
  38. Ævar Arnfjörð BjarmasonMar 4, 2022
  39. Derrick StoleeMar 4, 2022
  40. Derrick StoleeMar 8, 2022
  41. Johannes SchindelinMar 10, 2022
  42. Derrick StoleeApr 7, 2022
  43. Ævar Arnfjörð BjarmasonApr 8, 2022
  44. Derrick StoleeApr 8, 2022
  45. Junio C HamanoApr 8, 2022
  46. Teng LongMar 8, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.