git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 08/12] bundle: parse filter capability

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 8, 2022, 14:39 UTC
Message-ID
<025f38290f5a705c80854a42e1abcaba9a9f336d.1646750359.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1159.v3.git.1646750359.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

The v3 bundle format has capabilities, allowing newer versions of Git to create bundles with newer features. Older versions that do not understand these new capabilities will fail with a helpful warning.

Create a new capability allowing Git to understand that the contained pack-file is filtered according to some object filter. Typically, this filter will be "blob:none" for a blobless partial clone.

This change teaches Git to parse this capability, place its value in the bundle header, and demonstrate this understanding by adding a message to 'git bundle verify'.

Since we will use gently_parse_list_objects_filter() outside of list-objects-filter-options.c, make it an external method and move its API documentation to before its declaration.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 Documentation/technical/bundle-format.txt | 11 ++++++++---
 bundle.c                                  | 15 ++++++++++++++-
 bundle.h                                  |  2 ++
 list-objects-filter-options.c             | 17 +----------------
 list-objects-filter-options.h             | 20 ++++++++++++++++++++
 5 files changed, 45 insertions(+), 20 deletions(-)
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index bac558d049a..b9be8644cf5 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -71,6 +71,11 @@ and the Git bundle v2 format cannot represent a shallow clone repository.
 == Capabilities
 
 Because there is no opportunity for negotiation, unknown capabilities cause 'git
-bundle' to abort.  The only known capability is `object-format`, which specifies
-the hash algorithm in use, and can take the same values as the
-`extensions.objectFormat` configuration value.
+bundle' to abort.
+
+* `object-format` specifies the hash algorithm in use, and can take the same
+  values as the `extensions.objectFormat` configuration value.
+
+* `filter` specifies an object filter as in the `--filter` option in
+  linkgit:git-rev-list[1]. The resulting pack-file must be marked as a
+  `.promisor` pack-file after it is unbundled.
diff --git a/bundle.c b/bundle.c
index 7ba60a573d7..41922565627 100644
--- a/bundle.c
+++ b/bundle.c
@@ -11,7 +11,7 @@
 #include "run-command.h"
 #include "refs.h"
 #include "strvec.h"
-
+#include "list-objects-filter-options.h"
 
 static const char v2_bundle_signature[] = "# v2 git bundle\n";
 static const char v3_bundle_signature[] = "# v3 git bundle\n";
@@ -33,6 +33,7 @@ void bundle_header_release(struct bundle_header *header)
 {
 	string_list_clear(&header->prerequisites, 1);
 	string_list_clear(&header->references, 1);
+	list_objects_filter_release(&header->filter);
 }
 
 static int parse_capability(struct bundle_header *header, const char *capability)
@@ -45,6 +46,10 @@ static int parse_capability(struct bundle_header *header, const char *capability
 		header->hash_algo = &hash_algos[algo];
 		return 0;
 	}
+	if (skip_prefix(capability, "filter=", &arg)) {
+		parse_list_objects_filter(&header->filter, arg);
+		return 0;
+	}
 	return error(_("unknown capability '%s'"), capability);
 }
 
@@ -220,6 +225,8 @@ int verify_bundle(struct repository *r,
 	req_nr = revs.pending.nr;
 	setup_revisions(2, argv, &revs, NULL);
 
+	revs.filter = &header->filter;
+
 	if (prepare_revision_walk(&revs))
 		die(_("revision walk setup failed"));
 
@@ -259,6 +266,12 @@ int verify_bundle(struct repository *r,
 			     r->nr),
 			  r->nr);
 		list_refs(r, 0, NULL);
+
+		if (header->filter.choice != LOFC_DISABLED) {
+			printf_ln("The bundle uses this filter: %s",
+				  list_objects_filter_spec(&header->filter));
+		}
+
 		r = &header->prerequisites;
 		if (!r->nr) {
 			printf_ln(_("The bundle records a complete history."));
diff --git a/bundle.h b/bundle.h
index 06009fe6b1f..7fef2108f43 100644
--- a/bundle.h
+++ b/bundle.h
@@ -4,12 +4,14 @@
 #include "strvec.h"
 #include "cache.h"
 #include "string-list.h"
+#include "list-objects-filter-options.h"
 
 struct bundle_header {
 	unsigned version;
 	struct string_list prerequisites;
 	struct string_list references;
 	const struct git_hash_algo *hash_algo;
+	struct list_objects_filter_options filter;
 };
 
 #define BUNDLE_HEADER_INIT \
diff --git a/list-objects-filter-options.c b/list-objects-filter-options.c
index fd8d59f653a..d8597cdee36 100644
--- a/list-objects-filter-options.c
+++ b/list-objects-filter-options.c
@@ -40,22 +40,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c)
 	BUG("list_object_filter_config_name: invalid argument '%d'", c);
 }
 
-/*
- * Parse value of the argument to the "filter" keyword.
- * On the command line this looks like:
- *       --filter=<arg>
- * and in the pack protocol as:
- *       "filter" SP <arg>
- *
- * The filter keyword will be used by many commands.
- * See Documentation/rev-list-options.txt for allowed values for <arg>.
- *
- * Capture the given arg as the "filter_spec".  This can be forwarded to
- * subordinate commands when necessary (although it's better to pass it through
- * expand_list_objects_filter_spec() first).  We also "intern" the arg for the
- * convenience of the current command.
- */
-static int gently_parse_list_objects_filter(
+int gently_parse_list_objects_filter(
 	struct list_objects_filter_options *filter_options,
 	const char *arg,
 	struct strbuf *errbuf)
diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h
index da5b6737e27..f6fe6a3d2ca 100644
--- a/list-objects-filter-options.h
+++ b/list-objects-filter-options.h
@@ -72,6 +72,26 @@ struct list_objects_filter_options {
 /* Normalized command line arguments */
 #define CL_ARG__FILTER "filter"
 
+/*
+ * Parse value of the argument to the "filter" keyword.
+ * On the command line this looks like:
+ *       --filter=<arg>
+ * and in the pack protocol as:
+ *       "filter" SP <arg>
+ *
+ * The filter keyword will be used by many commands.
+ * See Documentation/rev-list-options.txt for allowed values for <arg>.
+ *
+ * Capture the given arg as the "filter_spec".  This can be forwarded to
+ * subordinate commands when necessary (although it's better to pass it through
+ * expand_list_objects_filter_spec() first).  We also "intern" the arg for the
+ * convenience of the current command.
+ */
+int gently_parse_list_objects_filter(
+	struct list_objects_filter_options *filter_options,
+	const char *arg,
+	struct strbuf *errbuf);
+
 void list_objects_filter_die_if_populated(
 	struct list_objects_filter_options *filter_options);
 
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Junio C Hamano
Message 80 of 114 in “Partial bundles”
  1. 00/11 Partial bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  2. 01/11 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  3. 02/11 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Feb 23, 2022
  4. Junio C HamanoMar 4, 2022
  5. Derrick StoleeMar 7, 2022
  6. Junio C HamanoMar 7, 2022
  7. 03/11 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Feb 23, 2022
  8. Junio C HamanoMar 4, 2022
  9. 04/11 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Feb 23, 2022
  10. Junio C HamanoMar 4, 2022
  11. 05/11 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Feb 23, 2022
  12. Junio C HamanoMar 4, 2022
  13. 06/11 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Feb 23, 2022
  14. Junio C HamanoMar 4, 2022
  15. Derrick StoleeMar 7, 2022
  16. Junio C HamanoMar 7, 2022
  17. 08/11 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  18. Ævar Arnfjörð BjarmasonMar 7, 2022
  19. Derrick StoleeMar 7, 2022
  20. Ævar Arnfjörð BjarmasonMar 7, 2022
  21. Derrick StoleeMar 7, 2022
  22. Ævar Arnfjörð BjarmasonMar 7, 2022
  23. 07/11 bundle: safely handle --objects optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  24. Jeff HostetlerFeb 28, 2022
  25. Junio C HamanoMar 4, 2022
  26. Derrick StoleeMar 7, 2022
  27. Junio C HamanoMar 4, 2022
  28. Ævar Arnfjörð BjarmasonMar 7, 2022
  29. 09/11 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Feb 23, 2022
  30. 10/11 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  31. Junio C HamanoMar 4, 2022
  32. Derrick StoleeMar 7, 2022
  33. Junio C HamanoMar 7, 2022
  34. Ævar Arnfjörð BjarmasonMar 7, 2022
  35. 11/11 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Feb 23, 2022
  36. Junio C HamanoMar 4, 2022
  37. Derrick StoleeMar 7, 2022
  38. Junio C HamanoMar 7, 2022
  39. Derrick StoleeMar 7, 2022
  40. Junio C HamanoMar 7, 2022
  41. Derrick StoleeMar 7, 2022
  42. Junio C HamanoMar 7, 2022
  43. Derrick StoleeMar 7, 2022
  44. Junio C HamanoMar 7, 2022
  45. Ævar Arnfjörð BjarmasonMar 7, 2022
  46. Derrick StoleeMar 7, 2022
  47. Jeff HostetlerFeb 28, 2022
  48. Derrick StoleeFeb 28, 2022
  49. Jeff HostetlerMar 1, 2022
  50. Derrick StoleeMar 4, 2022
  51. Ævar Arnfjörð BjarmasonMar 7, 2022
  52. Derrick StoleeMar 7, 2022
  53. 00/12 Partial bundlesDerrick Stolee via GitGitGadget, Mar 7, 2022
  54. 01/12 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 7, 2022
  55. 02/12 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 7, 2022
  56. 03/12 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 7, 2022
  57. 04/12 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 7, 2022
  58. 05/12 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 7, 2022
  59. 06/12 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 7, 2022
  60. 07/12 bundle: safely handle --objects optionDerrick Stolee via GitGitGadget, Mar 7, 2022
  61. Ævar Arnfjörð BjarmasonMar 8, 2022
  62. Derrick StoleeMar 8, 2022
  63. Ævar Arnfjörð BjarmasonMar 8, 2022
  64. 12/12 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 7, 2022
  65. 10/12 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 7, 2022
  66. 09/12 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 7, 2022
  67. 11/12 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 7, 2022
  68. 08/12 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 7, 2022
  69. Ævar Arnfjörð BjarmasonMar 8, 2022
  70. Derrick StoleeMar 8, 2022
  71. Junio C HamanoMar 7, 2022
  72. 00/12 Partial bundlesDerrick Stolee via GitGitGadget, Mar 8, 2022
  73. 01/12 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 8, 2022
  74. 02/12 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 8, 2022
  75. 03/12 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 8, 2022
  76. 04/12 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 8, 2022
  77. 05/12 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 8, 2022
  78. Ævar Arnfjörð BjarmasonMar 9, 2022
  79. 06/12 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 8, 2022
  80. 08/12 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 8, 2022
  81. Junio C HamanoMar 8, 2022
  82. Derrick StoleeMar 9, 2022
  83. Ævar Arnfjörð BjarmasonMar 9, 2022
  84. 07/12 list-objects: handle NULL function pointersÆvar Arnfjörð Bjarmason via GitGitGadget, Mar 8, 2022
  85. Junio C HamanoMar 8, 2022
  86. Ævar Arnfjörð BjarmasonMar 9, 2022
  87. Derrick StoleeMar 9, 2022
  88. Junio C HamanoMar 9, 2022
  89. 09/12 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 8, 2022
  90. 10/12 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 8, 2022
  91. 11/12 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 8, 2022
  92. 12/12 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 8, 2022
  93. Derrick StoleeMar 8, 2022
  94. Junio C HamanoMar 8, 2022
  95. 00/13 Partial bundlesDerrick Stolee via GitGitGadget, Mar 9, 2022
  96. 09/13 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 9, 2022
  97. Junio C HamanoMar 9, 2022
  98. Derrick StoleeMar 9, 2022
  99. 06/13 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 9, 2022
  100. 03/13 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 9, 2022
  101. Junio C HamanoMar 9, 2022
  102. 01/13 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 9, 2022
  103. 07/13 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 9, 2022
  104. 11/13 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 9, 2022
  105. 04/13 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 9, 2022
  106. Ævar Arnfjörð BjarmasonMar 10, 2022
  107. Derrick StoleeMar 10, 2022
  108. Ævar Arnfjörð BjarmasonMar 10, 2022
  109. 08/13 list-objects: handle NULL function pointersÆvar Arnfjörð Bjarmason via GitGitGadget, Mar 9, 2022
  110. 05/13 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 9, 2022
  111. 02/13 list-objects-filter-options: create copy helperDerrick Stolee via GitGitGadget, Mar 9, 2022
  112. 10/13 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 9, 2022
  113. 12/13 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 9, 2022
  114. 13/13 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 9, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.