git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 08/11] bundle: parse filter capability

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 7, 2022, 15:38 UTC
Message-ID
<220307.86bkyhzs3r.gmgdl@evledraar.gmail.com>
In-Reply-To
<e7dbb46e6acb5c22a0b456135f3af42b974e0268.1645638911.git.gitgitgadget@gmail.com>
On Wed, Feb 23 2022, Derrick Stolee via GitGitGadget wrote:
Show 88 quoted lines
> From: Derrick Stolee <derrickstolee@github.com>
>
> The v3 bundle format has capabilities, allowing newer versions of Git to
> create bundles with newer features. Older versions that do not
> understand these new capabilities will fail with a helpful warning.
>
> Create a new capability allowing Git to understand that the contained
> pack-file is filtered according to some object filter. Typically, this
> filter will be "blob:none" for a blobless partial clone.
>
> This change teaches Git to parse this capability, place its value in the
> bundle header, and demonstrate this understanding by adding a message to
> 'git bundle verify'.
>
> Signed-off-by: Derrick Stolee <derrickstolee@github.com>
> ---
>  bundle.c                      | 17 ++++++++++++++++-
>  bundle.h                      |  3 +++
>  list-objects-filter-options.c |  2 +-
>  list-objects-filter-options.h |  5 +++++
>  4 files changed, 25 insertions(+), 2 deletions(-)
>
> diff --git a/bundle.c b/bundle.c
> index dc56db9a50a..2afced4d991 100644
> --- a/bundle.c
> +++ b/bundle.c
> @@ -11,7 +11,7 @@
>  #include "run-command.h"
>  #include "refs.h"
>  #include "strvec.h"
> -
> +#include "list-objects-filter-options.h"
>  
>  static const char v2_bundle_signature[] = "# v2 git bundle\n";
>  static const char v3_bundle_signature[] = "# v3 git bundle\n";
> @@ -33,6 +33,8 @@ void bundle_header_release(struct bundle_header *header)
>  {
>  	string_list_clear(&header->prerequisites, 1);
>  	string_list_clear(&header->references, 1);
> +	list_objects_filter_release(header->filter);
> +	free(header->filter);
>  }
>  
>  static int parse_capability(struct bundle_header *header, const char *capability)
> @@ -45,6 +47,11 @@ static int parse_capability(struct bundle_header *header, const char *capability
>  		header->hash_algo = &hash_algos[algo];
>  		return 0;
>  	}
> +	if (skip_prefix(capability, "filter=", &arg)) {
> +		CALLOC_ARRAY(header->filter, 1);
> +		parse_list_objects_filter(header->filter, arg);
> +		return 0;
> +	}
>  	return error(_("unknown capability '%s'"), capability);
>  }
>  
> @@ -220,6 +227,8 @@ int verify_bundle(struct repository *r,
>  	req_nr = revs.pending.nr;
>  	setup_revisions(2, argv, &revs, NULL);
>  
> +	revs.filter = header->filter;
> +
>  	if (prepare_revision_walk(&revs))
>  		die(_("revision walk setup failed"));
>  
> @@ -259,6 +268,12 @@ int verify_bundle(struct repository *r,
>  			     r->nr),
>  			  r->nr);
>  		list_refs(r, 0, NULL);
> +
> +		if (header->filter) {
> +			printf_ln("The bundle uses this filter: %s",
> +				  list_objects_filter_spec(header->filter));
> +		}
> +
>  		r = &header->prerequisites;
>  		if (!r->nr) {
>  			printf_ln(_("The bundle records a complete history."));
> diff --git a/bundle.h b/bundle.h
> index 06009fe6b1f..eb026153d56 100644
> --- a/bundle.h
> +++ b/bundle.h
> @@ -5,11 +5,14 @@
>  #include "cache.h"
>  #include "string-list.h"
>  
> +struct list_objects_filter_options;
> +

For the other ones we include the relevant header, do the same here (or if there's a need to not do it, do we need it for the rest too?)

Show 7 quoted lines
>  struct bundle_header {
>  	unsigned version;
>  	struct string_list prerequisites;
>  	struct string_list references;
>  	const struct git_hash_algo *hash_algo;
> +	struct list_objects_filter_options *filter;
>  };

I haven't tried, but any reason this needs to be a *filter v.s. embedding it in the struct?

Then we'd just need list_objects_filter_release() and not the free() as well.

Is it because you're piggy-backing on "if (header->filter)" as "do we have it" state, better to check .nr?

> @@ -55,7 +55,7 @@ const char *list_object_filter_config_name(enum list_objects_filter_choice c)
>   * expand_list_objects_filter_spec() first).  We also "intern" the arg for the
>   * convenience of the current command.
>   */
These API docs....
Show 12 quoted lines
> -static int gently_parse_list_objects_filter(
> +int gently_parse_list_objects_filter(
>  	struct list_objects_filter_options *filter_options,
>  	const char *arg,
>  	struct strbuf *errbuf)
> diff --git a/list-objects-filter-options.h b/list-objects-filter-options.h
> index da5b6737e27..347a99c28cf 100644
> --- a/list-objects-filter-options.h
> +++ b/list-objects-filter-options.h
> @@ -72,6 +72,11 @@ struct list_objects_filter_options {
>  /* Normalized command line arguments */
>  #define CL_ARG__FILTER "filter"
...should be moved here, presumably.
Show 5 quoted lines
> +int gently_parse_list_objects_filter(
> +	struct list_objects_filter_options *filter_options,
> +	const char *arg,
> +	struct strbuf *errbuf);
> +
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee
Message 18 of 114 in “Partial bundles”
  1. 00/11 Partial bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  2. 01/11 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  3. 02/11 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Feb 23, 2022
  4. Junio C HamanoMar 4, 2022
  5. Derrick StoleeMar 7, 2022
  6. Junio C HamanoMar 7, 2022
  7. 03/11 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Feb 23, 2022
  8. Junio C HamanoMar 4, 2022
  9. 04/11 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Feb 23, 2022
  10. Junio C HamanoMar 4, 2022
  11. 05/11 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Feb 23, 2022
  12. Junio C HamanoMar 4, 2022
  13. 06/11 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Feb 23, 2022
  14. Junio C HamanoMar 4, 2022
  15. Derrick StoleeMar 7, 2022
  16. Junio C HamanoMar 7, 2022
  17. 08/11 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  18. Ævar Arnfjörð BjarmasonMar 7, 2022
  19. Derrick StoleeMar 7, 2022
  20. Ævar Arnfjörð BjarmasonMar 7, 2022
  21. Derrick StoleeMar 7, 2022
  22. Ævar Arnfjörð BjarmasonMar 7, 2022
  23. 07/11 bundle: safely handle --objects optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  24. Jeff HostetlerFeb 28, 2022
  25. Junio C HamanoMar 4, 2022
  26. Derrick StoleeMar 7, 2022
  27. Junio C HamanoMar 4, 2022
  28. Ævar Arnfjörð BjarmasonMar 7, 2022
  29. 09/11 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Feb 23, 2022
  30. 10/11 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  31. Junio C HamanoMar 4, 2022
  32. Derrick StoleeMar 7, 2022
  33. Junio C HamanoMar 7, 2022
  34. Ævar Arnfjörð BjarmasonMar 7, 2022
  35. 11/11 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Feb 23, 2022
  36. Junio C HamanoMar 4, 2022
  37. Derrick StoleeMar 7, 2022
  38. Junio C HamanoMar 7, 2022
  39. Derrick StoleeMar 7, 2022
  40. Junio C HamanoMar 7, 2022
  41. Derrick StoleeMar 7, 2022
  42. Junio C HamanoMar 7, 2022
  43. Derrick StoleeMar 7, 2022
  44. Junio C HamanoMar 7, 2022
  45. Ævar Arnfjörð BjarmasonMar 7, 2022
  46. Derrick StoleeMar 7, 2022
  47. Jeff HostetlerFeb 28, 2022
  48. Derrick StoleeFeb 28, 2022
  49. Jeff HostetlerMar 1, 2022
  50. Derrick StoleeMar 4, 2022
  51. Ævar Arnfjörð BjarmasonMar 7, 2022
  52. Derrick StoleeMar 7, 2022
  53. 00/12 Partial bundlesDerrick Stolee via GitGitGadget, Mar 7, 2022
  54. 01/12 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 7, 2022
  55. 02/12 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 7, 2022
  56. 03/12 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 7, 2022
  57. 04/12 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 7, 2022
  58. 05/12 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 7, 2022
  59. 06/12 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 7, 2022
  60. 07/12 bundle: safely handle --objects optionDerrick Stolee via GitGitGadget, Mar 7, 2022
  61. Ævar Arnfjörð BjarmasonMar 8, 2022
  62. Derrick StoleeMar 8, 2022
  63. Ævar Arnfjörð BjarmasonMar 8, 2022
  64. 12/12 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 7, 2022
  65. 10/12 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 7, 2022
  66. 09/12 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 7, 2022
  67. 11/12 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 7, 2022
  68. 08/12 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 7, 2022
  69. Ævar Arnfjörð BjarmasonMar 8, 2022
  70. Derrick StoleeMar 8, 2022
  71. Junio C HamanoMar 7, 2022
  72. 00/12 Partial bundlesDerrick Stolee via GitGitGadget, Mar 8, 2022
  73. 01/12 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 8, 2022
  74. 02/12 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 8, 2022
  75. 03/12 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 8, 2022
  76. 04/12 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 8, 2022
  77. 05/12 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 8, 2022
  78. Ævar Arnfjörð BjarmasonMar 9, 2022
  79. 06/12 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 8, 2022
  80. 08/12 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 8, 2022
  81. Junio C HamanoMar 8, 2022
  82. Derrick StoleeMar 9, 2022
  83. Ævar Arnfjörð BjarmasonMar 9, 2022
  84. 07/12 list-objects: handle NULL function pointersÆvar Arnfjörð Bjarmason via GitGitGadget, Mar 8, 2022
  85. Junio C HamanoMar 8, 2022
  86. Ævar Arnfjörð BjarmasonMar 9, 2022
  87. Derrick StoleeMar 9, 2022
  88. Junio C HamanoMar 9, 2022
  89. 09/12 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 8, 2022
  90. 10/12 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 8, 2022
  91. 11/12 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 8, 2022
  92. 12/12 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 8, 2022
  93. Derrick StoleeMar 8, 2022
  94. Junio C HamanoMar 8, 2022
  95. 00/13 Partial bundlesDerrick Stolee via GitGitGadget, Mar 9, 2022
  96. 09/13 bundle: parse filter capabilityDerrick Stolee via GitGitGadget, Mar 9, 2022
  97. Junio C HamanoMar 9, 2022
  98. Derrick StoleeMar 9, 2022
  99. 06/13 list-objects: consolidate traverse_commit_list[_filtered]Derrick Stolee via GitGitGadget, Mar 9, 2022
  100. 03/13 revision: put object filter into struct rev_infoDerrick Stolee via GitGitGadget, Mar 9, 2022
  101. Junio C HamanoMar 9, 2022
  102. 01/13 index-pack: document and test the --promisor optionDerrick Stolee via GitGitGadget, Mar 9, 2022
  103. 07/13 MyFirstObjectWalk: update recommended usageDerrick Stolee via GitGitGadget, Mar 9, 2022
  104. 11/13 bundle: create filtered bundlesDerrick Stolee via GitGitGadget, Mar 9, 2022
  105. 04/13 pack-objects: use rev.filter when possibleDerrick Stolee via GitGitGadget, Mar 9, 2022
  106. Ævar Arnfjörð BjarmasonMar 10, 2022
  107. Derrick StoleeMar 10, 2022
  108. Ævar Arnfjörð BjarmasonMar 10, 2022
  109. 08/13 list-objects: handle NULL function pointersÆvar Arnfjörð Bjarmason via GitGitGadget, Mar 9, 2022
  110. 05/13 pack-bitmap: drop filter in prepare_bitmap_walk()Derrick Stolee via GitGitGadget, Mar 9, 2022
  111. 02/13 list-objects-filter-options: create copy helperDerrick Stolee via GitGitGadget, Mar 9, 2022
  112. 10/13 rev-list: move --filter parsing into revision.cDerrick Stolee via GitGitGadget, Mar 9, 2022
  113. 12/13 bundle: unbundle promisor packsDerrick Stolee via GitGitGadget, Mar 9, 2022
  114. 13/13 clone: fail gracefully when cloning filtered bundleDerrick Stolee via GitGitGadget, Mar 9, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.