git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 03/10] path-walk: support blobless filter

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 4, 2026, 20:21 UTC
Message-ID
<ed4d277a2c7dff0566acaf7b902e7474214fbab4.1777926079.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2101.v2.git.1777926079.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

The 'git pack-objects' command can opt-in to using the path-walk API for scanning the objects. Currently, this option is dynamically disabled if combined with '--filter=<X>', even when using a simple filter such as 'blob:none' to signal a blobless packfile. This is a common scenario for repos at scale, so is worth integrating.

Also, users can opt-in to the '--path-walk' option by default through the pack.usePathWalk=true config option. When using that in a blobless partial clone, the following warning can appear even though the user did not specify either option directly:

  warning: cannot use --filter with --path-walk

Teach the path-walk API to handle the 'blob:none' object filter natively. When revs->filter.choice is LOFC_BLOB_NONE, the path-walk sets info->blobs to 0 (skipping all blob objects) and clears the filter from revs so that prepare_revision_walk() does not reject the configuration.

This check is implemented in the static prepare_filters() method, which will simultaneously check if the input filters are compatible and will make the appropriate mutations to the path_walk_info and filters if the path_walk_info is non-NULL. This allows us to use this logic both in the API method path_walk_filter_compatible() for use in builtin/pack-objects.c and as a prep step in walk_objects_by_path().

Update the test helper (test-path-walk) to accept --filter=<spec> as a test-tool option (before '--'), applying it to revs after setup_revisions() to avoid the --objects requirement check.

Also switch test-path-walk from REV_INFO_INIT with manual repo assignment to repo_init_revisions(), which properly initializes the filter_spec strbuf needed for filter parsing.

Add tests for blob:none with --all and with a single branch.

The performance test p5315 shows the impact of this change when using blobless filters:

Test HEAD~1 HEAD --------------------------------------------------------------------- 5315.6: repack (blob:none) 13.53 13.87 +2.5% 5315.7: repack size (blob:none) 137.7M 137.8M +0.1% 5315.8: repack (blob:none, --path-walk) 13.51 23.43 +73.4% 5315.9: repack size (blob:none, --path-walk) 137.7M 115.2M -16.3%

These performance tests were run on the Git repository. The --path-walk feature shows meaningful space savings (16% smaller for blobless packs) at the cost of increased computation time due to the two compression passes. This data demonstrates that the feature is engaged and provides real compression benefits when --no-reuse-delta forces fresh deltas.

Co-Authored-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 Documentation/git-pack-objects.adoc |  6 +--
 builtin/pack-objects.c              |  2 +-
 path-walk.c                         | 30 +++++++++++++++
 path-walk.h                         |  7 ++++
 t/helper/test-path-walk.c           | 11 +++++-
 t/t6601-path-walk.sh                | 60 +++++++++++++++++++++++++++++
 6 files changed, 111 insertions(+), 5 deletions(-)
diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index b78175fbe1..917045d5c3 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -402,9 +402,9 @@ will be automatically changed to version `1`.
 	of filenames that cause collisions in Git's default name-hash
 	algorithm.
 +
-Incompatible with `--delta-islands`, `--shallow`, or `--filter`. The
-`--use-bitmap-index` option will be ignored in the presence of
-`--path-walk.`
+Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
+ignored in the presence of `--path-walk`. Whe `--path-walk` option
+supports the `--filter=<spec>` form `blob:none`.
 
 
 DELTA ISLANDS
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 4338962904..bc9fb5b457 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -5177,7 +5177,7 @@ int cmd_pack_objects(int argc,
 
 	if (path_walk) {
 		const char *option = NULL;
-		if (filter_options.choice)
+		if (!path_walk_filter_compatible(&filter_options))
 			option = "--filter";
 		else if (use_delta_islands)
 			option = "--delta-islands";
diff --git a/path-walk.c b/path-walk.c
index 6e426af433..a4dd197c37 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -9,6 +9,7 @@
 #include "hashmap.h"
 #include "hex.h"
 #include "list-objects.h"
+#include "list-objects-filter-options.h"
 #include "object.h"
 #include "oid-array.h"
 #include "path.h"
@@ -485,6 +486,32 @@ static int setup_pending_objects(struct path_walk_info *info,
 	return 0;
 }
 
+static int prepare_filters(struct path_walk_info *info,
+			   struct list_objects_filter_options *options)
+{
+	switch (options->choice) {
+	case LOFC_DISABLED:
+		return 1;
+
+	case LOFC_BLOB_NONE:
+		if (info) {
+			info->blobs = 0;
+			list_objects_filter_release(options);
+		}
+		return 1;
+
+	default:
+		error(_("object filter '%s' not supported by the path-walk API"),
+		      list_objects_filter_spec(options));
+		return 0;
+	}
+}
+
+int path_walk_filter_compatible(struct list_objects_filter_options *options)
+{
+	return prepare_filters(NULL, options);
+}
+
 /**
  * Given the configuration of 'info', walk the commits based on 'info->revs' and
  * call 'info->path_fn' on each discovered path.
@@ -512,6 +539,9 @@ int walk_objects_by_path(struct path_walk_info *info)
 
 	trace2_region_enter("path-walk", "commit-walk", info->revs->repo);
 
+	if (!prepare_filters(info, &info->revs->filter))
+		return -1;
+
 	CALLOC_ARRAY(commit_list, 1);
 	commit_list->type = OBJ_COMMIT;
 
diff --git a/path-walk.h b/path-walk.h
index 5ef5a8440e..be8d27b398 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -85,3 +85,10 @@ void path_walk_info_clear(struct path_walk_info *info);
  * Returns nonzero on an error.
  */
 int walk_objects_by_path(struct path_walk_info *info);
+
+struct list_objects_filter_options;
+/**
+ * Given a set of options for filtering objects, return 1 if the options
+ * are compatible with the path-walk API and 0 otherwise.
+ */
+int path_walk_filter_compatible(struct list_objects_filter_options *options);
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index fe63002c2b..88f86ae0dc 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -4,6 +4,7 @@
 #include "dir.h"
 #include "environment.h"
 #include "hex.h"
+#include "list-objects-filter-options.h"
 #include "object-name.h"
 #include "object.h"
 #include "pretty.h"
@@ -71,6 +72,8 @@ int cmd__path_walk(int argc, const char **argv)
 	struct rev_info revs = REV_INFO_INIT;
 	struct path_walk_info info = PATH_WALK_INFO_INIT;
 	struct path_walk_test_data data = { 0 };
+	struct list_objects_filter_options filter_options =
+		LIST_OBJECTS_FILTER_INIT;
 	struct option options[] = {
 		OPT_BOOL(0, "blobs", &info.blobs,
 			 N_("toggle inclusion of blob objects")),
@@ -86,11 +89,12 @@ int cmd__path_walk(int argc, const char **argv)
 			 N_("toggle aggressive edge walk")),
 		OPT_BOOL(0, "stdin-pl", &stdin_pl,
 			 N_("read a pattern list over stdin")),
+		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
 		OPT_END(),
 	};
 
 	setup_git_directory();
-	revs.repo = the_repository;
+	repo_init_revisions(the_repository, &revs, NULL);
 
 	argc = parse_options(argc, argv, NULL,
 			     options, path_walk_usage,
@@ -101,6 +105,10 @@ int cmd__path_walk(int argc, const char **argv)
 	else
 		usage(path_walk_usage[0]);
 
+	/* Apply the filter after setup_revisions to avoid the --objects check. */
+	if (filter_options.choice)
+		list_objects_filter_copy(&revs.filter, &filter_options);
+
 	info.revs = &revs;
 	info.path_fn = emit_block;
 	info.path_fn_data = &data;
@@ -129,6 +137,7 @@ int cmd__path_walk(int argc, const char **argv)
 		free(info.pl);
 	}
 
+	list_objects_filter_release(&filter_options);
 	release_revisions(&revs);
 	return res;
 }
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 56bd1e3c5b..94df309987 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -415,4 +415,64 @@ test_expect_success 'trees are reported exactly once' '
 	test_line_count = 1 out-filtered
 '
 
+test_expect_success 'all, blob:none filter' '
+	test-tool path-walk --filter=blob:none -- --all >out &&
+
+	cat >expect <<-EOF &&
+	0:commit::$(git rev-parse topic)
+	0:commit::$(git rev-parse base)
+	0:commit::$(git rev-parse base~1)
+	0:commit::$(git rev-parse base~2)
+	1:tag:/tags:$(git rev-parse refs/tags/first)
+	1:tag:/tags:$(git rev-parse refs/tags/second.1)
+	1:tag:/tags:$(git rev-parse refs/tags/second.2)
+	1:tag:/tags:$(git rev-parse refs/tags/third)
+	1:tag:/tags:$(git rev-parse refs/tags/fourth)
+	1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+	1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+	2:tree::$(git rev-parse topic^{tree})
+	2:tree::$(git rev-parse base^{tree})
+	2:tree::$(git rev-parse base~1^{tree})
+	2:tree::$(git rev-parse base~2^{tree})
+	2:tree::$(git rev-parse refs/tags/tree-tag^{})
+	2:tree::$(git rev-parse refs/tags/tree-tag2^{})
+	3:tree:a/:$(git rev-parse base:a)
+	4:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+	5:tree:left/:$(git rev-parse base:left)
+	5:tree:left/:$(git rev-parse base~2:left)
+	6:tree:right/:$(git rev-parse topic:right)
+	6:tree:right/:$(git rev-parse base~1:right)
+	6:tree:right/:$(git rev-parse base~2:right)
+	blobs:0
+	commits:4
+	tags:7
+	trees:13
+	EOF
+
+	test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, blob:none filter' '
+	test-tool path-walk --filter=blob:none -- topic >out &&
+
+	cat >expect <<-EOF &&
+	0:commit::$(git rev-parse topic)
+	0:commit::$(git rev-parse base~1)
+	0:commit::$(git rev-parse base~2)
+	1:tree::$(git rev-parse topic^{tree})
+	1:tree::$(git rev-parse base~1^{tree})
+	1:tree::$(git rev-parse base~2^{tree})
+	2:tree:left/:$(git rev-parse base~2:left)
+	3:tree:right/:$(git rev-parse topic:right)
+	3:tree:right/:$(git rev-parse base~1:right)
+	3:tree:right/:$(git rev-parse base~2:right)
+	blobs:0
+	commits:3
+	tags:0
+	trees:7
+	EOF
+
+	test_cmp_sorted expect out
+'
+
 test_done
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 16 of 96 in “pack-objects: integrate --path-walk and some --filter options”
  1. 0/7 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  2. 1/7 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  3. Junio C HamanoMay 4, 2026
  4. Derrick StoleeMay 4, 2026
  5. 2/7 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 2, 2026
  6. 3/7 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 2, 2026
  7. 4/7 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  8. Junio C HamanoMay 3, 2026
  9. Derrick StoleeMay 4, 2026
  10. 5/7 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 2, 2026
  11. 6/7 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 2, 2026
  12. 7/7 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  13. 00/10 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  14. 01/10 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  15. 02/10 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 4, 2026
  16. 03/10 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 4, 2026
  17. 04/10 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  18. 05/10 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 4, 2026
  19. 06/10 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 4, 2026
  20. 07/10 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  21. 08/10 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 4, 2026
  22. 09/10 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 4, 2026
  23. 10/10 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 4, 2026
  24. Derrick StoleeMay 5, 2026
  25. Taylor BlauMay 5, 2026
  26. Derrick StoleeMay 5, 2026
  27. Taylor BlauMay 5, 2026
  28. Derrick StoleeMay 7, 2026
  29. Junio C HamanoMay 11, 2026
  30. Derrick StoleeMay 11, 2026
  31. 00/12 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  32. 01/12 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 11, 2026
  33. Taylor BlauMay 12, 2026
  34. 02/12 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  35. Taylor BlauMay 12, 2026
  36. 03/12 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 11, 2026
  37. Taylor BlauMay 12, 2026
  38. Derrick StoleeMay 13, 2026
  39. 04/12 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 11, 2026
  40. Taylor BlauMay 12, 2026
  41. Derrick StoleeMay 13, 2026
  42. 05/12 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 11, 2026
  43. Taylor BlauMay 11, 2026
  44. Derrick StoleeMay 11, 2026
  45. 06/12 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  46. Taylor BlauMay 12, 2026
  47. 07/12 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 11, 2026
  48. Taylor BlauMay 12, 2026
  49. Derrick StoleeMay 13, 2026
  50. 08/12 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 11, 2026
  51. 09/12 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  52. 10/12 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 11, 2026
  53. Taylor BlauMay 12, 2026
  54. Derrick StoleeMay 13, 2026
  55. 11/12 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 11, 2026
  56. 12/12 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 11, 2026
  57. Taylor BlauMay 12, 2026
  58. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  59. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 13, 2026
  60. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  61. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 13, 2026
  62. Taylor BlauMay 19, 2026
  63. Derrick StoleeMay 21, 2026
  64. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 13, 2026
  65. Taylor BlauMay 19, 2026
  66. Derrick StoleeMay 21, 2026
  67. Taylor BlauMay 22, 2026
  68. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 13, 2026
  69. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  70. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 13, 2026
  71. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 13, 2026
  72. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  73. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 13, 2026
  74. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 13, 2026
  75. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 13, 2026
  76. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 13, 2026
  77. Taylor BlauMay 19, 2026
  78. Derrick StoleeMay 21, 2026
  79. Taylor BlauMay 22, 2026
  80. Derrick StoleeMay 22, 2026
  81. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  82. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 22, 2026
  83. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  84. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 22, 2026
  85. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 22, 2026
  86. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 22, 2026
  87. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  88. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 22, 2026
  89. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 22, 2026
  90. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  91. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 22, 2026
  92. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 22, 2026
  93. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 22, 2026
  94. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 22, 2026
  95. Taylor BlauMay 22, 2026
  96. Junio C HamanoMay 24, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.