git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 08/13] path-walk: add pl_sparse_trees to control tree pruning

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 13, 2026, 21:18 UTC
Message-ID
<0f517be8e3853e05d313258133db4eab229d3835.1778707135.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2101.v4.git.1778707135.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

The path-walk API prunes trees and blobs when a sparse-checkout pattern list is provided, which is the correct behavior for 'git backfill --sparse' since it only needs to fill in objects at paths within the sparse cone.

However, a future change will use the path-walk API with a sparse:<oid> filter that restricts only blobs while retaining all reachable trees. To support both behaviors, add a 'pl_sparse_trees' flag to path_walk_info. When set (as in 'git backfill --sparse' and the --stdin-pl test helper mode), the sparse patterns prune both trees and blobs. When unset, only blobs are filtered and all trees are walked and reported.

Additionally, move the SEEN flag assignment in add_tree_entries() to after the sparse pattern and pathspec checks. Previously, SEEN was set immediately upon discovering an object, before checking whether its path matched the sparse patterns. When the same object ID appeared at multiple paths (e.g. sibling directories with identical contents), the first path to be visited would mark the object as SEEN. If that path was outside the sparse cone, the object would be skipped there but also never discovered at its in-cone path.

By deferring the SEEN flag until after the checks pass, objects that are skipped due to sparse filtering remain discoverable at other paths where they may be in scope.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 builtin/backfill.c        |  1 +
 path-walk.c               |  5 +++--
 path-walk.h               |  6 ++++++
 t/helper/test-path-walk.c |  6 +++++-
 t/t6601-path-walk.sh      | 37 +++++++++++++++++++++++++++++++++++++
 5 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 5254a42711..e71e0f4742 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -109,6 +109,7 @@ static int do_backfill(struct backfill_context *ctx)
 
 	if (ctx->sparse) {
 		CALLOC_ARRAY(info.pl, 1);
+		info.pl_sparse_trees = 1;
 		if (get_sparse_checkout_patterns(info.pl)) {
 			path_walk_info_clear(&info);
 			return error(_("problem loading sparse-checkout"));
diff --git a/path-walk.c b/path-walk.c
index 04b924d4de..225857bbc8 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -183,7 +183,6 @@ static int add_tree_entries(struct path_walk_context *ctx,
 		/* Skip this object if already seen. */
 		if (o->flags & SEEN)
 			continue;
-		o->flags |= SEEN;
 
 		strbuf_setlen(&path, base_len);
 		strbuf_add(&path, entry.path, entry.pathlen);
@@ -204,7 +203,8 @@ static int add_tree_entries(struct path_walk_context *ctx,
 							  ctx->repo->index);
 
 			if (ctx->info->pl->use_cone_patterns &&
-			    match == NOT_MATCHED)
+			    match == NOT_MATCHED &&
+			    (type == OBJ_BLOB || ctx->info->pl_sparse_trees))
 				continue;
 			else if (!ctx->info->pl->use_cone_patterns &&
 				 type == OBJ_BLOB &&
@@ -239,6 +239,7 @@ static int add_tree_entries(struct path_walk_context *ctx,
 				continue;
 		}
 
+		o->flags |= SEEN;
 		add_path_to_list(ctx, path.buf, type, &entry.oid,
 				 !(o->flags & UNINTERESTING));
 
diff --git a/path-walk.h b/path-walk.h
index 60ceb65433..7e57ae5f65 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -76,8 +76,14 @@ struct path_walk_info {
 	 * of the cone. If not in cone mode, then all tree paths will be
 	 * explored but the path_fn will only be called when the path matches
 	 * the sparse-checkout patterns.
+	 *
+	 * When 'pl_sparse_trees' is zero, the sparse patterns only restrict
+	 * blobs and all trees are included in the walk output. This matches
+	 * the behavior of the sparse:oid object filter. When nonzero, trees
+	 * are also pruned by the sparse patterns (as used by backfill).
 	 */
 	struct pattern_list *pl;
+	int pl_sparse_trees;
 };
 
 #define PATH_WALK_INFO_INIT {   \
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index 88f86ae0dc..3f2b50a9aa 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -68,7 +68,7 @@ static int emit_block(const char *path, struct oid_array *oids,
 
 int cmd__path_walk(int argc, const char **argv)
 {
-	int res, stdin_pl = 0;
+	int res, stdin_pl = 0, pl_sparse_trees = -1;
 	struct rev_info revs = REV_INFO_INIT;
 	struct path_walk_info info = PATH_WALK_INFO_INIT;
 	struct path_walk_test_data data = { 0 };
@@ -89,6 +89,8 @@ int cmd__path_walk(int argc, const char **argv)
 			 N_("toggle aggressive edge walk")),
 		OPT_BOOL(0, "stdin-pl", &stdin_pl,
 			 N_("read a pattern list over stdin")),
+		OPT_BOOL(0, "pl-sparse-trees", &pl_sparse_trees,
+			 N_("toggle pruning of trees by sparse patterns")),
 		OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
 		OPT_END(),
 	};
@@ -116,6 +118,8 @@ int cmd__path_walk(int argc, const char **argv)
 	if (stdin_pl) {
 		struct strbuf in = STRBUF_INIT;
 		CALLOC_ARRAY(info.pl, 1);
+		info.pl_sparse_trees = (pl_sparse_trees >= 0) ?
+			pl_sparse_trees : 1;
 
 		info.pl->use_cone_patterns = 1;
 
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 45f366d738..02ad83dfb0 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -206,6 +206,43 @@ test_expect_success 'base & topic, sparse' '
 	test_cmp_sorted expect out
 '
 
+test_expect_success 'base & topic, sparse, no tree pruning' '
+	cat >patterns <<-EOF &&
+	/*
+	!/*/
+	/left/
+	EOF
+
+	test-tool path-walk --stdin-pl --no-pl-sparse-trees \
+		-- base topic <patterns >out &&
+
+	cat >expect <<-EOF &&
+	0:commit::$(git rev-parse topic)
+	0:commit::$(git rev-parse base)
+	0:commit::$(git rev-parse base~1)
+	0:commit::$(git rev-parse base~2)
+	1:tree::$(git rev-parse topic^{tree})
+	1:tree::$(git rev-parse base^{tree})
+	1:tree::$(git rev-parse base~1^{tree})
+	1:tree::$(git rev-parse base~2^{tree})
+	2:blob:a:$(git rev-parse base~2:a)
+	3:tree:a/:$(git rev-parse base:a)
+	4:tree:left/:$(git rev-parse base:left)
+	4:tree:left/:$(git rev-parse base~2:left)
+	5:blob:left/b:$(git rev-parse base~2:left/b)
+	5:blob:left/b:$(git rev-parse base:left/b)
+	6:tree:right/:$(git rev-parse topic:right)
+	6:tree:right/:$(git rev-parse base~1:right)
+	6:tree:right/:$(git rev-parse base~2:right)
+	blobs:3
+	commits:4
+	tags:0
+	trees:10
+	EOF
+
+	test_cmp_sorted expect out
+'
+
 test_expect_success 'topic only' '
 	test-tool path-walk -- topic >out &&
 
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 71 of 96 in “pack-objects: integrate --path-walk and some --filter options”
  1. 0/7 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  2. 1/7 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  3. Junio C HamanoMay 4, 2026
  4. Derrick StoleeMay 4, 2026
  5. 2/7 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 2, 2026
  6. 3/7 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 2, 2026
  7. 4/7 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  8. Junio C HamanoMay 3, 2026
  9. Derrick StoleeMay 4, 2026
  10. 5/7 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 2, 2026
  11. 6/7 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 2, 2026
  12. 7/7 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  13. 00/10 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  14. 01/10 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  15. 02/10 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 4, 2026
  16. 03/10 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 4, 2026
  17. 04/10 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  18. 05/10 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 4, 2026
  19. 06/10 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 4, 2026
  20. 07/10 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  21. 08/10 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 4, 2026
  22. 09/10 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 4, 2026
  23. 10/10 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 4, 2026
  24. Derrick StoleeMay 5, 2026
  25. Taylor BlauMay 5, 2026
  26. Derrick StoleeMay 5, 2026
  27. Taylor BlauMay 5, 2026
  28. Derrick StoleeMay 7, 2026
  29. Junio C HamanoMay 11, 2026
  30. Derrick StoleeMay 11, 2026
  31. 00/12 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  32. 01/12 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 11, 2026
  33. Taylor BlauMay 12, 2026
  34. 02/12 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  35. Taylor BlauMay 12, 2026
  36. 03/12 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 11, 2026
  37. Taylor BlauMay 12, 2026
  38. Derrick StoleeMay 13, 2026
  39. 04/12 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 11, 2026
  40. Taylor BlauMay 12, 2026
  41. Derrick StoleeMay 13, 2026
  42. 05/12 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 11, 2026
  43. Taylor BlauMay 11, 2026
  44. Derrick StoleeMay 11, 2026
  45. 06/12 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  46. Taylor BlauMay 12, 2026
  47. 07/12 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 11, 2026
  48. Taylor BlauMay 12, 2026
  49. Derrick StoleeMay 13, 2026
  50. 08/12 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 11, 2026
  51. 09/12 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  52. 10/12 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 11, 2026
  53. Taylor BlauMay 12, 2026
  54. Derrick StoleeMay 13, 2026
  55. 11/12 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 11, 2026
  56. 12/12 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 11, 2026
  57. Taylor BlauMay 12, 2026
  58. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  59. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 13, 2026
  60. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  61. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 13, 2026
  62. Taylor BlauMay 19, 2026
  63. Derrick StoleeMay 21, 2026
  64. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 13, 2026
  65. Taylor BlauMay 19, 2026
  66. Derrick StoleeMay 21, 2026
  67. Taylor BlauMay 22, 2026
  68. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 13, 2026
  69. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  70. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 13, 2026
  71. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 13, 2026
  72. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  73. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 13, 2026
  74. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 13, 2026
  75. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 13, 2026
  76. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 13, 2026
  77. Taylor BlauMay 19, 2026
  78. Derrick StoleeMay 21, 2026
  79. Taylor BlauMay 22, 2026
  80. Derrick StoleeMay 22, 2026
  81. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  82. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 22, 2026
  83. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  84. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 22, 2026
  85. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 22, 2026
  86. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 22, 2026
  87. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  88. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 22, 2026
  89. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 22, 2026
  90. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  91. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 22, 2026
  92. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 22, 2026
  93. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 22, 2026
  94. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 22, 2026
  95. Taylor BlauMay 22, 2026
  96. Junio C HamanoMay 24, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.