git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 5/5] path-walk: support wildcard pathspecs for blob filtering

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 17, 2026, 00:29 UTC
Message-ID
<beb1c92554c76907315a4d1a7983226d2bf5a828.1773707361.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2070.git.1773707361.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

Previously, walk_objects_by_path() silently ignored pathspecs containing wildcards or magic by clearing them. This caused all blobs to be downloaded regardless of the given pathspec. Wildcard pathspecs like "d/file.*.txt" are useful for narrowing which blobs to process (e.g., during 'git backfill').

Support wildcard pathspecs by making three changes:
 1. Add an 'exact_pathspecs' flag to path_walk_context. When the
    pathspec has no wildcards or magic, set this flag and use the
    existing fast-path prefix matching in add_tree_entries(). When
    wildcards are present, skip that block since prefix matching
    cannot handle glob patterns.
 2. Disable revision-level commit pruning (revs->prune = 0) for
    wildcard pathspecs. The revision walk uses the pathspec to filter
    commits via TREESAME detection. For exact prefix pathspecs this
    works well, but wildcard pathspecs may fail to match through
    TREESAME because fnmatch with WM_PATHNAME does not cross directory
    boundaries. Disabling pruning ensures all commits are visited and
    their trees are available for the path-walk to filter.
 3. Add a match_pathspec() check in walk_path() to filter out blobs
    whose full path does not match the pathspec. This provides the
    actual blob-level filtering for wildcard pathspecs.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 path-walk.c         | 22 ++++++++++++++--------
 t/t5620-backfill.sh |  7 +++----
 2 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/path-walk.c b/path-walk.c
index e1ad4b0208..67fb0f7572 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -62,6 +62,8 @@ struct path_walk_context {
 	 */
 	struct prio_queue path_stack;
 	struct strset path_stack_pushed;
+
+	unsigned exact_pathspecs:1;
 };
 
 static int compare_by_type(const void *one, const void *two, void *cb_data)
@@ -206,7 +208,7 @@ static int add_tree_entries(struct path_walk_context *ctx,
 				 match != MATCHED)
 				continue;
 		}
-		if (ctx->revs->prune_data.nr) {
+		if (ctx->revs->prune_data.nr && ctx->exact_pathspecs) {
 			struct pathspec *pd = &ctx->revs->prune_data;
 			bool found = false;
 
@@ -317,6 +319,13 @@ static int walk_path(struct path_walk_context *ctx,
 			return 0;
 	}
 
+	if (list->type == OBJ_BLOB &&
+	    ctx->revs->prune_data.nr &&
+	    !match_pathspec(ctx->repo->index, &ctx->revs->prune_data,
+			   path, strlen(path), 0,
+			   NULL, 0))
+		return 0;
+
 	/* Evaluate function pointer on this data, if requested. */
 	if ((list->type == OBJ_TREE && ctx->info->trees) ||
 	    (list->type == OBJ_BLOB && ctx->info->blobs) ||
@@ -525,15 +534,12 @@ int walk_objects_by_path(struct path_walk_info *info)
 		info->revs->tag_objects = 1;
 
 	if (ctx.revs->prune_data.nr) {
-		/*
-		 * Only exact prefix pathspecs are currently supported.
-		 * Clear any wildcard or magic pathspecs to avoid
-		 * incorrect prefix matching.
-		 */
 		struct pathspec *pd = &ctx.revs->prune_data;
 
-		if (pd->has_wildcard || pd->magic)
-			pd->nr = 0;
+		if (!pd->has_wildcard && !pd->magic)
+			ctx.exact_pathspecs = 1;
+		else
+			ctx.revs->prune = 0;
 	}
 
 	/* Insert a single list for the root tree into the paths. */
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index 52f6484ca1..c6f54ee91c 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -307,12 +307,11 @@ test_expect_success 'backfill with wildcard pathspec' '
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
 	test_line_count = 48 missing &&
 
-	# TODO: The wildcard pathspec should limit downloaded blobs,
-	# but currently all blobs are downloaded.
-	git -C backfill-path backfill HEAD -- "d/file.*.txt" &&
+	git -C backfill-path backfill HEAD -- "d/file.*.txt" 2>err &&
+	test_must_be_empty err &&
 
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
-	test_line_count = 0 missing
+	test_line_count = 40 missing
 '
 
 test_expect_success 'backfill with --all' '
-- 
gitgitgadget
Previous: Derrick StoleeNext: Junio C Hamano
Message 18 of 46 in “backfill: accept revision arguments”
  1. 0/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  2. 1/5 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 17, 2026
  3. Junio C HamanoMar 17, 2026
  4. 2/5 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 17, 2026
  5. 3/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  6. Junio C HamanoMar 17, 2026
  7. Kristoffer HaugsbakkMar 18, 2026
  8. Derrick StoleeMar 23, 2026
  9. Patrick SteinhardtMar 19, 2026
  10. Derrick StoleeMar 23, 2026
  11. 4/5 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 17, 2026
  12. Junio C HamanoMar 17, 2026
  13. Derrick StoleeMar 18, 2026
  14. Patrick SteinhardtMar 19, 2026
  15. Patrick SteinhardtMar 19, 2026
  16. Patrick SteinhardtMar 19, 2026
  17. Derrick StoleeMar 23, 2026
  18. 5/5 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 17, 2026
  19. Junio C HamanoMar 17, 2026
  20. Derrick StoleeMar 18, 2026
  21. Derrick StoleeMar 23, 2026
  22. Junio C HamanoMar 17, 2026
  23. Patrick SteinhardtMar 19, 2026
  24. Derrick StoleeMar 19, 2026
  25. Patrick SteinhardtMar 20, 2026
  26. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  27. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 23, 2026
  28. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 23, 2026
  29. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  30. Patrick SteinhardtMar 24, 2026
  31. Derrick StoleeMar 26, 2026
  32. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 23, 2026
  33. Patrick SteinhardtMar 24, 2026
  34. Derrick StoleeMar 26, 2026
  35. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 23, 2026
  36. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 23, 2026
  37. Junio C HamanoMar 23, 2026
  38. Derrick StoleeMar 23, 2026
  39. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  40. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 26, 2026
  41. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 26, 2026
  42. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  43. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 26, 2026
  44. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 26, 2026
  45. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 26, 2026
  46. Patrick SteinhardtMar 27, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.