git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 4/5] backfill: work with prefix pathspecs

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 17, 2026, 00:29 UTC
Message-ID
<977f62faa550ecc4ef7774b4ecf53a9a520fc62a.1773707361.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2070.git.1773707361.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

The previous change allowed specifying revision arguments over the 'git backfill' command-line. This created the opportunity for pathspecs that specify a smaller set of starting commits, but otherwise did not restrict the blob paths that were downloaded.

Update the path-walk API to accept certain kinds of pathspecs and to silently ignore anything too complex. The current behavior focuses on pathspecs that match paths exactly. This includes exact filenames, including directory names as prefixes. Pathspecs containing wildcards or magic are cleared so the path walk downloads all blobs, as before.

The reason for this restriction is to allow for a faster execution by pruning the path walk to only trees that could contribute towards one of those paths as a parent directory.

The test directory 'd/f/' (next to 'd/file*.txt') was prepared in a previous commit to exercise the subtlety in prefix matching.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 path-walk.c         | 55 +++++++++++++++++++++++++++++++++++++++++++++
 t/t5620-backfill.sh | 16 +++++--------
 2 files changed, 61 insertions(+), 10 deletions(-)
diff --git a/path-walk.c b/path-walk.c
index 364e4cfa19..e1ad4b0208 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -206,6 +206,49 @@ static int add_tree_entries(struct path_walk_context *ctx,
 				 match != MATCHED)
 				continue;
 		}
+		if (ctx->revs->prune_data.nr) {
+			struct pathspec *pd = &ctx->revs->prune_data;
+			bool found = false;
+
+			for (int i = 0; i < pd->nr; i++) {
+				struct pathspec_item *item = &pd->items[i];
+
+				/*
+				 * Is this path a parent directory of
+				 * the pathspec item?
+				 */
+				if (path.len < (size_t)item->len &&
+				    !strncmp(path.buf, item->match, path.len) &&
+				    item->match[path.len - 1] == '/') {
+					found = true;
+					break;
+				}
+
+				/*
+				 * Or, is the pathspec an exact match?
+				 */
+				if (path.len == (size_t)item->len &&
+				    !strcmp(path.buf, item->match)) {
+					found = true;
+					break;
+				}
+
+				/*
+				 * Or, is the pathspec a directory prefix
+				 * match?
+				 */
+				if (path.len > (size_t)item->len &&
+				    !strncmp(path.buf, item->match, item->len) &&
+				    path.buf[item->len] == '/') {
+					found = true;
+					break;
+				}
+			}
+
+			/* Skip paths that do not match the prefix. */
+			if (!found)
+				continue;
+		}
 
 		add_path_to_list(ctx, path.buf, type, &entry.oid,
 				 !(o->flags & UNINTERESTING));
@@ -481,6 +524,18 @@ int walk_objects_by_path(struct path_walk_info *info)
 	if (info->tags)
 		info->revs->tag_objects = 1;
 
+	if (ctx.revs->prune_data.nr) {
+		/*
+		 * Only exact prefix pathspecs are currently supported.
+		 * Clear any wildcard or magic pathspecs to avoid
+		 * incorrect prefix matching.
+		 */
+		struct pathspec *pd = &ctx.revs->prune_data;
+
+		if (pd->has_wildcard || pd->magic)
+			pd->nr = 0;
+	}
+
 	/* Insert a single list for the root tree into the paths. */
 	CALLOC_ARRAY(root_tree_list, 1);
 	root_tree_list->type = OBJ_TREE;
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index db66d8b614..52f6484ca1 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -273,13 +273,11 @@ test_expect_success 'backfill with prefix pathspec' '
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
 	test_line_count = 48 missing &&
 
-	# TODO: The pathspec should limit the downloaded blobs to
-	# only those matching the prefix "d/f", but currently all
-	# blobs are downloaded.
-	git -C backfill-path backfill HEAD -- d/f &&
+	git -C backfill-path backfill HEAD -- d/f 2>err &&
+	test_must_be_empty err &&
 
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
-	test_line_count = 0 missing
+	test_line_count = 40 missing
 '
 
 test_expect_success 'backfill with multiple pathspecs' '
@@ -292,13 +290,11 @@ test_expect_success 'backfill with multiple pathspecs' '
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
 	test_line_count = 48 missing &&
 
-	# TODO: The pathspecs should limit the downloaded blobs to
-	# only those matching "d/f" or "a", but currently all blobs
-	# are downloaded.
-	git -C backfill-path backfill HEAD -- d/f a &&
+	git -C backfill-path backfill HEAD -- d/f a 2>err &&
+	test_must_be_empty err &&
 
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
-	test_line_count = 0 missing
+	test_line_count = 16 missing
 '
 
 test_expect_success 'backfill with wildcard pathspec' '
-- 
gitgitgadget
Previous: Derrick StoleeNext: Junio C Hamano
Message 11 of 46 in “backfill: accept revision arguments”
  1. 0/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  2. 1/5 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 17, 2026
  3. Junio C HamanoMar 17, 2026
  4. 2/5 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 17, 2026
  5. 3/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  6. Junio C HamanoMar 17, 2026
  7. Kristoffer HaugsbakkMar 18, 2026
  8. Derrick StoleeMar 23, 2026
  9. Patrick SteinhardtMar 19, 2026
  10. Derrick StoleeMar 23, 2026
  11. 4/5 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 17, 2026
  12. Junio C HamanoMar 17, 2026
  13. Derrick StoleeMar 18, 2026
  14. Patrick SteinhardtMar 19, 2026
  15. Patrick SteinhardtMar 19, 2026
  16. Patrick SteinhardtMar 19, 2026
  17. Derrick StoleeMar 23, 2026
  18. 5/5 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 17, 2026
  19. Junio C HamanoMar 17, 2026
  20. Derrick StoleeMar 18, 2026
  21. Derrick StoleeMar 23, 2026
  22. Junio C HamanoMar 17, 2026
  23. Patrick SteinhardtMar 19, 2026
  24. Derrick StoleeMar 19, 2026
  25. Patrick SteinhardtMar 20, 2026
  26. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  27. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 23, 2026
  28. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 23, 2026
  29. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  30. Patrick SteinhardtMar 24, 2026
  31. Derrick StoleeMar 26, 2026
  32. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 23, 2026
  33. Patrick SteinhardtMar 24, 2026
  34. Derrick StoleeMar 26, 2026
  35. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 23, 2026
  36. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 23, 2026
  37. Junio C HamanoMar 23, 2026
  38. Derrick StoleeMar 23, 2026
  39. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  40. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 26, 2026
  41. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 26, 2026
  42. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  43. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 26, 2026
  44. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 26, 2026
  45. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 26, 2026
  46. Patrick SteinhardtMar 27, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.