git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/6] backfill: work with prefix pathspecs

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Mar 23, 2026, 11:40 UTC
Message-ID
<f8f2c613260458e3c86109888ee1c51313095319.1774266019.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2070.v2.git.1774266019.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

The previous change allowed specifying revision arguments over the 'git backfill' command-line. This created the opportunity for restricting the initial commit set by filtering the revision walk through a pathspec. Other than filtering the commit set (and thereby the root trees), this did not restrict the path-walk implementation of 'git backfill' and did not restrict the blobs that were downloaded to only those matching the pathspec.

Update the path-walk API to accept certain kinds of pathspecs and to silently ignore anything too complex, for now. We will update this in the next change to properly restrict to even complex pathspecs.

The current behavior focuses on pathspecs that match paths exactly. This includes exact filenames, including directory names as prefixes. Pathspecs containing wildcards or magic are cleared so the path walk downloads all blobs, as before.

The reason for this restriction is to allow for a faster execution by pruning the path walk to only trees that could contribute towards one of those paths as a parent directory.

The test directory 'd/f/' (next to 'd/file*.txt') was prepared in a previous commit to exercise the subtlety in prefix matching.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 path-walk.c         | 40 ++++++++++++++++++++++++++++++++++++++++
 path.c              |  2 +-
 path.h              |  6 ++++++
 t/t5620-backfill.sh | 16 ++++++----------
 4 files changed, 53 insertions(+), 11 deletions(-)
diff --git a/path-walk.c b/path-walk.c
index 364e4cfa19..0d640e2f24 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -11,6 +11,7 @@
 #include "list-objects.h"
 #include "object.h"
 #include "oid-array.h"
+#include "path.h"
 #include "prio-queue.h"
 #include "repository.h"
 #include "revision.h"
@@ -206,6 +207,34 @@ static int add_tree_entries(struct path_walk_context *ctx,
 				 match != MATCHED)
 				continue;
 		}
+		if (ctx->revs->prune_data.nr) {
+			struct pathspec *pd = &ctx->revs->prune_data;
+			bool found = false;
+
+			/* remove '/' for these checks. */
+			path.buf[path.len - 1] = 0;
+
+			for (int i = 0; i < pd->nr; i++) {
+				struct pathspec_item *item = &pd->items[i];
+
+				/*
+				 * Continue if either is a directory prefix
+				 * of the other.
+				 */
+				if (dir_prefix(path.buf, item->match) ||
+				    dir_prefix(item->match, path.buf)) {
+					found = true;
+					break;
+				}
+			}
+
+			/* return '/' after these checks. */
+			path.buf[path.len - 1] = '/';
+
+			/* Skip paths that do not match the prefix. */
+			if (!found)
+				continue;
+		}
 
 		add_path_to_list(ctx, path.buf, type, &entry.oid,
 				 !(o->flags & UNINTERESTING));
@@ -481,6 +510,17 @@ int walk_objects_by_path(struct path_walk_info *info)
 	if (info->tags)
 		info->revs->tag_objects = 1;
 
+	if (ctx.revs->prune_data.nr) {
+		/*
+		 * Only exact prefix pathspecs are currently supported.
+		 * Clear any wildcard or magic pathspecs to avoid
+		 * incorrect prefix matching.
+		 */
+		if (ctx.revs->prune_data.has_wildcard ||
+		    ctx.revs->prune_data.magic)
+			clear_pathspec(&ctx.revs->prune_data);
+	}
+
 	/* Insert a single list for the root tree into the paths. */
 	CALLOC_ARRAY(root_tree_list, 1);
 	root_tree_list->type = OBJ_TREE;
diff --git a/path.c b/path.c
index d726537622..aebb10b2e9 100644
--- a/path.c
+++ b/path.c
@@ -57,7 +57,7 @@ static void strbuf_cleanup_path(struct strbuf *sb)
 		strbuf_remove(sb, 0, path - sb->buf);
 }
 
-static int dir_prefix(const char *buf, const char *dir)
+int dir_prefix(const char *buf, const char *dir)
 {
 	int len = strlen(dir);
 	return !strncmp(buf, dir, len) &&
diff --git a/path.h b/path.h
index 0ec95a0b07..829fafd7e9 100644
--- a/path.h
+++ b/path.h
@@ -114,6 +114,12 @@ const char *repo_submodule_path_replace(struct repository *repo,
 					const char *fmt, ...)
 	__attribute__((format (printf, 4, 5)));
 
+/*
+ * Given a directory name 'dir' (not ending with a trailing '/'),
+ * determine if 'buf' is equal to 'dir' or has prefix 'dir'+'/'.
+ */
+int dir_prefix(const char *buf, const char *dir);
+
 void report_linked_checkout_garbage(struct repository *r);
 
 /*
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index db66d8b614..52f6484ca1 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -273,13 +273,11 @@ test_expect_success 'backfill with prefix pathspec' '
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
 	test_line_count = 48 missing &&
 
-	# TODO: The pathspec should limit the downloaded blobs to
-	# only those matching the prefix "d/f", but currently all
-	# blobs are downloaded.
-	git -C backfill-path backfill HEAD -- d/f &&
+	git -C backfill-path backfill HEAD -- d/f 2>err &&
+	test_must_be_empty err &&
 
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
-	test_line_count = 0 missing
+	test_line_count = 40 missing
 '
 
 test_expect_success 'backfill with multiple pathspecs' '
@@ -292,13 +290,11 @@ test_expect_success 'backfill with multiple pathspecs' '
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
 	test_line_count = 48 missing &&
 
-	# TODO: The pathspecs should limit the downloaded blobs to
-	# only those matching "d/f" or "a", but currently all blobs
-	# are downloaded.
-	git -C backfill-path backfill HEAD -- d/f a &&
+	git -C backfill-path backfill HEAD -- d/f a 2>err &&
+	test_must_be_empty err &&
 
 	git -C backfill-path rev-list --quiet --objects --missing=print HEAD >missing &&
-	test_line_count = 0 missing
+	test_line_count = 16 missing
 '
 
 test_expect_success 'backfill with wildcard pathspec' '
-- 
gitgitgadget
Previous: Derrick StoleeNext: Patrick Steinhardt
Message 32 of 46 in “backfill: accept revision arguments”
  1. 0/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  2. 1/5 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 17, 2026
  3. Junio C HamanoMar 17, 2026
  4. 2/5 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 17, 2026
  5. 3/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  6. Junio C HamanoMar 17, 2026
  7. Kristoffer HaugsbakkMar 18, 2026
  8. Derrick StoleeMar 23, 2026
  9. Patrick SteinhardtMar 19, 2026
  10. Derrick StoleeMar 23, 2026
  11. 4/5 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 17, 2026
  12. Junio C HamanoMar 17, 2026
  13. Derrick StoleeMar 18, 2026
  14. Patrick SteinhardtMar 19, 2026
  15. Patrick SteinhardtMar 19, 2026
  16. Patrick SteinhardtMar 19, 2026
  17. Derrick StoleeMar 23, 2026
  18. 5/5 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 17, 2026
  19. Junio C HamanoMar 17, 2026
  20. Derrick StoleeMar 18, 2026
  21. Derrick StoleeMar 23, 2026
  22. Junio C HamanoMar 17, 2026
  23. Patrick SteinhardtMar 19, 2026
  24. Derrick StoleeMar 19, 2026
  25. Patrick SteinhardtMar 20, 2026
  26. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  27. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 23, 2026
  28. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 23, 2026
  29. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  30. Patrick SteinhardtMar 24, 2026
  31. Derrick StoleeMar 26, 2026
  32. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 23, 2026
  33. Patrick SteinhardtMar 24, 2026
  34. Derrick StoleeMar 26, 2026
  35. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 23, 2026
  36. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 23, 2026
  37. Junio C HamanoMar 23, 2026
  38. Derrick StoleeMar 23, 2026
  39. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  40. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 26, 2026
  41. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 26, 2026
  42. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  43. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 26, 2026
  44. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 26, 2026
  45. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 26, 2026
  46. Patrick SteinhardtMar 27, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.