git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 04/13] path-walk: always emit directly-requested objects

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 22, 2026, 18:24 UTC
Message-ID
<91845bcef0692bf15b41af1dbf01dcc02baf42ee.1779474277.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2101.v5.git.1779474277.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>

We are preparing to integrate the path-walk API with some --filter options in 'git pack-objects', but there is a subtle issue that is revealed when those are put together and the test suite is run with GIT_TEST_PACK_PATH_WALK=1.

When a filter reduces the set of requested objects, this results in filtering out directly-requested objects, such as in the download of needed blobs in a blobless partial clone.

The root cause is that the scan of pending objects in the path-walk API respects the filters set in the path_walk_info instead of overriding them for pending objects.

We can tell that a path is part of the directly-referenced objects if its path name starts with '/' (other paths, including root trees never have this starting character). Create a path_is_for_direct_objects() to make this meaning clear, especially as we add more references in the future as we integrate the path-walk API with partial clone filter options.

Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
 Documentation/technical/api-path-walk.adoc |  7 ++++
 path-walk.c                                | 42 ++++++++++++++--------
 path-walk.h                                |  5 +++
 3 files changed, 39 insertions(+), 15 deletions(-)
diff --git a/Documentation/technical/api-path-walk.adoc b/Documentation/technical/api-path-walk.adoc
index a67de1b143..6e17b13d61 100644
--- a/Documentation/technical/api-path-walk.adoc
+++ b/Documentation/technical/api-path-walk.adoc
@@ -48,6 +48,13 @@ commits.
 	applications could disable some options to make it simpler to walk
 	the objects or to have fewer calls to `path_fn`.
 +
+Note that objects directly requested as pending objects (such as targets
+of lightweight tags or other ref tips) are always emitted to `path_fn`,
+even when the corresponding type flag is disabled. Only objects
+discovered during the tree walk are subject to these type filters. This
+ensures that objects specifically requested through the revision input
+are never silently dropped.
++
 While it is possible to walk only commits in this way, consumers would be
 better off using the revision walk API instead.
 
diff --git a/path-walk.c b/path-walk.c
index 6e426af433..05bfc1c114 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -248,6 +248,17 @@ static int add_tree_entries(struct path_walk_context *ctx,
 	return 0;
 }
 
+/*
+ * Paths starting with '/' (e.g., "/tags", "/tagged-blobs") hold objects that
+ * were directly requested by 'pending' objects rather than discovered during
+ * tree traversal.
+ */
+static int path_is_for_direct_objects(const char *path)
+{
+	ASSERT(path);
+	return path[0] == '/';
+}
+
 /*
  * For each path in paths_to_explore, walk the trees another level
  * and add any found blobs to the batch (but only if they exist and
@@ -306,14 +317,19 @@ static int walk_path(struct path_walk_context *ctx,
 
 	if (list->type == OBJ_BLOB &&
 	    ctx->revs->prune_data.nr &&
+	    !path_is_for_direct_objects(path) &&
 	    !match_pathspec(ctx->repo->index, &ctx->revs->prune_data,
 			   path, strlen(path), 0,
 			   NULL, 0))
 		return 0;
 
-	/* Evaluate function pointer on this data, if requested. */
-	if ((list->type == OBJ_TREE && ctx->info->trees) ||
-	    (list->type == OBJ_BLOB && ctx->info->blobs) ||
+	/*
+	 * Evaluate function pointer on this data, if requested.
+	 * Ignore object type filters for tagged objects (path starts
+	 * with `/`).
+	 */
+	if ((list->type == OBJ_TREE && (ctx->info->trees || path_is_for_direct_objects(path))) ||
+	    (list->type == OBJ_BLOB && (ctx->info->blobs || path_is_for_direct_objects(path))) ||
 	    (list->type == OBJ_TAG && ctx->info->tags))
 		ret = ctx->info->path_fn(path, &list->oids, list->type,
 					ctx->info->path_fn_data);
@@ -374,10 +390,8 @@ static int setup_pending_objects(struct path_walk_info *info,
 
 	if (info->tags)
 		CALLOC_ARRAY(tags, 1);
-	if (info->blobs)
-		CALLOC_ARRAY(tagged_blobs, 1);
-	if (info->trees)
-		root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
+	CALLOC_ARRAY(tagged_blobs, 1);
+	root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
 
 	/*
 	 * Pending objects include:
@@ -421,8 +435,6 @@ static int setup_pending_objects(struct path_walk_info *info,
 
 		switch (obj->type) {
 		case OBJ_TREE:
-			if (!info->trees)
-				continue;
 			if (pending->path) {
 				char *path = *pending->path ? xstrfmt("%s/", pending->path)
 							    : xstrdup("");
@@ -435,8 +447,6 @@ static int setup_pending_objects(struct path_walk_info *info,
 			break;
 
 		case OBJ_BLOB:
-			if (!info->blobs)
-				continue;
 			if (pending->path)
 				add_path_to_list(ctx, pending->path, OBJ_BLOB, &obj->oid, 1);
 			else
@@ -532,15 +542,17 @@ int walk_objects_by_path(struct path_walk_info *info)
 	push_to_stack(&ctx, root_path);
 
 	/*
-	 * Set these values before preparing the walk to catch
-	 * lightweight tags pointing to non-commits and indexed objects.
+	 * Ensure that prepare_revision_walk() keeps all pending objects
+	 * even through an object type filter.
 	 */
-	info->revs->blob_objects = info->blobs;
-	info->revs->tree_objects = info->trees;
+	info->revs->blob_objects = info->revs->tree_objects = 1;
 
 	if (prepare_revision_walk(info->revs))
 		die(_("failed to setup revision walk"));
 
+	info->revs->blob_objects = info->blobs;
+	info->revs->tree_objects = info->trees;
+
 	/*
 	 * Walk trees to mark them as UNINTERESTING.
 	 * This is particularly important when 'edge_aggressive' is set.
diff --git a/path-walk.h b/path-walk.h
index 5ef5a8440e..657eeda8ec 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -36,6 +36,11 @@ struct path_walk_info {
 	/**
 	 * Initialize which object types the path_fn should be called on. This
 	 * could also limit the walk to skip blobs if not set.
+	 *
+	 * Note: even when 'blobs' or 'trees' is disabled, objects that are
+	 * directly requested as pending objects will still be emitted to
+	 * path_fn. Only objects discovered during the tree walk are filtered by
+	 * these flags.
 	 */
 	int commits;
 	int trees;
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Derrick Stolee via GitGitGadget
Message 85 of 96 in “pack-objects: integrate --path-walk and some --filter options”
  1. 0/7 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  2. 1/7 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  3. Junio C HamanoMay 4, 2026
  4. Derrick StoleeMay 4, 2026
  5. 2/7 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 2, 2026
  6. 3/7 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 2, 2026
  7. 4/7 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  8. Junio C HamanoMay 3, 2026
  9. Derrick StoleeMay 4, 2026
  10. 5/7 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 2, 2026
  11. 6/7 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 2, 2026
  12. 7/7 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  13. 00/10 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  14. 01/10 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  15. 02/10 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 4, 2026
  16. 03/10 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 4, 2026
  17. 04/10 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  18. 05/10 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 4, 2026
  19. 06/10 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 4, 2026
  20. 07/10 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  21. 08/10 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 4, 2026
  22. 09/10 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 4, 2026
  23. 10/10 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 4, 2026
  24. Derrick StoleeMay 5, 2026
  25. Taylor BlauMay 5, 2026
  26. Derrick StoleeMay 5, 2026
  27. Taylor BlauMay 5, 2026
  28. Derrick StoleeMay 7, 2026
  29. Junio C HamanoMay 11, 2026
  30. Derrick StoleeMay 11, 2026
  31. 00/12 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  32. 01/12 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 11, 2026
  33. Taylor BlauMay 12, 2026
  34. 02/12 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  35. Taylor BlauMay 12, 2026
  36. 03/12 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 11, 2026
  37. Taylor BlauMay 12, 2026
  38. Derrick StoleeMay 13, 2026
  39. 04/12 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 11, 2026
  40. Taylor BlauMay 12, 2026
  41. Derrick StoleeMay 13, 2026
  42. 05/12 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 11, 2026
  43. Taylor BlauMay 11, 2026
  44. Derrick StoleeMay 11, 2026
  45. 06/12 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  46. Taylor BlauMay 12, 2026
  47. 07/12 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 11, 2026
  48. Taylor BlauMay 12, 2026
  49. Derrick StoleeMay 13, 2026
  50. 08/12 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 11, 2026
  51. 09/12 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  52. 10/12 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 11, 2026
  53. Taylor BlauMay 12, 2026
  54. Derrick StoleeMay 13, 2026
  55. 11/12 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 11, 2026
  56. 12/12 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 11, 2026
  57. Taylor BlauMay 12, 2026
  58. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  59. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 13, 2026
  60. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  61. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 13, 2026
  62. Taylor BlauMay 19, 2026
  63. Derrick StoleeMay 21, 2026
  64. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 13, 2026
  65. Taylor BlauMay 19, 2026
  66. Derrick StoleeMay 21, 2026
  67. Taylor BlauMay 22, 2026
  68. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 13, 2026
  69. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  70. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 13, 2026
  71. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 13, 2026
  72. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  73. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 13, 2026
  74. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 13, 2026
  75. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 13, 2026
  76. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 13, 2026
  77. Taylor BlauMay 19, 2026
  78. Derrick StoleeMay 21, 2026
  79. Taylor BlauMay 22, 2026
  80. Derrick StoleeMay 22, 2026
  81. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  82. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 22, 2026
  83. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  84. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 22, 2026
  85. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 22, 2026
  86. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 22, 2026
  87. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  88. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 22, 2026
  89. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 22, 2026
  90. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  91. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 22, 2026
  92. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 22, 2026
  93. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 22, 2026
  94. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 22, 2026
  95. Taylor BlauMay 22, 2026
  96. Junio C HamanoMay 24, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.