git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 00/13] pack-objects: integrate --path-walk and some --filter options

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
May 13, 2026, 21:18 UTC
Message-ID
<pull.2101.v4.git.1778707135.gitgitgadget@gmail.com>
In-Reply-To
<pull.2101.v3.git.1778523189.gitgitgadget@gmail.com>
NOTE: This series is based on en/backfill-fixes-and-edges.

The 'git pack-objects' command has a '--path-walk' option that uses the path-walk API instead of a typical revision walk to group objects into chunks by path name instead of relying solely on name-hashes to group similar files together. (It also does a second compression pass looking for better deltas after the first pass that is focused within chunks per path.)

The '--path-walk' feature was not previously integrated with the '--filter' feature, so a warning would appear and disable the path-walk API when a filter is given. This patch series integrates these together in the following ways:

 * --filter=blob:none updates the path-walk API options to skip blobs.
 * --filter=blob:limit=<size> adds a scan to a list of blob objects to
   remove objects that are too large.
 * --filter=sparse:<oid> adds a scan to the chunks to validate that the
   paths match the sparse-checkout patterns.

In particular, this last check is significantly faster than the previous algorithm because it can check all objects at a given path simultaneously instead of checking all sparse-checkout patterns for each discovered blob object.

A subtlety must be added here, in that we must change how we mark an object as "seen" during the path-walk. We may need to add an object to multiple paths and only mark it as "seen" if it indeed matched the sparse-checkout patterns as the path is accepted for emitting to the callback. This adds a new filter that the "seen" objects must also be removed from later chunks to avoid sending the same object as grouped to multiple chunks.

There's also a subtle detail here in that the path-walk API also prunes tree paths based on cone-mode sparse-checkouts, to enable 'git backfill --sparse' operating quickly for small sparse-checkout scopes. But the --filter=sparse:<oid> feature doesn't prune trees!

As a future step, I do plan to recommend that we add a treesparse:<oid> setting that does allow us to trim the tree set by cone-mode sparse patterns. At the time that partial clone filters were being created, cone mode sparse-checkout didn't exist and neither did the sparse index. Those features together make a smaller tree set possible, assuming the user never needs to change their scope. This would be a significant change so it is not implemented here, though the git pack-objects integration would be quick after this series completes.

Neither the sparse:<oid> or hypothetical treesparse:<oid> options are or should necessarily be supported by Git servers. It's too expensive to compute dynamically and it doesn't work well with reachability bitmaps. What becomes possible with this change is that it becomes reasonably fast to construct bundles with these filters that can bootstrap a working environment with the full history of all files within a given sparse-checkout scope.

Performance Results ===================

Since the '--path-walk' option is ignored in today's Git version when a '--filter' is added, the performance matches the behavior without '--path-walk'. For the tables below, you can compare the rows against each other (time and then packfile size) for the mode without and then with '--path-walk' as a representation of "before" and "after". (These tables are repeated in the commit messages as new implementations improve specific rows.)

I chose a number of open source repositories of various sizes and shapes:

git/git =======

Test HEAD ------------------------------------------------------------------- 5315.2: repack (no filter) 27.73 5315.3: repack size (no filter) 250.6M 5315.4: repack (no filter, --path-walk) 35.19 5315.5: repack size (no filter, --path-walk) 220.1M 5315.6: repack (blob:none) 13.42 5315.7: repack size (blob:none) 137.6M 5315.8: repack (blob:none, --path-walk) 20.98 5315.9: repack size (blob:none, --path-walk) 115.2M 5315.10: repack (sparse:oid) 72.53 5315.11: repack size (sparse:oid) 187.5M 5315.12: repack (sparse:oid, --path-walk) 29.00 5315.13: repack size (sparse:oid, --path-walk) 161.0M

nodejs/node ===========

Test HEAD -------------------------------------------------------------------- 5315.2: repack (no filter) 75.53 5315.3: repack size (no filter) 0.9G 5315.4: repack (no filter, --path-walk) 80.54 5315.5: repack size (no filter, --path-walk) 885.7M 5315.6: repack (blob:none) 12.65 5315.7: repack size (blob:none) 148.6M 5315.8: repack (blob:none, --path-walk) 17.60 5315.9: repack size (blob:none, --path-walk) 134.6M 5315.10: repack (sparse:oid) 518.84 5315.11: repack size (sparse:oid) 153.4M 5315.12: repack (sparse:oid, --path-walk) 27.99 5315.13: repack size (sparse:oid, --path-walk) 139.4M

microsoft/fluentui ==================

Test HEAD -------------------------------------------------------------------- 5315.2: repack (no filter) 146.77 5315.3: repack size (no filter) 562.1M 5315.4: repack (no filter, --path-walk) 72.82 5315.5: repack size (no filter, --path-walk) 172.6M 5315.6: repack (blob:none) 4.84 5315.7: repack size (blob:none) 62.7M 5315.8: repack (blob:none, --path-walk) 5.19 5315.9: repack size (blob:none, --path-walk) 59.9M 5315.10: repack (sparse:oid) 59.95 5315.11: repack size (sparse:oid) 85.6M 5315.12: repack (sparse:oid, --path-walk) 15.16 5315.13: repack size (sparse:oid, --path-walk) 72.5M

microsoftdocs/azure-devops-docs ===============================

Test HEAD --------------------------------------------------------------------- 5315.2: repack (no filter) 4.41 5315.3: repack size (no filter) 1.6G 5315.4: repack (no filter, --path-walk) 6.00 5315.5: repack size (no filter, --path-walk) 1.6G 5315.6: repack (blob:none) 1.35 5315.7: repack size (blob:none) 60.0M 5315.8: repack (blob:none, --path-walk) 1.23 5315.9: repack size (blob:none, --path-walk) 60.0M 5315.10: repack (sparse:oid) 138.24 5315.11: repack size (sparse:oid) 84.4M 5315.12: repack (sparse:oid, --path-walk) 1.86 5315.13: repack size (sparse:oid, --path-walk) 84.4M

Performance conclusions =======================

As seen in earlier series around the '--path-walk' feature, the space savings can be valuable but is not always guaranteed. When the space savings doesn't happen, then the time spent is generally slower because of the two-pass mechanism. The microsoftdocs/azure-devops-docs repo demonstrates this case quite clearly.

However, even in these cases the 'sparse:<oid>' filters are much faster because of the ability to check an entire set of objects against the sparse-checkout patterns only once.

Thanks, -Stolee

UPDATES IN V2 =============

 * Rebased onto en/backfill-fixes-and-edges to properly integrate with the
   incompatible rev-list options logic in that series.
 * Updated documentation as behavior changes. Credit to Taylor Blau for
   presenting these suggestions in his RFC [2].
 * Added three patches of Taylor's to extend more filter options.

UPDATES IN V3 =============

Upon realizing that the tests were not passing with GIT_TEST_PACK_PATH_WALK=1, I spent a lot of time reworking each patch to pass all tests with that variable enabled. This led to a lot of meaningful changes:

 * A new patch updates t5620-backfill.sh because they are currently failing
   due to a check for quiet stderr checks. These changes are reverted later
   when the filters are integrated so the warning stops being written.
 * I move the logic for the path-walk API emitting "directly requested"
   objects (non-commits in the 'pending' list). This is substantial enough
   to be its own patch.
 * The filtering logic is pulled entirely within the path-walk API instead
   of needing integration within builtin/pack-objects.c.
 * The tree:0 filter had a lot to be desired when fetching missing objects,
   so is substantially updated.
 * The object-type filter requires a change to the typical direct-request
   behavior, including a new 'strict_types' member that prevents ever
   allowing objects against type.
 * The combine filter needed better logic around multiple blob size limits,
   to take the smaller of the two.
 * The t6601-path-walk.sh script has many test updates to better reflect the
   new behavior, as required by the other partial clone tests under
   GIT_TEST_PACK_PATH_WALK=1.
 * Doc updates for 'form' to 'forms' when multiple forms are supported.

I've also updated Taylor's bitmap-related patches into three commits on top of this series (see [3]).

UPDATES IN V4 =============

Thanks, Taylor for the careful review.
 * Several typos are fixed.
 * The performance test is corrected for issues around piping Git commands
   and made more robust to the existence of submodules.
 * BIG: The tree:0 patch is significantly updated in this version. Taylor
   correctly smelled a problem with the new logic to emit the /tagged-trees
   object set, and that signaled that those trees were previously never
   emitted. I update the test to demonstrate that changing the data shape
   (including tagged trees that are otherwise-unreachable) doesn't change
   the test behavior, signaling a bug. The behavior change details all the
   complexities of visiting only directly-requested trees under a tree:0
   filter and recursing on all trees in other cases.

P.S. I've CC'd the folks who were on the original path-walk feature thread [1]

[1] https://lore.kernel.org/git/pull.1819.git.1741571455.gitgitgadget@gmail.com/

[2] https://lore.kernel.org/git/cover.1777853408.git.me@ttaylorr.com/

[3] https://github.com/derrickstolee/git/compare/path-walk-filters...derrickstolee:git:path-walk-bitmaps

Derrick Stolee (10):
  t5620: make test work with path-walk var
  pack-objects: pass --objects with --path-walk
  t/perf: add pack-objects filter and path-walk benchmark
  path-walk: always emit directly-requested objects
  path-walk: support blobless filter
  backfill: die on incompatible filter options
  path-walk: support blob size limit filter
  path-walk: add pl_sparse_trees to control tree pruning
  pack-objects: support sparse:oid filter with path-walk
  t6601: tag otherwise-unreachable trees
Taylor Blau (3):
  path-walk: support `tree:0` filter
  path-walk: support `object:type` filter
  path-walk: support `combine` filter
 Documentation/git-backfill.adoc            |   4 +
 Documentation/git-pack-objects.adoc        |   8 +-
 Documentation/technical/api-path-walk.adoc |   7 +
 builtin/backfill.c                         |   8 +-
 builtin/pack-objects.c                     |  23 +-
 path-walk.c                                | 270 ++++++++--
 path-walk.h                                |  31 ++
 t/helper/test-path-walk.c                  |  17 +-
 t/perf/p5315-pack-objects-filter.sh        | 131 +++++
 t/t5317-pack-objects-filter-objects.sh     | 125 +++++
 t/t5620-backfill.sh                        |   8 +
 t/t6601-path-walk.sh                       | 572 +++++++++++++++++++--
 12 files changed, 1126 insertions(+), 78 deletions(-)
 create mode 100755 t/perf/p5315-pack-objects-filter.sh
base-commit: a1ad4a0fca14cdeb55ab9fb065551b15cafa8a4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2101%2Fderrickstolee%2Fpath-walk-filters-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2101/derrickstolee/path-walk-filters-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/2101
Range-diff vs v3:
  1:  0840110116 =  1:  0840110116 t5620: make test work with path-walk var
  2:  5cc6383380 !  2:  d7c87545f3 pack-objects: pass --objects with --path-walk
     @@ Commit message
          pack-objects: pass --objects with --path-walk
      
          When 'git pack-objects' has the --path-walk option enabled, it uses a
     -    different set of revision walk parameters than normal. For once,
     -    --objects was previously assumed by the path-walk API and was not needed
     -    to be added. We also needed --boundary to allow discovering
     -    UNINTERESTING objects to use as delta bases.
     +    different set of revision walk parameters than normal. For one,
     +    --objects was previously assumed by the path-walk API and could be
     +    omitted. We also needed --boundary to allow discovering UNINTERESTING
     +    objects to use as delta bases.
      
          We will be updating the path-walk API soon to work with some filter
          options. However, the revision machinery will trigger a fatal error:
  3:  77329cf8f4 !  3:  fb8a0f9c43 t/perf: add pack-objects filter and path-walk benchmark
     @@ t/perf/p5315-pack-objects-filter.sh (new)
      +	# directories at evenly-spaced positions so the choice is stable
      +	# and scales to repos of any shape.
      +
     -+	git ls-tree -d --name-only HEAD >top-dirs &&
     ++	git ls-tree -d HEAD >top-entries &&
     ++	grep "^040000" top-entries |
     ++		awk "{print \$4;}" >top-dirs &&
      +	top_nr=$(wc -l <top-dirs) &&
      +
      +	>depth2-dirs &&
      +	while read tdir
      +	do
     -+		git ls-tree -d --name-only "HEAD:$tdir" 2>/dev/null |
     -+		sed "s|^|$tdir/|" >>depth2-dirs || return 1
     -+	done <top-dirs &&
     ++		git ls-tree -d --name-only "HEAD:$tdir" 2>/dev/null || return 1
     ++	done <top-dirs >depth2-dirs.raw &&
     ++	sed "s|^|$tdir/|" <depth2-dirs.raw >depth2-dirs &&
      +
      +	d2_nr=$(wc -l <depth2-dirs) &&
      +
  4:  50933cccf5 !  4:  e77c8a6bbc path-walk: always emit directly-requested objects
     @@ path-walk.c: static int add_tree_entries(struct path_walk_context *ctx,
      + */
      +static int path_is_for_direct_objects(const char *path)
      +{
     ++	ASSERT(path);
      +	return path[0] == '/';
      +}
      +
  5:  b2deb7f819 !  5:  f4904f81e0 path-walk: support blobless filter
     @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1
      -`--use-bitmap-index` option will be ignored in the presence of
      -`--path-walk.`
      +Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     -+ignored in the presence of `--path-walk`. Whe `--path-walk` option
     ++ignored in the presence of `--path-walk`. The `--path-walk` option
      +supports the `--filter=<spec>` form `blob:none`.
       
       
  6:  da191e29e6 =  6:  f37467e46f backfill: die on incompatible filter options
  7:  a1ab70405d !  7:  133c1b156c path-walk: support blob size limit filter
     @@ Documentation/git-pack-objects.adoc
      @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
       +
       Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     - ignored in the presence of `--path-walk`. Whe `--path-walk` option
     + ignored in the presence of `--path-walk`. The `--path-walk` option
      -supports the `--filter=<spec>` form `blob:none`.
      +supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
       
  8:  2360a5be81 =  8:  0f517be8e3 path-walk: add pl_sparse_trees to control tree pruning
  9:  d9f5a98830 !  9:  b4dc09ab69 pack-objects: support sparse:oid filter with path-walk
     @@ Documentation/git-pack-objects.adoc
      @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
       +
       Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     - ignored in the presence of `--path-walk`. Whe `--path-walk` option
     + ignored in the presence of `--path-walk`. The `--path-walk` option
      -supports the `--filter=<spec>` forms `blob:none` and `blob:limit=<n>`.
      +supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
      +`sparse:<oid>`.
  -:  ---------- > 10:  0b1eed0790 t6601: tag otherwise-unreachable trees
 10:  c9efff0ea8 ! 11:  b23244c4c2 path-walk: support `tree:0` filter
     @@ Commit message
          at all under `tree:0`, which matches the intended behavior of the
          filter.
      
     +    However, this requires first fixing some issues with how the path-walk
     +    API handles directly-requested trees _and_ trees requested through
     +    lightweight tags. These changes create substantial updates to
     +    t6601-path-walk.sh, which the previous change highlighted as a problem
     +    by tagging otherwise-unreachable trees and having them not appear in the
     +    output.
     +
          Non-zero tree-depth filters are not supported. Those depend on the depth
          at which a tree is visited, which is a path-walk concept the filter
          machinery doesn't currently share with the path-walk API. Reject them in
     @@ Documentation/git-pack-objects.adoc
      @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
       +
       Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     - ignored in the presence of `--path-walk`. Whe `--path-walk` option
     + ignored in the presence of `--path-walk`. The `--path-walk` option
      -supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and
      -`sparse:<oid>`.
      +supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
     @@ path-walk.c: static int walk_path(struct path_walk_context *ctx,
       
      -	/* Expand data for children. */
      -	if (list->type == OBJ_TREE) {
     -+	/* Expand data for children, unless this is a direct-object path. */
     -+	if (list->type == OBJ_TREE && !path_is_for_direct_objects(path)) {
     ++	/*
     ++	 * Expand tree children, except when the set is directly requested
     ++	 * _and_ we are otherwise filtering out trees.
     ++	 */
     ++	if (list->type == OBJ_TREE &&
     ++	    (!path_is_for_direct_objects(path) || ctx->info->trees)) {
     ++		/* Use root path if expanding from tagged/direct trees. */
     ++		const char *expand_path = !strcmp(path, "/tagged-trees")
     ++					  ? root_path : path;
       		for (size_t i = 0; i < list->oids.nr; i++) {
       			ret |= add_tree_entries(ctx,
     - 					    path,
     +-					    path,
     ++					    expand_path,
     + 					    &list->oids.oid[i]);
     + 		}
     + 	}
      @@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
       {
       	struct type_and_oid_list *tags = NULL;
       	struct type_and_oid_list *tagged_blobs = NULL;
     +-	struct type_and_oid_list *root_tree_list = NULL;
      +	struct type_and_oid_list *tagged_trees = NULL;
     - 	struct type_and_oid_list *root_tree_list = NULL;
       
       	if (info->tags)
       		CALLOC_ARRAY(tags, 1);
       	CALLOC_ARRAY(tagged_blobs, 1);
     -+	if (!info->trees)
     -+		CALLOC_ARRAY(tagged_trees, 1);
     - 	root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
     +-	root_tree_list = strmap_get(&ctx->paths_to_lists, root_path);
     ++	CALLOC_ARRAY(tagged_trees, 1);
       
       	/*
     + 	 * Pending objects include:
      @@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
       
       		switch (obj->type) {
       		case OBJ_TREE:
      -			if (pending->path) {
     -+			if (tagged_trees) {
     -+				/*
     -+				 * Trees are disabled but pending trees
     -+				 * should still be emitted. Collect them
     -+				 * into a "/tagged-trees" list that
     -+				 * bypasses the object type filter.
     -+				 */
     -+				oid_array_append(&tagged_trees->oids, &obj->oid);
     -+			} else if (pending->path) {
     - 				char *path = *pending->path ? xstrfmt("%s/", pending->path)
     - 							    : xstrdup("");
     +-				char *path = *pending->path ? xstrfmt("%s/", pending->path)
     +-							    : xstrdup("");
     ++			if (pending->path && *pending->path) {
     ++				char *path = xstrfmt("%s/", pending->path);
       				add_path_to_list(ctx, path, OBJ_TREE, &obj->oid, 1);
     + 				free(path);
     ++			} else if (!pending->path || !info->trees) {
     ++				oid_array_append(&tagged_trees->oids, &obj->oid);
     + 			} else {
     +-				/* assume a root tree, such as a lightweight tag. */
     +-				oid_array_append(&root_tree_list->oids, &obj->oid);
     ++				add_path_to_list(ctx, root_path, OBJ_TREE,
     ++						 &obj->oid, 1);
     + 			}
     + 			break;
     + 
      @@ path-walk.c: static int setup_pending_objects(struct path_walk_info *info,
       			free(tagged_blobs);
       		}
     @@ path-walk.c: static int prepare_filters(struct path_walk_info *info,
       			struct object_id sparse_oid;
      
       ## t/t6601-path-walk.sh ##
     +@@ t/t6601-path-walk.sh: test_expect_success 'all' '
     + 	3:tree::$(git rev-parse base^{tree})
     + 	3:tree::$(git rev-parse base~1^{tree})
     + 	3:tree::$(git rev-parse base~2^{tree})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag^{})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag2^{})
     + 	4:blob:a:$(git rev-parse base~2:a)
     +-	5:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
     +-	6:tree:a/:$(git rev-parse base:a)
     +-	7:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     +-	8:blob:child/file:$(git rev-parse refs/tags/tree-tag:child/file)
     +-	9:tree:left/:$(git rev-parse base:left)
     +-	9:tree:left/:$(git rev-parse base~2:left)
     +-	10:blob:left/b:$(git rev-parse base~2:left/b)
     +-	10:blob:left/b:$(git rev-parse base:left/b)
     +-	11:tree:right/:$(git rev-parse topic:right)
     +-	11:tree:right/:$(git rev-parse base~1:right)
     +-	11:tree:right/:$(git rev-parse base~2:right)
     +-	12:blob:right/c:$(git rev-parse base~2:right/c)
     +-	12:blob:right/c:$(git rev-parse topic:right/c)
     +-	13:blob:right/d:$(git rev-parse base~1:right/d)
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	6:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
     ++	7:tree:a/:$(git rev-parse base:a)
     ++	8:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	9:blob:child/file:$(git rev-parse refs/tags/tree-tag:child/file)
     ++	10:tree:left/:$(git rev-parse base:left)
     ++	10:tree:left/:$(git rev-parse base~2:left)
     ++	11:blob:left/b:$(git rev-parse base~2:left/b)
     ++	11:blob:left/b:$(git rev-parse base:left/b)
     ++	12:tree:right/:$(git rev-parse topic:right)
     ++	12:tree:right/:$(git rev-parse base~1:right)
     ++	12:tree:right/:$(git rev-parse base~2:right)
     ++	13:blob:right/c:$(git rev-parse base~2:right/c)
     ++	13:blob:right/c:$(git rev-parse topic:right/c)
     ++	14:blob:right/d:$(git rev-parse base~1:right/d)
     + 	blobs:10
     + 	commits:4
     + 	tags:7
     +@@ t/t6601-path-walk.sh: test_expect_success 'all, blob:none filter' '
     + 	3:tree::$(git rev-parse base^{tree})
     + 	3:tree::$(git rev-parse base~1^{tree})
     + 	3:tree::$(git rev-parse base~2^{tree})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag^{})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag2^{})
     +-	4:tree:a/:$(git rev-parse base:a)
     +-	5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     +-	6:tree:left/:$(git rev-parse base:left)
     +-	6:tree:left/:$(git rev-parse base~2:left)
     +-	7:tree:right/:$(git rev-parse topic:right)
     +-	7:tree:right/:$(git rev-parse base~1:right)
     +-	7:tree:right/:$(git rev-parse base~2:right)
     ++	4:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	4:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	5:tree:a/:$(git rev-parse base:a)
     ++	6:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	7:tree:left/:$(git rev-parse base:left)
     ++	7:tree:left/:$(git rev-parse base~2:left)
     ++	8:tree:right/:$(git rev-parse topic:right)
     ++	8:tree:right/:$(git rev-parse base~1:right)
     ++	8:tree:right/:$(git rev-parse base~2:right)
     + 	blobs:2
     + 	commits:4
     + 	tags:7
     +@@ t/t6601-path-walk.sh: test_expect_success 'all, blob:limit=0 filter' '
     + 	3:tree::$(git rev-parse base^{tree})
     + 	3:tree::$(git rev-parse base~1^{tree})
     + 	3:tree::$(git rev-parse base~2^{tree})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag^{})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag2^{})
     +-	4:tree:a/:$(git rev-parse base:a)
     +-	5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     +-	6:tree:left/:$(git rev-parse base:left)
     +-	6:tree:left/:$(git rev-parse base~2:left)
     +-	7:tree:right/:$(git rev-parse topic:right)
     +-	7:tree:right/:$(git rev-parse base~1:right)
     +-	7:tree:right/:$(git rev-parse base~2:right)
     ++	4:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	4:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	5:tree:a/:$(git rev-parse base:a)
     ++	6:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	7:tree:left/:$(git rev-parse base:left)
     ++	7:tree:left/:$(git rev-parse base~2:left)
     ++	8:tree:right/:$(git rev-parse topic:right)
     ++	8:tree:right/:$(git rev-parse base~1:right)
     ++	8:tree:right/:$(git rev-parse base~2:right)
     + 	blobs:2
     + 	commits:4
     + 	tags:7
     +@@ t/t6601-path-walk.sh: test_expect_success 'all, blob:limit=3 filter' '
     + 	3:tree::$(git rev-parse base^{tree})
     + 	3:tree::$(git rev-parse base~1^{tree})
     + 	3:tree::$(git rev-parse base~2^{tree})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag^{})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag2^{})
     + 	4:blob:a:$(git rev-parse base~2:a)
     +-	5:tree:a/:$(git rev-parse base:a)
     +-	6:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     +-	7:tree:left/:$(git rev-parse base:left)
     +-	7:tree:left/:$(git rev-parse base~2:left)
     +-	8:blob:left/b:$(git rev-parse base~2:left/b)
     +-	9:tree:right/:$(git rev-parse topic:right)
     +-	9:tree:right/:$(git rev-parse base~1:right)
     +-	9:tree:right/:$(git rev-parse base~2:right)
     +-	10:blob:right/c:$(git rev-parse base~2:right/c)
     +-	11:blob:right/d:$(git rev-parse base~1:right/d)
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	6:tree:a/:$(git rev-parse base:a)
     ++	7:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	8:tree:left/:$(git rev-parse base:left)
     ++	8:tree:left/:$(git rev-parse base~2:left)
     ++	9:blob:left/b:$(git rev-parse base~2:left/b)
     ++	10:tree:right/:$(git rev-parse topic:right)
     ++	10:tree:right/:$(git rev-parse base~1:right)
     ++	10:tree:right/:$(git rev-parse base~2:right)
     ++	11:blob:right/c:$(git rev-parse base~2:right/c)
     ++	12:blob:right/d:$(git rev-parse base~1:right/d)
     + 	blobs:6
     + 	commits:4
     + 	tags:7
      @@ t/t6601-path-walk.sh: test_expect_success 'all, blob:limit=3 filter' '
       	test_cmp_sorted expect out
       '
     @@ t/t6601-path-walk.sh: test_expect_success 'all, blob:limit=3 filter' '
       test_expect_success 'setup sparse filter blob' '
       	# Cone-mode patterns: include root, exclude all dirs, include left/
       	cat >patterns <<-\EOF &&
     +@@ t/t6601-path-walk.sh: test_expect_success 'all, sparse:oid filter' '
     + 	3:tree::$(git rev-parse base^{tree})
     + 	3:tree::$(git rev-parse base~1^{tree})
     + 	3:tree::$(git rev-parse base~2^{tree})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag^{})
     +-	3:tree::$(git rev-parse refs/tags/tree-tag2^{})
     + 	4:blob:a:$(git rev-parse base~2:a)
     +-	5:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
     +-	6:tree:a/:$(git rev-parse base:a)
     +-	7:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     +-	8:tree:left/:$(git rev-parse base:left)
     +-	8:tree:left/:$(git rev-parse base~2:left)
     +-	9:blob:left/b:$(git rev-parse base~2:left/b)
     +-	9:blob:left/b:$(git rev-parse base:left/b)
     +-	10:tree:right/:$(git rev-parse topic:right)
     +-	10:tree:right/:$(git rev-parse base~1:right)
     +-	10:tree:right/:$(git rev-parse base~2:right)
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	5:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	6:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
     ++	7:tree:a/:$(git rev-parse base:a)
     ++	8:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	9:tree:left/:$(git rev-parse base:left)
     ++	9:tree:left/:$(git rev-parse base~2:left)
     ++	10:blob:left/b:$(git rev-parse base~2:left/b)
     ++	10:blob:left/b:$(git rev-parse base:left/b)
     ++	11:tree:right/:$(git rev-parse topic:right)
     ++	11:tree:right/:$(git rev-parse base~1:right)
     ++	11:tree:right/:$(git rev-parse base~2:right)
     + 	blobs:6
     + 	commits:4
     + 	tags:7
 11:  b221ea4712 ! 12:  7e1e503361 path-walk: support `object:type` filter
     @@ Commit message
       ## Documentation/git-pack-objects.adoc ##
      @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
       Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     - ignored in the presence of `--path-walk`. Whe `--path-walk` option
     + ignored in the presence of `--path-walk`. The `--path-walk` option
       supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
      -`tree:0`, and `sparse:<oid>`.
      +`tree:0`, `object:type=<type>`, and `sparse:<oid>`.
     @@ t/t6601-path-walk.sh: test_expect_success 'tree:1 filter is rejected' '
      +	0:tree::$(git rev-parse base^{tree})
      +	0:tree::$(git rev-parse base~1^{tree})
      +	0:tree::$(git rev-parse base~2^{tree})
     -+	0:tree::$(git rev-parse refs/tags/tree-tag^{})
     -+	0:tree::$(git rev-parse refs/tags/tree-tag2^{})
     -+	1:tree:a/:$(git rev-parse base:a)
     -+	2:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     -+	3:tree:left/:$(git rev-parse base:left)
     -+	3:tree:left/:$(git rev-parse base~2:left)
     -+	4:tree:right/:$(git rev-parse topic:right)
     -+	4:tree:right/:$(git rev-parse base~1:right)
     -+	4:tree:right/:$(git rev-parse base~2:right)
     ++	1:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag^{})
     ++	1:tree:/tagged-trees:$(git rev-parse refs/tags/tree-tag2^{})
     ++	2:tree:a/:$(git rev-parse base:a)
     ++	3:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
     ++	4:tree:left/:$(git rev-parse base:left)
     ++	4:tree:left/:$(git rev-parse base~2:left)
     ++	5:tree:right/:$(git rev-parse topic:right)
     ++	5:tree:right/:$(git rev-parse base~1:right)
     ++	5:tree:right/:$(git rev-parse base~2:right)
      +	blobs:0
      +	commits:0
      +	tags:0
 12:  16bd3c3161 ! 13:  a615b1a707 path-walk: support `combine` filter
     @@ Commit message
       ## Documentation/git-pack-objects.adoc ##
      @@ Documentation/git-pack-objects.adoc: will be automatically changed to version `1`.
       Incompatible with `--delta-islands`. The `--use-bitmap-index` option is
     - ignored in the presence of `--path-walk`. Whe `--path-walk` option
     + ignored in the presence of `--path-walk`. The `--path-walk` option
       supports the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,
      -`tree:0`, `object:type=<type>`, and `sparse:<oid>`.
      +`tree:0`, `object:type=<type>`, and `sparse:<oid>`. These supported filter
-- 
gitgitgadget
Previous: Taylor BlauNext: Derrick Stolee via GitGitGadget
Message 58 of 96 in “pack-objects: integrate --path-walk and some --filter options”
  1. 0/7 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  2. 1/7 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  3. Junio C HamanoMay 4, 2026
  4. Derrick StoleeMay 4, 2026
  5. 2/7 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 2, 2026
  6. 3/7 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 2, 2026
  7. 4/7 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 2, 2026
  8. Junio C HamanoMay 3, 2026
  9. Derrick StoleeMay 4, 2026
  10. 5/7 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 2, 2026
  11. 6/7 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 2, 2026
  12. 7/7 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 2, 2026
  13. 00/10 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  14. 01/10 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  15. 02/10 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 4, 2026
  16. 03/10 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 4, 2026
  17. 04/10 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 4, 2026
  18. 05/10 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 4, 2026
  19. 06/10 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 4, 2026
  20. 07/10 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 4, 2026
  21. 08/10 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 4, 2026
  22. 09/10 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 4, 2026
  23. 10/10 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 4, 2026
  24. Derrick StoleeMay 5, 2026
  25. Taylor BlauMay 5, 2026
  26. Derrick StoleeMay 5, 2026
  27. Taylor BlauMay 5, 2026
  28. Derrick StoleeMay 7, 2026
  29. Junio C HamanoMay 11, 2026
  30. Derrick StoleeMay 11, 2026
  31. 00/12 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  32. 01/12 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 11, 2026
  33. Taylor BlauMay 12, 2026
  34. 02/12 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  35. Taylor BlauMay 12, 2026
  36. 03/12 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 11, 2026
  37. Taylor BlauMay 12, 2026
  38. Derrick StoleeMay 13, 2026
  39. 04/12 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 11, 2026
  40. Taylor BlauMay 12, 2026
  41. Derrick StoleeMay 13, 2026
  42. 05/12 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 11, 2026
  43. Taylor BlauMay 11, 2026
  44. Derrick StoleeMay 11, 2026
  45. 06/12 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 11, 2026
  46. Taylor BlauMay 12, 2026
  47. 07/12 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 11, 2026
  48. Taylor BlauMay 12, 2026
  49. Derrick StoleeMay 13, 2026
  50. 08/12 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 11, 2026
  51. 09/12 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 11, 2026
  52. 10/12 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 11, 2026
  53. Taylor BlauMay 12, 2026
  54. Derrick StoleeMay 13, 2026
  55. 11/12 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 11, 2026
  56. 12/12 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 11, 2026
  57. Taylor BlauMay 12, 2026
  58. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  59. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 13, 2026
  60. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  61. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 13, 2026
  62. Taylor BlauMay 19, 2026
  63. Derrick StoleeMay 21, 2026
  64. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 13, 2026
  65. Taylor BlauMay 19, 2026
  66. Derrick StoleeMay 21, 2026
  67. Taylor BlauMay 22, 2026
  68. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 13, 2026
  69. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 13, 2026
  70. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 13, 2026
  71. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 13, 2026
  72. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 13, 2026
  73. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 13, 2026
  74. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 13, 2026
  75. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 13, 2026
  76. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 13, 2026
  77. Taylor BlauMay 19, 2026
  78. Derrick StoleeMay 21, 2026
  79. Taylor BlauMay 22, 2026
  80. Derrick StoleeMay 22, 2026
  81. 00/13 pack-objects: integrate --path-walk and some --filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  82. 01/13 t5620: make test work with path-walk varDerrick Stolee via GitGitGadget, May 22, 2026
  83. 02/13 pack-objects: pass --objects with --path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  84. 03/13 t/perf: add pack-objects filter and path-walk benchmarkDerrick Stolee via GitGitGadget, May 22, 2026
  85. 04/13 path-walk: always emit directly-requested objectsDerrick Stolee via GitGitGadget, May 22, 2026
  86. 05/13 path-walk: support blobless filterDerrick Stolee via GitGitGadget, May 22, 2026
  87. 06/13 backfill: die on incompatible filter optionsDerrick Stolee via GitGitGadget, May 22, 2026
  88. 07/13 path-walk: support blob size limit filterDerrick Stolee via GitGitGadget, May 22, 2026
  89. 08/13 path-walk: add pl_sparse_trees to control tree pruningDerrick Stolee via GitGitGadget, May 22, 2026
  90. 09/13 pack-objects: support sparse:oid filter with path-walkDerrick Stolee via GitGitGadget, May 22, 2026
  91. 10/13 t6601: tag otherwise-unreachable treesDerrick Stolee via GitGitGadget, May 22, 2026
  92. 11/13 path-walk: support `tree:0` filterTaylor Blau via GitGitGadget, May 22, 2026
  93. 12/13 path-walk: support `object:type` filterTaylor Blau via GitGitGadget, May 22, 2026
  94. 13/13 path-walk: support `combine` filterTaylor Blau via GitGitGadget, May 22, 2026
  95. Taylor BlauMay 22, 2026
  96. Junio C HamanoMay 24, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.