{"thread":{"id":"65584","subject":"[RFC PATCH 0/7] pack-bitmap: resolve various `--path-walk` incompatibilities","startedAt":"2026-05-04T00:11:16Z","lastAt":"2026-05-04T21:56:18Z","messageCount":13,"participants":["Taylor Blau","Derrick Stolee","Kristoffer Haugsbakk"],"isPatch":true,"patchVersion":1,"patchTotal":7},"messages":[{"id":"542629","messageId":"cover.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":null,"subject":"[RFC PATCH 0/7] pack-bitmap: resolve various `--path-walk` incompatibilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:11Z","receivedAt":"2026-05-04T00:11:16Z","isPatch":true,"body":"(Note to the maintainer, this is built on top of 'ds/path-walk-filters').\n\nBetween other tasks, I have been working on trying to integrate\n`--path-walk` within GitHub's infrastructure. In order to do this,\n`--path-walk` must work with features that GitHub depends on, such as\nreachability bitmaps and delta-islands (along with filters, shallow,\netc., though more on that below).\n\nI had been sitting on these patches for a few days in my fork before\nStolee sent his series in [1] which resolves incompatibilities between\nthe `--path-walk` option and various filter types. Since I figured that\nothers are working in this area I wanted to send a reworked version of\nmy series for a couple of reasons:\n\n 1. Since reviewers are already looking at this area as a consequence of\n    Stolee's series, this topic should be slightly easier to review\n    while the area is fresh.\n\n 2. In case Stolee (or others) are working on resolving the\n    incompatibility between `--path-walk` and either delta-islands or\n    reachability bitmaps, this series can either combine with those (if\n    any) or serve as inspiration (if others are in the process of\n    writing such series).\n\nWhen writing this originally, I had borrowed the same filter-application\nmechanism from bitmaps, which supports trivial filters (e.g., blob:none,\ntree:0, and combinations therein). Stolee's series is a strict\nimprovement on that approach supporting sparse:<oid> filters as well, so\nI reworked my filtering-related patches based on that.\n\nThe patches surrounding bitmaps and delta-islands are largely\nunchanged from when I had originally written them:\n\n * Supporting bitmaps with `--path-walk` is mostly straightforward, and\n   boils down to ensuring that the path-walk-specific object callback\n   indexes any commit(s) it sees for bitmapping.\n\n * Supporting delta-islands with `--path-walk` required a bit more\n   surgery, and involves propagating island marks for commits in the\n   path-walk-specific callback, as well as recording tree depth\n   information in the same spot.\n\nI'm submitting these patches as an RFC, since (a) I haven't thought\ndeeply about the approach taken here and could very well be on the wrong\ntrack, and (b) in case Stolee or others want to combine forces here\nand/or coordinate around each other.\n\nThanks in advance for your review!\n\n[1]: https://lore.kernel.org/git/pull.2101.git.1777731354.gitgitgadget@gmail.com/\n\nTaylor Blau (7):\n  pack-objects: update `--path-walk`'s existing incompatibilities\n  path-walk: support `tree:0` filter\n  path-walk: support `object:type` filter\n  path-walk: support `combine` filter\n  pack-objects: support reachability bitmaps with `--path-walk`\n  pack-objects: extract `record_tree_depth()` helper\n  pack-objects: support `--delta-islands` with `--path-walk`\n\n Documentation/git-pack-objects.adoc |  10 +-\n builtin/pack-objects.c              |  62 ++++++---\n path-walk.c                         |  63 +++++++--\n t/t5310-pack-bitmaps.sh             |  36 +++++\n t/t5320-delta-islands.sh            |  29 ++++\n t/t6601-path-walk.sh                | 196 ++++++++++++++++++++++++++++\n 6 files changed, 365 insertions(+), 31 deletions(-)\n\n\nbase-commit: 465ceb37112ddfc6338727887f4431e755bf1831\n-- \n2.54.0.4.g6aa0d38a4ec\n"},{"id":"542630","messageId":"babe1596161365209c226d374db70a1bdc284a1c.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 1/7] pack-objects: update `--path-walk`'s existing incompatibilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:17Z","receivedAt":"2026-05-04T00:11:20Z","isPatch":true,"body":"The documentation in git-pack-objects(1) claims that `--path-walk` is\nincompatible with `-shallow`. However, commit c178b02e29f (pack-objects:\nallow --shallow and --path-walk, 2025-05-16) resolves this\nincompatibility, leaving the documentation stale.\n\nLikewise, this documentation claims that `--filter` is incompatible, but\n`blob:none`, `blob:limit=<n>`, and `sparse:oid=<blob>` already work via\npath-walk.\n\nList the supported `--filter` forms explicitly and note that other forms\nfall back to the regular object traversal. Also remove the\nincompatibility notice with `--shallow`.\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc | 8 +++++---\n 1 file changed, 5 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex b78175fbe1b..8dea8259787 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -402,9 +402,11 @@ will be automatically changed to version `1`.\n \tof filenames that cause collisions in Git's default name-hash\n \talgorithm.\n +\n-Incompatible with `--delta-islands`, `--shallow`, or `--filter`. The\n-`--use-bitmap-index` option will be ignored in the presence of\n-`--path-walk.`\n+Incompatible with `--delta-islands`. Path-walk supports the\n+`--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and\n+`sparse:oid=<blob>`. Other filter forms fall back to the regular object\n+traversal. The `--use-bitmap-index` option will be ignored in the\n+presence of `--path-walk`.\n \n \n DELTA ISLANDS\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542631","messageId":"e1b7fd3cb2a2bba5f6404ac5f8ac3487a46d51b5.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 2/7] path-walk: support `tree:0` filter","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:20Z","receivedAt":"2026-05-04T00:11:23Z","isPatch":true,"body":"The `tree:0` object filter omits all trees and blobs from the result,\nkeeping only commits and tags. Consequently, this filter type should\nhas a fairly straightforward integration with path-walk, as the decision\nto include an object depends only on its type and does not depend on any\npath-sensitive state.\n\nMapping it onto `path_walk_info` is direct: set `info->trees = 0` and\n`info->blobs = 0` in `prepare_filters()` when the `LOFC_TREE_DEPTH`\nchoice is requested with depth zero. The existing code already plumbs\nthose flags through the rest of the walk:\n\n - 'walk_objects_by_path()' sets `revs->blob_objects = info->blobs` and\n   `revs->tree_objects = info->trees` before `prepare_revision_walk()`,\n   so the revision walk doesn't try to enumerate trees or blobs itself.\n\n - The commit-walk loop short-circuits the root-tree fetch with\n   \"if (!info->trees && !info->blobs) continue;\", so we never even\n   look up the root tree, let alone descend into it.\n\n - `setup_pending_objects()` skips pending trees and blobs based on\n   the same flags.\n\nThis means the path-walk doesn't allocate or expand any tree structures\nat all under `tree:0`, which matches the intended behavior of the\nfilter.\n\nNon-zero tree-depth filters are not supported. Those depend on the depth\nat which a tree is visited, which is a path-walk concept the filter\nmachinery doesn't currently share with the path-walk API. Reject them in\n`prepare_filters()` with a helpful error and let pack-objects fall back\nto the regular traversal, the same way it already does for unsupported\nfilters.\n\nAdd coverage in t6601 for both `--all` and a single-branch case to\nconfirm that no trees or blobs are emitted, and a separate test that\n`tree:1` is rejected with the expected error message. Place the new\ntests before \"setup sparse filter blob\" so they run on the original set\nof refs, before the orphan branch that the sparse-tree tests create.\n\nUpdate Documentation/git-pack-objects.adoc to drop --filter from\nthe unconditional incompatibility list and call out the supported\nsubset (which already includes the filters added by Stolee's\nearlier patches: blob:none, blob:limit, and sparse:oid).\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc | 10 +++----\n path-walk.c                         | 13 +++++++++\n t/t6601-path-walk.sh                | 45 +++++++++++++++++++++++++++++\n 3 files changed, 63 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex 8dea8259787..cfb5bc0ae16 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -402,11 +402,11 @@ will be automatically changed to version `1`.\n \tof filenames that cause collisions in Git's default name-hash\n \talgorithm.\n +\n-Incompatible with `--delta-islands`. Path-walk supports the\n-`--filter=<spec>` forms `blob:none`, `blob:limit=<n>`, and\n-`sparse:oid=<blob>`. Other filter forms fall back to the regular object\n-traversal. The `--use-bitmap-index` option will be ignored in the\n-presence of `--path-walk`.\n+Incompatible with `--delta-islands`. Path-walk supports\n+the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,\n+`sparse:oid=<blob>`, and `tree:0`. Other filter forms fall back to the\n+regular object traversal. The `--use-bitmap-index` option will be\n+ignored in the presence of `--path-walk`.\n \n \n DELTA ISLANDS\ndiff --git a/path-walk.c b/path-walk.c\nindex 700617ee2fe..36a1e5b967a 100644\n--- a/path-walk.c\n+++ b/path-walk.c\n@@ -564,6 +564,19 @@ static int prepare_filters(struct path_walk_info *info,\n \t\t}\n \t\treturn 1;\n \n+\tcase LOFC_TREE_DEPTH:\n+\t\tif (options->tree_exclude_depth) {\n+\t\t\terror(_(\"tree:%lu filter not supported by the path-walk API\"),\n+\t\t\t      options->tree_exclude_depth);\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (info) {\n+\t\t\tinfo->trees = 0;\n+\t\t\tinfo->blobs = 0;\n+\t\t\tlist_objects_filter_release(options);\n+\t\t}\n+\t\treturn 1;\n+\n \tcase LOFC_SPARSE_OID:\n \t\tif (info) {\n \t\t\tstruct object_id sparse_oid;\ndiff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh\nindex 520269dfc65..72e09211e63 100755\n--- a/t/t6601-path-walk.sh\n+++ b/t/t6601-path-walk.sh\n@@ -590,6 +590,51 @@ test_expect_success 'all, blob:limit=3 filter' '\n \ttest_cmp_sorted expect out\n '\n \n+test_expect_success 'all, tree:0 filter' '\n+\ttest-tool path-walk --filter=tree:0 -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:commit::$(git rev-parse topic)\n+\t0:commit::$(git rev-parse base)\n+\t0:commit::$(git rev-parse base~1)\n+\t0:commit::$(git rev-parse base~2)\n+\t1:tag:/tags:$(git rev-parse refs/tags/first)\n+\t1:tag:/tags:$(git rev-parse refs/tags/second.1)\n+\t1:tag:/tags:$(git rev-parse refs/tags/second.2)\n+\t1:tag:/tags:$(git rev-parse refs/tags/third)\n+\t1:tag:/tags:$(git rev-parse refs/tags/fourth)\n+\t1:tag:/tags:$(git rev-parse refs/tags/tree-tag)\n+\t1:tag:/tags:$(git rev-parse refs/tags/blob-tag)\n+\tblobs:0\n+\tcommits:4\n+\ttags:7\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'topic only, tree:0 filter' '\n+\ttest-tool path-walk --filter=tree:0 -- topic >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:commit::$(git rev-parse topic)\n+\t0:commit::$(git rev-parse base~1)\n+\t0:commit::$(git rev-parse base~2)\n+\tblobs:0\n+\tcommits:3\n+\ttags:0\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'tree:1 filter is rejected' '\n+\ttest_must_fail test-tool path-walk --filter=tree:1 -- --all 2>err &&\n+\ttest_grep \"tree:1 filter not supported by the path-walk API\" err\n+'\n+\n test_expect_success 'setup sparse filter blob' '\n \t# Cone-mode patterns: include root, exclude all dirs, include left/\n \tcat >patterns <<-\\EOF &&\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542632","messageId":"db46c1248ece57476b369a9bff920facab24be04.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 3/7] path-walk: support `object:type` filter","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:23Z","receivedAt":"2026-05-04T00:11:25Z","isPatch":true,"body":"The `object:type` filter accepts only objects of a single type; it is\nthe second member of the object-info-only filter family that bitmap\ntraversal already supports.\n\nLike `blob:none` and `tree:0`, it can be evaluated with nothing more\nthan the object's type, which is exactly the granularity path-walk's\nexisting info->{commits,trees,blobs,tags} flags already control.\n\nMap `LOFC_OBJECT_TYPE` in `prepare_filters()` by AND-ing each flag\nagainst the filtered type. A single `object:type=X` filter\napplied to the default info (all flags = 1) leaves `info->X = 1` and\nall the others 0, which is what we want.\n\nUsing an AND rather than straight assignment prepares us for a\nsubsequent change to implement combined object filters.\n\nThe path-walk machinery is mostly already wired for the per-type\ndistinction:\n\n - `walk_path()` calls `path_fn` for a batch only when the corresponding\n   `info->X` flag is set, so unwanted types are silently not reported.\n\n - `add_tree_entries()` skips tree entries of type `OBJ_BLOB` when\n   `info->blobs` is unset, so we don't even allocate paths for them.\n\n - The commit-walk loop short-circuits the root-tree fetch when\n   `!info->trees && !info->blobs`, so commit-only filters don't descend\n   into trees at all.\n\nBut there are a couple of side effects of the \"trees off, blobs on\" case\nthat need fixing:\n\n 1. 'setup_pending_objects()' previously skipped pending trees as soon\n    as `info->trees` was zero. For 'object:type=blob' the call site\n    needs those pending trees: a lightweight tag pointing to a tree, or\n    an annotated tag whose peeled target is a tree, can both reach\n    blobs that are otherwise unreachable from any commit's root tree.\n    Loosen the gate to \"if (!info->trees && !info->blobs) continue\" and\n    similarly retrieve the root_tree_list whenever either trees or\n    blobs are wanted.\n\n 2. The revision machinery's `handle_commit()` drops pending trees when\n    `revs->tree_objects` is zero (see the 'OBJ_TREE' handler in\n    revision.c), so by the time path-walk sees the pending list\n    after `prepare_revision_walk()` the tree-bearing pendings would\n    already be gone. Fix this by setting\n\n        revs->tree_objects = info->trees || info->blobs\n\n    so pending trees survive `prepare_revision_walk()` whenever we\n    need to walk into them. Path-walk still resets tree_objects to\n    zero immediately after `prepare_revision_walk()` returns, so the\n    rev-walk itself never enumerates trees redundantly with\n    path-walk's own descent.\n\nAdd coverage in t6601 for each of the four `object:type` values. The\n'object:type=blob' test in particular asserts that file2 and child/file\n(both reachable only through tag-pointed trees) show up in the output,\nexercising the pending-tree fix.\n\nUpdate Documentation/git-pack-objects.adoc to add object:type to\nthe list of supported --filter forms.\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc |  7 ++-\n path-walk.c                         | 23 +++++++-\n t/t6601-path-walk.sh                | 86 +++++++++++++++++++++++++++++\n 3 files changed, 110 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex cfb5bc0ae16..22c782611d2 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -404,9 +404,10 @@ will be automatically changed to version `1`.\n +\n Incompatible with `--delta-islands`. Path-walk supports\n the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,\n-`sparse:oid=<blob>`, and `tree:0`. Other filter forms fall back to the\n-regular object traversal. The `--use-bitmap-index` option will be\n-ignored in the presence of `--path-walk`.\n+`sparse:oid=<blob>`, `tree:0`, and `object:type=<type>`. Other filter\n+forms fall back to the regular object traversal. The\n+`--use-bitmap-index` option will be ignored in the presence of\n+`--path-walk`.\n \n \n DELTA ISLANDS\ndiff --git a/path-walk.c b/path-walk.c\nindex 36a1e5b967a..b9902abbb75 100644\n--- a/path-walk.c\n+++ b/path-walk.c\n@@ -430,7 +430,7 @@ static int setup_pending_objects(struct path_walk_info *info,\n \t\tCALLOC_ARRAY(tags, 1);\n \tif (info->blobs)\n \t\tCALLOC_ARRAY(tagged_blobs, 1);\n-\tif (info->trees)\n+\tif (info->trees || info->blobs)\n \t\troot_tree_list = strmap_get(&ctx->paths_to_lists, root_path);\n \n \t/*\n@@ -475,7 +475,7 @@ static int setup_pending_objects(struct path_walk_info *info,\n \n \t\tswitch (obj->type) {\n \t\tcase OBJ_TREE:\n-\t\t\tif (!info->trees)\n+\t\t\tif (!info->trees && !info->blobs)\n \t\t\t\tcontinue;\n \t\t\tif (pending->path) {\n \t\t\t\tchar *path = *pending->path ? xstrfmt(\"%s/\", pending->path)\n@@ -577,6 +577,16 @@ static int prepare_filters(struct path_walk_info *info,\n \t\t}\n \t\treturn 1;\n \n+\tcase LOFC_OBJECT_TYPE:\n+\t\tif (info) {\n+\t\t\tinfo->commits &= options->object_type == OBJ_COMMIT;\n+\t\t\tinfo->tags &= options->object_type == OBJ_TAG;\n+\t\t\tinfo->trees &= options->object_type == OBJ_TREE;\n+\t\t\tinfo->blobs &= options->object_type == OBJ_BLOB;\n+\t\t\tlist_objects_filter_release(options);\n+\t\t}\n+\t\treturn 1;\n+\n \tcase LOFC_SPARSE_OID:\n \t\tif (info) {\n \t\t\tstruct object_id sparse_oid;\n@@ -683,9 +693,16 @@ int walk_objects_by_path(struct path_walk_info *info)\n \t/*\n \t * Set these values before preparing the walk to catch\n \t * lightweight tags pointing to non-commits and indexed objects.\n+\t *\n+\t * Keep tree_objects set whenever blobs are wanted: blobs may\n+\t * be reachable through trees that show up as pending objects\n+\t * (e.g., via lightweight tags pointing to trees, or annotated\n+\t * tags whose peeled target is a tree). Without tree_objects,\n+\t * prepare_revision_walk() would discard those pending trees\n+\t * and we would never descend into them.\n \t */\n \tinfo->revs->blob_objects = info->blobs;\n-\tinfo->revs->tree_objects = info->trees;\n+\tinfo->revs->tree_objects = info->trees || info->blobs;\n \n \tif (prepare_revision_walk(info->revs))\n \t\tdie(_(\"failed to setup revision walk\"));\ndiff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh\nindex 72e09211e63..13016e62ab1 100755\n--- a/t/t6601-path-walk.sh\n+++ b/t/t6601-path-walk.sh\n@@ -635,6 +635,92 @@ test_expect_success 'tree:1 filter is rejected' '\n \ttest_grep \"tree:1 filter not supported by the path-walk API\" err\n '\n \n+test_expect_success 'all, object:type=commit filter' '\n+\ttest-tool path-walk --filter=object:type=commit -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:commit::$(git rev-parse topic)\n+\t0:commit::$(git rev-parse base)\n+\t0:commit::$(git rev-parse base~1)\n+\t0:commit::$(git rev-parse base~2)\n+\tblobs:0\n+\tcommits:4\n+\ttags:0\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'all, object:type=tag filter' '\n+\ttest-tool path-walk --filter=object:type=tag -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:tag:/tags:$(git rev-parse refs/tags/first)\n+\t0:tag:/tags:$(git rev-parse refs/tags/second.1)\n+\t0:tag:/tags:$(git rev-parse refs/tags/second.2)\n+\t0:tag:/tags:$(git rev-parse refs/tags/third)\n+\t0:tag:/tags:$(git rev-parse refs/tags/fourth)\n+\t0:tag:/tags:$(git rev-parse refs/tags/tree-tag)\n+\t0:tag:/tags:$(git rev-parse refs/tags/blob-tag)\n+\tblobs:0\n+\tcommits:0\n+\ttags:7\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'all, object:type=tree filter' '\n+\ttest-tool path-walk --filter=object:type=tree -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:tree::$(git rev-parse topic^{tree})\n+\t0:tree::$(git rev-parse base^{tree})\n+\t0:tree::$(git rev-parse base~1^{tree})\n+\t0:tree::$(git rev-parse base~2^{tree})\n+\t0:tree::$(git rev-parse refs/tags/tree-tag^{})\n+\t0:tree::$(git rev-parse refs/tags/tree-tag2^{})\n+\t1:tree:a/:$(git rev-parse base:a)\n+\t2:tree:child/:$(git rev-parse refs/tags/tree-tag:child)\n+\t3:tree:left/:$(git rev-parse base:left)\n+\t3:tree:left/:$(git rev-parse base~2:left)\n+\t4:tree:right/:$(git rev-parse topic:right)\n+\t4:tree:right/:$(git rev-parse base~1:right)\n+\t4:tree:right/:$(git rev-parse base~2:right)\n+\tblobs:0\n+\tcommits:0\n+\ttags:0\n+\ttrees:13\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'all, object:type=blob filter' '\n+\ttest-tool path-walk --filter=object:type=blob -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})\n+\t0:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})\n+\t1:blob:a:$(git rev-parse base~2:a)\n+\t2:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)\n+\t3:blob:child/file:$(git rev-parse refs/tags/tree-tag:child/file)\n+\t4:blob:left/b:$(git rev-parse base:left/b)\n+\t4:blob:left/b:$(git rev-parse base~2:left/b)\n+\t5:blob:right/c:$(git rev-parse base~2:right/c)\n+\t5:blob:right/c:$(git rev-parse topic:right/c)\n+\t6:blob:right/d:$(git rev-parse base~1:right/d)\n+\tblobs:10\n+\tcommits:0\n+\ttags:0\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n test_expect_success 'setup sparse filter blob' '\n \t# Cone-mode patterns: include root, exclude all dirs, include left/\n \tcat >patterns <<-\\EOF &&\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542633","messageId":"5a4c39d7ae18c2dafa0e9d80ce5aad9ee6db4245.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 4/7] path-walk: support `combine` filter","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:26Z","receivedAt":"2026-05-04T00:11:28Z","isPatch":true,"body":"The `combine` filter takes the intersection of its children, that is:\nobjects are shown only when all child filters would admit the object.\n\nThe preceding patches added support for many individual filter types.\nEnable users to compose these filters by implementing support for the\n`combine` filter type.\n\nMapping intersection onto path_walk_info works because every supported\nchild filter is a monotonic restriction:\n\n - `blob:none`, `tree:0` unconditionally clear `info->blobs` and (for\n   `tree:0`) `info->trees`; clearing an already-cleared flag is a\n   no-op.\n\n - `object:type=X` is now expressed as an AND of each type flag with the\n   filtered type, so applying multiple such filters only refines the\n   existing set rather than overwrites it.\n\n - `blob:limit=N` has to compose too: the intersection of \"size < L1\"\n   and \"size < L2\" is \"size < min(L1, L2)\".\n\n   Update the `LOFC_BLOB_LIMIT` handler to take the running minimum when\n   `info->blob_limit` is already set, so a combined filter with, e.g.,\n   both \"blob:limit=10\" and \"blob:limit=5\" produces a limit of 5\n   regardless of ordering.\n\n - `sparse:oid` is left unchanged. A `combine` filter that includes a\n   `sparse:oid` is allowed at most once, since the existing handler\n   refuses to overwrite `info->pl`. Two `sparse:oid` filters in a single\n   `combine` would be unusual and are rejected with a warning, matching\n   the standalone `sparse:oid` behavior.\n\nImplementation-wise, the existing `prepare_filters()` called\n`list_objects_filter_release()` inside each case branch. That works fine\nfor top-level filters, but `combine` filters need to recurse over its\n  child filters without releasing each one in turn (since the parent's\n  release iterates the sub array). Split `prepare_filters()` into a\n  recursive helper that performs only the mutation, plus a thin wrapper\n  that calls the helper and then releases the top-level filter once.\n\nThe `LOFC_COMBINE` case in the helper just walks `sub_nr` and recurses;\nchild filters are released by the wrapper's single\n`list_objects_filter_release()` call on the parent (which itself\nrecursively releases each sub-filter, the same way it always has).\n\nIf any sub-filter is unsupported (e.g. \"tree:1\", \"sparse:<path>\", or a\nnot-yet-supported choice), the recursion bubbles a failure up and the\nexisting pack-objects/backfill fallback paths kick in.\n\nAdd coverage in t6601:\n\n  - \"combine:blob:none+tree:0\" collapses to \"tree:0\"\n\n  - \"combine:object:type=blob+blob:limit=3\" yields only the blobs\n    smaller than three bytes\n\n  - \"combine:object:type=blob+object:type=tree\" intersects to empty\n\n  - \"combine:tree:1+blob:none\" reports the \"tree:1\" error.\n\nUpdate Documentation/git-pack-objects.adoc to add combine to the\nlist of supported --filter forms.\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc |  8 ++--\n path-walk.c                         | 31 +++++++++-----\n t/t6601-path-walk.sh                | 65 +++++++++++++++++++++++++++++\n 3 files changed, 90 insertions(+), 14 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex 22c782611d2..6c7bbff5be5 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -404,10 +404,10 @@ will be automatically changed to version `1`.\n +\n Incompatible with `--delta-islands`. Path-walk supports\n the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,\n-`sparse:oid=<blob>`, `tree:0`, and `object:type=<type>`. Other filter\n-forms fall back to the regular object traversal. The\n-`--use-bitmap-index` option will be ignored in the presence of\n-`--path-walk`.\n+`sparse:oid=<blob>`, `tree:0`, `object:type=<type>`, and `combine:`\n+over any of those. Other filter forms fall back to the regular object\n+traversal. The `--use-bitmap-index` option will be ignored in the\n+presence of `--path-walk`.\n \n \n DELTA ISLANDS\ndiff --git a/path-walk.c b/path-walk.c\nindex b9902abbb75..6d66da3dc3b 100644\n--- a/path-walk.c\n+++ b/path-walk.c\n@@ -539,28 +539,26 @@ static int setup_pending_objects(struct path_walk_info *info,\n \treturn 0;\n }\n \n-static int prepare_filters(struct path_walk_info *info,\n-\t\t\t   struct list_objects_filter_options *options)\n+static int prepare_filters_one(struct path_walk_info *info,\n+\t\t\t       struct list_objects_filter_options *options)\n {\n \tswitch (options->choice) {\n \tcase LOFC_DISABLED:\n \t\treturn 1;\n \n \tcase LOFC_BLOB_NONE:\n-\t\tif (info) {\n+\t\tif (info)\n \t\t\tinfo->blobs = 0;\n-\t\t\tlist_objects_filter_release(options);\n-\t\t}\n \t\treturn 1;\n \n \tcase LOFC_BLOB_LIMIT:\n \t\tif (info) {\n \t\t\tif (!options->blob_limit_value) {\n \t\t\t\tinfo->blobs = 0;\n-\t\t\t} else {\n+\t\t\t} else if (!info->blob_limit ||\n+\t\t\t\t   options->blob_limit_value < info->blob_limit) {\n \t\t\t\tinfo->blob_limit = options->blob_limit_value;\n \t\t\t}\n-\t\t\tlist_objects_filter_release(options);\n \t\t}\n \t\treturn 1;\n \n@@ -573,7 +571,6 @@ static int prepare_filters(struct path_walk_info *info,\n \t\tif (info) {\n \t\t\tinfo->trees = 0;\n \t\t\tinfo->blobs = 0;\n-\t\t\tlist_objects_filter_release(options);\n \t\t}\n \t\treturn 1;\n \n@@ -583,7 +580,6 @@ static int prepare_filters(struct path_walk_info *info,\n \t\t\tinfo->tags &= options->object_type == OBJ_TAG;\n \t\t\tinfo->trees &= options->object_type == OBJ_TREE;\n \t\t\tinfo->blobs &= options->object_type == OBJ_BLOB;\n-\t\t\tlist_objects_filter_release(options);\n \t\t}\n \t\treturn 1;\n \n@@ -624,8 +620,13 @@ static int prepare_filters(struct path_walk_info *info,\n \t\t\t\twarning(_(\"sparse filter is not cone-mode compatible\"));\n \t\t\t\treturn 0;\n \t\t\t}\n+\t\t}\n+\t\treturn 1;\n \n-\t\t\tlist_objects_filter_release(options);\n+\tcase LOFC_COMBINE:\n+\t\tfor (size_t i = 0; i < options->sub_nr; i++) {\n+\t\t\tif (!prepare_filters_one(info, &options->sub[i]))\n+\t\t\t\treturn 0;\n \t\t}\n \t\treturn 1;\n \n@@ -636,6 +637,16 @@ static int prepare_filters(struct path_walk_info *info,\n \t}\n }\n \n+static int prepare_filters(struct path_walk_info *info,\n+\t\t\t   struct list_objects_filter_options *options)\n+{\n+\tif (!prepare_filters_one(info, options))\n+\t\treturn 0;\n+\tif (info)\n+\t\tlist_objects_filter_release(options);\n+\treturn 1;\n+}\n+\n int path_walk_filter_compatible(struct list_objects_filter_options *options)\n {\n \treturn prepare_filters(NULL, options);\ndiff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh\nindex 13016e62ab1..a7d5f0de4ec 100755\n--- a/t/t6601-path-walk.sh\n+++ b/t/t6601-path-walk.sh\n@@ -721,6 +721,71 @@ test_expect_success 'all, object:type=blob filter' '\n \ttest_cmp_sorted expect out\n '\n \n+test_expect_success 'all, combine:blob:none+tree:0 filter' '\n+\ttest-tool path-walk \\\n+\t\t--filter=combine:blob:none+tree:0 -- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:commit::$(git rev-parse topic)\n+\t0:commit::$(git rev-parse base)\n+\t0:commit::$(git rev-parse base~1)\n+\t0:commit::$(git rev-parse base~2)\n+\t1:tag:/tags:$(git rev-parse refs/tags/first)\n+\t1:tag:/tags:$(git rev-parse refs/tags/second.1)\n+\t1:tag:/tags:$(git rev-parse refs/tags/second.2)\n+\t1:tag:/tags:$(git rev-parse refs/tags/third)\n+\t1:tag:/tags:$(git rev-parse refs/tags/fourth)\n+\t1:tag:/tags:$(git rev-parse refs/tags/tree-tag)\n+\t1:tag:/tags:$(git rev-parse refs/tags/blob-tag)\n+\tblobs:0\n+\tcommits:4\n+\ttags:7\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'all, combine:object:type=blob+blob:limit=3 filter' '\n+\ttest-tool path-walk \\\n+\t\t--filter=combine:object:type=blob+blob:limit=3 \\\n+\t\t-- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\t0:blob:a:$(git rev-parse base~2:a)\n+\t1:blob:left/b:$(git rev-parse base~2:left/b)\n+\t2:blob:right/c:$(git rev-parse base~2:right/c)\n+\t3:blob:right/d:$(git rev-parse base~1:right/d)\n+\tblobs:4\n+\tcommits:0\n+\ttags:0\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'all, combine of disjoint object:types is empty' '\n+\ttest-tool path-walk \\\n+\t\t--filter=combine:object:type=blob+object:type=tree \\\n+\t\t-- --all >out &&\n+\n+\tcat >expect <<-EOF &&\n+\tblobs:0\n+\tcommits:0\n+\ttags:0\n+\ttrees:0\n+\tEOF\n+\n+\ttest_cmp_sorted expect out\n+'\n+\n+test_expect_success 'combine: rejects unsupported subfilters' '\n+\ttest_must_fail test-tool path-walk \\\n+\t\t--filter=combine:tree:1+blob:none -- --all 2>err &&\n+\ttest_grep \"tree:1 filter not supported by the path-walk API\" err\n+'\n+\n test_expect_success 'setup sparse filter blob' '\n \t# Cone-mode patterns: include root, exclude all dirs, include left/\n \tcat >patterns <<-\\EOF &&\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542634","messageId":"f50f8df01a9f216d5b4388b2fe4ff58077b574f3.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 5/7] pack-objects: support reachability bitmaps with `--path-walk`","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:29Z","receivedAt":"2026-05-04T00:11:31Z","isPatch":true,"body":"When 'pack-objects' is invoked with '--path-walk', it prevents us from\nusing reachability bitmaps.\n\nThis behavior dates back to 70664d2865c (pack-objects: add --path-walk\noption, 2025-05-16), which included a comment in the relevant portion of\nthe command-line arguments handling that read as follows:\n\n    /*\n     * We must disable the bitmaps because we are removing\n     * the --objects / --objects-edge[-aggressive] options.\n     */\n\nIn fb2c309b7d3 (pack-objects: pass --objects with --path-walk,\n2026-05-02), we adjusted this behavior to also pass \"--objects\", but\nstill disable use of reachability bitmaps.\n\nFortunately, disabling reachability bitmaps is not strictly necessary.\nConsider a couple of pack-objects use-cases: one during repacking, when\nwe would ordinarily generate reachability bitmaps, and another for\nserving fetches and clones, when we would ordinarily read existing\nbitmaps:\n\n - When attempting to generate reachability bitmaps, we would fail to do\n   so since path-walk reveals objects through the\n   `add_objects_by_path()` callback rather than, e.g., `show_commit()`,\n   so the bitmap selector's `index_commit_for_bitmap()` was never called\n   for any commit.\n\n   The selection routine then had no candidates and bitmap writing was\n   effectively a no-op.\n\n - On the bitmap-reading side, an invocation like \"git pack-objects\n   --use-bitmap-index --path-walk\" never even tried to consult a bitmap,\n   even when one was sitting on disk that could have answered the\n   request.\n\nNeither restriction is required. They are discussed in turn:\n\n - For bitmap-writing, all we need is for `index_commit_for_bitmap()` to\n   see each commit that path-walk visits. The path-walk callback already\n   groups commits into a single batch keyed by `OBJ_COMMIT`, so invoking\n   `index_commit_for_bitmap()` from there gives the bitmap selection\n   routine the same input it would have gotten from `show_commit()` in\n   the regular traversal.\n\n   The candidate set of commits is identical, though the ordering\n   differs. Bitmap selection is sensitive to commit ordering, but\n   commits are visited in the same order as we see them from\n   `get_revision()` so bitmap selection should be identical with or\n   without `--path-walk`.\n\n - For bitmap-reading, all we need is for `revs->tree_objects` (and so\n   on for blobs and tags) to be set, otherwise bitmap traversal would\n   only emit commit objects.\n\n   In commit fb2c309b7d3, those flags are set via passing \"--objects\",\n   so bitmap traversal under \"--path-walk\" packs everything just like\n   any other \"--use-bitmap-index\" invocation.\n\nIf an existing reachability bitmap is unable to satisfy the request (no\nbitmap on disk, haves not in the bitmapped pack, etc.) we fall through\nto path-walk's own enumeration, just as the regular traversal falls back\nwhen a bitmap is unavailable.\n\nIn other words: \"--path-walk --use-bitmap-index\" uses reachability\nbitmaps when available, and otherwise enumerates via path-walk.\n\nThe regression in t5310 deserves a word about pack-reuse. With\npack-reuse enabled (the default), the output pack copies whole regions\nof the existing bitmapped pack before `traverse_bitmap_commit_list()`\neven runs, so a naive test would happily pass even if, say,\n`revs->blob_objects` is set to 0. We test both with and without\npack-reuse enabled. When pack-reuse is disabled, pack-objects must\nenumerate the resulting bitmap without copying any existing on-disk in\nthe rev_info setup that pack-reuse would otherwise paper over.\n\nUpdate Documentation/git-pack-objects.adoc to drop the \"ignored\"\nclaim about --use-bitmap-index in favor of describing the new\nfallback chain.\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc |  6 +++--\n builtin/pack-objects.c              | 10 +++++++-\n t/t5310-pack-bitmaps.sh             | 36 +++++++++++++++++++++++++++++\n 3 files changed, 49 insertions(+), 3 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex 6c7bbff5be5..60e594c7bc4 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -406,8 +406,10 @@ Incompatible with `--delta-islands`. Path-walk supports\n the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,\n `sparse:oid=<blob>`, `tree:0`, `object:type=<type>`, and `combine:`\n over any of those. Other filter forms fall back to the regular object\n-traversal. The `--use-bitmap-index` option will be ignored in the\n-presence of `--path-walk`.\n+traversal. When `--use-bitmap-index` is specified with `--path-walk`, a\n+successful bitmap traversal is used for object enumeration, with\n+path-walk remaining as the fallback traversal when the bitmap cannot\n+satisfy the request.\n \n \n DELTA ISLANDS\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ba00d8148ab..1a5f1afd32e 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -4732,6 +4732,15 @@ static int add_objects_by_path(const char *path,\n \t\t\tcontinue;\n \n \t\tadd_object_entry(oid, type, path, exclude);\n+\n+\t\tif (type == OBJ_COMMIT && write_bitmap_index) {\n+\t\t\tstruct commit *commit;\n+\n+\t\t\tcommit = lookup_commit(the_repository, oid);\n+\t\t\tif (!commit)\n+\t\t\t\tdie(_(\"could not find commit %s\"), oid_to_hex(oid));\n+\t\t\tindex_commit_for_bitmap(commit);\n+\t\t}\n \t}\n \n \toe_end = to_pack.nr_objects;\n@@ -5193,7 +5202,6 @@ int cmd_pack_objects(int argc,\n \tif (path_walk) {\n \t\tstrvec_push(&rp, \"--boundary\");\n \t\tstrvec_push(&rp, \"--objects\");\n-\t\tuse_bitmap_index = 0;\n \t} else if (thin) {\n \t\tuse_internal_rev_list = 1;\n \t\tstrvec_push(&rp, shallow\ndiff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh\nindex f693cb56691..69c5da1580a 100755\n--- a/t/t5310-pack-bitmaps.sh\n+++ b/t/t5310-pack-bitmaps.sh\n@@ -577,6 +577,42 @@ test_bitmap_cases\n \n sane_unset GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL\n \n+test_expect_success 'path-walk repack can write and use bitmap indexes' '\n+\ttest_when_finished \"rm -rf path-walk-bitmap\" &&\n+\tgit init path-walk-bitmap &&\n+\t(\n+\t\tcd path-walk-bitmap &&\n+\t\ttest_commit first &&\n+\t\ttest_commit second &&\n+\t\ttest_commit third &&\n+\n+\t\tgit repack -a -d -b --path-walk &&\n+\t\tgit rev-list --test-bitmap --use-bitmap-index HEAD &&\n+\n+\t\tgit rev-parse HEAD >in &&\n+\n+\t\tgit rev-list --objects --no-object-names HEAD >expect.raw &&\n+\t\tsort expect.raw >expect &&\n+\n+\t\tfor reuse in true false\n+\t\tdo\n+\t\t\t: >trace.txt &&\n+\n+\t\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace.txt\" \\\n+\t\t\tgit -c pack.allowPackReuse=$reuse pack-objects \\\n+\t\t\t\t--stdout --revs --path-walk --use-bitmap-index \\\n+\t\t\t\t<in >out.pack &&\n+\t\t\tgrep \"\\\"category\\\":\\\"bitmap\\\",\\\"key\\\":\\\"bitmap/hits\\\"\" trace.txt &&\n+\n+\t\t\tgit index-pack out.pack &&\n+\n+\t\t\tlist_packed_objects out.idx >actual.raw &&\n+\t\t\tsort actual.raw >actual &&\n+\t\t\ttest_cmp expect actual || return 1\n+\t\tdone\n+\t)\n+'\n+\n test_expect_success 'incremental repack fails when bitmaps are requested' '\n \ttest_commit more-1 &&\n \ttest_must_fail git repack -d 2>err &&\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542635","messageId":"35b4485fa5258f63d0e0996be7760ec83e9adac6.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 6/7] pack-objects: extract `record_tree_depth()` helper","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:32Z","receivedAt":"2026-05-04T00:11:34Z","isPatch":true,"body":"Prepare for a future change that needs to record tree depths from a\nsecond call site by factoring out the delta islands-specific portion of\n`show_object()` out into a helper, `record_tree_depth()`.\n\n`record_tree_depth()` takes a tree OID along with the path that\n`show_object()` received, and computes the directory depth from the\nslash count in the path.\n\nWhile we're in the area, make a few minor clean-ups:\n\n - Gate the call on `obj->type == OBJ_TREE`, as we only care to compute\n   the depth for tree objects. The sole caller of `oe_tree_depth()`\n   resides in `delta-islands.c::resolve_tree_islands()`, and only calls\n   `oe_tree_depth()` behind `oe_type(...) == OBJ_TREE`.\n\n - Defer computing the depth for an object until we know it is in the\n   `to_pack` list.\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n builtin/pack-objects.c | 34 ++++++++++++++++++++--------------\n 1 file changed, 20 insertions(+), 14 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 1a5f1afd32e..842d1fcac29 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -2722,6 +2722,24 @@ static inline void oe_set_tree_depth(struct packing_data *pack,\n \tpack->tree_depth[e - pack->objects] = tree_depth;\n }\n \n+static void record_tree_depth(const struct object_id *oid, const char *name)\n+{\n+\tconst char *p;\n+\tunsigned depth;\n+\tstruct object_entry *ent = packlist_find(&to_pack, oid);\n+\n+\tif (!ent)\n+\t\treturn;\n+\n+\t/* the empty string is a root tree, which is depth 0 */\n+\tdepth = *name ? 1 : 0;\n+\tfor (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))\n+\t\tdepth++;\n+\n+\tif (depth > oe_tree_depth(&to_pack, ent))\n+\t\toe_set_tree_depth(&to_pack, ent, depth);\n+}\n+\n /*\n  * Return the size of the object without doing any delta\n  * reconstruction (so non-deltas are true object sizes, but deltas\n@@ -4375,20 +4393,8 @@ static void show_object(struct object *obj, const char *name,\n \tadd_preferred_base_object(name);\n \tadd_object_entry(&obj->oid, obj->type, name, 0);\n \n-\tif (use_delta_islands) {\n-\t\tconst char *p;\n-\t\tunsigned depth;\n-\t\tstruct object_entry *ent;\n-\n-\t\t/* the empty string is a root tree, which is depth 0 */\n-\t\tdepth = *name ? 1 : 0;\n-\t\tfor (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))\n-\t\t\tdepth++;\n-\n-\t\tent = packlist_find(&to_pack, &obj->oid);\n-\t\tif (ent && depth > oe_tree_depth(&to_pack, ent))\n-\t\t\toe_set_tree_depth(&to_pack, ent, depth);\n-\t}\n+\tif (use_delta_islands && obj->type == OBJ_TREE)\n+\t\trecord_tree_depth(&obj->oid, name);\n }\n \n static void show_object__ma_allow_any(struct object *obj, const char *name, void *data)\n-- \n2.54.0.4.g6aa0d38a4ec\n\n"},{"id":"542636","messageId":"0a1a9ed1e3c8f883587800c232e504937d706bc0.1777853408.git.me@ttaylorr.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"[RFC PATCH 7/7] pack-objects: support `--delta-islands` with `--path-walk`","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2026-05-04T00:11:34Z","receivedAt":"2026-05-04T00:11:37Z","isPatch":true,"body":"Since the inception of `--path-walk`, this option has a documented\nincompatibility with `--delta-islands`.\n\nWhen discussing those original patches on the list, a message from\nStolee in [1] noted the following:\n\n    this could be remedied by [...] doing a separate walk to identify\n    islands using the normal method\n\nIn a related portion of the thread, Peff explains[2]:\n\n    The delta islands code already does its own tree walk to propagate\n    the bits down (it does rely on the base walk's show_commit() to\n    propagate through the commits).\n\n    Once each object has its island bitmaps, I think however you\n    choose to come up with delta candidates [...] you should be able\n    to use it. It's fundamentally just answering the question of \"am\n    I allowed to delta between these two objects\".\n\nThat is similar to what this patch does, and it turns out the cheaper\noption (do the side-effects inside the path-walk callback rather than\nvia a second walk) is sufficient.\n\nRecall how delta-islands are computed during a normal repack:\n\n - `show_commit()` calls `propagate_island_marks()` for each commit,\n   which merges the commit's island bitset onto its root tree object and\n   onto each of its parent commits.\n\n - `show_object()` for a tree records the tree's depth derived from the\n   slash-separated pathname. Subsequent `resolve_tree_islands()` uses\n   that depth to walk trees in increasing-depth order, propagating each\n   tree's marks to its children.\n\n - At delta-search time, `in_same_island()` enforces that a delta\n   target's island bitmap is a subset of its base's: every island\n   that reaches the target must also reach the base.\n\nPath-walk's enumeration callback is `add_objects_by_path()`. It already\nadds objects to `to_pack', but until now did not perform any of the\nisland-related side effects. Two things are needed:\n\n - For each commit batch, call `propagate_island_marks()` on the commit,\n   exactly as show_commit() does.\n\n   Order matters here. `mark_remote_island_1()` only seeds marks on\n   tip commits, so a non-tip commit has marks in the `island_marks` map\n   only after some descendant has already had `propagate_island_marks()`\n   run on it. If we see a commit before its descendants, its\n   `island_marks` entry would still be empty, the call would be a no-op,\n   and that commit's root tree would never receive any marks at all.\n\n   As a consequence, `resolve_tree_islands()` would later look up the\n   tree, find nothing, and propagate nothing. The traversal must visit\n   children before parents.\n\n   The path-walk batch preserves that order mechanically. Path-walk\n   appends commits to its `OBJ_COMMIT` batch as they come back from the\n   same `get_revision()` loop the regular traversal uses, and\n   `add_objects_by_path()` iterates the batch in array order. So every\n   commit reaches `propagate_island_marks()` in the same sequence that\n   `show_commit()` would have seen it, and the descendant-first chain\n   that the algorithm relies on is intact.\n\n   Skip the call for boundary commits to match `show_commit()`, which is\n   only invoked for interesting commits (this call is a no-op anyway for\n   boundary commits since they are not in 'island_marks', but matching\n   `show_commit()` exactly keeps the two enumeration modes tidy).\n\n - For each tree batch, record the tree's depth from the path. Use the\n   `record_tree_depth()` helper from the previous commit so both\n   callbacks behave identically, including the \"max-depth-wins\" behavior\n   when a tree is reached via more than one path. The helper accepts\n   both the show_object() path shape (\"foo\", \"foo/bar\") and the\n   path-walk shape with a trailing '/' (\"foo/\", \"foo/bar/\"), so depths\n   recorded from either traversal mode are directly comparable.\n\n   This is implicit in the implementation sketch from Peff above.\n   `resolve_tree_islands()` sorts trees by `oe->tree_depth` (ascending)\n   before propagating marks down, so that a parent tree's marks are\n   finalized before its children inherit them. Without recording the\n   depth at path-walk time, every path-walk-discovered tree would land\n   at depth 0 in `to_pack`, the sort would lose its ordering, and\n   children could inherit marks from parents whose own contributions had\n   not yet been merged in.\n\nWith those two pieces in place, `resolve_tree_islands()` receives\nidentical input to a normal traversal, so the existing correctness\nargument carries over verbatim: depth-ordered processing guarantees that\na parent tree's marks are propagated to a child only after the parent\nitself has been finalized, and the \"is-this-a-subset\" check at delta\ntime is the same regardless of how the marks got there.\n\nCoverage in t5320 exercises both repack flavors (with and without '-b'),\nconfirms that cross-island deltas remain forbidden, and that\nintra-island deltas are still allowed.\n\n[1]: https://lore.kernel.org/git/9aa2471b-0850-4707-9733-d3b33609f5f2@gmail.com/\n[2]: https://lore.kernel.org/git/20240911063203.GA1538586@coredump.intra.peff.net/\n\nSigned-off-by: Taylor Blau <me@ttaylorr.com>\n---\n Documentation/git-pack-objects.adoc | 15 +++++++--------\n builtin/pack-objects.c              | 22 ++++++++++++++++++----\n t/t5320-delta-islands.sh            | 29 +++++++++++++++++++++++++++++\n 3 files changed, 54 insertions(+), 12 deletions(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex 60e594c7bc4..aa7a9721203 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -402,14 +402,13 @@ will be automatically changed to version `1`.\n \tof filenames that cause collisions in Git's default name-hash\n \talgorithm.\n +\n-Incompatible with `--delta-islands`. Path-walk supports\n-the `--filter=<spec>` forms `blob:none`, `blob:limit=<n>`,\n-`sparse:oid=<blob>`, `tree:0`, `object:type=<type>`, and `combine:`\n-over any of those. Other filter forms fall back to the regular object\n-traversal. When `--use-bitmap-index` is specified with `--path-walk`, a\n-successful bitmap traversal is used for object enumeration, with\n-path-walk remaining as the fallback traversal when the bitmap cannot\n-satisfy the request.\n+Path-walk supports the `--filter=<spec>` forms `blob:none`,\n+`blob:limit=<n>`, `sparse:oid=<blob>`, `tree:0`, `object:type=<type>`,\n+and `combine:` over any of those. Other filter forms fall back to the\n+regular object traversal. When `--use-bitmap-index` is specified with\n+`--path-walk`, a successful bitmap traversal is used for object\n+enumeration, with path-walk remaining as the fallback traversal when\n+the bitmap cannot satisfy the request.\n \n \n DELTA ISLANDS\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 842d1fcac29..d79366db3de 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -4739,13 +4739,29 @@ static int add_objects_by_path(const char *path,\n \n \t\tadd_object_entry(oid, type, path, exclude);\n \n-\t\tif (type == OBJ_COMMIT && write_bitmap_index) {\n+\t\tif (type == OBJ_COMMIT) {\n \t\t\tstruct commit *commit;\n \n+\t\t\tif (!write_bitmap_index && !use_delta_islands)\n+\t\t\t\tcontinue;\n+\n \t\t\tcommit = lookup_commit(the_repository, oid);\n \t\t\tif (!commit)\n \t\t\t\tdie(_(\"could not find commit %s\"), oid_to_hex(oid));\n-\t\t\tindex_commit_for_bitmap(commit);\n+\t\t\tif (write_bitmap_index)\n+\t\t\t\tindex_commit_for_bitmap(commit);\n+\t\t\t/*\n+\t\t\t * Skip island propagation for boundary commits.\n+\t\t\t * The regular traversal's show_commit() is only\n+\t\t\t * called for interesting commits; matching that\n+\t\t\t * here keeps path-walk from doing extra work that\n+\t\t\t * would only be a no-op anyway (boundary commits\n+\t\t\t * are not in island_marks).\n+\t\t\t */\n+\t\t\tif (use_delta_islands && !exclude)\n+\t\t\t\tpropagate_island_marks(the_repository, commit);\n+\t\t} else if (type == OBJ_TREE && use_delta_islands) {\n+\t\t\trecord_tree_depth(oid, path);\n \t\t}\n \t}\n \n@@ -5196,8 +5212,6 @@ int cmd_pack_objects(int argc,\n \t\tconst char *option = NULL;\n \t\tif (!path_walk_filter_compatible(&filter_options))\n \t\t\toption = \"--filter\";\n-\t\telse if (use_delta_islands)\n-\t\t\toption = \"--delta-islands\";\n \n \t\tif (option) {\n \t\t\twarning(_(\"cannot use %s with %s\"),\ndiff --git a/t/t5320-delta-islands.sh b/t/t5320-delta-islands.sh\nindex 2c961c70963..9b28344a0a3 100755\n--- a/t/t5320-delta-islands.sh\n+++ b/t/t5320-delta-islands.sh\n@@ -53,6 +53,35 @@ test_expect_success 'separate islands disallows delta' '\n \t! is_delta_base $two $one\n '\n \n+test_expect_success 'path-walk island repack respects islands' '\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace.path-walk-islands\" \\\n+\t\tgit -c \"pack.island=refs/heads/(.*)\" repack -adfi \\\n+\t\t--path-walk 2>err &&\n+\ttest_region pack-objects path-walk trace.path-walk-islands &&\n+\ttest_grep ! \"cannot use --delta-islands with --path-walk\" err &&\n+\t! is_delta_base $one $two &&\n+\t! is_delta_base $two $one\n+'\n+\n+test_expect_success 'path-walk island bitmap repack respects islands' '\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace.path-walk-island-bitmap\" \\\n+\t\tgit -c \"pack.island=refs/heads/(.*)\" repack -a -d -f -i -b \\\n+\t\t--path-walk 2>err &&\n+\ttest_region pack-objects path-walk trace.path-walk-island-bitmap &&\n+\ttest_path_is_file .git/objects/pack/*.bitmap &&\n+\tgit rev-list --test-bitmap --use-bitmap-index one &&\n+\ttest_grep ! \"cannot use --delta-islands with --path-walk\" err &&\n+\t! is_delta_base $one $two &&\n+\t! is_delta_base $two $one\n+'\n+\n+test_expect_success 'path-walk same island allows delta' '\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace.path-walk-same-island\" \\\n+\t\tgit -c \"pack.island=refs/heads\" repack -adfi --path-walk &&\n+\ttest_region pack-objects path-walk trace.path-walk-same-island &&\n+\tis_delta_base $one $two\n+'\n+\n test_expect_success 'same island allows delta' '\n \tgit -c \"pack.island=refs/heads\" repack -adfi &&\n \tis_delta_base $one $two\n-- \n2.54.0.4.g6aa0d38a4ec\n"},{"id":"542663","messageId":"e70eefe6-9c18-4643-a995-69fe99edd1e3@gmail.com","threadId":"65584","inReplyTo":"cover.1777853408.git.me@ttaylorr.com","subject":"Re: [RFC PATCH 0/7] pack-bitmap: resolve various `--path-walk` incompatibilities","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-05-04T12:13:30Z","receivedAt":"2026-05-04T12:13:33Z","isPatch":true,"body":"On 5/3/2026 8:11 PM, Taylor Blau wrote:\n> (Note to the maintainer, this is built on top of 'ds/path-walk-filters').\n> \n> Between other tasks, I have been working on trying to integrate\n> `--path-walk` within GitHub's infrastructure. In order to do this,\n> `--path-walk` must work with features that GitHub depends on, such as\n> reachability bitmaps and delta-islands (along with filters, shallow,\n> etc., though more on that below).\n> \n> I had been sitting on these patches for a few days in my fork before\n> Stolee sent his series in [1] which resolves incompatibilities between\n> the `--path-walk` option and various filter types. Since I figured that\n> others are working in this area I wanted to send a reworked version of\n> my series for a couple of reasons:\n> \n>  1. Since reviewers are already looking at this area as a consequence of\n>     Stolee's series, this topic should be slightly easier to review\n>     while the area is fresh.\n\nI agree that we should review both series together. It's been a while\nsince the original path-walk API series, so it may require some refresh\nof all its nuances.\n \n>  2. In case Stolee (or others) are working on resolving the\n>     incompatibility between `--path-walk` and either delta-islands or\n>     reachability bitmaps, this series can either combine with those (if\n>     any) or serve as inspiration (if others are in the process of\n>     writing such series).\n\nI was _not_ working on bitmap compatibility, but I'm grateful to see it!\n> When writing this originally, I had borrowed the same filter-application\n> mechanism from bitmaps, which supports trivial filters (e.g., blob:none,\n> tree:0, and combinations therein). Stolee's series is a strict\n> improvement on that approach supporting sparse:<oid> filters as well, so\n> I reworked my filtering-related patches based on that.\n\nYou have some new filters that I had not considered, so they are welcome\nadditions. If you don't mind, I could add them into my series, as they\nmay be more appropriate grouped with the other filter changes. \n> The patches surrounding bitmaps and delta-islands are largely\n> unchanged from when I had originally written them:\n> \n>  * Supporting bitmaps with `--path-walk` is mostly straightforward, and\n>    boils down to ensuring that the path-walk-specific object callback\n>    indexes any commit(s) it sees for bitmapping.\n> \n>  * Supporting delta-islands with `--path-walk` required a bit more\n>    surgery, and involves propagating island marks for commits in the\n>    path-walk-specific callback, as well as recording tree depth\n>    information in the same spot.\n> \n> I'm submitting these patches as an RFC, since (a) I haven't thought\n> deeply about the approach taken here and could very well be on the wrong\n> track, and (b) in case Stolee or others want to combine forces here\n> and/or coordinate around each other.\n\nI'll definitely take a very close read of these patches, as there are\nsome interesting interactions here.\n\nThanks,\n-Stolee\n\n"},{"id":"542664","messageId":"180bf883-2941-424a-a01f-6a75d18823b4@gmail.com","threadId":"65584","inReplyTo":"babe1596161365209c226d374db70a1bdc284a1c.1777853408.git.me@ttaylorr.com","subject":"Re: [RFC PATCH 1/7] pack-objects: update `--path-walk`'s existing incompatibilities","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-05-04T12:22:16Z","receivedAt":"2026-05-04T12:22:18Z","isPatch":true,"body":"On 5/3/2026 8:11 PM, Taylor Blau wrote:\n> The documentation in git-pack-objects(1) claims that `--path-walk` is\n> incompatible with `-shallow`. However, commit c178b02e29f (pack-objects:\n> allow --shallow and --path-walk, 2025-05-16) resolves this\n> incompatibility, leaving the documentation stale.\n> \n> Likewise, this documentation claims that `--filter` is incompatible, but\n> `blob:none`, `blob:limit=<n>`, and `sparse:oid=<blob>` already work via\n> path-walk.\n> \n> List the supported `--filter` forms explicitly and note that other forms\n> fall back to the regular object traversal. Also remove the\n> incompatibility notice with `--shallow`.\n \nThanks for pointing out that I didn't update the docs in my series.\n\nI should incorporate the appropriate language for these changes in my\npatches and give you co-authorship. That will also help avoid using\ncommit references that are impermanent until the series lands.\n\nThanks,\n-Stolee\n\n"},{"id":"542665","messageId":"bfb6d757-7b13-4267-9fd5-8739c7395378@gmail.com","threadId":"65584","inReplyTo":"e1b7fd3cb2a2bba5f6404ac5f8ac3487a46d51b5.1777853408.git.me@ttaylorr.com","subject":"Re: [RFC PATCH 2/7] path-walk: support `tree:0` filter","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-05-04T12:30:23Z","receivedAt":"2026-05-04T12:30:29Z","isPatch":true,"body":"On 5/3/2026 8:11 PM, Taylor Blau wrote:\n> The `tree:0` object filter omits all trees and blobs from the result,\n> keeping only commits and tags. Consequently, this filter type should\n> has a fairly straightforward integration with path-walk, as the decision\n> to include an object depends only on its type and does not depend on any\n> path-sensitive state.\n\nI agree that the implementation here is straight-forward. It's something\nwhere I could easily see wanting to disable the path-walk API because it\nis no longer contributing much value, but perhaps the caller wants a\nconsistent callback that provides all commits and tags in different\nchunks.\n> Non-zero tree-depth filters are not supported. Those depend on the depth\n> at which a tree is visited, which is a path-walk concept the filter\n> machinery doesn't currently share with the path-walk API. Reject them in\n> `prepare_filters()` with a helpful error and let pack-objects fall back\n> to the regular traversal, the same way it already does for unsupported\n> filters.\n\nI think that this could be remedied with some tweaks to the internal\nmethods and data within the path-walk API to track a depth. This could\nbe handled later, if there was enough demand for nonzero tree-depth.\n\nThe diff itself looks good.\n\nThanks,\n-Stolee\n\n"},{"id":"542666","messageId":"f8a376a2-33dc-4e9a-9365-ae453c1452c5@gmail.com","threadId":"65584","inReplyTo":"db46c1248ece57476b369a9bff920facab24be04.1777853408.git.me@ttaylorr.com","subject":"Re: [RFC PATCH 3/7] path-walk: support `object:type` filter","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-05-04T12:32:48Z","receivedAt":"2026-05-04T12:32:51Z","isPatch":true,"body":"On 5/3/2026 8:11 PM, Taylor Blau wrote:\n> The `object:type` filter accepts only objects of a single type; it is\n> the second member of the object-info-only filter family that bitmap\n> traversal already supports.\n\n...\n\n> But there are a couple of side effects of the \"trees off, blobs on\" case\n> that need fixing:\n> \n>  1. 'setup_pending_objects()' previously skipped pending trees as soon\n>     as `info->trees` was zero. For 'object:type=blob' the call site\n>     needs those pending trees: a lightweight tag pointing to a tree, or\n>     an annotated tag whose peeled target is a tree, can both reach\n>     blobs that are otherwise unreachable from any commit's root tree.\n>     Loosen the gate to \"if (!info->trees && !info->blobs) continue\" and\n>     similarly retrieve the root_tree_list whenever either trees or\n>     blobs are wanted.\n> \n>  2. The revision machinery's `handle_commit()` drops pending trees when\n>     `revs->tree_objects` is zero (see the 'OBJ_TREE' handler in\n>     revision.c), so by the time path-walk sees the pending list\n>     after `prepare_revision_walk()` the tree-bearing pendings would\n>     already be gone. Fix this by setting\n> \n>         revs->tree_objects = info->trees || info->blobs\n> \n>     so pending trees survive `prepare_revision_walk()` whenever we\n>     need to walk into them. Path-walk still resets tree_objects to\n>     zero immediately after `prepare_revision_walk()` returns, so the\n>     rev-walk itself never enumerates trees redundantly with\n>     path-walk's own descent.\n\nBoth of these changes are very valuable bug fixes for the path-walk API!\nThanks for catching the distinction here where we should still be\nwalking trees in order to find the blobs we want.\n\nThanks,\n-Stolee\n\n"},{"id":"542731","messageId":"0b7fbb5f-bfab-44c4-8bb6-11d6f9978779@app.fastmail.com","threadId":"65584","inReplyTo":"e1b7fd3cb2a2bba5f6404ac5f8ac3487a46d51b5.1777853408.git.me@ttaylorr.com","subject":"Re: [RFC PATCH 2/7] path-walk: support `tree:0` filter","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2026-05-04T21:55:56Z","receivedAt":"2026-05-04T21:56:18Z","isPatch":true,"body":"On Mon, May 4, 2026, at 02:11, Taylor Blau wrote:\n> The `tree:0` object filter omits all trees and blobs from the result,\n> keeping only commits and tags. Consequently, this filter type should\n> has a fairly straightforward integration with path-walk, as the decision\n\ns/has a/have a/\n\n> to include an object depends only on its type and does not depend on any\n> path-sensitive state.\n>\n>[snip]\n"}]}