{"thread":{"id":"66326","subject":"[PATCH 0/2] connected: add incremental connectivity check","startedAt":"2026-09-14T09:47:59Z","lastAt":"2026-10-23T18:52:09Z","messageCount":60,"participants":["Kristofer Karlsson via GitGitGadget","Junio C Hamano","Kristofer Karlsson","Patrick Steinhardt"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"552689","messageId":"pull.2211.git.1789379276.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":null,"subject":"[PATCH 0/2] connected: add incremental connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-14T09:47:54Z","receivedAt":"2026-09-14T09:47:59Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"This series adds an incremental mode for the connectivity check, gated\nbehind transfer.connectivityCheck=incremental (no expected changes unless\nyou opt in).\n\nThe intent is to solve the problem of the connectivity check slowing down as\nthe number of reachable objects from the boundary grows.\n\nIt relates to the RFC I sent out earlier:\n\n[RFC] check_connected: toward incoming-proportional cost\nhttps://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/\n\n\n\nDesign\n======\n\nThe verifier runs inside the same rev-list subprocess that check_connected()\nalready spawns, triggered by a new internal flag --verify-trees-incremental.\nAfter get_revision() collects the incoming commits, the verifier processes\nthem in topological order (ancestors before descendants).\n\nThe idea is to keep a set of trusted objects, shared across the incoming\ncommits, that grows over time. We visit the new/untrusted commit trees and\ndo a comparison walk over the trees of their parents. Entries discovered on\nthe trusted parent side are remembered as trusted, which lets later\nverification skip matching objects and avoid descending into unchanged\nsubtrees.\n\nMore algorithmic details are in\nDocumentation/technical/connectivity-check.adoc.\n\n\nBenchmarks\n==========\n\nI'll just mention a short summary here, to avoid repeating what's already in\nthe commit message. The incremental mode is faster than the full mode when\nthere are few commits to verify and when the active object tree is large. In\nthe happy case, the work tracks the changed paths and their comparison trees\nrather than the full reachable object closure, which substantially reduces\nthe dependence on total repository size. I've seen speedups up to around 20x\nfor the synthetic perf tests.\n\nRunning against a large real-world repo (3.4 GB boundary closure), the\nnumbers are more dramatic. All timings use rev-list directly with --not\nHEADN, isolating the tree verification cost from the boundary-finding cost:\n\ncommits    full  incr.  speedup   full RSS  incr. RSS\n      1    1.9s  0.01s    190x      3.4 GB     14 MB\n     10    1.9s  0.04s     48x      3.4 GB    125 MB\n    100    1.9s  0.37s      5x      3.4 GB    1.1 GB\n\n\nThe full mode takes ~1.9s regardless of commit count because it is dominated\nby walking the boundary closure. Incremental scales with the number of\nincoming commits and the paths they touch. Memory follows the same pattern:\nincremental uses a fraction of the full mode's RSS for small pushes,\nconverging only when many commits are verified.\n\nThere are also regression cases in the synthetic fixtures. With long\nincoming histories, the extra parent-tree scans accumulate; in the synthetic\nfixture incremental is about 1.5x slower at 10000 commits. Per-commit\nchanges have less impact than expected: even when every directory is\ntouched, incremental remains competitive; bypassing the object cache for\ntree reads likely helps here.\n\nI cannot establish how common these regression cases are. In the cases I\nhave tested, however, the regression has remained modest; I have not been\nable to provoke a substantially larger slowdown. My feeling is that this is\nan acceptable tradeoff behind the opt-in config, since the target case\n(small pushes to large repos) sees the largest speedup, while the regression\nappears with long incoming histories.\n\nA safety net for this regression could be to dynamically disable the\nincremental mode if the number of incoming commits is too large, but this is\nleft out of the initial version to avoid overly speculative code.\n\nDeepening fetches currently fall back to the full check because the full\ncheck omits --not --all for deepening -- there is no existing-reference\nboundary at which the walk can stop. An incremental approach is possible\nhere too -- using the old shallow roots as the trusted boundary and walking\nthe deepened ancestry forward -- but that is a separate change and left for\nfuture work. Deepening is also less common than regular fetch and\nreceive-pack, where the speedup matters most.\n\n\nTest coverage\n=============\n\nMost correctness cases in t5412-connectivity-check.sh are run in both full\nand incremental modes to check semantic equivalence. Selected cases\nadditionally assert trace2 tree/blob counts for the incremental mode, to\nverify that unchanged portions of the object graph are actually skipped. It\ncovers:\n\n * Corruption detection: missing blobs, missing trees, type mismatches,\n   malformed trees (unparseable, mid-tree corruption)\n * Tree optimization: trace2 assertions confirm unchanged subtrees are\n   skipped, subtree moves, merge parent boundaries\n * Root commits (no parents -- verifies full tree closure)\n * Partial clones: missing promised blobs, missing promised trees,\n   verification of local commits\n * Replacement objects (with and without GIT_NO_REPLACE_OBJECTS)\n * Shallow boundaries\n * Deepening fetches (falls back to full check)\n * Integration: real push, fetch, and clone\n\nMost tests call git rev-list directly with the appropriate flags;\nintegration tests exercise the full check_connected() path through push,\nfetch, and clone.\n\n\nAlternatives considered\n=======================\n\nMy first prototype ran the verifier in-process inside connected.c. This\nrequired a second rev-list subprocess just for boundary finding, _nofetch\nvariants of several object-reading functions to prevent lazy fetches in\npartial clones, explicit shallow-file plumbing, and careful avoidance of\ndie() in all code paths reachable from the verifier. The result worked but\nwas fragile and touched many files.\n\nMoving the verifier into the rev-list subprocess eliminated all of those\nproblems: in partial clones the existing --exclude-promisor-objects handling\nalready disables lazy fetching, die() is isolated by the process boundary,\nshallow and replacement semantics are established before the verifier runs,\nand error routing comes for free via stderr.\n\nSo while I liked the idea of being less reliant on checking within a\nsubprocess, making that work ended up being a lot more complex.\n\n\nNext steps\n==========\n\nThis series only addresses tree verification; the other significant cost is\nfinding the commit boundary, especially for repos with many refs. I already\nhave some prototypes for optimizing that too, and if this ends up landing,\nthat would be something I would start polishing up.\n\nThanks, Kristofer\n\nKristofer Karlsson (2):\n  Documentation: describe connectivity checking\n  connected: add incremental connectivity check via rev-list\n\n Documentation/config/transfer.adoc            |  20 +\n Documentation/rev-list-options.adoc           |   6 +\n .../technical/connectivity-check.adoc         | 243 +++++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  18 +\n connected.c                                   |  24 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5412-connectivity-check.perl |  44 ++\n t/perf/p5412-connectivity-check.sh            |  92 +++\n t/t5412-connectivity-check.sh                 | 655 ++++++++++++++++++\n tree-verify.c                                 | 306 ++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1426 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n create mode 100644 t/perf/generate-repo-p5412-connectivity-check.perl\n create mode 100755 t/perf/p5412-connectivity-check.sh\n create mode 100755 t/t5412-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\n\nbase-commit: 47ce80527c56f462cb97db4ca8125342204d3783\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2211%2Fspkrka%2Ftree-diff-connectivity-v1-clean-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2211/spkrka/tree-diff-connectivity-v1-clean-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2211\n-- \ngitgitgadget\n"},{"id":"552690","messageId":"e55c5452db0b7cb683d4e2ad51cd8f44046c23bd.1789379276.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH 1/2] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-14T09:47:55Z","receivedAt":"2026-09-14T09:48:00Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd Documentation/technical/connectivity-check.adoc describing\nthe connectivity invariant and the full connectivity check.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n .../technical/connectivity-check.adoc         | 109 ++++++++++++++++++\n 1 file changed, 109 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nnew file mode 100644\nindex 0000000000..d20bff6af6\n--- /dev/null\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -0,0 +1,109 @@\n+Connectivity checking\n+=====================\n+\n+After receiving new objects via fetch, push (receive-pack), clone,\n+or bundle, Git verifies that the new reference tips do not leave\n+the repository in a state where reachable objects are missing.\n+This verification is called the connectivity check.\n+\n+Connectivity invariant\n+----------------------\n+\n+A repository is connected when every object reachable from its\n+references is available locally (with exceptions noted below).\n+\n+The connectivity check maintains this invariant when references\n+are updated.  It trusts the existing connected state and verifies\n+that the new reference tips do not introduce references to\n+unavailable objects.  Verification is permitted to stop when it\n+reaches objects already reachable from trusted existing\n+references, since their closure is already connected.  These\n+trusted references include local references and references from\n+alternate object stores.\n+\n+Without this check, a truncated or corrupted transfer could leave\n+a repository in a state where later history walks encounter\n+missing objects.\n+\n+Exceptions\n+~~~~~~~~~~\n+\n+Gitlink entries (submodule references) are excluded from\n+connectivity checking.  Their target objects belong to a separate\n+repository.\n+\n+In partial clones, objects promised by a promisor remote are\n+accepted as connected without requiring local existence.  The\n+check excludes promisor objects from traversal so that it does\n+not trigger on-demand fetches for them.\n+\n+Full connectivity check\n+-----------------------\n+\n+`check_connected()` (see `connected.c`) normally performs the\n+connectivity check using a `rev-list` subprocess, feeding the\n+new reference tips via stdin.  A normal invocation is roughly:\n+\n+    git rev-list --objects --stdin --not --all --quiet\n+        --alternate-refs [--exclude-promisor-objects]\n+\n+When promisor remotes are configured, `check_connected()` first\n+attempts a fast path based on promisor packfiles.  If it falls\n+back to the `rev-list` check, `--exclude-promisor-objects` is\n+added so that the traversal does not trigger on-demand fetches.\n+\n+Consider the following graph after a fetch, where all reference\n+tips point directly to commits.  For simplicity, only local\n+references appear on the already-connected side; alternate refs\n+play the same role.  N3 is a merge commit:\n+\n+            /-------------L2\n+           /\n+    C1---B1---C2---B2-----L1\n+          \\         \\\n+           N1        N3---T2\n+            \\       /\n+             N2-----------T1\n+\n+    L1, L2:         local refs\n+    T1, T2:         incoming tips (new refs)\n+    N1, N2, N3:     incoming commits (N3 is a merge)\n+    B1, B2:         boundary commits (already connected)\n+    C1, C2:         already connected (but not boundary)\n+\n+The incoming set is the commits reachable from the incoming\n+tips but not from the already-connected side.  Boundary commits\n+are the already-connected commits at the edge of that set.  Here\n+B1 is an ancestor of B2, which happens when incoming branches\n+fork at different depths in the existing history.\n+\n+The check proceeds in three phases:\n+\n+1. Walk from the incoming tips (T1, T2) against the trusted\n+   refs (L1, L2) to find the incoming set ({N1, N2, N3, T1, T2}).\n+\n+2. Walk the trees of the boundary commits (B1, B2) and mark\n+   those objects uninteresting.  These trees are already trusted\n+   because their commits are on the already-connected side.\n+\n+3. Walk the trees of each incoming commit and verify that every\n+   referenced object is connected, stopping at objects already\n+   marked uninteresting in phase 2.\n+\n+Deepening fetches\n+~~~~~~~~~~~~~~~~~\n+\n+For deepening fetches (where the shallow boundary moves), the\n+full check omits `--not --all`.  There is no existing-reference\n+boundary at which the walk can stop.  Instead, traversal follows\n+the effective shallow boundary supplied for the deepened\n+repository.  The new content may be below the old shallow\n+boundary even when the tips themselves have not changed.\n+\n+Non-commit tips\n+~~~~~~~~~~~~~~~\n+\n+When a new reference points to a non-commit object, such as a\n+tag, tree, or blob, that object is not part of the commit walk.\n+These non-commit tips are handled by the subsequent object\n+traversal.\n-- \ngitgitgadget\n\n"},{"id":"552691","messageId":"ebe6c90cc58b9e1f64c9bec4a18e8cb3ce9be1b2.1789379276.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH 2/2] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-14T09:47:56Z","receivedAt":"2026-09-14T09:48:03Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe full connectivity check uses rev-list to find commits\nreachable from the incoming tips but not from the\nalready-connected side, then walks their object closure.  Commit\ntraversal stops at the connectivity boundary, but trees and blobs\nreachable from that boundary still need to be walked so they can\nbe marked uninteresting, allocating a struct object for each one.\nOn repositories where the boundary commits have large trees, the\nconnectivity check for small incoming changes visits and tracks\nmore objects than needed.\n\nAdd an alternative connectivity check that verifies incoming\ncommits incrementally against their parents.\n\nThe verifier processes incoming commits with ancestors first and\ncompares each new tree against its trusted parent trees.  Entries\nalready seen on the trusted side are skipped by OID, so unchanged\nsubtrees need not remain at the same path to be recognized.\nChanged subtrees are recursively compared against same-path parent\nsubtrees, and new subtrees without a comparison base are verified\nfrom scratch.\n\nSee Documentation/technical/connectivity-check.adoc for the trust\ninvariants and detailed algorithm.\n\nThe incremental check runs as an internal\n--verify-trees-incremental mode in rev-list.  After the normal\nrevision walk identifies the incoming commits, the verifier\nconsumes them before the usual object traversal; any pending\nnon-commit tips are still handled by the existing traversal.\n\nPartial-clone semantics are preserved: objects promised by a\npromisor remote are accepted as connected, and on-demand fetching\nis prevented by excluding promisor objects from traversal.  The\nnew mode is selected by transfer.connectivityCheck=incremental,\nwith full remaining the default.  Deepening fetches fall back to\nthe full check because they do not have the normal\nexisting-reference boundary, and traversal instead follows the\neffective shallow boundary.\n\np5412 results (median of 3), scaling one dimension at a time.\nEach fixture is a repository with a flat tree of many\ndirectories containing 100 files each.  The incoming set is a\nchain of commits on top of the existing history, with each\ncommit changing files in different directories round-robin.\nThe three axes vary the total repository tree size, the number\nof incoming commits, and the number of files changed per\nincoming commit.\n\nScaling tree size (10 incoming commits, 10 files/commit):\n\n    files    full  incr.  full/incr\n      5K    0.01s  0.01s    1.0x\n     50K    0.04s  0.01s    4.0x\n    200K    0.15s  0.01s   15.0x\n    800K    0.61s  0.03s   20.3x\n\nThe full mode must walk the trees of boundary commits, which\ngrows with overall tree size.  Incremental still scans the\nroot trees, but avoids descending into unchanged subtrees,\nso it grows much more slowly with repository size.\n\nScaling incoming commit count (200K files, 10 files/commit):\n\n    commits    full  incr.  full/incr\n          1    0.14s  0.01s   14.0x\n         10    0.15s  0.01s   15.0x\n        100    0.19s  0.05s    3.8x\n        500    0.32s  0.27s    1.2x\n       3000    1.23s  1.52s    0.8x\n       5000    1.88s  2.43s    0.8x\n      10000    3.72s  5.36s    0.7x\n\nWith many commits the per-commit overhead of scanning both\nthe new and parent root trees accumulates and incremental\nbecomes slower.  Breakeven is between 500 and 3000 commits\nand the ratio stabilizes near 0.7x for this fixture.\n\nScaling files per incoming commit (200K files, 10 commits):\n\n    files/commit    full  incr.  full/incr\n               1    0.14s  0.01s   14.0x\n              10    0.15s  0.01s   15.0x\n             100    0.16s  0.04s    4.0x\n             500    0.23s  0.14s    1.6x\n            1000    0.34s  0.28s    1.2x\n            2000    0.70s  0.63s    1.1x\n\nBreakeven is around 1000 files/commit.  At 2000 files/commit\n(every directory touched), incremental remains slightly faster;\nbypassing the object cache for tree reads likely helps here.\n\nFor small repositories both modes are fast enough that the\ndifference is difficult to measure reliably.\n\nSelected peak RSS measurements from the same fixtures:\n\n    case                        full    incr.  ratio\n    200K files, 10 commits      31 MB    12 MB   0.4x\n    800K files, 10 commits     110 MB    26 MB   0.2x\n    200K files, 5000 commits   155 MB   151 MB   1.0x\n\nIncremental uses substantially less memory when it can prune\nmost of the boundary tree walk.  In the long-history case the\ntwo modes visit similar object sets and memory converges.\n\nThe regression cases in the CPU benchmarks are in wall-clock\ntime rather than memory, primarily from scanning some trees\nmore than once.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc            |  20 +\n Documentation/rev-list-options.adoc           |   6 +\n .../technical/connectivity-check.adoc         | 134 ++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  18 +\n connected.c                                   |  24 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5412-connectivity-check.perl |  44 ++\n t/perf/p5412-connectivity-check.sh            |  92 +++\n t/t5412-connectivity-check.sh                 | 655 ++++++++++++++++++\n tree-verify.c                                 | 306 ++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1317 insertions(+)\n create mode 100644 t/perf/generate-repo-p5412-connectivity-check.perl\n create mode 100755 t/perf/p5412-connectivity-check.sh\n create mode 100755 t/t5412-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex f1ce50f4a6..b9939d3bde 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -1,3 +1,23 @@\n+transfer.connectivityCheck::\n+\tChoose which algorithm to use for the connectivity check\n+\tperformed during object transfer operations such as\n+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n+\tThe connectivity check verifies that all objects reachable\n+\tfrom the incoming tips are available locally or, in a partial\n+\tclone, promised by a promisor remote.\n+\tThe variants are as follows:\n++\n+--\n+`full` (default);;\n+\tWalk the full object closure of the boundary commits.\n+`incremental`;;\n+\tVerify incoming commits by diffing their trees against parent\n+\ttrees, recursively descending only into entries that differ.\n+\tThe largest benefits occur when incoming commits change a\n+\tsmall fraction of a large tree closure.\n+\tFalls back to `full` for deepening fetches.\n+--\n+\n transfer.credentialsInUrl::\n \tA configured URL can contain plaintext credentials in the form\n \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex fd831f0ec6..7964b691c4 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1089,6 +1089,12 @@ we cannot get their Object ID though, an error will be raised.\n \tstronger than `--missing=allow-promisor` because it limits the\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n+\n+`--verify-trees-incremental`::\n+\t(For internal use only.)  Verify tree connectivity\n+\tincrementally by comparing each commit's tree against its\n+\tparent trees.  Used by `check_connected()` when\n+\t`transfer.connectivityCheck` is set to `incremental`.\n endif::git-rev-list[]\n \n `--no-walk[=(sorted|unsorted)]`::\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nindex d20bff6af6..0a8f370546 100644\n--- a/Documentation/technical/connectivity-check.adoc\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -107,3 +107,137 @@ When a new reference points to a non-commit object, such as a\n tag, tree, or blob, that object is not part of the commit walk.\n These non-commit tips are handled by the subsequent object\n traversal.\n+\n+Incremental connectivity check\n+------------------------------\n+\n+The incremental mode, selected by\n+`transfer.connectivityCheck=incremental`, avoids traversing the\n+full tree walk of the boundary commits.  Instead, it verifies\n+each incoming commit's tree against the already-trusted trees of\n+its parents.\n+\n+Trust model\n+~~~~~~~~~~~\n+\n+A tree is trusted when its transitive object closure is known to\n+be connected.  Trees reachable from commits on the\n+already-connected side of the boundary are therefore trusted.\n+\n+Incoming commits are processed with ancestors before descendants.\n+Once an incoming commit's tree has been verified, it is trusted\n+and can be used as a comparison base for later descendants.\n+\n+This gives an inductive correctness argument: every parent of the\n+commit currently being verified is either already connected or is\n+an earlier incoming commit whose tree has already been verified.\n+\n+Tree states\n+~~~~~~~~~~~\n+\n+The verifier tracks tree OIDs in three states:\n+\n+untrusted::\n+\tThe tree has not yet been established as connected.  This\n+\tis the implicit state of an OID not present in the state\n+\tmap.\n+\n+trusted::\n+\tThe tree is known to have a connected transitive closure,\n+\tbut its direct entries have not yet been published into the\n+\tverifier's trusted object sets.\n+\n+expanded::\n+\tThe tree is trusted and its direct non-gitlink entries\n+\thave also been published into the trusted object sets.\n+\n+State transitions are monotonic: a tree may move from untrusted\n+to trusted to expanded, but never backwards.  An untrusted tree\n+that is successfully verified goes directly to expanded.\n+\n+Blobs require only trusted/untrusted state: a blob becomes\n+trusted when it is found in a trusted tree or when its existence\n+and type have been verified directly.\n+\n+The trusted/expanded distinction is an optimization.  An expanded\n+parent does not need to be reread merely to publish entries that\n+are already trusted, though it may still be read when same-path\n+subtree bases are needed for recursive comparison.\n+\n+Algorithm\n+~~~~~~~~~\n+\n+At a high level:\n+\n+    verify(commits):\n+        sort topologically (ancestors first)\n+        for each commit:\n+            mark parent root trees as trusted\n+            verify_tree(commit.tree, parent root trees)\n+\n+    verify_tree(tree, base_trees):\n+        if tree already trusted: return\n+        read tree, collect entries not already trusted\n+        for each available base tree:\n+            publish its entries as trusted\n+            record same-path subtrees as bases\n+        for each collected entry:\n+            if now trusted: skip\n+            if blob: verify blob connectivity and type\n+            if tree: verify_tree(entry, its recorded bases)\n+        mark tree as expanded\n+\n+The important ordering within `verify_tree` is that entries from\n+the new tree are collected before the base trees are scanned, but\n+are processed only afterwards.  Trust learned from any base can\n+therefore eliminate work before recursive verification begins.\n+\n+Same-path parent subtrees are passed down as comparison bases\n+during recursive descent.  If no comparison base is available,\n+the new subtree is verified from scratch.\n+\n+Worked example\n+~~~~~~~~~~~~~~\n+\n+Consider a commit that changes one file under `lib/` and moves an\n+unchanged subtree from `src/` to `dev/`:\n+\n+    Parent tree              New tree\n+    +-- src/   (aaa)         +-- dev/   (aaa)\n+    +-- lib/   (bbb)         +-- lib/   (ccc)\n+         +-- foo.c (ddd)          +-- foo.c (ddd)\n+         +-- bar.c (eee)          +-- bar.c (fff)\n+\n+Scanning the parent makes `aaa` trusted even though it moved from\n+`src/` to `dev/`, so that subtree is skipped.  The changed `ccc`\n+subtree is compared against its same-path parent `bbb`; scanning\n+`bbb` makes `ddd` and `eee` trusted, leaving only the new `fff`\n+blob to be checked.\n+\n+This illustrates two properties:\n+\n+* Trust is OID-based rather than path-based.  An unchanged subtree\n+  is recognized after a move.\n+\n+* Same-path parent subtrees provide recursive comparison bases.\n+  These bases improve pruning efficiency but are not required for\n+  correctness; a new subtree can always be verified from scratch.\n+\n+Multiple parents\n+~~~~~~~~~~~~~~~~\n+\n+For a merge commit, root trees from all parents are comparison\n+bases.  When several parents contain a same-path subtree, each\n+matching subtree is collected as a recursive comparison base.\n+\n+Entries published from any trusted parent become globally trusted,\n+so a matching tree or blob entry present in any parent can be\n+skipped while verifying the merge tree.\n+\n+Missing comparison bases\n+~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+A promised base tree may not be locally available for comparison.\n+In that case the verifier skips that base and verifies the new\n+subtree without it.  The missing comparison can reduce pruning but\n+does not affect correctness.\ndiff --git a/Makefile b/Makefile\nindex d4b775953d..aa5b4e2b84 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1357,6 +1357,7 @@ LIB_OBJS += trailer.o\n LIB_OBJS += transport-helper.o\n LIB_OBJS += transport.o\n LIB_OBJS += tree-diff.o\n+LIB_OBJS += tree-verify.o\n LIB_OBJS += tree-walk.o\n LIB_OBJS += tree.o\n LIB_OBJS += unpack-trees.o\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 6b596231ab..7741fdec9f 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -28,6 +28,7 @@\n #include \"commit-reach.h\"\n #include \"quote.h\"\n #include \"strbuf.h\"\n+#include \"tree-verify.h\"\n \n struct rev_list_info {\n \tstruct rev_info *revs;\n@@ -706,6 +707,7 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n+\tint verify_trees_incremental = 0;\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -748,6 +750,8 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n+\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n+\t\t\tverify_trees_incremental = 1;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n \t\t} else if (!strcmp(arg, \"-z\")) {\n@@ -822,6 +826,8 @@ int cmd_rev_list(int argc,\n \n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n \t\t\tcontinue; /* already handled above */\n+\t\tif (!strcmp(arg, \"--verify-trees-incremental\"))\n+\t\t\tcontinue; /* already handled above */\n \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n \t\t\tcontinue; /* already handled above */\n \n@@ -935,6 +941,18 @@ int cmd_rev_list(int argc,\n \n \tprepare_maximal_independent(&revs);\n \n+\tif (verify_trees_incremental) {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *new_commits = NULL;\n+\n+\t\twhile ((commit = get_revision(&revs)) != NULL)\n+\t\t\tcommit_list_insert(commit, &new_commits);\n+\n+\t\tverify_commits_incremental(repo, &new_commits,\n+\t\t\t\t\t   revs.exclude_promisor_objects);\n+\t\tcommit_list_free(new_commits);\n+\t}\n+\n \tif (revs.tree_objects)\n \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n \ndiff --git a/connected.c b/connected.c\nindex 929b9bd28d..e3dcd8e2b0 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"git-compat-util.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n@@ -67,6 +68,26 @@ static int check_connected_promisor(oid_iterate_fn fn,\n \treturn 1;\n }\n \n+static int incremental_check_applicable(struct check_connected_options *opt)\n+{\n+\tconst char *algorithm = NULL;\n+\n+\tif (repo_config_get_string_tmp(the_repository,\n+\t\t\t\t       \"transfer.connectivitycheck\",\n+\t\t\t\t       &algorithm))\n+\t\treturn 0;\n+\tif (!strcasecmp(algorithm, \"full\"))\n+\t\treturn 0;\n+\tif (strcasecmp(algorithm, \"incremental\"))\n+\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n+\t\t    algorithm);\n+\n+\tif (opt->is_deepening_fetch)\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n /*\n  * If we feed all the commits we want to verify to this command\n  *\n@@ -133,6 +154,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n+\tif (incremental_check_applicable(opt))\n+\t\tstrvec_push(&rev_list.args,\n+\t\t\t    \"--verify-trees-incremental\");\n \n \trev_list.git_cmd = 1;\n \tif (opt->env)\ndiff --git a/meson.build b/meson.build\nindex 0a95d90d21..d94dedc26c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -562,6 +562,7 @@ libgit_sources = [\n   'transport-helper.c',\n   'transport.c',\n   'tree-diff.c',\n+  'tree-verify.c',\n   'tree-walk.c',\n   'tree.c',\n   'unpack-trees.c',\ndiff --git a/t/meson.build b/t/meson.build\nindex 7f53cca7d1..19a8246c44 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -652,6 +652,7 @@ integration_tests = [\n   't5409-colorize-remote-messages.sh',\n   't5410-receive-pack.sh',\n   't5411-proc-receive-hook.sh',\n+  't5412-connectivity-check.sh',\n   't5500-fetch-pack.sh',\n   't5501-fetch-push-alternates.sh',\n   't5502-quickfetch.sh',\ndiff --git a/t/perf/generate-repo-p5412-connectivity-check.perl b/t/perf/generate-repo-p5412-connectivity-check.perl\nnew file mode 100644\nindex 0000000000..9546996280\n--- /dev/null\n+++ b/t/perf/generate-repo-p5412-connectivity-check.perl\n@@ -0,0 +1,44 @@\n+#!/usr/bin/perl\n+#\n+# Generate a fast-import stream for p5412 connectivity check benchmarks.\n+#\n+# Usage: generate-repo-p5412-connectivity-check.perl\n+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n+#\n+# Creates one initial commit with dirs*files_per_dir files, then\n+# <commits> additional commits each modifying <files_per_commit>\n+# files in directories chosen round-robin from 1..<hot_dirs>.\n+\n+use strict;\n+use warnings;\n+\n+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n+$hot = $nd if !$hot || $hot > $nd;\n+$fpc = 1   if !$fpc;\n+\n+sub data {\n+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n+}\n+\n+# Initial tree: one commit with nd*nf files.\n+printf \"commit refs/heads/main\\n\";\n+printf \"committer perf <perf\\@test.com> now\\n\";\n+data(\"initial\");\n+for my $d (1..$nd) {\n+\tfor my $f (1..$nf) {\n+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n+\t}\n+}\n+\n+# Subsequent commits (auto-chained by fast-import).\n+for my $i (1..$nc) {\n+\tprintf \"commit refs/heads/main\\n\";\n+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n+\tdata(sprintf \"change-%03d\", $i);\n+\tfor my $j (0..$fpc-1) {\n+\t\tmy $d = (($i + $j) % $hot) + 1;\n+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n+\t}\n+}\ndiff --git a/t/perf/p5412-connectivity-check.sh b/t/perf/p5412-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..827643a847\n--- /dev/null\n+++ b/t/perf/p5412-connectivity-check.sh\n@@ -0,0 +1,92 @@\n+#!/bin/sh\n+\n+test_description='performance of connectivity check modes\n+\n+Compare the default and incremental rev-list connectivity modes\n+directly, avoiding pack transfer noise.\n+\n+Each repository has a flat tree of many directories with 100 files\n+in each.  Three axes are scaled independently: tree size, commit\n+count, and files changed per commit.'\n+\n+. ./perf-lib.sh\n+\n+test_perf_fresh_repo\n+\n+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5412-connectivity-check.perl\"\n+\n+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n+# $5=files_per_commit (optional, default=1)\n+test_perf_conn () {\n+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n+\tlocal total=$(($nd * $nf))\n+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n+\tlocal label=\"${total} files, ${nc} commits\"\n+\tif test \"$hot\" -lt \"$nd\"\n+\tthen\n+\t\tlabel=\"$label (${hot} hot dirs)\"\n+\tfi\n+\tif test \"$fpc\" -gt 1\n+\tthen\n+\t\tlabel=\"$label (${fpc} files/commit)\"\n+\tfi\n+\n+\ttest_expect_success \"setup $label\" '\n+\t\tif test -d '\"$name\"'\n+\t\tthen\n+\t\t\ttrue\n+\t\telse\n+\t\t\tgit init '\"$name\"' &&\n+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n+\t\t\t(\n+\t\t\t\tcd '\"$name\"' &&\n+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n+\t\t\t\tgit update-ref refs/heads/main \\\n+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n+\t\t\t\tgit repack -ad &&\n+\t\t\t\tgit config gc.auto 0\n+\t\t\t)\n+\t\tfi\n+\t'\n+\n+\ttest_perf \"$label (full)\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+\n+\ttest_perf \"$label (incremental)\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list --verify-trees-incremental \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+}\n+\n+# Scaling tree size (10 commits, 10 files/commit).\n+test_perf_conn   50 100 10   50 10\n+test_perf_conn  500 100 10  500 10\n+test_perf_conn 2000 100 10 2000 10\n+test_perf_conn 8000 100 10 8000 10\n+\n+# Scaling commit count (200K files, 10 files/commit).\n+test_perf_conn 2000 100    1 2000 10\n+test_perf_conn 2000 100   10 2000 10\n+test_perf_conn 2000 100  100 2000 10\n+test_perf_conn 2000 100   500 2000 10\n+test_perf_conn 2000 100  3000 2000 10\n+test_perf_conn 2000 100  5000 2000 10\n+test_perf_conn 2000 100 10000 2000 10\n+\n+# Scaling files per commit (200K files, 10 commits).\n+test_perf_conn 2000 100 10 2000   1\n+test_perf_conn 2000 100 10 2000  10\n+test_perf_conn 2000 100 10 2000  100\n+test_perf_conn 2000 100 10 2000  500\n+test_perf_conn 2000 100 10 2000 1000\n+test_perf_conn 2000 100 10 2000 2000\n+\n+test_done\ndiff --git a/t/t5412-connectivity-check.sh b/t/t5412-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..e276085e83\n--- /dev/null\n+++ b/t/t5412-connectivity-check.sh\n@@ -0,0 +1,655 @@\n+#!/bin/sh\n+\n+test_description='connectivity check (transfer.connectivityCheck)'\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_oid_cache <<-\\EOF\n+missing sha1:0000000000000000000000000000000000000001\n+missing sha256:0000000000000000000000000000000000000000000000000000000000000001\n+EOF\n+\n+set_connectivity_check () {\n+\tgit -C \"$1\" config transfer.connectivityCheck \"$2\"\n+}\n+\n+# Run a connectivity check via rev-list directly.  Uses $mode\n+# (set by the enclosing for-loop) to choose full or incremental.\n+check_connected () {\n+\tflags= &&\n+\tif test \"$mode\" = incremental\n+\tthen\n+\t\tflags=--verify-trees-incremental\n+\tfi &&\n+\tprintf '%s\\n' \"$@\" |\n+\tgit rev-list $flags \\\n+\t\t--objects --stdin --not --all --quiet \\\n+\t\t--exclude-promisor-objects\n+}\n+\n+# Run git with a temporary index, leaving the real index untouched.\n+tmpgit () {\n+\tGIT_INDEX_FILE=.git/tmp-idx git \"$@\"\n+}\n+\n+# Create a commit with one file changed, without modifying HEAD,\n+# index, or worktree.  Prints the new commit OID on stdout.\n+# Usage: commit_with_change <parent> <path> <content>\n+commit_with_change () {\n+\tnew_blob=$(echo \"$3\" | git hash-object -w --stdin) &&\n+\ttmpgit read-tree \"$1\" &&\n+\ttmpgit update-index --replace \\\n+\t\t--cacheinfo \"100644,$new_blob,$2\" &&\n+\tnew_tree=$(tmpgit write-tree) &&\n+\trm -f .git/tmp-idx &&\n+\tgit commit-tree \"$new_tree\" -p \"$1\" -m \"modify $2\"\n+}\n+\n+# Check OIDs and optionally verify trace2 counts.\n+# Usage: check_connected_trace <trace-file> <trees> <blobs> <oid>...\n+# An empty string for <trees> or <blobs> skips that assertion.\n+check_connected_trace () {\n+\ttrace_file=$1 trees=$2 blobs=$3 &&\n+\tshift 3 &&\n+\ttest_env GIT_TRACE2_EVENT=\"$(pwd)/$trace_file\" \\\n+\t\tcheck_connected \"$@\" &&\n+\tif test \"$mode\" != incremental\n+\tthen\n+\t\treturn\n+\tfi &&\n+\tif test -n \"$trees\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity trees_loaded \"$trees\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi &&\n+\tif test -n \"$blobs\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity blobs_checked \"$blobs\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi\n+}\n+\n+# Shared setup: a repo with several root-level files and nested dirs.\n+# The unchanged/ subtree (10 dirs x 10 files = 100 blobs, 11 trees)\n+# acts as a canary: any test asserting small tree/blob counts would\n+# fail dramatically if incremental accidentally walked into it.\n+\n+test_expect_success 'setup main repo' '\n+\tgit init main-repo &&\n+\t(\n+\t\tcd main-repo &&\n+\t\tfor i in $(test_seq 1 5)\n+\t\tdo\n+\t\t\techo \"file $i\" >\"file-$i.txt\" || return 1\n+\t\tdone &&\n+\t\tgit add file-*.txt &&\n+\t\tgit commit -m \"initial\" &&\n+\n+\t\tmkdir -p a/b/c &&\n+\t\techo deep >a/b/c/deep.txt &&\n+\t\techo other >a/other.txt &&\n+\t\tgit add a/b/c/deep.txt a/other.txt &&\n+\t\tgit commit -m \"add nested dirs\" &&\n+\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\td=\"unchanged/dir-$i\" &&\n+\t\t\tmkdir -p \"$d\" &&\n+\t\t\tfor j in $(test_seq 1 10)\n+\t\t\tdo\n+\t\t\t\techo \"$i $j\" >\"$d/file-$j.txt\" || return 1\n+\t\t\tdone\n+\t\tdone &&\n+\t\tgit add unchanged/ &&\n+\t\tgit commit -m \"add unchanged canary subtree\"\n+\t)\n+'\n+\n+test_expect_success 'setup replacement object repo' '\n+\tgit init replace-test &&\n+\t(\n+\t\tcd replace-test &&\n+\n+\t\ttest_commit --no-tag original file.txt &&\n+\t\toriginal=$(git rev-parse HEAD) &&\n+\t\torig_blob=$(git rev-parse HEAD:file.txt) &&\n+\n+\t\t# Orphan replacement commit with a different tree\n+\t\treplacement_tree=$(echo replaced | git hash-object -w --stdin |\n+\t\t\txargs -I{} git mktree <<-EOF\n+\t\t\t100644 blob {}\tfile.txt\n+\t\t\tEOF\n+\t\t) &&\n+\t\treplacement=$(git commit-tree -m \"replacement\" \\\n+\t\t\t\"$replacement_tree\") &&\n+\n+\t\tgit replace \"$original\" \"$replacement\" &&\n+\n+\t\t# Remove the original blob so only the replacement\n+\t\t# tree is complete.\n+\t\trm .git/objects/$(test_oid_to_path \"$orig_blob\") &&\n+\n+\t\t# Drop branch and HEAD so --not --all does not\n+\t\t# exclude the original commit.\n+\t\tgit update-ref -d refs/heads/main &&\n+\t\tgit update-ref -d HEAD &&\n+\n+\t\techo \"$original\" >.git/test-oid\n+\t)\n+'\n+\n+missing_oid=$(test_oid missing)\n+original_oid=$(cat replace-test/.git/test-oid)\n+\n+for mode in full incremental\n+do\n+\n+# Corruption detection: craft broken object graphs and verify detection.\n+# All tests use main-repo without modifying its refs or worktree.\n+\n+test_expect_success \"$mode: rejects commit with missing blob\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects commit with missing subtree\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"40000 tree ${missing_oid}\\tdir\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"bad tree object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct tree tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"hello\" | git hash-object -w --stdin) &&\n+\t\tcheck_connected \"$blob_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects missing direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$missing_oid\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects blob OID reused as tree entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\tblob_oid=$(git rev-parse HEAD:file-1.txt) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"40000 subdir\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a tree\" err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects tree OID reused as blob entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\ttree_oid=$(git rev-parse HEAD:a) &&\n+\t\tbin_oid=$(echo \"$tree_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"100644 fakefile\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a blob\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: checks multiple tips\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"tip-a\") &&\n+\t\tc2=$(commit_with_change HEAD file-2.txt \"tip-b\") &&\n+\t\tgit tag -a -m \"tagged\" multi-tag \"$c1\" &&\n+\t\ttag_oid=$(git rev-parse multi-tag) &&\n+\t\tgit tag -d multi-tag &&\n+\t\tcheck_connected \"$c2\" \"$tag_oid\"\n+\t)\n+'\n+\n+# Tree-diff optimization: verify trace2 counts.\n+\n+test_expect_success \"$mode: handles root commit (no parents)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"root-content\" | git hash-object -w --stdin) &&\n+\t\ttree_oid=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$blob_oid\" | git mktree) &&\n+\t\troot_oid=$(git commit-tree \"$tree_oid\" -m \"root\") &&\n+\n+\t\t# No parent trees, so the full tree is verified.\n+\t\t# 1 tree loaded (root), 1 blob checked.\n+\t\tcheck_connected_trace trace-root.txt 1 1 \"$root_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles single file change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD file-1.txt \"changed\") &&\n+\n+\t\t# 2 trees loaded (new root + parent root), 1 blob checked.\n+\t\tcheck_connected_trace trace-flat.txt 2 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles nested change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD a/b/c/deep.txt \"deep-changed\") &&\n+\t\t# 4 new trees + 4 parent trees = 8 loaded, 1 blob checked.\n+\t\tcheck_connected_trace trace-nested.txt 8 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles change-then-revert\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"revert-tmp\") &&\n+\t\tc2=$(commit_with_change \"$c1\" file-1.txt \"file 1\") &&\n+\t\tc3=$(commit_with_change \"$c2\" file-1.txt \"revert-final\") &&\n+\n+\t\t# c1: new root + parent root = 2 loads.  c2: root matches\n+\t\t# HEAD (already trusted), skipped.  c3: new root + parent\n+\t\t# already expanded = 1 load.  Total: 3 trees, 2 blobs.\n+\t\tcheck_connected_trace trace-revert.txt 3 2 \"$c3\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles subtree moved to another path\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tmoved_tree=$(git ls-tree HEAD |\n+\t\t\tsed \"s/\ta$/\tmoved/\" |\n+\t\t\tgit mktree) &&\n+\t\tmoved=$(git commit-tree \"$moved_tree\" -p HEAD -m move) &&\n+\t\t# New root + parent root scanned, but the moved subtree\n+\t\t# (same OID) is trusted and not descended into.\n+\t\tcheck_connected_trace trace-move.txt 2 0 \"$moved\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles multi-parent merge\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tleft=$(commit_with_change HEAD file-1.txt left) &&\n+\t\tright=$(commit_with_change HEAD file-2.txt right) &&\n+\t\tgit update-ref refs/heads/left \"$left\" &&\n+\t\tgit update-ref refs/heads/right \"$right\" &&\n+\t\tleft_blob=$(git rev-parse \"$left:file-1.txt\") &&\n+\t\tright_blob=$(git rev-parse \"$right:file-2.txt\") &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$left_blob,file-1.txt\" &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$right_blob,file-2.txt\" &&\n+\t\tmerge_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tmerge=$(git commit-tree \"$merge_tree\" \\\n+\t\t\t-p \"$left\" -p \"$right\" -m merge) &&\n+\t\t# Both parent root trees are scanned as bases, so\n+\t\t# blobs from each parent are trusted without ODB checks.\n+\t\tcheck_connected_trace trace-merge.txt 3 0 \"$merge\" &&\n+\t\tgit update-ref -d refs/heads/left &&\n+\t\tgit update-ref -d refs/heads/right\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles merge with incoming and boundary parents\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# parent1 is incoming (not a ref), HEAD is boundary.\n+\t\tparent1=$(commit_with_change HEAD file-1.txt merge-inc) &&\n+\t\tmerge=$(git commit-tree \\\n+\t\t\t\"$(git rev-parse \"$parent1^{tree}\")\" \\\n+\t\t\t-p \"$parent1\" -p HEAD -m merge-mixed) &&\n+\t\t# parent1 verified first (topo order): 2 trees, 1 blob.\n+\t\t# merge tree = parent1 tree (already trusted): 0 extra.\n+\t\tcheck_connected_trace trace-merge-inc.txt 2 1 \"$merge\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles gitlink entries (submodules)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"160000,$missing_oid,my-submodule\" &&\n+\t\tgitlink_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tgitlink_commit=$(git commit-tree \"$gitlink_tree\" -p HEAD \\\n+\t\t\t-m \"add gitlink\") &&\n+\n+\t\t# Gitlink entries are skipped -- the missing submodule\n+\t\t# commit OID does not cause a failure.\n+\t\tcheck_connected_trace trace-gitlink.txt 2 0 \"$gitlink_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles file-to-directory transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"foo\" is a blob at root.\n+\t\tblob_a=$(echo \"file-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,foo\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add foo as file\") &&\n+\n+\t\t# Child: \"foo\" becomes a directory (foo/bar.txt).\n+\t\tblob_b=$(echo \"dir-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove foo &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,foo/bar.txt\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"foo: file to directory\") &&\n+\t\tcheck_connected_trace trace-f2d.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles directory-to-file transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"bar/baz.txt\" exists (bar is a directory).\n+\t\tblob_a=$(echo \"nested\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,bar/baz.txt\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add bar as directory\") &&\n+\n+\t\t# Child: \"bar\" becomes a plain file.\n+\t\tblob_b=$(echo \"flat\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove bar/baz.txt &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,bar\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"bar: directory to file\") &&\n+\t\tcheck_connected_trace trace-d2f.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+# Replacement objects.\n+\n+test_expect_success \"$mode: accepts with replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tcheck_connected \"$original_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects without replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tGIT_NO_REPLACE_OBJECTS=1 &&\n+\t\texport GIT_NO_REPLACE_OBJECTS &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$original_oid\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised blob\" '\n+\ttest_when_finished \"rm -rf prom-src prom-server.git prom-client\" &&\n+\tgit init prom-src &&\n+\ttest_commit -C prom-src --no-tag base file.txt original &&\n+\ttest_commit -C prom-src --no-tag \"add file2\" file2.txt extra &&\n+\tgit clone --bare prom-src prom-server.git &&\n+\tgit -C prom-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/prom-server.git\" prom-client &&\n+\t(\n+\t\tcd prom-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file2.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tnewname.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised tree\" '\n+\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n+\tgit init prom-tree-src &&\n+\tmkdir -p prom-tree-src/a/b &&\n+\ttest_commit -C prom-tree-src --no-tag \"nested dirs\" a/b/file.txt deep &&\n+\tgit clone --bare prom-tree-src prom-tree-server.git &&\n+\tgit -C prom-tree-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-tree-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-tree-server.git\" prom-tree-client &&\n+\t(\n+\t\tcd prom-tree-client &&\n+\t\tpromised_tree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\" &&\n+\t\tnew_tree=$(printf \"40000 tree %s\\trenamed\\n\" \\\n+\t\t\t\"$promised_tree\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised tree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies new subtree when parent subtree is promised\" '\n+\ttest_when_finished \"rm -rf prom-base-src prom-base-server.git prom-base-client\" &&\n+\tgit init prom-base-src &&\n+\tmkdir -p prom-base-src/a &&\n+\ttest_commit -C prom-base-src --no-tag \"base\" a/file.txt deep &&\n+\tgit clone --bare prom-base-src prom-base-server.git &&\n+\tgit -C prom-base-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-base-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-base-server.git\" prom-base-client &&\n+\t(\n+\t\tcd prom-base-client &&\n+\t\tparent_subtree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\" &&\n+\t\tnew_blob=$(echo \"local-content\" | git hash-object -w --stdin) &&\n+\t\tnew_subtree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$new_blob\" | git mktree) &&\n+\t\tnew_root=$(printf \"40000 tree %s\\ta\\n\" \\\n+\t\t\t\"$new_subtree\" | git mktree) &&\n+\t\tnew_commit=$(git commit-tree \"$new_root\" \\\n+\t\t\t-p HEAD -m \"replace promised subtree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies local commit in partial clone\" '\n+\ttest_when_finished \"rm -rf pc-src pc-server.git pc-client\" &&\n+\tgit init pc-src &&\n+\ttest_commit -C pc-src --no-tag base file.txt &&\n+\tgit clone --bare pc-src pc-server.git &&\n+\tgit -C pc-server.git config uploadpack.allowfilter true &&\n+\tgit -C pc-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/pc-server.git\" pc-client &&\n+\t(\n+\t\tcd pc-client &&\n+\t\tlocal_commit=$(commit_with_change HEAD file.txt local-content) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: respects shallow boundary\" '\n+\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n+\tgit init shallow-src &&\n+\ttest_commit -C shallow-src --no-tag base file content-1 &&\n+\tmkdir shallow-src/sub &&\n+\ttest_commit -C shallow-src --no-tag change sub/other content-2 &&\n+\tgit clone --depth=1 \"file://$(pwd)/shallow-src\" shallow &&\n+\t(\n+\t\tcd shallow &&\n+\t\ttip=$(git rev-parse HEAD) &&\n+\t\tgit for-each-ref --format=\"delete %(refname)\" |\n+\t\t\tgit update-ref --no-deref --stdin &&\n+\t\tcheck_connected \"$tip\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: deepening fetch succeeds\" '\n+\ttest_when_finished \"rm -rf deepen-src deepen-server.git deepen-client\" &&\n+\tgit init deepen-src &&\n+\ttest_commit -C deepen-src --no-tag c1 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c2 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c3 file.txt &&\n+\tgit clone --bare deepen-src deepen-server.git &&\n+\tgit clone --depth=1 \"file://$(pwd)/deepen-server.git\" deepen-client &&\n+\tset_connectivity_check deepen-client $mode &&\n+\ttest -f deepen-client/.git/shallow &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/deepen-trace.txt\" \\\n+\t\tgit -C deepen-client fetch --deepen=2 origin main &&\n+\t# Incremental falls back to full for deepening fetches,\n+\t# so the trees_loaded event should not appear.\n+\ttest_grep ! trees_loaded deepen-trace.txt\n+'\n+\n+test_expect_success \"$mode: malformed tree detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\techo abc >malformed-tree &&\n+\t\tmalformed_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tmalformed-tree) &&\n+\t\tmalformed_commit=$(git commit-tree \"$malformed_tree\" \\\n+\t\t\t-p HEAD -m malformed) &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$malformed_commit\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: mid-tree corruption detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# Build a tree with one valid entry followed by garbage.\n+\t\tblob_oid=$(echo \"valid\" | git hash-object -w --stdin) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\t\tprintf \"100644 good\\0${bin_oid}GARBAGE\" >corrupt-mid-tree &&\n+\t\tcorrupt_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tcorrupt-mid-tree) &&\n+\t\tcorrupt_commit=$(git commit-tree \"$corrupt_tree\" \\\n+\t\t\t-p HEAD -m \"mid-tree corruption\") &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$corrupt_commit\" 2>err &&\n+\t\ttest_grep \"too-short tree object\" err\n+\t)\n+'\n+\n+done\n+\n+# Algorithm selection.\n+\n+test_expect_success 'invalid transfer.connectivityCheck is rejected' '\n+\ttest_when_finished \"rm -rf invalid-cfg-src invalid-cfg-dst\" &&\n+\tgit init invalid-cfg-src &&\n+\ttest_commit -C invalid-cfg-src --no-tag base file.txt &&\n+\tgit clone invalid-cfg-src invalid-cfg-dst &&\n+\ttest_commit -C invalid-cfg-src --no-tag update file.txt updated &&\n+\tgit -C invalid-cfg-dst config transfer.connectivityCheck bogus &&\n+\ttest_must_fail git -C invalid-cfg-dst fetch origin main 2>err &&\n+\ttest_grep \"unknown transfer.connectivityCheck\" err\n+'\n+\n+test_expect_success 'push uses incremental when configured' '\n+\ttest_when_finished \"rm -rf int-src int-dst.git\" &&\n+\tgit init int-src &&\n+\ttest_commit -C int-src --no-tag base file.txt &&\n+\tgit clone --bare int-src int-dst.git &&\n+\ttest_commit -C int-src --no-tag update file.txt updated &&\n+\tset_connectivity_check int-dst.git incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/push-trace.txt\" \\\n+\t\tgit -C int-src push ../int-dst.git main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<push-trace.txt\n+'\n+\n+test_expect_success 'fetch uses incremental when configured' '\n+\ttest_when_finished \"rm -rf fetch-src fetch-dst\" &&\n+\tgit init fetch-src &&\n+\ttest_commit -C fetch-src --no-tag base file.txt &&\n+\tgit clone fetch-src fetch-dst &&\n+\ttest_commit -C fetch-src --no-tag update file.txt updated &&\n+\tset_connectivity_check fetch-dst incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/fetch-trace.txt\" \\\n+\t\tgit -C fetch-dst fetch origin main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<fetch-trace.txt\n+'\n+\n+test_expect_success 'clone respects transfer.connectivityCheck' '\n+\ttest_when_finished \"rm -rf clone-src clone-dst\" &&\n+\tgit init clone-src &&\n+\ttest_commit -C clone-src --no-tag base file.txt &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/clone-trace.txt\" \\\n+\t\tgit -c transfer.connectivityCheck=incremental \\\n+\t\tclone --no-local clone-src clone-dst &&\n+\ttest_trace2_data_singular connectivity trees_loaded 0 \\\n+\t\t<clone-trace.txt\n+'\n+\n+test_done\ndiff --git a/tree-verify.c b/tree-verify.c\nnew file mode 100644\nindex 0000000000..16acc9b16d\n--- /dev/null\n+++ b/tree-verify.c\n@@ -0,0 +1,306 @@\n+#include \"git-compat-util.h\"\n+#include \"commit.h\"\n+#include \"gettext.h\"\n+#include \"hex.h\"\n+#include \"khash.h\"\n+#include \"object.h\"\n+#include \"odb.h\"\n+#include \"oid-array.h\"\n+#include \"oidset.h\"\n+#include \"tree.h\"\n+#include \"tree-walk.h\"\n+#include \"tree-verify.h\"\n+#include \"packfile.h\"\n+#include \"trace2.h\"\n+\n+enum tree_state {\n+\tTREE_UNTRUSTED = 0,\n+\tTREE_TRUSTED   = 1,\n+\tTREE_EXPANDED  = 2,\n+};\n+\n+KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n+\t   oidhash_by_value, oideq_by_value)\n+\n+static enum tree_state tree_map_get(kh_oid_tree_t *m,\n+\t\t\t\t    const struct object_id *oid)\n+{\n+\tkhint_t pos = kh_get_oid_tree(m, *oid);\n+\tif (pos == kh_end(m))\n+\t\treturn TREE_UNTRUSTED;\n+\treturn kh_val(m, pos);\n+}\n+\n+static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n+\t\t\t enum tree_state state)\n+{\n+\tint added;\n+\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n+\tif (added)\n+\t\tkh_val(m, pos) = state;\n+\telse if (state > kh_val(m, pos))\n+\t\tkh_val(m, pos) = state;\n+}\n+\n+struct work_item {\n+\tstruct name_entry entry;\n+\tstruct oid_array parent_trees;\n+};\n+\n+struct verify_state {\n+\tkh_oid_tree_t *trees;\n+\tstruct oidset trusted_blobs;\n+\tint trees_loaded;\n+\tint blobs_checked;\n+\tint exclude_promisor_objects;\n+};\n+\n+/*\n+ * Merge-walk the work list against one base tree entry, recording\n+ * same-path parent subtrees as recursive comparison bases.\n+ * Returns the updated work-list cursor.\n+ */\n+static size_t collect_subtree_bases(struct work_item *work, size_t nr_work,\n+\t\t\t\t    size_t wi, const struct name_entry *entry)\n+{\n+\twhile (wi < nr_work) {\n+\t\tint cmp = base_name_compare(\n+\t\t\twork[wi].entry.path, work[wi].entry.pathlen,\n+\t\t\twork[wi].entry.mode,\n+\t\t\tentry->path, entry->pathlen,\n+\t\t\tentry->mode);\n+\t\tif (cmp > 0)\n+\t\t\tbreak;\n+\t\tif (cmp < 0) {\n+\t\t\twi++;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (S_ISDIR(work[wi].entry.mode) &&\n+\t\t    S_ISDIR(entry->mode))\n+\t\t\toid_array_append(&work[wi].parent_trees,\n+\t\t\t\t\t &entry->oid);\n+\t\treturn wi + 1;\n+\t}\n+\treturn wi;\n+}\n+\n+static void verify_blob(struct repository *repo,\n+\t\t\tconst struct object_id *oid,\n+\t\t\tstruct verify_state *vs)\n+{\n+\tint type;\n+\n+\tif (oidset_contains(&vs->trusted_blobs, oid))\n+\t\treturn;\n+\n+\tvs->blobs_checked++;\n+\ttype = odb_read_object_info(repo->objects, oid, NULL);\n+\tif (type == OBJ_BLOB) {\n+\t\toidset_insert(&vs->trusted_blobs, oid);\n+\t\treturn;\n+\t}\n+\tif (type >= 0)\n+\t\tdie(_(\"object %s is a %s, not a blob\"),\n+\t\t    oid_to_hex(oid), type_name(type));\n+\tif (vs->exclude_promisor_objects &&\n+\t    is_promisor_object(repo, oid))\n+\t\treturn;\n+\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n+}\n+\n+static void *read_tree_object(struct object_database *odb,\n+\t\t\t      const struct object_id *oid,\n+\t\t\t      size_t *sizep)\n+{\n+\tenum object_type type;\n+\tvoid *buf = odb_read_object(odb, oid, &type, sizep);\n+\n+\tif (buf && type != OBJ_TREE) {\n+\t\tfree(buf);\n+\t\tdie(_(\"object %s is a %s, not a tree\"),\n+\t\t    oid_to_hex(oid), type_name(type));\n+\t}\n+\treturn buf;\n+}\n+\n+static void verify_tree(struct repository *repo,\n+\t\t\tconst struct object_id *new_tree_oid,\n+\t\t\tconst struct oid_array *base_trees,\n+\t\t\tstruct verify_state *vs, int depth)\n+{\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tstruct work_item *work = NULL;\n+\tsize_t nr_work = 0, alloc_work = 0;\n+\tint need_subtree_bases = 0;\n+\tsize_t i, tree_size;\n+\tvoid *tree_buf;\n+\n+\tif (depth > repo->settings.max_allowed_tree_depth)\n+\t\tdie(_(\"exceeded maximum allowed tree depth\"));\n+\n+\tif (tree_map_get(vs->trees, new_tree_oid) >= TREE_TRUSTED)\n+\t\treturn;\n+\n+\ttree_buf = read_tree_object(repo->objects, new_tree_oid, &tree_size);\n+\tif (!tree_buf) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, new_tree_oid))\n+\t\t\treturn;\n+\t\tdie(_(\"bad tree object %s\"),\n+\t\t    oid_to_hex(new_tree_oid));\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tinit_tree_desc(&desc, new_tree_oid, tree_buf, tree_size);\n+\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (S_ISGITLINK(entry.mode))\n+\t\t\tcontinue;\n+\t\tif (S_ISDIR(entry.mode)) {\n+\t\t\tif (tree_map_get(vs->trees, &entry.oid) >= TREE_TRUSTED)\n+\t\t\t\tcontinue;\n+\t\t\tneed_subtree_bases = 1;\n+\t\t} else {\n+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry.oid))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\tALLOC_GROW(work, nr_work + 1, alloc_work);\n+\t\tmemset(&work[nr_work], 0, sizeof(work[nr_work]));\n+\t\twork[nr_work].entry = entry;\n+\t\tnr_work++;\n+\t}\n+\n+\tif (!nr_work) {\n+\t\tfree(tree_buf);\n+\t\tgoto done;\n+\t}\n+\n+\tfor (i = 0; base_trees && i < base_trees->nr; i++) {\n+\t\tconst struct object_id *base_oid = &base_trees->oid[i];\n+\t\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n+\t\tstruct tree_desc base_desc;\n+\t\tstruct name_entry scan_entry;\n+\t\tsize_t wi = 0, base_size;\n+\t\tvoid *base_buf;\n+\n+\t\tif (expanded && !need_subtree_bases)\n+\t\t\tcontinue;\n+\n+\t\tbase_buf = read_tree_object(repo->objects, base_oid,\n+\t\t\t\t\t    &base_size);\n+\t\tif (!base_buf) {\n+\t\t\tif (vs->exclude_promisor_objects &&\n+\t\t\t    is_promisor_object(repo, base_oid))\n+\t\t\t\tcontinue;\n+\t\t\tdie(_(\"bad tree object %s\"),\n+\t\t\t    oid_to_hex(base_oid));\n+\t\t}\n+\n+\t\tvs->trees_loaded++;\n+\t\tinit_tree_desc(&base_desc, base_oid, base_buf, base_size);\n+\n+\t\twhile (tree_entry(&base_desc, &scan_entry)) {\n+\t\t\tif (S_ISGITLINK(scan_entry.mode))\n+\t\t\t\tcontinue;\n+\n+\t\t\tif (need_subtree_bases)\n+\t\t\t\twi = collect_subtree_bases(work, nr_work,\n+\t\t\t\t\t\t\t   wi, &scan_entry);\n+\n+\t\t\tif (!expanded) {\n+\t\t\t\tif (S_ISDIR(scan_entry.mode))\n+\t\t\t\t\ttree_map_add(vs->trees,\n+\t\t\t\t\t\t     &scan_entry.oid,\n+\t\t\t\t\t\t     TREE_TRUSTED);\n+\t\t\t\telse\n+\t\t\t\t\toidset_insert(&vs->trusted_blobs,\n+\t\t\t\t\t\t      &scan_entry.oid);\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!expanded)\n+\t\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n+\n+\t\tfree(base_buf);\n+\t}\n+\n+\tfor (i = 0; i < nr_work; i++) {\n+\t\tif (S_ISDIR(work[i].entry.mode))\n+\t\t\tverify_tree(repo, &work[i].entry.oid,\n+\t\t\t\t    &work[i].parent_trees, vs,\n+\t\t\t\t    depth + 1);\n+\t\telse\n+\t\t\tverify_blob(repo, &work[i].entry.oid, vs);\n+\t}\n+\n+\tfree(tree_buf);\n+\n+done:\n+\ttree_map_add(vs->trees, new_tree_oid, TREE_EXPANDED);\n+\tfor (i = 0; i < nr_work; i++)\n+\t\toid_array_clear(&work[i].parent_trees);\n+\tfree(work);\n+}\n+\n+static void verify_commit_tree(struct repository *repo,\n+\t\t\t       struct commit *commit,\n+\t\t\t       struct verify_state *vs)\n+{\n+\tstruct oid_array base_trees = OID_ARRAY_INIT;\n+\tstruct commit_list *p;\n+\n+\t/*\n+\t * Parent trees are trusted: boundary parents are already\n+\t * connected, and earlier incoming parents were verified\n+\t * first due to the topological processing order.\n+\t */\n+\tfor (p = commit->parents; p; p = p->next) {\n+\t\tconst struct object_id *tree_oid;\n+\t\tparse_commit_or_die(p->item);\n+\t\ttree_oid = get_commit_tree_oid(p->item);\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n+\t\toid_array_append(&base_trees, tree_oid);\n+\t}\n+\n+\tverify_tree(repo, get_commit_tree_oid(commit),\n+\t\t    &base_trees, vs, 0);\n+\toid_array_clear(&base_trees);\n+}\n+\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects)\n+{\n+\tstruct verify_state vs = { 0 };\n+\tstruct commit_list *iter;\n+\tunsigned nr_before;\n+\n+\tvs.trees = kh_init_oid_tree();\n+\tvs.exclude_promisor_objects = exclude_promisor_objects;\n+\n+\t/*\n+\t * Ancestors must be verified before descendants so that parent\n+\t * trees can be trusted without re-verification.  Sort explicitly\n+\t * rather than relying on the caller's ordering.\n+\t *\n+\t * sort_in_topological_order() silently drops cycle members,\n+\t * so explicitly check if the size has changed.\n+\t */\n+\tnr_before = commit_list_count(*commits);\n+\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n+\tif (commit_list_count(*commits) < nr_before)\n+\t\tdie(_(\"cycle detected in incoming commit graph\"));\n+\n+\t*commits = commit_list_reverse(*commits);\n+\n+\tfor (iter = *commits; iter; iter = iter->next)\n+\t\tverify_commit_tree(repo, iter->item, &vs);\n+\n+\tkh_destroy_oid_tree(vs.trees);\n+\toidset_clear(&vs.trusted_blobs);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"trees_loaded\", vs.trees_loaded);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"blobs_checked\", vs.blobs_checked);\n+}\ndiff --git a/tree-verify.h b/tree-verify.h\nnew file mode 100644\nindex 0000000000..6aadadff70\n--- /dev/null\n+++ b/tree-verify.h\n@@ -0,0 +1,15 @@\n+#ifndef TREE_VERIFY_H\n+#define TREE_VERIFY_H\n+\n+struct commit_list;\n+struct repository;\n+\n+/*\n+ * Verify trees of commits incrementally against their parents.\n+ * Dies on verification failure.\n+ */\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects);\n+\n+#endif /* TREE_VERIFY_H */\n-- \ngitgitgadget\n"},{"id":"552711","messageId":"xmqqmrtj7tp2.fsf@gitster.g","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/2] connected: add incremental connectivity check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-09-14T15:12:41Z","receivedAt":"2026-09-14T15:12:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> This series adds an incremental mode for the connectivity check, gated\n> behind transfer.connectivityCheck=incremental (no expected changes unless\n> you opt in).\n>\n> The intent is to solve the problem of the connectivity check slowing down as\n> the number of reachable objects from the boundary grows.\n\nExciting benchmarks.\n"},{"id":"552712","messageId":"xmqqh5jr7t1h.fsf@gitster.g","threadId":"66326","inReplyTo":"ebe6c90cc58b9e1f64c9bec4a18e8cb3ce9be1b2.1789379276.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/2] connected: add incremental connectivity check via rev-list","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-09-14T15:26:50Z","receivedAt":"2026-09-14T15:26:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> +static void verify_blob(struct repository *repo,\n> +\t\t\tconst struct object_id *oid,\n> +\t\t\tstruct verify_state *vs)\n> +{\n> +\tint type;\n> +\n> +\tif (oidset_contains(&vs->trusted_blobs, oid))\n> +\t\treturn;\n> +\n> +\tvs->blobs_checked++;\n> +\ttype = odb_read_object_info(repo->objects, oid, NULL);\n> +\tif (type == OBJ_BLOB) {\n> +\t\toidset_insert(&vs->trusted_blobs, oid);\n> +\t\treturn;\n> +\t}\n> +\tif (type >= 0)\n> +\t\tdie(_(\"object %s is a %s, not a blob\"),\n> +\t\t    oid_to_hex(oid), type_name(type));\n> +\tif (vs->exclude_promisor_objects &&\n> +\t    is_promisor_object(repo, oid))\n> +\t\treturn;\n> +\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n> +}\n\nI wonder if this is_promisor_object() call comes a bit too late, as\nwe earlier already have called odb_read_object_info() which may have\nfetched it lazily from the promisor remote?  Or do we globally\ndisable promisor_remote_get_direct() call somehow without having to\npass OBJECT_INFO_SKIP_FETCH_OBJECT flag?\n\n> +static void verify_commit_tree(struct repository *repo,\n> +\t\t\t       struct commit *commit,\n> +\t\t\t       struct verify_state *vs)\n> +{\n> +\tstruct oid_array base_trees = OID_ARRAY_INIT;\n> +\tstruct commit_list *p;\n> +\n> +\t/*\n> +\t * Parent trees are trusted: boundary parents are already\n> +\t * connected, and earlier incoming parents were verified\n> +\t * first due to the topological processing order.\n> +\t */\n> +\tfor (p = commit->parents; p; p = p->next) {\n> +\t\tconst struct object_id *tree_oid;\n> +\t\tparse_commit_or_die(p->item);\n> +\t\ttree_oid = get_commit_tree_oid(p->item);\n> +\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n> +\t\toid_array_append(&base_trees, tree_oid);\n> +\t}\n> +\n> +\tverify_tree(repo, get_commit_tree_oid(commit),\n> +\t\t    &base_trees, vs, 0);\n> +\toid_array_clear(&base_trees);\n> +}\n\nDo we assume that we do not have to deal with repository corruption\nin any graceful way?  I am just wondering what happens when\nget_commit_tree_oid() yields NULL after parse_commit_or_die() finds\np->item is a valid-looking commit object but the tree within it is\nnot, and we end up passing NULL to tree_map_add(), perhaps?\n\nThe same potential issue may exist in the get_commit_tree_oid() call\noutside the look at the end on the incoming commit's tree.\n"},{"id":"552723","messageId":"CAL71e4My+maYAtWbkoHXvsm=qhCmao7K_7mLV5wg1FyKTa3u8A@mail.gmail.com","threadId":"66326","inReplyTo":"xmqqh5jr7t1h.fsf@gitster.g","subject":"Re: [PATCH 2/2] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-09-14T17:46:05Z","receivedAt":"2026-09-14T17:46:19Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 14 Sept 2026 at 17:26, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> I wonder if this is_promisor_object() call comes a bit too late, as\n> we earlier already have called odb_read_object_info() which may have\n> fetched it lazily from the promisor remote?  Or do we globally\n> disable promisor_remote_get_direct() call somehow without having to\n> pass OBJECT_INFO_SKIP_FETCH_OBJECT flag?\n\nYes, I think it's safe due to the following mechanism:\n\n1. If promisors exist, the connectivity-check will invoke\n   rev-list with --exclude-promisor-objects.\n2. rev-list in turn sets repo->fetch_if_missing = 0 on startup.\n3. Then the odb read goes down into do_oid_object_info_extended()\n   which respects that flag.\n\nHowever, my paranoia kicked in so I re-ran my test for this,\nafter adding some temporary code inside\nverify_commits_incremental():\n\n    repo->fetch_if_missing = 1;\n\nAnd fortunately, one of the tests failed as expected.\n\n    Exactly 1 failure out of 62 tests: test 53\n      \"incremental: verifies new subtree when parent subtree is\n       promised\".\n\nAnd the relevant assertion is this one:\n\n    test_must_fail env GIT_NO_LAZY_FETCH=1 \\\n        git cat-file -e \"$parent_subtree\"\n\nwhich ensures that the object was never fetched.\n\nHowever, the test only catches this scenario for trees,\nnot blobs -- that's an oversight, I will add a matching\ntest for blobs too.\n\nI think the code technically works as-is, but I could also try\nto rewrite the code to stop depending on odb_read_object_info()\nand instead use odb_read_object_info_extended() which allows\nme to pass the flags.  That gives us belts and suspenders, which\nmay be nicer here.\n\n> Do we assume that we do not have to deal with repository corruption\n> in any graceful way?  I am just wondering what happens when\n> get_commit_tree_oid() yields NULL after parse_commit_or_die() finds\n> p->item is a valid-looking commit object but the tree within it is\n> not, and we end up passing NULL to tree_map_add(), perhaps?\n>\n> The same potential issue may exist in the get_commit_tree_oid() call\n> outside the look at the end on the incoming commit's tree.\n\nYou're right, this is an oversight.\nI think I incorrectly assumed that parse_commit_or_die()\nwould catch any malformed commit.\n\nI will add a NULL check and a die()-exit at the two call sites\nin verify_commit_tree()\n\n    die(_(\"unable to load root tree for commit %s\"),\n        oid_to_hex(&commit->object.oid));\n\nThanks for spotting these errors,\nKristofer\n"},{"id":"553462","messageId":"pull.2211.v2.git.1790600552.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH v2 0/2] connected: add incremental connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-28T13:02:30Z","receivedAt":"2026-09-28T13:02:39Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"This series adds an incremental mode for the connectivity check, gated\nbehind transfer.connectivityCheck=incremental (no expected changes unless\nyou opt in).\n\nThe intent is to solve the problem of the connectivity check slowing down as\nthe number of reachable objects from the boundary grows.\n\nIt relates to the RFC I sent out earlier:\n\n[RFC] check_connected: toward incoming-proportional cost\nhttps://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/\n\n\n\nDesign\n======\n\nThe verifier runs inside the same rev-list subprocess that check_connected()\nalready spawns, triggered by a new internal flag --verify-trees-incremental.\nAfter get_revision() collects the incoming commits, the verifier processes\nthem in topological order (ancestors before descendants).\n\nThe idea is to keep a set of trusted objects, shared across the incoming\ncommits, that grows over time. We visit the new/untrusted commit trees and\ndo a comparison walk over the trees of their parents. Entries discovered on\nthe trusted parent side are remembered as trusted, which lets later\nverification skip matching objects and avoid descending into unchanged\nsubtrees.\n\nMore algorithmic details are in\nDocumentation/technical/connectivity-check.adoc.\n\n\nBenchmarks\n==========\n\nI'll just mention a short summary here, to avoid repeating what's already in\nthe commit message. The incremental mode is faster than the full mode when\nthere are few commits to verify and when the active object tree is large. In\nthe happy case, the work tracks the changed paths and their comparison trees\nrather than the full reachable object closure, which substantially reduces\nthe dependence on total repository size. I've seen speedups up to around 20x\nfor the synthetic perf tests.\n\nRunning against a large real-world repo (3.4 GB boundary closure), the\nnumbers are more dramatic. All timings use rev-list directly with --not\nHEADN, isolating the tree verification cost from the boundary-finding cost:\n\ncommits    full  incr.  speedup   full RSS  incr. RSS\n      1    1.9s  0.01s    190x      3.4 GB     14 MB\n     10    1.9s  0.04s     48x      3.4 GB    125 MB\n    100    1.9s  0.37s      5x      3.4 GB    1.1 GB\n\n\nThe full mode takes ~1.9s regardless of commit count because it is dominated\nby walking the boundary closure. Incremental scales with the number of\nincoming commits and the paths they touch. Memory follows the same pattern:\nincremental uses a fraction of the full mode's RSS for small pushes,\nconverging only when many commits are verified.\n\nThere are also regression cases in the synthetic fixtures. With long\nincoming histories, the extra parent-tree scans accumulate; in the synthetic\nfixture incremental is about 1.5x slower at 10000 commits. Per-commit\nchanges have less impact than expected: even when every directory is\ntouched, incremental remains competitive; bypassing the object cache for\ntree reads likely helps here.\n\nI cannot establish how common these regression cases are. In the cases I\nhave tested, however, the regression has remained modest; I have not been\nable to provoke a substantially larger slowdown. My feeling is that this is\nan acceptable tradeoff behind the opt-in config, since the target case\n(small pushes to large repos) sees the largest speedup, while the regression\nappears with long incoming histories.\n\nA safety net for this regression could be to dynamically disable the\nincremental mode if the number of incoming commits is too large, but this is\nleft out of the initial version to avoid overly speculative code.\n\nDeepening fetches currently fall back to the full check because the full\ncheck omits --not --all for deepening -- there is no existing-reference\nboundary at which the walk can stop. An incremental approach is possible\nhere too -- using the old shallow roots as the trusted boundary and walking\nthe deepened ancestry forward -- but that is a separate change and left for\nfuture work. Deepening is also less common than regular fetch and\nreceive-pack, where the speedup matters most.\n\n\nTest coverage\n=============\n\nMost correctness cases in t5412-connectivity-check.sh are run in both full\nand incremental modes to check semantic equivalence. Selected cases\nadditionally assert trace2 tree/blob counts for the incremental mode, to\nverify that unchanged portions of the object graph are actually skipped. It\ncovers:\n\n * Corruption detection: missing blobs, missing trees, type mismatches,\n   malformed trees (unparseable, mid-tree corruption)\n * Tree optimization: trace2 assertions confirm unchanged subtrees are\n   skipped, subtree moves, merge parent boundaries\n * Root commits (no parents -- verifies full tree closure)\n * Partial clones: missing promised blobs, missing promised trees,\n   verification of local commits\n * Replacement objects (with and without GIT_NO_REPLACE_OBJECTS)\n * Shallow boundaries\n * Deepening fetches (falls back to full check)\n * Integration: real push, fetch, and clone\n\nMost tests call git rev-list directly with the appropriate flags;\nintegration tests exercise the full check_connected() path through push,\nfetch, and clone.\n\n\nAlternatives considered\n=======================\n\nMy first prototype ran the verifier in-process inside connected.c. This\nrequired a second rev-list subprocess just for boundary finding, _nofetch\nvariants of several object-reading functions to prevent lazy fetches in\npartial clones, explicit shallow-file plumbing, and careful avoidance of\ndie() in all code paths reachable from the verifier. The result worked but\nwas fragile and touched many files.\n\nMoving the verifier into the rev-list subprocess eliminated all of those\nproblems: in partial clones the existing --exclude-promisor-objects handling\nalready disables lazy fetching, die() is isolated by the process boundary,\nshallow and replacement semantics are established before the verifier runs,\nand error routing comes for free via stderr.\n\nSo while I liked the idea of being less reliant on checking within a\nsubprocess, making that work ended up being a lot more complex.\n\n\nNext steps\n==========\n\nThis series only addresses tree verification; the other significant cost is\nfinding the commit boundary, especially for repos with many refs. I already\nhave some prototypes for optimizing that too, and if this ends up landing,\nthat would be something I would start polishing up.\n\n\nChanges since v1\n================\n\n * Guard against get_commit_tree_oid() returning NULL in\n   verify_commit_tree(), dying with a clear message instead of a NULL\n   dereference. (Thanks Junio for catching this)\n * Set fetch_if_missing = 0 in the rev-list early option parse when\n   --verify-trees-incremental is seen, and add a BUG() assertion in\n   verify_commits_incremental() to catch any future caller that forgets.\n   This makes the lazy-fetch protection explicit rather than relying solely\n   on --exclude-promisor-objects having set it earlier. (Thanks again to\n   Junio)\n * Add a test that verifies promised blobs are not lazy-fetched during the\n   incremental check.\n * Reuse the existing 3-argument type-mismatch format string to avoid adding\n   new strings for localization.\n\nThanks, Kristofer\n\nKristofer Karlsson (2):\n  Documentation: describe connectivity checking\n  connected: add incremental connectivity check via rev-list\n\n Documentation/config/transfer.adoc            |  20 +\n Documentation/rev-list-options.adoc           |   6 +\n .../technical/connectivity-check.adoc         | 243 +++++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  19 +\n connected.c                                   |  24 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5412-connectivity-check.perl |  44 ++\n t/perf/p5412-connectivity-check.sh            |  92 +++\n t/t5412-connectivity-check.sh                 | 680 ++++++++++++++++++\n tree-verify.c                                 | 316 ++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1462 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n create mode 100644 t/perf/generate-repo-p5412-connectivity-check.perl\n create mode 100755 t/perf/p5412-connectivity-check.sh\n create mode 100755 t/t5412-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\n\nbase-commit: 47ce80527c56f462cb97db4ca8125342204d3783\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2211%2Fspkrka%2Ftree-diff-connectivity-v1-clean-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2211/spkrka/tree-diff-connectivity-v1-clean-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/2211\n\nRange-diff vs v1:\n\n 1:  e55c5452db = 1:  97c11449ae Documentation: describe connectivity checking\n 2:  ebe6c90cc5 ! 2:  6ad528f4bb connected: add incremental connectivity check via rev-list\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \t\t\trevs.exclude_promisor_objects = 1;\n      +\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n      +\t\t\tverify_trees_incremental = 1;\n     ++\t\t\trepo->fetch_if_missing = 0;\n       \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n       \t\t\tparse_missing_action_value(repo, arg);\n       \t\t} else if (!strcmp(arg, \"-z\")) {\n     @@ t/t5412-connectivity-check.sh (new)\n      +\t)\n      +'\n      +\n     ++test_expect_success \"$mode: does not lazy-fetch promised blob\" '\n     ++\ttest_when_finished \"rm -rf nofetch-src nofetch-server.git nofetch-client\" &&\n     ++\tgit init nofetch-src &&\n     ++\ttest_commit -C nofetch-src --no-tag base file.txt content &&\n     ++\tgit clone --bare nofetch-src nofetch-server.git &&\n     ++\tgit -C nofetch-server.git config uploadpack.allowfilter true &&\n     ++\tgit -C nofetch-server.git config uploadpack.allowanysha1inwant true &&\n     ++\tgit clone --no-checkout --filter=blob:none \\\n     ++\t\t\"file://$(pwd)/nofetch-server.git\" nofetch-client &&\n     ++\t(\n     ++\t\tcd nofetch-client &&\n     ++\t\tpromised_blob=$(git rev-parse HEAD:file.txt) &&\n     ++\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n     ++\t\t\tgit cat-file -e \"$promised_blob\" &&\n     ++\t\tnew_tree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n     ++\t\t\t\"$promised_blob\" |\n     ++\t\t\tgit mktree --missing) &&\n     ++\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n     ++\t\t\t-m \"root commit with promised blob\") &&\n     ++\t\tcheck_connected \"$new_commit\" &&\n     ++\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n     ++\t\t\tgit cat-file -e \"$promised_blob\"\n     ++\t)\n     ++'\n     ++\n      +test_expect_success \"$mode: accepts missing promised tree\" '\n      +\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n      +\tgit init prom-tree-src &&\n     @@ tree-verify.c (new)\n      +\t\treturn;\n      +\t}\n      +\tif (type >= 0)\n     -+\t\tdie(_(\"object %s is a %s, not a blob\"),\n     -+\t\t    oid_to_hex(oid), type_name(type));\n     ++\t\tdie(_(\"object %s is a %s, not a %s\"),\n     ++\t\t    oid_to_hex(oid), type_name(type), \"blob\");\n      +\tif (vs->exclude_promisor_objects &&\n      +\t    is_promisor_object(repo, oid))\n      +\t\treturn;\n     @@ tree-verify.c (new)\n      +\n      +\tif (buf && type != OBJ_TREE) {\n      +\t\tfree(buf);\n     -+\t\tdie(_(\"object %s is a %s, not a tree\"),\n     -+\t\t    oid_to_hex(oid), type_name(type));\n     ++\t\tdie(_(\"object %s is a %s, not a %s\"),\n     ++\t\t    oid_to_hex(oid), type_name(type), \"tree\");\n      +\t}\n      +\treturn buf;\n      +}\n     @@ tree-verify.c (new)\n      +\t\tconst struct object_id *tree_oid;\n      +\t\tparse_commit_or_die(p->item);\n      +\t\ttree_oid = get_commit_tree_oid(p->item);\n     ++\t\tif (!tree_oid)\n     ++\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n     ++\t\t\t    oid_to_hex(&p->item->object.oid));\n      +\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n      +\t\toid_array_append(&base_trees, tree_oid);\n      +\t}\n      +\n     ++\tif (!get_commit_tree_oid(commit))\n     ++\t\tdie(_(\"unable to load root tree for commit %s\"),\n     ++\t\t    oid_to_hex(&commit->object.oid));\n      +\tverify_tree(repo, get_commit_tree_oid(commit),\n      +\t\t    &base_trees, vs, 0);\n      +\toid_array_clear(&base_trees);\n     @@ tree-verify.c (new)\n      +\tstruct commit_list *iter;\n      +\tunsigned nr_before;\n      +\n     ++\tif (repo->fetch_if_missing)\n     ++\t\tBUG(\"verify_commits_incremental must not be called \"\n     ++\t\t    \"with fetch_if_missing set\");\n     ++\n      +\tvs.trees = kh_init_oid_tree();\n      +\tvs.exclude_promisor_objects = exclude_promisor_objects;\n      +\n\n-- \ngitgitgadget\n"},{"id":"553463","messageId":"97c11449aeae924436ba22a00a2545254e988a58.1790600552.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v2.git.1790600552.gitgitgadget@gmail.com","subject":"[PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-28T13:02:31Z","receivedAt":"2026-09-28T13:02:51Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd Documentation/technical/connectivity-check.adoc describing\nthe connectivity invariant and the full connectivity check.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n .../technical/connectivity-check.adoc         | 109 ++++++++++++++++++\n 1 file changed, 109 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nnew file mode 100644\nindex 0000000000..d20bff6af6\n--- /dev/null\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -0,0 +1,109 @@\n+Connectivity checking\n+=====================\n+\n+After receiving new objects via fetch, push (receive-pack), clone,\n+or bundle, Git verifies that the new reference tips do not leave\n+the repository in a state where reachable objects are missing.\n+This verification is called the connectivity check.\n+\n+Connectivity invariant\n+----------------------\n+\n+A repository is connected when every object reachable from its\n+references is available locally (with exceptions noted below).\n+\n+The connectivity check maintains this invariant when references\n+are updated.  It trusts the existing connected state and verifies\n+that the new reference tips do not introduce references to\n+unavailable objects.  Verification is permitted to stop when it\n+reaches objects already reachable from trusted existing\n+references, since their closure is already connected.  These\n+trusted references include local references and references from\n+alternate object stores.\n+\n+Without this check, a truncated or corrupted transfer could leave\n+a repository in a state where later history walks encounter\n+missing objects.\n+\n+Exceptions\n+~~~~~~~~~~\n+\n+Gitlink entries (submodule references) are excluded from\n+connectivity checking.  Their target objects belong to a separate\n+repository.\n+\n+In partial clones, objects promised by a promisor remote are\n+accepted as connected without requiring local existence.  The\n+check excludes promisor objects from traversal so that it does\n+not trigger on-demand fetches for them.\n+\n+Full connectivity check\n+-----------------------\n+\n+`check_connected()` (see `connected.c`) normally performs the\n+connectivity check using a `rev-list` subprocess, feeding the\n+new reference tips via stdin.  A normal invocation is roughly:\n+\n+    git rev-list --objects --stdin --not --all --quiet\n+        --alternate-refs [--exclude-promisor-objects]\n+\n+When promisor remotes are configured, `check_connected()` first\n+attempts a fast path based on promisor packfiles.  If it falls\n+back to the `rev-list` check, `--exclude-promisor-objects` is\n+added so that the traversal does not trigger on-demand fetches.\n+\n+Consider the following graph after a fetch, where all reference\n+tips point directly to commits.  For simplicity, only local\n+references appear on the already-connected side; alternate refs\n+play the same role.  N3 is a merge commit:\n+\n+            /-------------L2\n+           /\n+    C1---B1---C2---B2-----L1\n+          \\         \\\n+           N1        N3---T2\n+            \\       /\n+             N2-----------T1\n+\n+    L1, L2:         local refs\n+    T1, T2:         incoming tips (new refs)\n+    N1, N2, N3:     incoming commits (N3 is a merge)\n+    B1, B2:         boundary commits (already connected)\n+    C1, C2:         already connected (but not boundary)\n+\n+The incoming set is the commits reachable from the incoming\n+tips but not from the already-connected side.  Boundary commits\n+are the already-connected commits at the edge of that set.  Here\n+B1 is an ancestor of B2, which happens when incoming branches\n+fork at different depths in the existing history.\n+\n+The check proceeds in three phases:\n+\n+1. Walk from the incoming tips (T1, T2) against the trusted\n+   refs (L1, L2) to find the incoming set ({N1, N2, N3, T1, T2}).\n+\n+2. Walk the trees of the boundary commits (B1, B2) and mark\n+   those objects uninteresting.  These trees are already trusted\n+   because their commits are on the already-connected side.\n+\n+3. Walk the trees of each incoming commit and verify that every\n+   referenced object is connected, stopping at objects already\n+   marked uninteresting in phase 2.\n+\n+Deepening fetches\n+~~~~~~~~~~~~~~~~~\n+\n+For deepening fetches (where the shallow boundary moves), the\n+full check omits `--not --all`.  There is no existing-reference\n+boundary at which the walk can stop.  Instead, traversal follows\n+the effective shallow boundary supplied for the deepened\n+repository.  The new content may be below the old shallow\n+boundary even when the tips themselves have not changed.\n+\n+Non-commit tips\n+~~~~~~~~~~~~~~~\n+\n+When a new reference points to a non-commit object, such as a\n+tag, tree, or blob, that object is not part of the commit walk.\n+These non-commit tips are handled by the subsequent object\n+traversal.\n-- \ngitgitgadget\n\n"},{"id":"553464","messageId":"6ad528f4bb42a960910eb4fe917a3766fa55d598.1790600552.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v2.git.1790600552.gitgitgadget@gmail.com","subject":"[PATCH v2 2/2] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-09-28T13:02:32Z","receivedAt":"2026-09-28T13:02:54Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe full connectivity check uses rev-list to find commits\nreachable from the incoming tips but not from the\nalready-connected side, then walks their object closure.  Commit\ntraversal stops at the connectivity boundary, but trees and blobs\nreachable from that boundary still need to be walked so they can\nbe marked uninteresting, allocating a struct object for each one.\nOn repositories where the boundary commits have large trees, the\nconnectivity check for small incoming changes visits and tracks\nmore objects than needed.\n\nAdd an alternative connectivity check that verifies incoming\ncommits incrementally against their parents.\n\nThe verifier processes incoming commits with ancestors first and\ncompares each new tree against its trusted parent trees.  Entries\nalready seen on the trusted side are skipped by OID, so unchanged\nsubtrees need not remain at the same path to be recognized.\nChanged subtrees are recursively compared against same-path parent\nsubtrees, and new subtrees without a comparison base are verified\nfrom scratch.\n\nSee Documentation/technical/connectivity-check.adoc for the trust\ninvariants and detailed algorithm.\n\nThe incremental check runs as an internal\n--verify-trees-incremental mode in rev-list.  After the normal\nrevision walk identifies the incoming commits, the verifier\nconsumes them before the usual object traversal; any pending\nnon-commit tips are still handled by the existing traversal.\n\nPartial-clone semantics are preserved: objects promised by a\npromisor remote are accepted as connected, and on-demand fetching\nis prevented by excluding promisor objects from traversal.  The\nnew mode is selected by transfer.connectivityCheck=incremental,\nwith full remaining the default.  Deepening fetches fall back to\nthe full check because they do not have the normal\nexisting-reference boundary, and traversal instead follows the\neffective shallow boundary.\n\np5412 results (median of 3), scaling one dimension at a time.\nEach fixture is a repository with a flat tree of many\ndirectories containing 100 files each.  The incoming set is a\nchain of commits on top of the existing history, with each\ncommit changing files in different directories round-robin.\nThe three axes vary the total repository tree size, the number\nof incoming commits, and the number of files changed per\nincoming commit.\n\nScaling tree size (10 incoming commits, 10 files/commit):\n\n    files    full  incr.  full/incr\n      5K    0.01s  0.01s    1.0x\n     50K    0.04s  0.01s    4.0x\n    200K    0.15s  0.01s   15.0x\n    800K    0.61s  0.03s   20.3x\n\nThe full mode must walk the trees of boundary commits, which\ngrows with overall tree size.  Incremental still scans the\nroot trees, but avoids descending into unchanged subtrees,\nso it grows much more slowly with repository size.\n\nScaling incoming commit count (200K files, 10 files/commit):\n\n    commits    full  incr.  full/incr\n          1    0.14s  0.01s   14.0x\n         10    0.15s  0.01s   15.0x\n        100    0.19s  0.05s    3.8x\n        500    0.32s  0.27s    1.2x\n       3000    1.23s  1.52s    0.8x\n       5000    1.88s  2.43s    0.8x\n      10000    3.72s  5.36s    0.7x\n\nWith many commits the per-commit overhead of scanning both\nthe new and parent root trees accumulates and incremental\nbecomes slower.  Breakeven is between 500 and 3000 commits\nand the ratio stabilizes near 0.7x for this fixture.\n\nScaling files per incoming commit (200K files, 10 commits):\n\n    files/commit    full  incr.  full/incr\n               1    0.14s  0.01s   14.0x\n              10    0.15s  0.01s   15.0x\n             100    0.16s  0.04s    4.0x\n             500    0.23s  0.14s    1.6x\n            1000    0.34s  0.28s    1.2x\n            2000    0.70s  0.63s    1.1x\n\nBreakeven is around 1000 files/commit.  At 2000 files/commit\n(every directory touched), incremental remains slightly faster;\nbypassing the object cache for tree reads likely helps here.\n\nFor small repositories both modes are fast enough that the\ndifference is difficult to measure reliably.\n\nSelected peak RSS measurements from the same fixtures:\n\n    case                        full    incr.  ratio\n    200K files, 10 commits      31 MB    12 MB   0.4x\n    800K files, 10 commits     110 MB    26 MB   0.2x\n    200K files, 5000 commits   155 MB   151 MB   1.0x\n\nIncremental uses substantially less memory when it can prune\nmost of the boundary tree walk.  In the long-history case the\ntwo modes visit similar object sets and memory converges.\n\nThe regression cases in the CPU benchmarks are in wall-clock\ntime rather than memory, primarily from scanning some trees\nmore than once.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc            |  20 +\n Documentation/rev-list-options.adoc           |   6 +\n .../technical/connectivity-check.adoc         | 134 ++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  19 +\n connected.c                                   |  24 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5412-connectivity-check.perl |  44 ++\n t/perf/p5412-connectivity-check.sh            |  92 +++\n t/t5412-connectivity-check.sh                 | 680 ++++++++++++++++++\n tree-verify.c                                 | 316 ++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1353 insertions(+)\n create mode 100644 t/perf/generate-repo-p5412-connectivity-check.perl\n create mode 100755 t/perf/p5412-connectivity-check.sh\n create mode 100755 t/t5412-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex f1ce50f4a6..b9939d3bde 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -1,3 +1,23 @@\n+transfer.connectivityCheck::\n+\tChoose which algorithm to use for the connectivity check\n+\tperformed during object transfer operations such as\n+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n+\tThe connectivity check verifies that all objects reachable\n+\tfrom the incoming tips are available locally or, in a partial\n+\tclone, promised by a promisor remote.\n+\tThe variants are as follows:\n++\n+--\n+`full` (default);;\n+\tWalk the full object closure of the boundary commits.\n+`incremental`;;\n+\tVerify incoming commits by diffing their trees against parent\n+\ttrees, recursively descending only into entries that differ.\n+\tThe largest benefits occur when incoming commits change a\n+\tsmall fraction of a large tree closure.\n+\tFalls back to `full` for deepening fetches.\n+--\n+\n transfer.credentialsInUrl::\n \tA configured URL can contain plaintext credentials in the form\n \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex fd831f0ec6..7964b691c4 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1089,6 +1089,12 @@ we cannot get their Object ID though, an error will be raised.\n \tstronger than `--missing=allow-promisor` because it limits the\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n+\n+`--verify-trees-incremental`::\n+\t(For internal use only.)  Verify tree connectivity\n+\tincrementally by comparing each commit's tree against its\n+\tparent trees.  Used by `check_connected()` when\n+\t`transfer.connectivityCheck` is set to `incremental`.\n endif::git-rev-list[]\n \n `--no-walk[=(sorted|unsorted)]`::\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nindex d20bff6af6..0a8f370546 100644\n--- a/Documentation/technical/connectivity-check.adoc\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -107,3 +107,137 @@ When a new reference points to a non-commit object, such as a\n tag, tree, or blob, that object is not part of the commit walk.\n These non-commit tips are handled by the subsequent object\n traversal.\n+\n+Incremental connectivity check\n+------------------------------\n+\n+The incremental mode, selected by\n+`transfer.connectivityCheck=incremental`, avoids traversing the\n+full tree walk of the boundary commits.  Instead, it verifies\n+each incoming commit's tree against the already-trusted trees of\n+its parents.\n+\n+Trust model\n+~~~~~~~~~~~\n+\n+A tree is trusted when its transitive object closure is known to\n+be connected.  Trees reachable from commits on the\n+already-connected side of the boundary are therefore trusted.\n+\n+Incoming commits are processed with ancestors before descendants.\n+Once an incoming commit's tree has been verified, it is trusted\n+and can be used as a comparison base for later descendants.\n+\n+This gives an inductive correctness argument: every parent of the\n+commit currently being verified is either already connected or is\n+an earlier incoming commit whose tree has already been verified.\n+\n+Tree states\n+~~~~~~~~~~~\n+\n+The verifier tracks tree OIDs in three states:\n+\n+untrusted::\n+\tThe tree has not yet been established as connected.  This\n+\tis the implicit state of an OID not present in the state\n+\tmap.\n+\n+trusted::\n+\tThe tree is known to have a connected transitive closure,\n+\tbut its direct entries have not yet been published into the\n+\tverifier's trusted object sets.\n+\n+expanded::\n+\tThe tree is trusted and its direct non-gitlink entries\n+\thave also been published into the trusted object sets.\n+\n+State transitions are monotonic: a tree may move from untrusted\n+to trusted to expanded, but never backwards.  An untrusted tree\n+that is successfully verified goes directly to expanded.\n+\n+Blobs require only trusted/untrusted state: a blob becomes\n+trusted when it is found in a trusted tree or when its existence\n+and type have been verified directly.\n+\n+The trusted/expanded distinction is an optimization.  An expanded\n+parent does not need to be reread merely to publish entries that\n+are already trusted, though it may still be read when same-path\n+subtree bases are needed for recursive comparison.\n+\n+Algorithm\n+~~~~~~~~~\n+\n+At a high level:\n+\n+    verify(commits):\n+        sort topologically (ancestors first)\n+        for each commit:\n+            mark parent root trees as trusted\n+            verify_tree(commit.tree, parent root trees)\n+\n+    verify_tree(tree, base_trees):\n+        if tree already trusted: return\n+        read tree, collect entries not already trusted\n+        for each available base tree:\n+            publish its entries as trusted\n+            record same-path subtrees as bases\n+        for each collected entry:\n+            if now trusted: skip\n+            if blob: verify blob connectivity and type\n+            if tree: verify_tree(entry, its recorded bases)\n+        mark tree as expanded\n+\n+The important ordering within `verify_tree` is that entries from\n+the new tree are collected before the base trees are scanned, but\n+are processed only afterwards.  Trust learned from any base can\n+therefore eliminate work before recursive verification begins.\n+\n+Same-path parent subtrees are passed down as comparison bases\n+during recursive descent.  If no comparison base is available,\n+the new subtree is verified from scratch.\n+\n+Worked example\n+~~~~~~~~~~~~~~\n+\n+Consider a commit that changes one file under `lib/` and moves an\n+unchanged subtree from `src/` to `dev/`:\n+\n+    Parent tree              New tree\n+    +-- src/   (aaa)         +-- dev/   (aaa)\n+    +-- lib/   (bbb)         +-- lib/   (ccc)\n+         +-- foo.c (ddd)          +-- foo.c (ddd)\n+         +-- bar.c (eee)          +-- bar.c (fff)\n+\n+Scanning the parent makes `aaa` trusted even though it moved from\n+`src/` to `dev/`, so that subtree is skipped.  The changed `ccc`\n+subtree is compared against its same-path parent `bbb`; scanning\n+`bbb` makes `ddd` and `eee` trusted, leaving only the new `fff`\n+blob to be checked.\n+\n+This illustrates two properties:\n+\n+* Trust is OID-based rather than path-based.  An unchanged subtree\n+  is recognized after a move.\n+\n+* Same-path parent subtrees provide recursive comparison bases.\n+  These bases improve pruning efficiency but are not required for\n+  correctness; a new subtree can always be verified from scratch.\n+\n+Multiple parents\n+~~~~~~~~~~~~~~~~\n+\n+For a merge commit, root trees from all parents are comparison\n+bases.  When several parents contain a same-path subtree, each\n+matching subtree is collected as a recursive comparison base.\n+\n+Entries published from any trusted parent become globally trusted,\n+so a matching tree or blob entry present in any parent can be\n+skipped while verifying the merge tree.\n+\n+Missing comparison bases\n+~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+A promised base tree may not be locally available for comparison.\n+In that case the verifier skips that base and verifies the new\n+subtree without it.  The missing comparison can reduce pruning but\n+does not affect correctness.\ndiff --git a/Makefile b/Makefile\nindex d4b775953d..aa5b4e2b84 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1357,6 +1357,7 @@ LIB_OBJS += trailer.o\n LIB_OBJS += transport-helper.o\n LIB_OBJS += transport.o\n LIB_OBJS += tree-diff.o\n+LIB_OBJS += tree-verify.o\n LIB_OBJS += tree-walk.o\n LIB_OBJS += tree.o\n LIB_OBJS += unpack-trees.o\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 6b596231ab..beaec67d16 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -28,6 +28,7 @@\n #include \"commit-reach.h\"\n #include \"quote.h\"\n #include \"strbuf.h\"\n+#include \"tree-verify.h\"\n \n struct rev_list_info {\n \tstruct rev_info *revs;\n@@ -706,6 +707,7 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n+\tint verify_trees_incremental = 0;\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -748,6 +750,9 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n+\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n+\t\t\tverify_trees_incremental = 1;\n+\t\t\trepo->fetch_if_missing = 0;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n \t\t} else if (!strcmp(arg, \"-z\")) {\n@@ -822,6 +827,8 @@ int cmd_rev_list(int argc,\n \n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n \t\t\tcontinue; /* already handled above */\n+\t\tif (!strcmp(arg, \"--verify-trees-incremental\"))\n+\t\t\tcontinue; /* already handled above */\n \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n \t\t\tcontinue; /* already handled above */\n \n@@ -935,6 +942,18 @@ int cmd_rev_list(int argc,\n \n \tprepare_maximal_independent(&revs);\n \n+\tif (verify_trees_incremental) {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *new_commits = NULL;\n+\n+\t\twhile ((commit = get_revision(&revs)) != NULL)\n+\t\t\tcommit_list_insert(commit, &new_commits);\n+\n+\t\tverify_commits_incremental(repo, &new_commits,\n+\t\t\t\t\t   revs.exclude_promisor_objects);\n+\t\tcommit_list_free(new_commits);\n+\t}\n+\n \tif (revs.tree_objects)\n \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n \ndiff --git a/connected.c b/connected.c\nindex 929b9bd28d..e3dcd8e2b0 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"git-compat-util.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n@@ -67,6 +68,26 @@ static int check_connected_promisor(oid_iterate_fn fn,\n \treturn 1;\n }\n \n+static int incremental_check_applicable(struct check_connected_options *opt)\n+{\n+\tconst char *algorithm = NULL;\n+\n+\tif (repo_config_get_string_tmp(the_repository,\n+\t\t\t\t       \"transfer.connectivitycheck\",\n+\t\t\t\t       &algorithm))\n+\t\treturn 0;\n+\tif (!strcasecmp(algorithm, \"full\"))\n+\t\treturn 0;\n+\tif (strcasecmp(algorithm, \"incremental\"))\n+\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n+\t\t    algorithm);\n+\n+\tif (opt->is_deepening_fetch)\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n /*\n  * If we feed all the commits we want to verify to this command\n  *\n@@ -133,6 +154,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n+\tif (incremental_check_applicable(opt))\n+\t\tstrvec_push(&rev_list.args,\n+\t\t\t    \"--verify-trees-incremental\");\n \n \trev_list.git_cmd = 1;\n \tif (opt->env)\ndiff --git a/meson.build b/meson.build\nindex 0a95d90d21..d94dedc26c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -562,6 +562,7 @@ libgit_sources = [\n   'transport-helper.c',\n   'transport.c',\n   'tree-diff.c',\n+  'tree-verify.c',\n   'tree-walk.c',\n   'tree.c',\n   'unpack-trees.c',\ndiff --git a/t/meson.build b/t/meson.build\nindex 7f53cca7d1..19a8246c44 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -652,6 +652,7 @@ integration_tests = [\n   't5409-colorize-remote-messages.sh',\n   't5410-receive-pack.sh',\n   't5411-proc-receive-hook.sh',\n+  't5412-connectivity-check.sh',\n   't5500-fetch-pack.sh',\n   't5501-fetch-push-alternates.sh',\n   't5502-quickfetch.sh',\ndiff --git a/t/perf/generate-repo-p5412-connectivity-check.perl b/t/perf/generate-repo-p5412-connectivity-check.perl\nnew file mode 100644\nindex 0000000000..9546996280\n--- /dev/null\n+++ b/t/perf/generate-repo-p5412-connectivity-check.perl\n@@ -0,0 +1,44 @@\n+#!/usr/bin/perl\n+#\n+# Generate a fast-import stream for p5412 connectivity check benchmarks.\n+#\n+# Usage: generate-repo-p5412-connectivity-check.perl\n+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n+#\n+# Creates one initial commit with dirs*files_per_dir files, then\n+# <commits> additional commits each modifying <files_per_commit>\n+# files in directories chosen round-robin from 1..<hot_dirs>.\n+\n+use strict;\n+use warnings;\n+\n+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n+$hot = $nd if !$hot || $hot > $nd;\n+$fpc = 1   if !$fpc;\n+\n+sub data {\n+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n+}\n+\n+# Initial tree: one commit with nd*nf files.\n+printf \"commit refs/heads/main\\n\";\n+printf \"committer perf <perf\\@test.com> now\\n\";\n+data(\"initial\");\n+for my $d (1..$nd) {\n+\tfor my $f (1..$nf) {\n+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n+\t}\n+}\n+\n+# Subsequent commits (auto-chained by fast-import).\n+for my $i (1..$nc) {\n+\tprintf \"commit refs/heads/main\\n\";\n+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n+\tdata(sprintf \"change-%03d\", $i);\n+\tfor my $j (0..$fpc-1) {\n+\t\tmy $d = (($i + $j) % $hot) + 1;\n+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n+\t}\n+}\ndiff --git a/t/perf/p5412-connectivity-check.sh b/t/perf/p5412-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..827643a847\n--- /dev/null\n+++ b/t/perf/p5412-connectivity-check.sh\n@@ -0,0 +1,92 @@\n+#!/bin/sh\n+\n+test_description='performance of connectivity check modes\n+\n+Compare the default and incremental rev-list connectivity modes\n+directly, avoiding pack transfer noise.\n+\n+Each repository has a flat tree of many directories with 100 files\n+in each.  Three axes are scaled independently: tree size, commit\n+count, and files changed per commit.'\n+\n+. ./perf-lib.sh\n+\n+test_perf_fresh_repo\n+\n+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5412-connectivity-check.perl\"\n+\n+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n+# $5=files_per_commit (optional, default=1)\n+test_perf_conn () {\n+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n+\tlocal total=$(($nd * $nf))\n+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n+\tlocal label=\"${total} files, ${nc} commits\"\n+\tif test \"$hot\" -lt \"$nd\"\n+\tthen\n+\t\tlabel=\"$label (${hot} hot dirs)\"\n+\tfi\n+\tif test \"$fpc\" -gt 1\n+\tthen\n+\t\tlabel=\"$label (${fpc} files/commit)\"\n+\tfi\n+\n+\ttest_expect_success \"setup $label\" '\n+\t\tif test -d '\"$name\"'\n+\t\tthen\n+\t\t\ttrue\n+\t\telse\n+\t\t\tgit init '\"$name\"' &&\n+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n+\t\t\t(\n+\t\t\t\tcd '\"$name\"' &&\n+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n+\t\t\t\tgit update-ref refs/heads/main \\\n+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n+\t\t\t\tgit repack -ad &&\n+\t\t\t\tgit config gc.auto 0\n+\t\t\t)\n+\t\tfi\n+\t'\n+\n+\ttest_perf \"$label (full)\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+\n+\ttest_perf \"$label (incremental)\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list --verify-trees-incremental \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+}\n+\n+# Scaling tree size (10 commits, 10 files/commit).\n+test_perf_conn   50 100 10   50 10\n+test_perf_conn  500 100 10  500 10\n+test_perf_conn 2000 100 10 2000 10\n+test_perf_conn 8000 100 10 8000 10\n+\n+# Scaling commit count (200K files, 10 files/commit).\n+test_perf_conn 2000 100    1 2000 10\n+test_perf_conn 2000 100   10 2000 10\n+test_perf_conn 2000 100  100 2000 10\n+test_perf_conn 2000 100   500 2000 10\n+test_perf_conn 2000 100  3000 2000 10\n+test_perf_conn 2000 100  5000 2000 10\n+test_perf_conn 2000 100 10000 2000 10\n+\n+# Scaling files per commit (200K files, 10 commits).\n+test_perf_conn 2000 100 10 2000   1\n+test_perf_conn 2000 100 10 2000  10\n+test_perf_conn 2000 100 10 2000  100\n+test_perf_conn 2000 100 10 2000  500\n+test_perf_conn 2000 100 10 2000 1000\n+test_perf_conn 2000 100 10 2000 2000\n+\n+test_done\ndiff --git a/t/t5412-connectivity-check.sh b/t/t5412-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..1c84cfe452\n--- /dev/null\n+++ b/t/t5412-connectivity-check.sh\n@@ -0,0 +1,680 @@\n+#!/bin/sh\n+\n+test_description='connectivity check (transfer.connectivityCheck)'\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_oid_cache <<-\\EOF\n+missing sha1:0000000000000000000000000000000000000001\n+missing sha256:0000000000000000000000000000000000000000000000000000000000000001\n+EOF\n+\n+set_connectivity_check () {\n+\tgit -C \"$1\" config transfer.connectivityCheck \"$2\"\n+}\n+\n+# Run a connectivity check via rev-list directly.  Uses $mode\n+# (set by the enclosing for-loop) to choose full or incremental.\n+check_connected () {\n+\tflags= &&\n+\tif test \"$mode\" = incremental\n+\tthen\n+\t\tflags=--verify-trees-incremental\n+\tfi &&\n+\tprintf '%s\\n' \"$@\" |\n+\tgit rev-list $flags \\\n+\t\t--objects --stdin --not --all --quiet \\\n+\t\t--exclude-promisor-objects\n+}\n+\n+# Run git with a temporary index, leaving the real index untouched.\n+tmpgit () {\n+\tGIT_INDEX_FILE=.git/tmp-idx git \"$@\"\n+}\n+\n+# Create a commit with one file changed, without modifying HEAD,\n+# index, or worktree.  Prints the new commit OID on stdout.\n+# Usage: commit_with_change <parent> <path> <content>\n+commit_with_change () {\n+\tnew_blob=$(echo \"$3\" | git hash-object -w --stdin) &&\n+\ttmpgit read-tree \"$1\" &&\n+\ttmpgit update-index --replace \\\n+\t\t--cacheinfo \"100644,$new_blob,$2\" &&\n+\tnew_tree=$(tmpgit write-tree) &&\n+\trm -f .git/tmp-idx &&\n+\tgit commit-tree \"$new_tree\" -p \"$1\" -m \"modify $2\"\n+}\n+\n+# Check OIDs and optionally verify trace2 counts.\n+# Usage: check_connected_trace <trace-file> <trees> <blobs> <oid>...\n+# An empty string for <trees> or <blobs> skips that assertion.\n+check_connected_trace () {\n+\ttrace_file=$1 trees=$2 blobs=$3 &&\n+\tshift 3 &&\n+\ttest_env GIT_TRACE2_EVENT=\"$(pwd)/$trace_file\" \\\n+\t\tcheck_connected \"$@\" &&\n+\tif test \"$mode\" != incremental\n+\tthen\n+\t\treturn\n+\tfi &&\n+\tif test -n \"$trees\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity trees_loaded \"$trees\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi &&\n+\tif test -n \"$blobs\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity blobs_checked \"$blobs\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi\n+}\n+\n+# Shared setup: a repo with several root-level files and nested dirs.\n+# The unchanged/ subtree (10 dirs x 10 files = 100 blobs, 11 trees)\n+# acts as a canary: any test asserting small tree/blob counts would\n+# fail dramatically if incremental accidentally walked into it.\n+\n+test_expect_success 'setup main repo' '\n+\tgit init main-repo &&\n+\t(\n+\t\tcd main-repo &&\n+\t\tfor i in $(test_seq 1 5)\n+\t\tdo\n+\t\t\techo \"file $i\" >\"file-$i.txt\" || return 1\n+\t\tdone &&\n+\t\tgit add file-*.txt &&\n+\t\tgit commit -m \"initial\" &&\n+\n+\t\tmkdir -p a/b/c &&\n+\t\techo deep >a/b/c/deep.txt &&\n+\t\techo other >a/other.txt &&\n+\t\tgit add a/b/c/deep.txt a/other.txt &&\n+\t\tgit commit -m \"add nested dirs\" &&\n+\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\td=\"unchanged/dir-$i\" &&\n+\t\t\tmkdir -p \"$d\" &&\n+\t\t\tfor j in $(test_seq 1 10)\n+\t\t\tdo\n+\t\t\t\techo \"$i $j\" >\"$d/file-$j.txt\" || return 1\n+\t\t\tdone\n+\t\tdone &&\n+\t\tgit add unchanged/ &&\n+\t\tgit commit -m \"add unchanged canary subtree\"\n+\t)\n+'\n+\n+test_expect_success 'setup replacement object repo' '\n+\tgit init replace-test &&\n+\t(\n+\t\tcd replace-test &&\n+\n+\t\ttest_commit --no-tag original file.txt &&\n+\t\toriginal=$(git rev-parse HEAD) &&\n+\t\torig_blob=$(git rev-parse HEAD:file.txt) &&\n+\n+\t\t# Orphan replacement commit with a different tree\n+\t\treplacement_tree=$(echo replaced | git hash-object -w --stdin |\n+\t\t\txargs -I{} git mktree <<-EOF\n+\t\t\t100644 blob {}\tfile.txt\n+\t\t\tEOF\n+\t\t) &&\n+\t\treplacement=$(git commit-tree -m \"replacement\" \\\n+\t\t\t\"$replacement_tree\") &&\n+\n+\t\tgit replace \"$original\" \"$replacement\" &&\n+\n+\t\t# Remove the original blob so only the replacement\n+\t\t# tree is complete.\n+\t\trm .git/objects/$(test_oid_to_path \"$orig_blob\") &&\n+\n+\t\t# Drop branch and HEAD so --not --all does not\n+\t\t# exclude the original commit.\n+\t\tgit update-ref -d refs/heads/main &&\n+\t\tgit update-ref -d HEAD &&\n+\n+\t\techo \"$original\" >.git/test-oid\n+\t)\n+'\n+\n+missing_oid=$(test_oid missing)\n+original_oid=$(cat replace-test/.git/test-oid)\n+\n+for mode in full incremental\n+do\n+\n+# Corruption detection: craft broken object graphs and verify detection.\n+# All tests use main-repo without modifying its refs or worktree.\n+\n+test_expect_success \"$mode: rejects commit with missing blob\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects commit with missing subtree\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"40000 tree ${missing_oid}\\tdir\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"bad tree object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct tree tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"hello\" | git hash-object -w --stdin) &&\n+\t\tcheck_connected \"$blob_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects missing direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$missing_oid\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects blob OID reused as tree entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\tblob_oid=$(git rev-parse HEAD:file-1.txt) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"40000 subdir\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a tree\" err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects tree OID reused as blob entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\ttree_oid=$(git rev-parse HEAD:a) &&\n+\t\tbin_oid=$(echo \"$tree_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"100644 fakefile\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a blob\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: checks multiple tips\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"tip-a\") &&\n+\t\tc2=$(commit_with_change HEAD file-2.txt \"tip-b\") &&\n+\t\tgit tag -a -m \"tagged\" multi-tag \"$c1\" &&\n+\t\ttag_oid=$(git rev-parse multi-tag) &&\n+\t\tgit tag -d multi-tag &&\n+\t\tcheck_connected \"$c2\" \"$tag_oid\"\n+\t)\n+'\n+\n+# Tree-diff optimization: verify trace2 counts.\n+\n+test_expect_success \"$mode: handles root commit (no parents)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"root-content\" | git hash-object -w --stdin) &&\n+\t\ttree_oid=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$blob_oid\" | git mktree) &&\n+\t\troot_oid=$(git commit-tree \"$tree_oid\" -m \"root\") &&\n+\n+\t\t# No parent trees, so the full tree is verified.\n+\t\t# 1 tree loaded (root), 1 blob checked.\n+\t\tcheck_connected_trace trace-root.txt 1 1 \"$root_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles single file change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD file-1.txt \"changed\") &&\n+\n+\t\t# 2 trees loaded (new root + parent root), 1 blob checked.\n+\t\tcheck_connected_trace trace-flat.txt 2 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles nested change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD a/b/c/deep.txt \"deep-changed\") &&\n+\t\t# 4 new trees + 4 parent trees = 8 loaded, 1 blob checked.\n+\t\tcheck_connected_trace trace-nested.txt 8 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles change-then-revert\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"revert-tmp\") &&\n+\t\tc2=$(commit_with_change \"$c1\" file-1.txt \"file 1\") &&\n+\t\tc3=$(commit_with_change \"$c2\" file-1.txt \"revert-final\") &&\n+\n+\t\t# c1: new root + parent root = 2 loads.  c2: root matches\n+\t\t# HEAD (already trusted), skipped.  c3: new root + parent\n+\t\t# already expanded = 1 load.  Total: 3 trees, 2 blobs.\n+\t\tcheck_connected_trace trace-revert.txt 3 2 \"$c3\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles subtree moved to another path\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tmoved_tree=$(git ls-tree HEAD |\n+\t\t\tsed \"s/\ta$/\tmoved/\" |\n+\t\t\tgit mktree) &&\n+\t\tmoved=$(git commit-tree \"$moved_tree\" -p HEAD -m move) &&\n+\t\t# New root + parent root scanned, but the moved subtree\n+\t\t# (same OID) is trusted and not descended into.\n+\t\tcheck_connected_trace trace-move.txt 2 0 \"$moved\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles multi-parent merge\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tleft=$(commit_with_change HEAD file-1.txt left) &&\n+\t\tright=$(commit_with_change HEAD file-2.txt right) &&\n+\t\tgit update-ref refs/heads/left \"$left\" &&\n+\t\tgit update-ref refs/heads/right \"$right\" &&\n+\t\tleft_blob=$(git rev-parse \"$left:file-1.txt\") &&\n+\t\tright_blob=$(git rev-parse \"$right:file-2.txt\") &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$left_blob,file-1.txt\" &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$right_blob,file-2.txt\" &&\n+\t\tmerge_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tmerge=$(git commit-tree \"$merge_tree\" \\\n+\t\t\t-p \"$left\" -p \"$right\" -m merge) &&\n+\t\t# Both parent root trees are scanned as bases, so\n+\t\t# blobs from each parent are trusted without ODB checks.\n+\t\tcheck_connected_trace trace-merge.txt 3 0 \"$merge\" &&\n+\t\tgit update-ref -d refs/heads/left &&\n+\t\tgit update-ref -d refs/heads/right\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles merge with incoming and boundary parents\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# parent1 is incoming (not a ref), HEAD is boundary.\n+\t\tparent1=$(commit_with_change HEAD file-1.txt merge-inc) &&\n+\t\tmerge=$(git commit-tree \\\n+\t\t\t\"$(git rev-parse \"$parent1^{tree}\")\" \\\n+\t\t\t-p \"$parent1\" -p HEAD -m merge-mixed) &&\n+\t\t# parent1 verified first (topo order): 2 trees, 1 blob.\n+\t\t# merge tree = parent1 tree (already trusted): 0 extra.\n+\t\tcheck_connected_trace trace-merge-inc.txt 2 1 \"$merge\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles gitlink entries (submodules)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"160000,$missing_oid,my-submodule\" &&\n+\t\tgitlink_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tgitlink_commit=$(git commit-tree \"$gitlink_tree\" -p HEAD \\\n+\t\t\t-m \"add gitlink\") &&\n+\n+\t\t# Gitlink entries are skipped -- the missing submodule\n+\t\t# commit OID does not cause a failure.\n+\t\tcheck_connected_trace trace-gitlink.txt 2 0 \"$gitlink_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles file-to-directory transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"foo\" is a blob at root.\n+\t\tblob_a=$(echo \"file-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,foo\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add foo as file\") &&\n+\n+\t\t# Child: \"foo\" becomes a directory (foo/bar.txt).\n+\t\tblob_b=$(echo \"dir-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove foo &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,foo/bar.txt\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"foo: file to directory\") &&\n+\t\tcheck_connected_trace trace-f2d.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles directory-to-file transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"bar/baz.txt\" exists (bar is a directory).\n+\t\tblob_a=$(echo \"nested\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,bar/baz.txt\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add bar as directory\") &&\n+\n+\t\t# Child: \"bar\" becomes a plain file.\n+\t\tblob_b=$(echo \"flat\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove bar/baz.txt &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,bar\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"bar: directory to file\") &&\n+\t\tcheck_connected_trace trace-d2f.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+# Replacement objects.\n+\n+test_expect_success \"$mode: accepts with replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tcheck_connected \"$original_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects without replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tGIT_NO_REPLACE_OBJECTS=1 &&\n+\t\texport GIT_NO_REPLACE_OBJECTS &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$original_oid\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised blob\" '\n+\ttest_when_finished \"rm -rf prom-src prom-server.git prom-client\" &&\n+\tgit init prom-src &&\n+\ttest_commit -C prom-src --no-tag base file.txt original &&\n+\ttest_commit -C prom-src --no-tag \"add file2\" file2.txt extra &&\n+\tgit clone --bare prom-src prom-server.git &&\n+\tgit -C prom-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/prom-server.git\" prom-client &&\n+\t(\n+\t\tcd prom-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file2.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tnewname.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: does not lazy-fetch promised blob\" '\n+\ttest_when_finished \"rm -rf nofetch-src nofetch-server.git nofetch-client\" &&\n+\tgit init nofetch-src &&\n+\ttest_commit -C nofetch-src --no-tag base file.txt content &&\n+\tgit clone --bare nofetch-src nofetch-server.git &&\n+\tgit -C nofetch-server.git config uploadpack.allowfilter true &&\n+\tgit -C nofetch-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/nofetch-server.git\" nofetch-client &&\n+\t(\n+\t\tcd nofetch-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-m \"root commit with promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised tree\" '\n+\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n+\tgit init prom-tree-src &&\n+\tmkdir -p prom-tree-src/a/b &&\n+\ttest_commit -C prom-tree-src --no-tag \"nested dirs\" a/b/file.txt deep &&\n+\tgit clone --bare prom-tree-src prom-tree-server.git &&\n+\tgit -C prom-tree-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-tree-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-tree-server.git\" prom-tree-client &&\n+\t(\n+\t\tcd prom-tree-client &&\n+\t\tpromised_tree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\" &&\n+\t\tnew_tree=$(printf \"40000 tree %s\\trenamed\\n\" \\\n+\t\t\t\"$promised_tree\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised tree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies new subtree when parent subtree is promised\" '\n+\ttest_when_finished \"rm -rf prom-base-src prom-base-server.git prom-base-client\" &&\n+\tgit init prom-base-src &&\n+\tmkdir -p prom-base-src/a &&\n+\ttest_commit -C prom-base-src --no-tag \"base\" a/file.txt deep &&\n+\tgit clone --bare prom-base-src prom-base-server.git &&\n+\tgit -C prom-base-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-base-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-base-server.git\" prom-base-client &&\n+\t(\n+\t\tcd prom-base-client &&\n+\t\tparent_subtree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\" &&\n+\t\tnew_blob=$(echo \"local-content\" | git hash-object -w --stdin) &&\n+\t\tnew_subtree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$new_blob\" | git mktree) &&\n+\t\tnew_root=$(printf \"40000 tree %s\\ta\\n\" \\\n+\t\t\t\"$new_subtree\" | git mktree) &&\n+\t\tnew_commit=$(git commit-tree \"$new_root\" \\\n+\t\t\t-p HEAD -m \"replace promised subtree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies local commit in partial clone\" '\n+\ttest_when_finished \"rm -rf pc-src pc-server.git pc-client\" &&\n+\tgit init pc-src &&\n+\ttest_commit -C pc-src --no-tag base file.txt &&\n+\tgit clone --bare pc-src pc-server.git &&\n+\tgit -C pc-server.git config uploadpack.allowfilter true &&\n+\tgit -C pc-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/pc-server.git\" pc-client &&\n+\t(\n+\t\tcd pc-client &&\n+\t\tlocal_commit=$(commit_with_change HEAD file.txt local-content) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: respects shallow boundary\" '\n+\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n+\tgit init shallow-src &&\n+\ttest_commit -C shallow-src --no-tag base file content-1 &&\n+\tmkdir shallow-src/sub &&\n+\ttest_commit -C shallow-src --no-tag change sub/other content-2 &&\n+\tgit clone --depth=1 \"file://$(pwd)/shallow-src\" shallow &&\n+\t(\n+\t\tcd shallow &&\n+\t\ttip=$(git rev-parse HEAD) &&\n+\t\tgit for-each-ref --format=\"delete %(refname)\" |\n+\t\t\tgit update-ref --no-deref --stdin &&\n+\t\tcheck_connected \"$tip\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: deepening fetch succeeds\" '\n+\ttest_when_finished \"rm -rf deepen-src deepen-server.git deepen-client\" &&\n+\tgit init deepen-src &&\n+\ttest_commit -C deepen-src --no-tag c1 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c2 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c3 file.txt &&\n+\tgit clone --bare deepen-src deepen-server.git &&\n+\tgit clone --depth=1 \"file://$(pwd)/deepen-server.git\" deepen-client &&\n+\tset_connectivity_check deepen-client $mode &&\n+\ttest -f deepen-client/.git/shallow &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/deepen-trace.txt\" \\\n+\t\tgit -C deepen-client fetch --deepen=2 origin main &&\n+\t# Incremental falls back to full for deepening fetches,\n+\t# so the trees_loaded event should not appear.\n+\ttest_grep ! trees_loaded deepen-trace.txt\n+'\n+\n+test_expect_success \"$mode: malformed tree detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\techo abc >malformed-tree &&\n+\t\tmalformed_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tmalformed-tree) &&\n+\t\tmalformed_commit=$(git commit-tree \"$malformed_tree\" \\\n+\t\t\t-p HEAD -m malformed) &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$malformed_commit\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: mid-tree corruption detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# Build a tree with one valid entry followed by garbage.\n+\t\tblob_oid=$(echo \"valid\" | git hash-object -w --stdin) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\t\tprintf \"100644 good\\0${bin_oid}GARBAGE\" >corrupt-mid-tree &&\n+\t\tcorrupt_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tcorrupt-mid-tree) &&\n+\t\tcorrupt_commit=$(git commit-tree \"$corrupt_tree\" \\\n+\t\t\t-p HEAD -m \"mid-tree corruption\") &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$corrupt_commit\" 2>err &&\n+\t\ttest_grep \"too-short tree object\" err\n+\t)\n+'\n+\n+done\n+\n+# Algorithm selection.\n+\n+test_expect_success 'invalid transfer.connectivityCheck is rejected' '\n+\ttest_when_finished \"rm -rf invalid-cfg-src invalid-cfg-dst\" &&\n+\tgit init invalid-cfg-src &&\n+\ttest_commit -C invalid-cfg-src --no-tag base file.txt &&\n+\tgit clone invalid-cfg-src invalid-cfg-dst &&\n+\ttest_commit -C invalid-cfg-src --no-tag update file.txt updated &&\n+\tgit -C invalid-cfg-dst config transfer.connectivityCheck bogus &&\n+\ttest_must_fail git -C invalid-cfg-dst fetch origin main 2>err &&\n+\ttest_grep \"unknown transfer.connectivityCheck\" err\n+'\n+\n+test_expect_success 'push uses incremental when configured' '\n+\ttest_when_finished \"rm -rf int-src int-dst.git\" &&\n+\tgit init int-src &&\n+\ttest_commit -C int-src --no-tag base file.txt &&\n+\tgit clone --bare int-src int-dst.git &&\n+\ttest_commit -C int-src --no-tag update file.txt updated &&\n+\tset_connectivity_check int-dst.git incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/push-trace.txt\" \\\n+\t\tgit -C int-src push ../int-dst.git main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<push-trace.txt\n+'\n+\n+test_expect_success 'fetch uses incremental when configured' '\n+\ttest_when_finished \"rm -rf fetch-src fetch-dst\" &&\n+\tgit init fetch-src &&\n+\ttest_commit -C fetch-src --no-tag base file.txt &&\n+\tgit clone fetch-src fetch-dst &&\n+\ttest_commit -C fetch-src --no-tag update file.txt updated &&\n+\tset_connectivity_check fetch-dst incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/fetch-trace.txt\" \\\n+\t\tgit -C fetch-dst fetch origin main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<fetch-trace.txt\n+'\n+\n+test_expect_success 'clone respects transfer.connectivityCheck' '\n+\ttest_when_finished \"rm -rf clone-src clone-dst\" &&\n+\tgit init clone-src &&\n+\ttest_commit -C clone-src --no-tag base file.txt &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/clone-trace.txt\" \\\n+\t\tgit -c transfer.connectivityCheck=incremental \\\n+\t\tclone --no-local clone-src clone-dst &&\n+\ttest_trace2_data_singular connectivity trees_loaded 0 \\\n+\t\t<clone-trace.txt\n+'\n+\n+test_done\ndiff --git a/tree-verify.c b/tree-verify.c\nnew file mode 100644\nindex 0000000000..5c11c2251a\n--- /dev/null\n+++ b/tree-verify.c\n@@ -0,0 +1,316 @@\n+#include \"git-compat-util.h\"\n+#include \"commit.h\"\n+#include \"gettext.h\"\n+#include \"hex.h\"\n+#include \"khash.h\"\n+#include \"object.h\"\n+#include \"odb.h\"\n+#include \"oid-array.h\"\n+#include \"oidset.h\"\n+#include \"tree.h\"\n+#include \"tree-walk.h\"\n+#include \"tree-verify.h\"\n+#include \"packfile.h\"\n+#include \"trace2.h\"\n+\n+enum tree_state {\n+\tTREE_UNTRUSTED = 0,\n+\tTREE_TRUSTED   = 1,\n+\tTREE_EXPANDED  = 2,\n+};\n+\n+KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n+\t   oidhash_by_value, oideq_by_value)\n+\n+static enum tree_state tree_map_get(kh_oid_tree_t *m,\n+\t\t\t\t    const struct object_id *oid)\n+{\n+\tkhint_t pos = kh_get_oid_tree(m, *oid);\n+\tif (pos == kh_end(m))\n+\t\treturn TREE_UNTRUSTED;\n+\treturn kh_val(m, pos);\n+}\n+\n+static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n+\t\t\t enum tree_state state)\n+{\n+\tint added;\n+\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n+\tif (added)\n+\t\tkh_val(m, pos) = state;\n+\telse if (state > kh_val(m, pos))\n+\t\tkh_val(m, pos) = state;\n+}\n+\n+struct work_item {\n+\tstruct name_entry entry;\n+\tstruct oid_array parent_trees;\n+};\n+\n+struct verify_state {\n+\tkh_oid_tree_t *trees;\n+\tstruct oidset trusted_blobs;\n+\tint trees_loaded;\n+\tint blobs_checked;\n+\tint exclude_promisor_objects;\n+};\n+\n+/*\n+ * Merge-walk the work list against one base tree entry, recording\n+ * same-path parent subtrees as recursive comparison bases.\n+ * Returns the updated work-list cursor.\n+ */\n+static size_t collect_subtree_bases(struct work_item *work, size_t nr_work,\n+\t\t\t\t    size_t wi, const struct name_entry *entry)\n+{\n+\twhile (wi < nr_work) {\n+\t\tint cmp = base_name_compare(\n+\t\t\twork[wi].entry.path, work[wi].entry.pathlen,\n+\t\t\twork[wi].entry.mode,\n+\t\t\tentry->path, entry->pathlen,\n+\t\t\tentry->mode);\n+\t\tif (cmp > 0)\n+\t\t\tbreak;\n+\t\tif (cmp < 0) {\n+\t\t\twi++;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (S_ISDIR(work[wi].entry.mode) &&\n+\t\t    S_ISDIR(entry->mode))\n+\t\t\toid_array_append(&work[wi].parent_trees,\n+\t\t\t\t\t &entry->oid);\n+\t\treturn wi + 1;\n+\t}\n+\treturn wi;\n+}\n+\n+static void verify_blob(struct repository *repo,\n+\t\t\tconst struct object_id *oid,\n+\t\t\tstruct verify_state *vs)\n+{\n+\tint type;\n+\n+\tif (oidset_contains(&vs->trusted_blobs, oid))\n+\t\treturn;\n+\n+\tvs->blobs_checked++;\n+\ttype = odb_read_object_info(repo->objects, oid, NULL);\n+\tif (type == OBJ_BLOB) {\n+\t\toidset_insert(&vs->trusted_blobs, oid);\n+\t\treturn;\n+\t}\n+\tif (type >= 0)\n+\t\tdie(_(\"object %s is a %s, not a %s\"),\n+\t\t    oid_to_hex(oid), type_name(type), \"blob\");\n+\tif (vs->exclude_promisor_objects &&\n+\t    is_promisor_object(repo, oid))\n+\t\treturn;\n+\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n+}\n+\n+static void *read_tree_object(struct object_database *odb,\n+\t\t\t      const struct object_id *oid,\n+\t\t\t      size_t *sizep)\n+{\n+\tenum object_type type;\n+\tvoid *buf = odb_read_object(odb, oid, &type, sizep);\n+\n+\tif (buf && type != OBJ_TREE) {\n+\t\tfree(buf);\n+\t\tdie(_(\"object %s is a %s, not a %s\"),\n+\t\t    oid_to_hex(oid), type_name(type), \"tree\");\n+\t}\n+\treturn buf;\n+}\n+\n+static void verify_tree(struct repository *repo,\n+\t\t\tconst struct object_id *new_tree_oid,\n+\t\t\tconst struct oid_array *base_trees,\n+\t\t\tstruct verify_state *vs, int depth)\n+{\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tstruct work_item *work = NULL;\n+\tsize_t nr_work = 0, alloc_work = 0;\n+\tint need_subtree_bases = 0;\n+\tsize_t i, tree_size;\n+\tvoid *tree_buf;\n+\n+\tif (depth > repo->settings.max_allowed_tree_depth)\n+\t\tdie(_(\"exceeded maximum allowed tree depth\"));\n+\n+\tif (tree_map_get(vs->trees, new_tree_oid) >= TREE_TRUSTED)\n+\t\treturn;\n+\n+\ttree_buf = read_tree_object(repo->objects, new_tree_oid, &tree_size);\n+\tif (!tree_buf) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, new_tree_oid))\n+\t\t\treturn;\n+\t\tdie(_(\"bad tree object %s\"),\n+\t\t    oid_to_hex(new_tree_oid));\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tinit_tree_desc(&desc, new_tree_oid, tree_buf, tree_size);\n+\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (S_ISGITLINK(entry.mode))\n+\t\t\tcontinue;\n+\t\tif (S_ISDIR(entry.mode)) {\n+\t\t\tif (tree_map_get(vs->trees, &entry.oid) >= TREE_TRUSTED)\n+\t\t\t\tcontinue;\n+\t\t\tneed_subtree_bases = 1;\n+\t\t} else {\n+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry.oid))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\tALLOC_GROW(work, nr_work + 1, alloc_work);\n+\t\tmemset(&work[nr_work], 0, sizeof(work[nr_work]));\n+\t\twork[nr_work].entry = entry;\n+\t\tnr_work++;\n+\t}\n+\n+\tif (!nr_work) {\n+\t\tfree(tree_buf);\n+\t\tgoto done;\n+\t}\n+\n+\tfor (i = 0; base_trees && i < base_trees->nr; i++) {\n+\t\tconst struct object_id *base_oid = &base_trees->oid[i];\n+\t\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n+\t\tstruct tree_desc base_desc;\n+\t\tstruct name_entry scan_entry;\n+\t\tsize_t wi = 0, base_size;\n+\t\tvoid *base_buf;\n+\n+\t\tif (expanded && !need_subtree_bases)\n+\t\t\tcontinue;\n+\n+\t\tbase_buf = read_tree_object(repo->objects, base_oid,\n+\t\t\t\t\t    &base_size);\n+\t\tif (!base_buf) {\n+\t\t\tif (vs->exclude_promisor_objects &&\n+\t\t\t    is_promisor_object(repo, base_oid))\n+\t\t\t\tcontinue;\n+\t\t\tdie(_(\"bad tree object %s\"),\n+\t\t\t    oid_to_hex(base_oid));\n+\t\t}\n+\n+\t\tvs->trees_loaded++;\n+\t\tinit_tree_desc(&base_desc, base_oid, base_buf, base_size);\n+\n+\t\twhile (tree_entry(&base_desc, &scan_entry)) {\n+\t\t\tif (S_ISGITLINK(scan_entry.mode))\n+\t\t\t\tcontinue;\n+\n+\t\t\tif (need_subtree_bases)\n+\t\t\t\twi = collect_subtree_bases(work, nr_work,\n+\t\t\t\t\t\t\t   wi, &scan_entry);\n+\n+\t\t\tif (!expanded) {\n+\t\t\t\tif (S_ISDIR(scan_entry.mode))\n+\t\t\t\t\ttree_map_add(vs->trees,\n+\t\t\t\t\t\t     &scan_entry.oid,\n+\t\t\t\t\t\t     TREE_TRUSTED);\n+\t\t\t\telse\n+\t\t\t\t\toidset_insert(&vs->trusted_blobs,\n+\t\t\t\t\t\t      &scan_entry.oid);\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!expanded)\n+\t\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n+\n+\t\tfree(base_buf);\n+\t}\n+\n+\tfor (i = 0; i < nr_work; i++) {\n+\t\tif (S_ISDIR(work[i].entry.mode))\n+\t\t\tverify_tree(repo, &work[i].entry.oid,\n+\t\t\t\t    &work[i].parent_trees, vs,\n+\t\t\t\t    depth + 1);\n+\t\telse\n+\t\t\tverify_blob(repo, &work[i].entry.oid, vs);\n+\t}\n+\n+\tfree(tree_buf);\n+\n+done:\n+\ttree_map_add(vs->trees, new_tree_oid, TREE_EXPANDED);\n+\tfor (i = 0; i < nr_work; i++)\n+\t\toid_array_clear(&work[i].parent_trees);\n+\tfree(work);\n+}\n+\n+static void verify_commit_tree(struct repository *repo,\n+\t\t\t       struct commit *commit,\n+\t\t\t       struct verify_state *vs)\n+{\n+\tstruct oid_array base_trees = OID_ARRAY_INIT;\n+\tstruct commit_list *p;\n+\n+\t/*\n+\t * Parent trees are trusted: boundary parents are already\n+\t * connected, and earlier incoming parents were verified\n+\t * first due to the topological processing order.\n+\t */\n+\tfor (p = commit->parents; p; p = p->next) {\n+\t\tconst struct object_id *tree_oid;\n+\t\tparse_commit_or_die(p->item);\n+\t\ttree_oid = get_commit_tree_oid(p->item);\n+\t\tif (!tree_oid)\n+\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n+\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n+\t\toid_array_append(&base_trees, tree_oid);\n+\t}\n+\n+\tif (!get_commit_tree_oid(commit))\n+\t\tdie(_(\"unable to load root tree for commit %s\"),\n+\t\t    oid_to_hex(&commit->object.oid));\n+\tverify_tree(repo, get_commit_tree_oid(commit),\n+\t\t    &base_trees, vs, 0);\n+\toid_array_clear(&base_trees);\n+}\n+\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects)\n+{\n+\tstruct verify_state vs = { 0 };\n+\tstruct commit_list *iter;\n+\tunsigned nr_before;\n+\n+\tif (repo->fetch_if_missing)\n+\t\tBUG(\"verify_commits_incremental must not be called \"\n+\t\t    \"with fetch_if_missing set\");\n+\n+\tvs.trees = kh_init_oid_tree();\n+\tvs.exclude_promisor_objects = exclude_promisor_objects;\n+\n+\t/*\n+\t * Ancestors must be verified before descendants so that parent\n+\t * trees can be trusted without re-verification.  Sort explicitly\n+\t * rather than relying on the caller's ordering.\n+\t *\n+\t * sort_in_topological_order() silently drops cycle members,\n+\t * so explicitly check if the size has changed.\n+\t */\n+\tnr_before = commit_list_count(*commits);\n+\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n+\tif (commit_list_count(*commits) < nr_before)\n+\t\tdie(_(\"cycle detected in incoming commit graph\"));\n+\n+\t*commits = commit_list_reverse(*commits);\n+\n+\tfor (iter = *commits; iter; iter = iter->next)\n+\t\tverify_commit_tree(repo, iter->item, &vs);\n+\n+\tkh_destroy_oid_tree(vs.trees);\n+\toidset_clear(&vs.trusted_blobs);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"trees_loaded\", vs.trees_loaded);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"blobs_checked\", vs.blobs_checked);\n+}\ndiff --git a/tree-verify.h b/tree-verify.h\nnew file mode 100644\nindex 0000000000..6aadadff70\n--- /dev/null\n+++ b/tree-verify.h\n@@ -0,0 +1,15 @@\n+#ifndef TREE_VERIFY_H\n+#define TREE_VERIFY_H\n+\n+struct commit_list;\n+struct repository;\n+\n+/*\n+ * Verify trees of commits incrementally against their parents.\n+ * Dies on verification failure.\n+ */\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects);\n+\n+#endif /* TREE_VERIFY_H */\n-- \ngitgitgadget\n"},{"id":"554151","messageId":"asNY7SfEohsOSf0J@pks.im","threadId":"66326","inReplyTo":"97c11449aeae924436ba22a00a2545254e988a58.1790600552.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-05T07:59:41Z","receivedAt":"2026-10-05T07:59:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Sep 28, 2026 at 01:02:31PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> new file mode 100644\n> index 0000000000..d20bff6af6\n> --- /dev/null\n> +++ b/Documentation/technical/connectivity-check.adoc\n> @@ -0,0 +1,109 @@\n> +Connectivity checking\n> +=====================\n> +\n> +After receiving new objects via fetch, push (receive-pack), clone,\n> +or bundle, Git verifies that the new reference tips do not leave\n> +the repository in a state where reachable objects are missing.\n> +This verification is called the connectivity check.\n> +\n> +Connectivity invariant\n> +----------------------\n> +\n> +A repository is connected when every object reachable from its\n> +references is available locally (with exceptions noted below).\n\nRight. I think it would also be important to spell out the reverse of\nthis, which is that nothing can be assumed about objects that aren't\nreachable by any reference. So even if an object already exists in the\nobject database, it is not safe to assume that it is fully connected\nunless it is referenced.\n\n> +The connectivity check maintains this invariant when references\n> +are updated.  It trusts the existing connected state and verifies\n\nNit: it's basically already implicit, but I'd clarify that \"existing\nconnected state\" is again just the connected state of objects reachable\nfrom reference tips. So maybe \"It trusts that all objects reachable from\nreferences are already fully connected and verifies...\"\n\n> +that the new reference tips do not introduce references to\n> +unavailable objects.  Verification is permitted to stop when it\n> +reaches objects already reachable from trusted existing\n> +references, since their closure is already connected.  These\n> +trusted references include local references and references from\n> +alternate object stores.\n> +\n> +Without this check, a truncated or corrupted transfer could leave\n> +a repository in a state where later history walks encounter\n> +missing objects.\n> +\n> +Exceptions\n> +~~~~~~~~~~\n> +\n> +Gitlink entries (submodule references) are excluded from\n> +connectivity checking.  Their target objects belong to a separate\n> +repository.\n> +\n> +In partial clones, objects promised by a promisor remote are\n> +accepted as connected without requiring local existence.  The\n> +check excludes promisor objects from traversal so that it does\n> +not trigger on-demand fetches for them.\n> +\n> +Full connectivity check\n> +-----------------------\n> +\n> +`check_connected()` (see `connected.c`) normally performs the\n\nI'm always a bit hesitant to directly refer to code in our docs. We\nshould either make this documentation part of \"connected.c\" directly, or\nwe should not refer to code. Otherwise, chances that this documentation\ngrows stale is very high.\n\n> +connectivity check using a `rev-list` subprocess, feeding the\n> +new reference tips via stdin.  A normal invocation is roughly:\n> +\n> +    git rev-list --objects --stdin --not --all --quiet\n> +        --alternate-refs [--exclude-promisor-objects]\n> +\n> +When promisor remotes are configured, `check_connected()` first\n> +attempts a fast path based on promisor packfiles.  If it falls\n> +back to the `rev-list` check, `--exclude-promisor-objects` is\n> +added so that the traversal does not trigger on-demand fetches.\n> +\n> +Consider the following graph after a fetch, where all reference\n> +tips point directly to commits.  For simplicity, only local\n> +references appear on the already-connected side; alternate refs\n> +play the same role.  N3 is a merge commit:\n> +\n> +            /-------------L2\n> +           /\n> +    C1---B1---C2---B2-----L1\n> +          \\         \\\n> +           N1        N3---T2\n> +            \\       /\n> +             N2-----------T1\n> +\n> +    L1, L2:         local refs\n> +    T1, T2:         incoming tips (new refs)\n> +    N1, N2, N3:     incoming commits (N3 is a merge)\n> +    B1, B2:         boundary commits (already connected)\n> +    C1, C2:         already connected (but not boundary)\n> +\n> +The incoming set is the commits reachable from the incoming\n> +tips but not from the already-connected side.  Boundary commits\n> +are the already-connected commits at the edge of that set.  Here\n> +B1 is an ancestor of B2, which happens when incoming branches\n> +fork at different depths in the existing history.\n> +\n> +The check proceeds in three phases:\n> +\n> +1. Walk from the incoming tips (T1, T2) against the trusted\n> +   refs (L1, L2) to find the incoming set ({N1, N2, N3, T1, T2}).\n> +\n> +2. Walk the trees of the boundary commits (B1, B2) and mark\n> +   those objects uninteresting.  These trees are already trusted\n> +   because their commits are on the already-connected side.\n> +\n> +3. Walk the trees of each incoming commit and verify that every\n> +   referenced object is connected, stopping at objects already\n> +   marked uninteresting in phase 2.\n\nI feel like these phases here basically just explain how revision walks\nwork without adding any more details that are specifically relevant to\nthe connectivity check.\n\n> +Deepening fetches\n> +~~~~~~~~~~~~~~~~~\n> +\n> +For deepening fetches (where the shallow boundary moves), the\n> +full check omits `--not --all`.  There is no existing-reference\n> +boundary at which the walk can stop.  Instead, traversal follows\n> +the effective shallow boundary supplied for the deepened\n> +repository.  The new content may be below the old shallow\n> +boundary even when the tips themselves have not changed.\n> +\n> +Non-commit tips\n> +~~~~~~~~~~~~~~~\n> +\n> +When a new reference points to a non-commit object, such as a\n> +tag, tree, or blob, that object is not part of the commit walk.\n> +These non-commit tips are handled by the subsequent object\n> +traversal.\n\nHuh, what subsequent object traversal? This part puzzles me a bit.\n\nPatrick\n"},{"id":"554152","messageId":"asNZD7AOC6QL9q1d@pks.im","threadId":"66326","inReplyTo":"6ad528f4bb42a960910eb4fe917a3766fa55d598.1790600552.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/2] connected: add incremental connectivity check via rev-list","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-05T08:00:15Z","receivedAt":"2026-10-05T08:00:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Sep 28, 2026 at 01:02:32PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson <krka@spotify.com>\n> \n> The full connectivity check uses rev-list to find commits\n> reachable from the incoming tips but not from the\n> already-connected side, then walks their object closure.  Commit\n> traversal stops at the connectivity boundary, but trees and blobs\n> reachable from that boundary still need to be walked so they can\n> be marked uninteresting, allocating a struct object for each one.\n> On repositories where the boundary commits have large trees, the\n> connectivity check for small incoming changes visits and tracks\n> more objects than needed.\n\nOkay. In the old world we basically mark evertyhing as uninteresting,\nincluding trees and blobs, ...\n\n> Add an alternative connectivity check that verifies incoming\n> commits incrementally against their parents.\n> \n> The verifier processes incoming commits with ancestors first and\n> compares each new tree against its trusted parent trees.  Entries\n> already seen on the trusted side are skipped by OID, so unchanged\n> subtrees need not remain at the same path to be recognized.\n> Changed subtrees are recursively compared against same-path parent\n> subtrees, and new subtrees without a comparison base are verified\n> from scratch.\n\n... whereas in the new world you propose to skip marking trees/blobs as\nuninteresting. Instead, the idea is to compare the trees/blobs of the\nold tips directly with the trees/blobs of the new tips and only verify\nthose parts that have changed between the two?\n\nThis can of course cause us to verify significantly more objects in some\nscenarios. But it does have the consequence that we scale with the\nnumber of changes, not with the number of preexisting objects in the\nrepository. And that's something I'd really appreciate, because marking\nreachable objects as uninteresting is extremely expensive.\n\nOne thing I wonder though... does this help with the scenario where we\nhave tons of references or do we still end up passing \"--not --all\"? I\nhave seen many times that parsing the refs by itself is dominating the\ntime of the connectivity check quite significantly. So ideally, I'd like\nto have a solution that also catches this case. Your benchmarks do not\ncover that scenario though.\n\n> Incremental uses substantially less memory when it can prune\n> most of the boundary tree walk.  In the long-history case the\n> two modes visit similar object sets and memory converges.\n> \n> The regression cases in the CPU benchmarks are in wall-clock\n> time rather than memory, primarily from scanning some trees\n> more than once.\n> \n> Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n> ---\n>  Documentation/config/transfer.adoc            |  20 +\n>  Documentation/rev-list-options.adoc           |   6 +\n>  .../technical/connectivity-check.adoc         | 134 ++++\n>  Makefile                                      |   1 +\n>  builtin/rev-list.c                            |  19 +\n>  connected.c                                   |  24 +\n>  meson.build                                   |   1 +\n>  t/meson.build                                 |   1 +\n>  ...enerate-repo-p5412-connectivity-check.perl |  44 ++\n>  t/perf/p5412-connectivity-check.sh            |  92 +++\n>  t/t5412-connectivity-check.sh                 | 680 ++++++++++++++++++\n>  tree-verify.c                                 | 316 ++++++++\n>  tree-verify.h                                 |  15 +\n>  13 files changed, 1353 insertions(+)\n>  create mode 100644 t/perf/generate-repo-p5412-connectivity-check.perl\n>  create mode 100755 t/perf/p5412-connectivity-check.sh\n>  create mode 100755 t/t5412-connectivity-check.sh\n>  create mode 100644 tree-verify.c\n>  create mode 100644 tree-verify.h\n\nMay I suggest splitting up this patch in the following way?\n\n  - One commit that introduces the new option, but for now only accepts\n    \"full\" as the algorithm.\n\n  - One commit that introduces the benchmark.\n\n  - One commit that introduces the new flag for git-rev-list(1).\n\n  - One commit that then introduces the new strategy.\n\nThat may make it a bit easier to focus on the actual change.\n\n> diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> index d20bff6af6..0a8f370546 100644\n> --- a/Documentation/technical/connectivity-check.adoc\n> +++ b/Documentation/technical/connectivity-check.adoc\n> @@ -107,3 +107,137 @@ When a new reference points to a non-commit object, such as a\n>  tag, tree, or blob, that object is not part of the commit walk.\n>  These non-commit tips are handled by the subsequent object\n>  traversal.\n> +\n> +Incremental connectivity check\n> +------------------------------\n> +\n> +The incremental mode, selected by\n> +`transfer.connectivityCheck=incremental`, avoids traversing the\n> +full tree walk of the boundary commits.  Instead, it verifies\n> +each incoming commit's tree against the already-trusted trees of\n> +its parents.\n\nCan we define \"parents\" here? Specifically, I wonder how you define\n\"parent\" in the case where you perform a force push or when creating a\nnew reference. Is it the parent of the first new commit? Is it the old\nstate of the ref, if it even exists?\n\n> +Trust model\n> +~~~~~~~~~~~\n> +\n> +A tree is trusted when its transitive object closure is known to\n> +be connected.  Trees reachable from commits on the\n> +already-connected side of the boundary are therefore trusted.\n\nWhere the \"already-connected side of the boundary\" is anything reachable\nvia a reference.\n\n> +Incoming commits are processed with ancestors before descendants.\n> +Once an incoming commit's tree has been verified, it is trusted\n> +and can be used as a comparison base for later descendants.\n> +\n> +This gives an inductive correctness argument: every parent of the\n> +commit currently being verified is either already connected or is\n> +an earlier incoming commit whose tree has already been verified.\n\nRight. The big question to me still is how you identify\nalready-connected trees without having to read all references.\n\n> +Worked example\n\nWorked?\n\n[snip]\n> diff --git a/tree-verify.c b/tree-verify.c\n> new file mode 100644\n> index 0000000000..5c11c2251a\n> --- /dev/null\n> +++ b/tree-verify.c\n> @@ -0,0 +1,316 @@\n[snip]\n> +static void verify_commit_tree(struct repository *repo,\n> +\t\t\t       struct commit *commit,\n> +\t\t\t       struct verify_state *vs)\n> +{\n> +\tstruct oid_array base_trees = OID_ARRAY_INIT;\n> +\tstruct commit_list *p;\n> +\n> +\t/*\n> +\t * Parent trees are trusted: boundary parents are already\n> +\t * connected, and earlier incoming parents were verified\n> +\t * first due to the topological processing order.\n> +\t */\n\nI feel like I still miss where exactly you establish the trust boundary\nbetween preexisting fully-connected commits and new commits.\n\n> +\tfor (p = commit->parents; p; p = p->next) {\n> +\t\tconst struct object_id *tree_oid;\n> +\t\tparse_commit_or_die(p->item);\n> +\t\ttree_oid = get_commit_tree_oid(p->item);\n> +\t\tif (!tree_oid)\n> +\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n> +\t\t\t    oid_to_hex(&p->item->object.oid));\n> +\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n> +\t\toid_array_append(&base_trees, tree_oid);\n> +\t}\n> +\n> +\tif (!get_commit_tree_oid(commit))\n> +\t\tdie(_(\"unable to load root tree for commit %s\"),\n> +\t\t    oid_to_hex(&commit->object.oid));\n> +\tverify_tree(repo, get_commit_tree_oid(commit),\n> +\t\t    &base_trees, vs, 0);\n> +\toid_array_clear(&base_trees);\n> +}\n> +\n> +void verify_commits_incremental(struct repository *repo,\n> +\t\t\t\tstruct commit_list **commits,\n> +\t\t\t\tint exclude_promisor_objects)\n> +{\n> +\tstruct verify_state vs = { 0 };\n> +\tstruct commit_list *iter;\n> +\tunsigned nr_before;\n> +\n> +\tif (repo->fetch_if_missing)\n> +\t\tBUG(\"verify_commits_incremental must not be called \"\n> +\t\t    \"with fetch_if_missing set\");\n> +\n> +\tvs.trees = kh_init_oid_tree();\n> +\tvs.exclude_promisor_objects = exclude_promisor_objects;\n> +\n> +\t/*\n> +\t * Ancestors must be verified before descendants so that parent\n> +\t * trees can be trusted without re-verification.  Sort explicitly\n> +\t * rather than relying on the caller's ordering.\n> +\t *\n> +\t * sort_in_topological_order() silently drops cycle members,\n> +\t * so explicitly check if the size has changed.\n> +\t */\n> +\tnr_before = commit_list_count(*commits);\n> +\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n> +\tif (commit_list_count(*commits) < nr_before)\n> +\t\tdie(_(\"cycle detected in incoming commit graph\"));\n\nI don't think we should just die, should we? That may not interact well\nwith git-receive-pack(1) and others that expect a broken connectivity\ncheck to bubble up errors so that they can properly report those to the\nclient and clean up their local state.\n\n> +\t*commits = commit_list_reverse(*commits);\n> +\n> +\tfor (iter = *commits; iter; iter = iter->next)\n> +\t\tverify_commit_tree(repo, iter->item, &vs);\n\n> +\tkh_destroy_oid_tree(vs.trees);\n> +\toidset_clear(&vs.trusted_blobs);\n> +\ttrace2_data_intmax(\"connectivity\", repo,\n> +\t\t\t   \"trees_loaded\", vs.trees_loaded);\n> +\ttrace2_data_intmax(\"connectivity\", repo,\n> +\t\t\t   \"blobs_checked\", vs.blobs_checked);\n> +}\n\nPatrick\n"},{"id":"554214","messageId":"xmqqece4j6t4.fsf@gitster.g","threadId":"66326","inReplyTo":"asNY7SfEohsOSf0J@pks.im","subject":"Re: [PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-05T19:17:27Z","receivedAt":"2026-10-05T19:17:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n>> +Full connectivity check\n>> +-----------------------\n>> +\n>> +`check_connected()` (see `connected.c`) normally performs the\n>\n> I'm always a bit hesitant to directly refer to code in our docs. We\n> should either make this documentation part of \"connected.c\" directly, or\n> we should not refer to code. Otherwise, chances that this documentation\n> grows stale is very high.\n\nThis is totally outside the topic of documentation updates, but it\nmakes me wonder if we should pay attention to connectivity roots\nother than refs (like index entries) that we use when we run fsck.\n\n"},{"id":"554237","messageId":"asSOJVUTS3BMq6kS@pks.im","threadId":"66326","inReplyTo":"xmqqece4j6t4.fsf@gitster.g","subject":"Re: [PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-06T05:59:01Z","receivedAt":"2026-10-06T05:59:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 05, 2026 at 12:17:27PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> >> +Full connectivity check\n> >> +-----------------------\n> >> +\n> >> +`check_connected()` (see `connected.c`) normally performs the\n> >\n> > I'm always a bit hesitant to directly refer to code in our docs. We\n> > should either make this documentation part of \"connected.c\" directly, or\n> > we should not refer to code. Otherwise, chances that this documentation\n> > grows stale is very high.\n> \n> This is totally outside the topic of documentation updates, but it\n> makes me wonder if we should pay attention to connectivity roots\n> other than refs (like index entries) that we use when we run fsck.\n\nHmm, I'm not sure. I guess performance of the connectivity check is\ntypically an issue on the server side only, much less so on the client\nside. And the server would of course typically not even have an index\nentry at all. Same for reflogs, at least in many setups.\n\nI also wonder whether that'd really speed things up if we add more data\nsources. At GitLab we typically have the problem that we have too many\nconnectivity roots with refs alone, and that is making the whole check\npainfully slow in some repositories. So adding more connectivity roots\nto it would probably be counterproductive.\n\nPatrick\n\n"},{"id":"554260","messageId":"CAL71e4PzQJz16BvP=zaVoS25R1ezai1R9UBZ2XDhNSYFYEpknA@mail.gmail.com","threadId":"66326","inReplyTo":"asSOJVUTS3BMq6kS@pks.im","subject":"Re: [PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-06T10:03:03Z","receivedAt":"2026-10-06T10:03:03Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Tue, 6 Oct 2026 at 07:59, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> On Mon, Oct 05, 2026 at 12:17:27PM -0700, Junio C Hamano wrote:\n> >\n> > This is totally outside the topic of documentation updates, but it\n> > makes me wonder if we should pay attention to connectivity roots\n> > other than refs (like index entries) that we use when we run fsck.\n>\n> Hmm, I'm not sure. I guess performance of the connectivity check is\n> typically an issue on the server side only, much less so on the client\n> side. And the server would of course typically not even have an index\n> entry at all. Same for reflogs, at least in many setups.\n>\n> I also wonder whether that'd really speed things up if we add more data\n> sources. At GitLab we typically have the problem that we have too many\n> connectivity roots with refs alone, and that is making the whole check\n> painfully slow in some repositories. So adding more connectivity roots\n> to it would probably be counterproductive.\n\nI have some followup work in this area (but I want to keep focus)\nand it's indeed true that number of refs used as seeds is also an\nimportant part of the problem.\n\nI see connectivity and reachability as two separate things,\nthough the current implementation has an invariant based on\nreachability implying (e.g. requiring) connectivity.\n\nI think we could build up a stronger connectivity invariant\nfor the common case, where we can cheaply check if an object\nis connectivity-closed regardless of its reachability state.\n\nOne idea is to utilize the fact that MIDX maps out a very\nspecific set of packs, so we could compute which set of packs\nform a connectivity closure (and cheaply update that information\nas the MIDX changes).\n\nAnother idea would be to dynamically load refs as seeds\nin parallel with walking the graph.  This creates a tradeoff\nbetween how many objects we need to check and how many refs\nwe need to load (if we load fewer refs we just need to walk\nfurther to reach common ground).\n\nSorry, this is a bit hand-wave:y but my point is that I think\nregardless of how this is followed up to solve the bigger\nproblems, I think optimizing the object-tree traversal is a\nuseful first step.\n\nThanks,\nKristofer\n\n"},{"id":"554261","messageId":"CAL71e4Pd3ziR-7bbBDUei-pcF69mEpt33fE51WC-H=7YgXq=NQ@mail.gmail.com","threadId":"66326","inReplyTo":"asNY7SfEohsOSf0J@pks.im","subject":"Re: [PATCH v2 1/2] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-06T10:12:40Z","receivedAt":"2026-10-06T10:12:40Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"Thanks, it's clear I'll need to do some more wordsmithing to\nimprove the clarity here.  I'll reroll/rewrite parts of it in\nsome way (and aligned with the answers below).\n\nOn Mon, 5 Oct 2026 at 09:59, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> > +A repository is connected when every object reachable from its\n> > +references is available locally (with exceptions noted below).\n>\n> Right. I think it would also be important to spell out the reverse of\n> this, which is that nothing can be assumed about objects that aren't\n> reachable by any reference. So even if an object already exists in the\n> object database, it is not safe to assume that it is fully connected\n> unless it is referenced.\n\nFair, I can make that more explicit in the text.\n\n>\n> > +The connectivity check maintains this invariant when references\n> > +are updated.  It trusts the existing connected state and verifies\n>\n> Nit: it's basically already implicit, but I'd clarify that \"existing\n> connected state\" is again just the connected state of objects reachable\n> from reference tips. So maybe \"It trusts that all objects reachable from\n> references are already fully connected and verifies...\"\n\nAgreed, I will make that more clear.\n\n> > +Full connectivity check\n> > +-----------------------\n> > +\n> > +`check_connected()` (see `connected.c`) normally performs the\n>\n> I'm always a bit hesitant to directly refer to code in our docs. We\n> should either make this documentation part of \"connected.c\" directly, or\n> we should not refer to code. Otherwise, chances that this documentation\n> grows stale is very high.\n\nGood point, I will remove it and make the logic more self contained.\n\n> > +The check proceeds in three phases:\n> > +\n> > +1. Walk from the incoming tips (T1, T2) against the trusted\n> > +   refs (L1, L2) to find the incoming set ({N1, N2, N3, T1, T2}).\n> > +\n> > +2. Walk the trees of the boundary commits (B1, B2) and mark\n> > +   those objects uninteresting.  These trees are already trusted\n> > +   because their commits are on the already-connected side.\n> > +\n> > +3. Walk the trees of each incoming commit and verify that every\n> > +   referenced object is connected, stopping at objects already\n> > +   marked uninteresting in phase 2.\n>\n> I feel like these phases here basically just explain how revision walks\n> work without adding any more details that are specifically relevant to\n> the connectivity check.\n\nYes, though I think this is important for understanding how the\nconnectivity-check is implemented to see the relation to the\nrev-list walk.  I have some hope that this dependency can go\naway in the future, since the general rev-list function is not\nnecessarily the most optimal way to reason about connectivity.\n\nThat said, I am happy to remove this part of it's not deemed\nuseful.\n\n> > +When a new reference points to a non-commit object, such as a\n> > +tag, tree, or blob, that object is not part of the commit walk.\n> > +These non-commit tips are handled by the subsequent object\n> > +traversal.\n>\n> Huh, what subsequent object traversal? This part puzzles me a bit.\n\nGood point, this is meant to reference back to the rev-list\nimplementation but that's not very clear.  And like\nthe previous feedback, perhaps this should also go away\n(or be reworded in a more self contained way and less tied\nto rev-list)\n\nThanks,\nKristofer\n\n"},{"id":"554266","messageId":"CAL71e4PFpMPoSxFdnscRFiMB3ozudr64TjeQc8VKcO_M_MfJ=w@mail.gmail.com","threadId":"66326","inReplyTo":"asNZD7AOC6QL9q1d@pks.im","subject":"Re: [PATCH v2 2/2] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-06T10:37:40Z","receivedAt":"2026-10-06T10:37:40Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 5 Oct 2026 at 10:00, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> ... whereas in the new world you propose to skip marking trees/blobs as\n> uninteresting. Instead, the idea is to compare the trees/blobs of the\n> old tips directly with the trees/blobs of the new tips and only verify\n> those parts that have changed between the two?\n\nYes, kind of, but I am not sure what you meant by old tips and\nnew tips -- to be clear (and I should also write this more\nclearly) we only verify the commit-trees for all incoming\ncommits, and the comparisons are always between a commit and\nits parents (i.e. X and X^1, X^2, ...).  The key insight that\nmakes the approach work is that we can skip any object we have\nseen from a trusted base (e.g. a commit that was already\nreachable before).\n\n>\n> This can of course cause us to verify significantly more objects in some\n> scenarios. But it does have the consequence that we scale with the\n> number of changes, not with the number of preexisting objects in the\n> repository. And that's something I'd really appreciate, because marking\n> reachable objects as uninteresting is extremely expensive.\n\nYes, that's the goal in the happy case -- and I would argue it\ndoes not produce significantly more work in the worst case,\ndue to caching and remembering everything we've seen so far\n(only more work within a 2x bound or so).\n\n> One thing I wonder though... does this help with the scenario where we\n> have tons of references or do we still end up passing \"--not --all\"? I\n> have seen many times that parsing the refs by itself is dominating the\n> time of the connectivity check quite significantly. So ideally, I'd like\n> to have a solution that also catches this case. Your benchmarks do not\n> cover that scenario though.\n\nYes, the problem of scaling out with number of refs is a real\nproblem (and often a bigger one) and I want to address that\ntoo, but it's out of scope for this particular patch series.\n\n> May I suggest splitting up this patch in the following way?\n>\n>   - One commit that introduces the new option, but for now only accepts\n>     \"full\" as the algorithm.\n>\n>   - One commit that introduces the benchmark.\n>\n>   - One commit that introduces the new flag for git-rev-list(1).\n>\n>   - One commit that then introduces the new strategy.\n>\n> That may make it a bit easier to focus on the actual change.\n\nYes, I can definitely do that.  Initially I was thinking that\nsince it was basically only additions no obvious good in-between\nstate, it wouldn't help much to split it up, but I think you\nconvinced me there.\n\n> > +The incremental mode, selected by\n> > +`transfer.connectivityCheck=incremental`, avoids traversing the\n> > +full tree walk of the boundary commits.  Instead, it verifies\n> > +each incoming commit's tree against the already-trusted trees of\n> > +its parents.\n>\n> Can we define \"parents\" here? Specifically, I wonder how you define\n> \"parent\" in the case where you perform a force push or when creating a\n> new reference. Is it the parent of the first new commit? Is it the old\n> state of the ref, if it even exists?\n\nParent is always defined relative to the commit we are\ncurrently verifying.  For example, a push may come with\n3 commits (let's call the tip T), and then we do the\nfollowing comparisons:\n\n    T   vs T^1, T^2\n    T~1 vs T~1^1, T~1^2\n    T~2 vs T~2^1, T~2^2\n\nT~2^1 and T~2^2 must already exist and be reachable and\nso we can trust them to be connected.  And since this is\nrelying on memoizing already seen results, it's important\nto run the checks bottom-up (reverse topological order).\n\nI will see if I can make this more obvious in the commit\nmessage or documentation somehow.\n\n> > +Trust model\n> > +~~~~~~~~~~~\n> > +\n> > +A tree is trusted when its transitive object closure is known to\n> > +be connected.  Trees reachable from commits on the\n> > +already-connected side of the boundary are therefore trusted.\n>\n> Where the \"already-connected side of the boundary\" is anything reachable\n> via a reference.\n\nYes, precisely.\n\n> > +Incoming commits are processed with ancestors before descendants.\n> > +Once an incoming commit's tree has been verified, it is trusted\n> > +and can be used as a comparison base for later descendants.\n> > +\n> > +This gives an inductive correctness argument: every parent of the\n> > +commit currently being verified is either already connected or is\n> > +an earlier incoming commit whose tree has already been verified.\n>\n> Right. The big question to me still is how you identify\n> already-connected trees without having to read all references.\n\nThat part works just as before -- rev-list finds the\nalready-connected commits implicitly with the --not --all query.\nIt actually finds all the new commits, but we can deduce the\nboundary from there (and the pre-existing rev-list code also does\nthat).\n\n> > +Worked example\n>\n> Worked?\n\nHm, I suppose I could just use the phrase \"Example\" here instead.\nWill change.\n\n>\n> [snip]\n> > diff --git a/tree-verify.c b/tree-verify.c\n> > new file mode 100644\n> > index 0000000000..5c11c2251a\n> > --- /dev/null\n> > +++ b/tree-verify.c\n> > @@ -0,0 +1,316 @@\n> [snip]\n> > +static void verify_commit_tree(struct repository *repo,\n> > +                            struct commit *commit,\n> > +                            struct verify_state *vs)\n> > +{\n> > +     struct oid_array base_trees = OID_ARRAY_INIT;\n> > +     struct commit_list *p;\n> > +\n> > +     /*\n> > +      * Parent trees are trusted: boundary parents are already\n> > +      * connected, and earlier incoming parents were verified\n> > +      * first due to the topological processing order.\n> > +      */\n>\n> I feel like I still miss where exactly you establish the trust boundary\n> between preexisting fully-connected commits and new commits.\n\nThis is the same as before -- git rev-list produces the trust\nboundary based on reachability.  I think the only new thing here\nis the inductive leap.  Once we have verified a commit just above\nthe trust boundary, that itself becomes a new trust boundary.\n\n> > +     if (commit_list_count(*commits) < nr_before)\n> > +             die(_(\"cycle detected in incoming commit graph\"));\n>\n> I don't think we should just die, should we? That may not interact well\n> with git-receive-pack(1) and others that expect a broken connectivity\n> check to bubble up errors so that they can properly report those to the\n> client and clean up their local state.\n\nThis is one of the advantages of running within a sub-process --\nwe can safely die without breaking things -- and this is in fact\nhow the existing rev-list based implementation work, it will also\ndie with an error message / return code that the parent process\npicks up.\n\nMy original implementation tried to do it all within a single\nprocess but it became painful because a lot of the internal\nmachinery did not have non-fatal variants.\n\nThat said, I think long term it would be good to rework it\ninto a single process and have the right infrastructure in place\nto avoid dying.\n\nThanks,\nKristofer\n\n"},{"id":"554271","messageId":"asTkyiIZvX1ztMrH@pks.im","threadId":"66326","inReplyTo":"CAL71e4PFpMPoSxFdnscRFiMB3ozudr64TjeQc8VKcO_M_MfJ=w@mail.gmail.com","subject":"Re: [PATCH v2 2/2] connected: add incremental connectivity check via rev-list","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-06T12:08:42Z","receivedAt":"2026-10-06T12:08:42Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Oct 06, 2026 at 12:37:40PM +0200, Kristofer Karlsson wrote:\n> On Mon, 5 Oct 2026 at 10:00, Patrick Steinhardt <ps@pks.im> wrote:\n[snip]\n> > > +The incremental mode, selected by\n> > > +`transfer.connectivityCheck=incremental`, avoids traversing the\n> > > +full tree walk of the boundary commits.  Instead, it verifies\n> > > +each incoming commit's tree against the already-trusted trees of\n> > > +its parents.\n> >\n> > Can we define \"parents\" here? Specifically, I wonder how you define\n> > \"parent\" in the case where you perform a force push or when creating a\n> > new reference. Is it the parent of the first new commit? Is it the old\n> > state of the ref, if it even exists?\n> \n> Parent is always defined relative to the commit we are\n> currently verifying.  For example, a push may come with\n> 3 commits (let's call the tip T), and then we do the\n> following comparisons:\n> \n>     T   vs T^1, T^2\n>     T~1 vs T~1^1, T~1^2\n>     T~2 vs T~2^1, T~2^2\n> \n> T~2^1 and T~2^2 must already exist and be reachable and\n> so we can trust them to be connected.  And since this is\n> relying on memoizing already seen results, it's important\n> to run the checks bottom-up (reverse topological order).\n\nThis is the part that still eludes me though. How do we know that T~2^1\nand T~2^2 must already exist and be reachable?\n\nI think I was coming in with a false expectation that we're somehow\ngetting rid of marking preexistingrefs as uninteresting, and that is\nwhere my confusion comes from. Because ultimately, that does not seem to\nbe the case -- we still mark reference tips as uninteresting, as far as\nI can see. And then we can of course easily determine whether a specific\ncommit is preexisting because we marked the boundary as uninteresting.\n\nI was probably primed by my own earlier patch series in this context\nthat focussed on refs, and that may be the reason why I had skewed\nexpectations.\n\n> > > +Incoming commits are processed with ancestors before descendants.\n> > > +Once an incoming commit's tree has been verified, it is trusted\n> > > +and can be used as a comparison base for later descendants.\n> > > +\n> > > +This gives an inductive correctness argument: every parent of the\n> > > +commit currently being verified is either already connected or is\n> > > +an earlier incoming commit whose tree has already been verified.\n> >\n> > Right. The big question to me still is how you identify\n> > already-connected trees without having to read all references.\n> \n> That part works just as before -- rev-list finds the\n> already-connected commits implicitly with the --not --all query.\n> It actually finds all the new commits, but we can deduce the\n> boundary from there (and the pre-existing rev-list code also does\n> that).\n\nYeah.\n\n> > > diff --git a/tree-verify.c b/tree-verify.c\n> > > new file mode 100644\n> > > index 0000000000..5c11c2251a\n> > > --- /dev/null\n> > > +++ b/tree-verify.c\n> > > @@ -0,0 +1,316 @@\n> > [snip]\n> > > +static void verify_commit_tree(struct repository *repo,\n> > > +                            struct commit *commit,\n> > > +                            struct verify_state *vs)\n> > > +{\n> > > +     struct oid_array base_trees = OID_ARRAY_INIT;\n> > > +     struct commit_list *p;\n> > > +\n> > > +     /*\n> > > +      * Parent trees are trusted: boundary parents are already\n> > > +      * connected, and earlier incoming parents were verified\n> > > +      * first due to the topological processing order.\n> > > +      */\n> >\n> > I feel like I still miss where exactly you establish the trust boundary\n> > between preexisting fully-connected commits and new commits.\n> \n> This is the same as before -- git rev-list produces the trust\n> boundary based on reachability.  I think the only new thing here\n> is the inductive leap.  Once we have verified a commit just above\n> the trust boundary, that itself becomes a new trust boundary.\n> \n> > > +     if (commit_list_count(*commits) < nr_before)\n> > > +             die(_(\"cycle detected in incoming commit graph\"));\n> >\n> > I don't think we should just die, should we? That may not interact well\n> > with git-receive-pack(1) and others that expect a broken connectivity\n> > check to bubble up errors so that they can properly report those to the\n> > client and clean up their local state.\n> \n> This is one of the advantages of running within a sub-process --\n> we can safely die without breaking things -- and this is in fact\n> how the existing rev-list based implementation work, it will also\n> die with an error message / return code that the parent process\n> picks up.\n\nAh, right, I forgot that we're running in a separate process. I think\nthis will also become a bit clearer once this series is split up into\nsmaller individual steps.\n\nThanks!\n\nPatrick\n\n"},{"id":"554277","messageId":"CAL71e4OCuwT=Wj8eMF0DJ3UXFocb24yssLf4buyFDGEnTR6mgw@mail.gmail.com","threadId":"66326","inReplyTo":"asTkyiIZvX1ztMrH@pks.im","subject":"Re: [PATCH v2 2/2] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-06T12:36:27Z","receivedAt":"2026-10-06T12:36:27Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Tue, 6 Oct 2026 at 14:08, Patrick Steinhardt <ps@pks.im> wrote:\n> > Parent is always defined relative to the commit we are\n> > currently verifying.  For example, a push may come with\n> > 3 commits (let's call the tip T), and then we do the\n> > following comparisons:\n> >\n> >     T   vs T^1, T^2\n> >     T~1 vs T~1^1, T~1^2\n> >     T~2 vs T~2^1, T~2^2\n> >\n> > T~2^1 and T~2^2 must already exist and be reachable and\n> > so we can trust them to be connected.  And since this is\n> > relying on memoizing already seen results, it's important\n> > to run the checks bottom-up (reverse topological order).\n>\n> This is the part that still eludes me though. How do we know that T~2^1\n> and T~2^2 must already exist and be reachable?\n\nSince T~2 is the bottom of the incoming commits, its parent\ncommits must be part of the boundary -- by definition, since\notherwise they would instead by included by rev-list --not --all.\n\nHere is a concrete example with a push of 3 commits, where N1\nis a merge commit and we have two pre-existing refs R1 and R2\n(T, T~1, T~2 maps to N3, N2 and N1 respectively here):\n\n    B1---R1\n     \\\n      N1---N2---N3\n     /\n    B2---R2\n\n    --not --all produces: {N1, N2, N3}\n\n    B1 and B2 are boundary (reachable from existing refs R1 and R2)\n\n    Verification (bottom-up):\n    N1: compare N1.tree vs B1.tree and B2.tree\n        (B1, B2 are boundary -- not in incoming set,\n         so connected by definition)\n    N2: compare N2.tree vs N1.tree\n        (N1 just verified, trusted inductively)\n    N3:  compare N3.tree vs N2.tree\n        (N2 just verified, trusted inductively)\n\nWe never walk B1 or B2's full trees to mark objects\nuninteresting.  We only read their root trees as comparison\nbases when they are direct parents of an incoming commit.\nThat is where the savings come from.\n\nSo to directly answer your question: we know T~2's parents\n(B1 and B2 in this example) are connected because --not --all\ntold us so -- they were not produced by the commit walk, which\nmeans they are reachable from existing refs.  This is the same\ntrust boundary the full check uses.  The only new idea is the\ninductive step: once we verify a commit just above the boundary,\nit itself becomes trusted and extends the boundary upward.\n\n>\n> I think I was coming in with a false expectation that we're somehow\n> getting rid of marking preexistingrefs as uninteresting, and that is\n> where my confusion comes from. Because ultimately, that does not seem to\n> be the case -- we still mark reference tips as uninteresting, as far as\n> I can see. And then we can of course easily determine whether a specific\n> commit is preexisting because we marked the boundary as uninteresting.\n\nYes, precisely.\n\n> I was probably primed by my own earlier patch series in this context\n> that focussed on refs, and that may be the reason why I had skewed\n> expectations.\n\nYes, I've noted a few other workstreams that are sort of\ntouching similar areas, though not directly the connectivity\ncheck, so it's a lot of context to juggle at the same time\n(and I will need to take that into account for the next steps).\n\nThanks,\nKristofer\n\n"},{"id":"554750","messageId":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH v3 0/4] connected: add incremental connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-12T12:07:53Z","receivedAt":"2026-10-12T12:07:53Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"This series adds an incremental mode for the connectivity check, gated\nbehind transfer.connectivityCheck=incremental (no expected changes unless\nyou opt in).\n\nThe intent is to solve the problem of the connectivity check slowing down as\nthe number of reachable objects from the boundary grows.\n\nIt relates to the RFC I sent out earlier:\n\n[RFC] check_connected: toward incoming-proportional cost\nhttps://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/\n\n\n\nDesign\n======\n\nThe verifier runs inside the same rev-list subprocess that check_connected()\nalready spawns, triggered by a new internal flag --verify-trees-incremental.\nAfter get_revision() collects the incoming commits, the verifier processes\nthem in topological order (ancestors before descendants).\n\nThe idea is to keep a set of trusted objects, shared across the incoming\ncommits, that grows over time. We visit the new/untrusted commit trees and\ndo a comparison walk over the trees of their parents. Entries discovered on\nthe trusted parent side are remembered as trusted, which lets later\nverification skip matching objects and avoid descending into unchanged\nsubtrees.\n\nMore algorithmic details are in\nDocumentation/technical/connectivity-check.adoc.\n\n\nBenchmarks\n==========\n\nI'll just mention a short summary here, to avoid repeating what's already in\nthe commit message. The incremental mode is faster than the full mode when\nthere are few commits to verify and when the active object tree is large. In\nthe happy case, the work tracks the changed paths and their comparison trees\nrather than the full reachable object closure, which substantially reduces\nthe dependence on total repository size. I've seen speedups up to around 20x\nfor the synthetic perf tests.\n\nRunning against a large real-world repo (3.4 GB boundary closure), the\nnumbers are more dramatic. All timings use rev-list directly with --not\nHEADN, isolating the tree verification cost from the boundary-finding cost:\n\ncommits    full  incr.  speedup   full RSS  incr. RSS\n      1    1.9s  0.01s    190x      3.4 GB     14 MB\n     10    1.9s  0.04s     48x      3.4 GB    125 MB\n    100    1.9s  0.37s      5x      3.4 GB    1.1 GB\n\n\nThe full mode takes ~1.9s regardless of commit count because it is dominated\nby walking the boundary closure. Incremental scales with the number of\nincoming commits and the paths they touch. Memory follows the same pattern:\nincremental uses a fraction of the full mode's RSS for small pushes,\nconverging only when many commits are verified.\n\nThere are also regression cases in the synthetic fixtures. With long\nincoming histories, the extra parent-tree scans accumulate; in the synthetic\nfixture incremental is about 1.5x slower at 10000 commits. Per-commit\nchanges have less impact than expected: even when every directory is\ntouched, incremental remains competitive; bypassing the object cache for\ntree reads likely helps here.\n\nI cannot establish how common these regression cases are. In the cases I\nhave tested, however, the regression has remained modest; I have not been\nable to provoke a substantially larger slowdown. My feeling is that this is\nan acceptable tradeoff behind the opt-in config, since the target case\n(small pushes to large repos) sees the largest speedup, while the regression\nappears with long incoming histories.\n\nA safety net for this regression could be to dynamically disable the\nincremental mode if the number of incoming commits is too large, but this is\nleft out of the initial version to avoid overly speculative code.\n\nDeepening fetches currently fall back to the full check because the full\ncheck omits --not --all for deepening -- there is no existing-reference\nboundary at which the walk can stop. An incremental approach is possible\nhere too -- using the old shallow roots as the trusted boundary and walking\nthe deepened ancestry forward -- but that is a separate change and left for\nfuture work. Deepening is also less common than regular fetch and\nreceive-pack, where the speedup matters most.\n\n\nTest coverage\n=============\n\nMost correctness cases in t5413-connectivity-check.sh are run in both full\nand incremental modes to check semantic equivalence. Selected cases\nadditionally assert trace2 tree/blob counts for the incremental mode, to\nverify that unchanged portions of the object graph are actually skipped. It\ncovers:\n\n * Corruption detection: missing blobs, missing trees, type mismatches,\n   malformed trees (unparseable, mid-tree corruption)\n * Tree optimization: trace2 assertions confirm unchanged subtrees are\n   skipped, subtree moves, merge parent boundaries\n * Root commits (no parents -- verifies full tree closure)\n * Partial clones: missing promised blobs, missing promised trees,\n   verification of local commits\n * Replacement objects (with and without GIT_NO_REPLACE_OBJECTS)\n * Shallow boundaries\n * Deepening fetches (falls back to full check)\n * Integration: real push, fetch, and clone\n\nMost tests call git rev-list directly with the appropriate flags;\nintegration tests exercise the full check_connected() path through push,\nfetch, and clone.\n\n\nAlternatives considered\n=======================\n\nMy first prototype ran the verifier in-process inside connected.c. This\nrequired a second rev-list subprocess just for boundary finding, _nofetch\nvariants of several object-reading functions to prevent lazy fetches in\npartial clones, explicit shallow-file plumbing, and careful avoidance of\ndie() in all code paths reachable from the verifier. The result worked but\nwas fragile and touched many files.\n\nMoving the verifier into the rev-list subprocess eliminated all of those\nproblems: in partial clones the existing --exclude-promisor-objects handling\nalready disables lazy fetching, die() is isolated by the process boundary,\nshallow and replacement semantics are established before the verifier runs,\nand error routing comes for free via stderr.\n\nSo while I liked the idea of being less reliant on checking within a\nsubprocess, making that work ended up being a lot more complex.\n\n\nNext steps\n==========\n\nThis series only addresses tree verification; the other significant cost is\nfinding the commit boundary, especially for repos with many refs. I already\nhave some prototypes for optimizing that too, and if this ends up landing,\nthat would be something I would start polishing up.\n\n\nChanges since v2\n================\n\nAddresses Patrick's review (thanks!).\n\n * Split from 2 commits to 4: documentation, perf test baseline, config\n   plumbing + rev-list flag, then the incremental implementation.\n * Documentation rewrites: added unreachable-objects caveat, removed code\n   references to reduce staleness risk, replaced three-phase list with\n   cost-focused prose, rewrote trust model to explicitly describe boundary\n   mechanism and inductive argument, and several smaller clarifications.\n * Extracted tree_oid variable in verify_commit_tree() to avoid calling\n   get_commit_tree_oid(commit) twice.\n * Rebased onto latest master to resolve a meson.build conflict (t5412 was\n   taken by another topic). Renumbered test and perf files to t5413/p5413.\n\n\nChanges since v1\n================\n\n * Guard against get_commit_tree_oid() returning NULL in\n   verify_commit_tree(), dying with a clear message instead of a NULL\n   dereference. (Thanks Junio for catching this)\n * Set fetch_if_missing = 0 in the rev-list early option parse when\n   --verify-trees-incremental is seen, and add a BUG() assertion in\n   verify_commits_incremental() to catch any future caller that forgets.\n   This makes the lazy-fetch protection explicit rather than relying solely\n   on --exclude-promisor-objects having set it earlier. (Thanks again to\n   Junio)\n * Add a test that verifies promised blobs are not lazy-fetched during the\n   incremental check.\n * Reuse the existing 3-argument type-mismatch format string to avoid adding\n   new strings for localization.\n\nThanks, Kristofer\n\nKristofer Karlsson (4):\n  Documentation: describe connectivity checking\n  p5413: add perf tests for connectivity check\n  connected: add transfer.connectivityCheck config and rev-list flag\n  connected: add incremental connectivity check via rev-list\n\n Documentation/config/transfer.adoc            |  20 +\n Documentation/rev-list-options.adoc           |   6 +\n .../technical/connectivity-check.adoc         | 255 +++++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  19 +\n connected.c                                   |  24 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5413-connectivity-check.perl |  44 ++\n t/perf/p5413-connectivity-check.sh            |  92 +++\n t/t5413-connectivity-check.sh                 | 680 ++++++++++++++++++\n tree-verify.c                                 | 316 ++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1474 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\n\nbase-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2211%2Fspkrka%2Ftree-diff-connectivity-v1-clean-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2211/spkrka/tree-diff-connectivity-v1-clean-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/2211\n\nRange-diff vs v2:\n\n 1:  97c11449ae ! 1:  19ea8cb534 Documentation: describe connectivity checking\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +\n      +A repository is connected when every object reachable from its\n      +references is available locally (with exceptions noted below).\n     ++The reverse does not hold: objects that exist in the object\n     ++database but are not reachable from any reference may be\n     ++incomplete or lack a connected closure.\n      +\n      +The connectivity check maintains this invariant when references\n     -+are updated.  It trusts the existing connected state and verifies\n     -+that the new reference tips do not introduce references to\n     -+unavailable objects.  Verification is permitted to stop when it\n     -+reaches objects already reachable from trusted existing\n     -+references, since their closure is already connected.  These\n     -+trusted references include local references and references from\n     -+alternate object stores.\n     ++are updated.  It trusts that all objects reachable from existing\n     ++reference tips are already fully connected and verifies that the\n     ++new reference tips do not introduce references to unavailable\n     ++objects.  Verification is permitted to stop when it reaches\n     ++objects already reachable from existing references, since their\n     ++closure is already connected.  These trusted references include\n     ++local references and references from alternate object stores.\n      +\n      +Without this check, a truncated or corrupted transfer could leave\n      +a repository in a state where later history walks encounter\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +Full connectivity check\n      +-----------------------\n      +\n     -+`check_connected()` (see `connected.c`) normally performs the\n     -+connectivity check using a `rev-list` subprocess, feeding the\n     -+new reference tips via stdin.  A normal invocation is roughly:\n     ++The full connectivity check runs a `rev-list` subprocess, feeding\n     ++the new reference tips via stdin.  A normal invocation is roughly:\n      +\n      +    git rev-list --objects --stdin --not --all --quiet\n      +        --alternate-refs [--exclude-promisor-objects]\n      +\n     -+When promisor remotes are configured, `check_connected()` first\n     -+attempts a fast path based on promisor packfiles.  If it falls\n     -+back to the `rev-list` check, `--exclude-promisor-objects` is\n     -+added so that the traversal does not trigger on-demand fetches.\n     ++When promisor remotes are configured, a fast path based on\n     ++promisor packfiles is attempted first.  If it falls back to\n     ++the `rev-list` check, `--exclude-promisor-objects` is added so\n     ++that the traversal does not trigger on-demand fetches.\n      +\n      +Consider the following graph after a fetch, where all reference\n      +tips point directly to commits.  For simplicity, only local\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +B1 is an ancestor of B2, which happens when incoming branches\n      +fork at different depths in the existing history.\n      +\n     -+The check proceeds in three phases:\n     ++The `--not --all` arguments establish the trust boundary: the\n     ++incoming set is the commits reachable from the new tips but not\n     ++from any existing reference.\n      +\n     -+1. Walk from the incoming tips (T1, T2) against the trusted\n     -+   refs (L1, L2) to find the incoming set ({N1, N2, N3, T1, T2}).\n     -+\n     -+2. Walk the trees of the boundary commits (B1, B2) and mark\n     -+   those objects uninteresting.  These trees are already trusted\n     -+   because their commits are on the already-connected side.\n     -+\n     -+3. Walk the trees of each incoming commit and verify that every\n     -+   referenced object is connected, stopping at objects already\n     -+   marked uninteresting in phase 2.\n     ++The traversal then walks the trees of the boundary commits and\n     ++marks those objects uninteresting.  Trees of incoming commits\n     ++are verified against these marks, stopping at objects already\n     ++covered.  The cost of marking boundary trees uninteresting\n     ++dominates when the boundary commits have large trees, even when\n     ++the incoming changes are small.\n      +\n      +Deepening fetches\n      +~~~~~~~~~~~~~~~~~\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +\n      +When a new reference points to a non-commit object, such as a\n      +tag, tree, or blob, that object is not part of the commit walk.\n     -+These non-commit tips are handled by the subsequent object\n     -+traversal.\n     ++These non-commit tips are verified during the tree and blob\n     ++traversal that follows the commit walk within the same rev-list\n     ++invocation.\n -:  ---------- > 2:  211794c925 p5413: add perf tests for connectivity check\n -:  ---------- > 3:  cdf28f1df6 connected: add transfer.connectivityCheck config and rev-list flag\n 2:  6ad528f4bb ! 4:  efaada0b6a connected: add incremental connectivity check via rev-list\n     @@ Commit message\n          existing-reference boundary, and traversal instead follows the\n          effective shallow boundary.\n      \n     -    p5412 results (median of 3), scaling one dimension at a time.\n     +    p5413 results (median of 3), scaling one dimension at a time.\n          Each fixture is a repository with a flat tree of many\n          directories containing 100 files each.  The incoming set is a\n          chain of commits on top of the existing history, with each\n     @@ Commit message\n      \n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n     - ## Documentation/config/transfer.adoc ##\n     -@@\n     -+transfer.connectivityCheck::\n     -+\tChoose which algorithm to use for the connectivity check\n     -+\tperformed during object transfer operations such as\n     -+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n     -+\tThe connectivity check verifies that all objects reachable\n     -+\tfrom the incoming tips are available locally or, in a partial\n     -+\tclone, promised by a promisor remote.\n     -+\tThe variants are as follows:\n     -++\n     -+--\n     -+`full` (default);;\n     -+\tWalk the full object closure of the boundary commits.\n     -+`incremental`;;\n     -+\tVerify incoming commits by diffing their trees against parent\n     -+\ttrees, recursively descending only into entries that differ.\n     -+\tThe largest benefits occur when incoming commits change a\n     -+\tsmall fraction of a large tree closure.\n     -+\tFalls back to `full` for deepening fetches.\n     -+--\n     -+\n     - transfer.credentialsInUrl::\n     - \tA configured URL can contain plaintext credentials in the form\n     - \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\n     -\n     - ## Documentation/rev-list-options.adoc ##\n     -@@ Documentation/rev-list-options.adoc: we cannot get their Object ID though, an error will be raised.\n     - \tstronger than `--missing=allow-promisor` because it limits the\n     - \ttraversal, rather than just silencing errors about missing\n     - \tobjects.\n     -+\n     -+`--verify-trees-incremental`::\n     -+\t(For internal use only.)  Verify tree connectivity\n     -+\tincrementally by comparing each commit's tree against its\n     -+\tparent trees.  Used by `check_connected()` when\n     -+\t`transfer.connectivityCheck` is set to `incremental`.\n     - endif::git-rev-list[]\n     - \n     - `--no-walk[=(sorted|unsorted)]`::\n     -\n       ## Documentation/technical/connectivity-check.adoc ##\n     -@@ Documentation/technical/connectivity-check.adoc: When a new reference points to a non-commit object, such as a\n     - tag, tree, or blob, that object is not part of the commit walk.\n     - These non-commit tips are handled by the subsequent object\n     - traversal.\n     +@@ Documentation/technical/connectivity-check.adoc: tag, tree, or blob, that object is not part of the commit walk.\n     + These non-commit tips are verified during the tree and blob\n     + traversal that follows the commit walk within the same rev-list\n     + invocation.\n      +\n      +Incremental connectivity check\n      +------------------------------\n     @@ Documentation/technical/connectivity-check.adoc: When a new reference points to\n      +`transfer.connectivityCheck=incremental`, avoids traversing the\n      +full tree walk of the boundary commits.  Instead, it verifies\n      +each incoming commit's tree against the already-trusted trees of\n     -+its parents.\n     ++its parent commits.\n     ++\n     ++The same `--not --all` boundary as the full check identifies the\n     ++incoming commits.  The incremental mode reuses this boundary but\n     ++replaces what happens afterwards: rather than walking boundary\n     ++trees to mark objects uninteresting, it compares each incoming\n     ++commit's tree against its parents' trees, processing commits in\n     ++topological order (ancestors first).\n      +\n      +Trust model\n      +~~~~~~~~~~~\n      +\n      +A tree is trusted when its transitive object closure is known to\n     -+be connected.  Trees reachable from commits on the\n     -+already-connected side of the boundary are therefore trusted.\n     ++be connected.\n      +\n     -+Incoming commits are processed with ancestors before descendants.\n     -+Once an incoming commit's tree has been verified, it is trusted\n     -+and can be used as a comparison base for later descendants.\n     ++Every parent of an incoming commit is either on the\n     ++already-connected side of the `--not --all` boundary (a boundary\n     ++parent whose tree is trusted by definition) or is an earlier\n     ++incoming commit that was already verified due to the topological\n     ++processing order.  In both cases the parent's tree is trusted, so\n     ++all parent trees can be unconditionally used as comparison bases.\n      +\n     -+This gives an inductive correctness argument: every parent of the\n     -+commit currently being verified is either already connected or is\n     -+an earlier incoming commit whose tree has already been verified.\n     ++Once an incoming commit's tree has been verified, it is trusted\n     ++and can serve as a comparison base for later descendants.  This\n     ++gives an inductive correctness argument: the base case is the\n     ++trust boundary established by `--not --all`, and the inductive\n     ++step is that verifying a commit extends the trusted set.\n      +\n      +Tree states\n      +~~~~~~~~~~~\n     @@ Documentation/technical/connectivity-check.adoc: When a new reference points to\n      +during recursive descent.  If no comparison base is available,\n      +the new subtree is verified from scratch.\n      +\n     -+Worked example\n     -+~~~~~~~~~~~~~~\n     ++Example\n     ++~~~~~~~\n      +\n      +Consider a commit that changes one file under `lib/` and moves an\n      +unchanged subtree from `src/` to `dev/`:\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \tint ret = 0;\n       \n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n     - \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n       \t\t\trepo->fetch_if_missing = 0;\n       \t\t\trevs.exclude_promisor_objects = 1;\n     -+\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n     + \t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n      +\t\t\tverify_trees_incremental = 1;\n     -+\t\t\trepo->fetch_if_missing = 0;\n     + \t\t\trepo->fetch_if_missing = 0;\n       \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n       \t\t\tparse_missing_action_value(repo, arg);\n     - \t\t} else if (!strcmp(arg, \"-z\")) {\n     -@@ builtin/rev-list.c: int cmd_rev_list(int argc,\n     - \n     - \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n     - \t\t\tcontinue; /* already handled above */\n     -+\t\tif (!strcmp(arg, \"--verify-trees-incremental\"))\n     -+\t\t\tcontinue; /* already handled above */\n     - \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n     - \t\t\tcontinue; /* already handled above */\n     - \n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \n       \tprepare_maximal_independent(&revs);\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n       \n      \n     - ## connected.c ##\n     -@@\n     - #define USE_THE_REPOSITORY_VARIABLE\n     - \n     - #include \"git-compat-util.h\"\n     -+#include \"config.h\"\n     - #include \"gettext.h\"\n     - #include \"hex.h\"\n     - #include \"odb.h\"\n     -@@ connected.c: static int check_connected_promisor(oid_iterate_fn fn,\n     - \treturn 1;\n     - }\n     - \n     -+static int incremental_check_applicable(struct check_connected_options *opt)\n     -+{\n     -+\tconst char *algorithm = NULL;\n     -+\n     -+\tif (repo_config_get_string_tmp(the_repository,\n     -+\t\t\t\t       \"transfer.connectivitycheck\",\n     -+\t\t\t\t       &algorithm))\n     -+\t\treturn 0;\n     -+\tif (!strcasecmp(algorithm, \"full\"))\n     -+\t\treturn 0;\n     -+\tif (strcasecmp(algorithm, \"incremental\"))\n     -+\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n     -+\t\t    algorithm);\n     -+\n     -+\tif (opt->is_deepening_fetch)\n     -+\t\treturn 0;\n     -+\n     -+\treturn 1;\n     -+}\n     -+\n     - /*\n     -  * If we feed all the commits we want to verify to this command\n     -  *\n     -@@ connected.c: int check_connected(oid_iterate_fn fn, void *cb_data,\n     - \tif (opt->progress)\n     - \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n     - \t\t\t     _(\"Checking connectivity\"));\n     -+\tif (incremental_check_applicable(opt))\n     -+\t\tstrvec_push(&rev_list.args,\n     -+\t\t\t    \"--verify-trees-incremental\");\n     - \n     - \trev_list.git_cmd = 1;\n     - \tif (opt->env)\n     -\n       ## meson.build ##\n      @@ meson.build: libgit_sources = [\n         'transport-helper.c',\n     @@ meson.build: libgit_sources = [\n      \n       ## t/meson.build ##\n      @@ t/meson.build: integration_tests = [\n     -   't5409-colorize-remote-messages.sh',\n         't5410-receive-pack.sh',\n         't5411-proc-receive-hook.sh',\n     -+  't5412-connectivity-check.sh',\n     +   't5412-receive-report-hook.sh',\n     ++  't5413-connectivity-check.sh',\n         't5500-fetch-pack.sh',\n         't5501-fetch-push-alternates.sh',\n         't5502-quickfetch.sh',\n      \n     - ## t/perf/generate-repo-p5412-connectivity-check.perl (new) ##\n     + ## t/perf/p5413-connectivity-check.sh ##\n      @@\n     -+#!/usr/bin/perl\n     -+#\n     -+# Generate a fast-import stream for p5412 connectivity check benchmarks.\n     -+#\n     -+# Usage: generate-repo-p5412-connectivity-check.perl\n     -+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n     -+#\n     -+# Creates one initial commit with dirs*files_per_dir files, then\n     -+# <commits> additional commits each modifying <files_per_commit>\n     -+# files in directories chosen round-robin from 1..<hot_dirs>.\n     -+\n     -+use strict;\n     -+use warnings;\n     -+\n     -+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n     -+$hot = $nd if !$hot || $hot > $nd;\n     -+$fpc = 1   if !$fpc;\n     -+\n     -+sub data {\n     -+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n     -+}\n     -+\n     -+# Initial tree: one commit with nd*nf files.\n     -+printf \"commit refs/heads/main\\n\";\n     -+printf \"committer perf <perf\\@test.com> now\\n\";\n     -+data(\"initial\");\n     -+for my $d (1..$nd) {\n     -+\tfor my $f (1..$nf) {\n     -+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n     -+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n     -+\t}\n     -+}\n     -+\n     -+# Subsequent commits (auto-chained by fast-import).\n     -+for my $i (1..$nc) {\n     -+\tprintf \"commit refs/heads/main\\n\";\n     -+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n     -+\tdata(sprintf \"change-%03d\", $i);\n     -+\tfor my $j (0..$fpc-1) {\n     -+\t\tmy $d = (($i + $j) % $hot) + 1;\n     -+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n     -+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n     -+\t}\n     -+}\n     -\n     - ## t/perf/p5412-connectivity-check.sh (new) ##\n     -@@\n     -+#!/bin/sh\n     -+\n     + #!/bin/sh\n     + \n     +-test_description='performance of connectivity check\n      +test_description='performance of connectivity check modes\n     -+\n     + \n     +-Benchmark the rev-list connectivity check, avoiding pack transfer\n     +-noise.\n      +Compare the default and incremental rev-list connectivity modes\n      +directly, avoiding pack transfer noise.\n     -+\n     -+Each repository has a flat tree of many directories with 100 files\n     -+in each.  Three axes are scaled independently: tree size, commit\n     -+count, and files changed per commit.'\n     -+\n     -+. ./perf-lib.sh\n     -+\n     -+test_perf_fresh_repo\n     -+\n     -+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5412-connectivity-check.perl\"\n     -+\n     -+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n     -+# $5=files_per_commit (optional, default=1)\n     -+test_perf_conn () {\n     -+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n     -+\tlocal total=$(($nd * $nf))\n     -+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n     -+\tlocal label=\"${total} files, ${nc} commits\"\n     -+\tif test \"$hot\" -lt \"$nd\"\n     -+\tthen\n     -+\t\tlabel=\"$label (${hot} hot dirs)\"\n     -+\tfi\n     -+\tif test \"$fpc\" -gt 1\n     -+\tthen\n     -+\t\tlabel=\"$label (${fpc} files/commit)\"\n     -+\tfi\n     -+\n     -+\ttest_expect_success \"setup $label\" '\n     -+\t\tif test -d '\"$name\"'\n     -+\t\tthen\n     -+\t\t\ttrue\n     -+\t\telse\n     -+\t\t\tgit init '\"$name\"' &&\n     -+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n     -+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n     -+\t\t\t(\n     -+\t\t\t\tcd '\"$name\"' &&\n     -+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n     -+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n     -+\t\t\t\tgit update-ref refs/heads/main \\\n     -+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n     -+\t\t\t\tgit repack -ad &&\n     -+\t\t\t\tgit config gc.auto 0\n     -+\t\t\t)\n     -+\t\tfi\n     -+\t'\n     -+\n     + \n     + Each repository has a flat tree of many directories with 100 files\n     + in each.  Three axes are scaled independently: tree size, commit\n     +@@ t/perf/p5413-connectivity-check.sh: test_perf_conn () {\n     + \t\tfi\n     + \t'\n     + \n     +-\ttest_perf \"$label\" '\n      +\ttest_perf \"$label (full)\" '\n     -+\t\tcat '\"${name}\"'_new |\n     -+\t\tgit -C '\"$name\"' rev-list \\\n     -+\t\t\t--objects --stdin --not --all --quiet \\\n     -+\t\t\t--exclude-promisor-objects\n     -+\t'\n     + \t\tcat '\"${name}\"'_new |\n     + \t\tgit -C '\"$name\"' rev-list \\\n     + \t\t\t--objects --stdin --not --all --quiet \\\n     + \t\t\t--exclude-promisor-objects\n     + \t'\n      +\n      +\ttest_perf \"$label (incremental)\" '\n      +\t\tcat '\"${name}\"'_new |\n     @@ t/perf/p5412-connectivity-check.sh (new)\n      +\t\t\t--objects --stdin --not --all --quiet \\\n      +\t\t\t--exclude-promisor-objects\n      +\t'\n     -+}\n     -+\n     -+# Scaling tree size (10 commits, 10 files/commit).\n     -+test_perf_conn   50 100 10   50 10\n     -+test_perf_conn  500 100 10  500 10\n     -+test_perf_conn 2000 100 10 2000 10\n     -+test_perf_conn 8000 100 10 8000 10\n     -+\n     -+# Scaling commit count (200K files, 10 files/commit).\n     -+test_perf_conn 2000 100    1 2000 10\n     -+test_perf_conn 2000 100   10 2000 10\n     -+test_perf_conn 2000 100  100 2000 10\n     -+test_perf_conn 2000 100   500 2000 10\n     -+test_perf_conn 2000 100  3000 2000 10\n     -+test_perf_conn 2000 100  5000 2000 10\n     -+test_perf_conn 2000 100 10000 2000 10\n     -+\n     -+# Scaling files per commit (200K files, 10 commits).\n     -+test_perf_conn 2000 100 10 2000   1\n     -+test_perf_conn 2000 100 10 2000  10\n     -+test_perf_conn 2000 100 10 2000  100\n     -+test_perf_conn 2000 100 10 2000  500\n     -+test_perf_conn 2000 100 10 2000 1000\n     -+test_perf_conn 2000 100 10 2000 2000\n     -+\n     -+test_done\n     + }\n     + \n     + # Scaling tree size (10 commits, 10 files/commit).\n      \n     - ## t/t5412-connectivity-check.sh (new) ##\n     + ## t/t5413-connectivity-check.sh (new) ##\n      @@\n      +#!/bin/sh\n      +\n     @@ tree-verify.c (new)\n      +\t\t\t       struct verify_state *vs)\n      +{\n      +\tstruct oid_array base_trees = OID_ARRAY_INIT;\n     ++\tconst struct object_id *tree_oid;\n      +\tstruct commit_list *p;\n      +\n      +\t/*\n     @@ tree-verify.c (new)\n      +\t * first due to the topological processing order.\n      +\t */\n      +\tfor (p = commit->parents; p; p = p->next) {\n     -+\t\tconst struct object_id *tree_oid;\n      +\t\tparse_commit_or_die(p->item);\n      +\t\ttree_oid = get_commit_tree_oid(p->item);\n      +\t\tif (!tree_oid)\n     @@ tree-verify.c (new)\n      +\t\toid_array_append(&base_trees, tree_oid);\n      +\t}\n      +\n     -+\tif (!get_commit_tree_oid(commit))\n     ++\ttree_oid = get_commit_tree_oid(commit);\n     ++\tif (!tree_oid)\n      +\t\tdie(_(\"unable to load root tree for commit %s\"),\n      +\t\t    oid_to_hex(&commit->object.oid));\n     -+\tverify_tree(repo, get_commit_tree_oid(commit),\n     -+\t\t    &base_trees, vs, 0);\n     ++\tverify_tree(repo, tree_oid, &base_trees, vs, 0);\n      +\toid_array_clear(&base_trees);\n      +}\n      +\n\n-- \ngitgitgadget\n\n"},{"id":"554751","messageId":"19ea8cb534f0aaabdf21f358364ad32295b6bfc8.1791806877.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","subject":"[PATCH v3 1/4] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-12T12:07:54Z","receivedAt":"2026-10-12T12:07:54Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd Documentation/technical/connectivity-check.adoc describing\nthe connectivity invariant and the full connectivity check.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n .../technical/connectivity-check.adoc         | 110 ++++++++++++++++++\n 1 file changed, 110 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nnew file mode 100644\nindex 0000000000..abef02debd\n--- /dev/null\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -0,0 +1,110 @@\n+Connectivity checking\n+=====================\n+\n+After receiving new objects via fetch, push (receive-pack), clone,\n+or bundle, Git verifies that the new reference tips do not leave\n+the repository in a state where reachable objects are missing.\n+This verification is called the connectivity check.\n+\n+Connectivity invariant\n+----------------------\n+\n+A repository is connected when every object reachable from its\n+references is available locally (with exceptions noted below).\n+The reverse does not hold: objects that exist in the object\n+database but are not reachable from any reference may be\n+incomplete or lack a connected closure.\n+\n+The connectivity check maintains this invariant when references\n+are updated.  It trusts that all objects reachable from existing\n+reference tips are already fully connected and verifies that the\n+new reference tips do not introduce references to unavailable\n+objects.  Verification is permitted to stop when it reaches\n+objects already reachable from existing references, since their\n+closure is already connected.  These trusted references include\n+local references and references from alternate object stores.\n+\n+Without this check, a truncated or corrupted transfer could leave\n+a repository in a state where later history walks encounter\n+missing objects.\n+\n+Exceptions\n+~~~~~~~~~~\n+\n+Gitlink entries (submodule references) are excluded from\n+connectivity checking.  Their target objects belong to a separate\n+repository.\n+\n+In partial clones, objects promised by a promisor remote are\n+accepted as connected without requiring local existence.  The\n+check excludes promisor objects from traversal so that it does\n+not trigger on-demand fetches for them.\n+\n+Full connectivity check\n+-----------------------\n+\n+The full connectivity check runs a `rev-list` subprocess, feeding\n+the new reference tips via stdin.  A normal invocation is roughly:\n+\n+    git rev-list --objects --stdin --not --all --quiet\n+        --alternate-refs [--exclude-promisor-objects]\n+\n+When promisor remotes are configured, a fast path based on\n+promisor packfiles is attempted first.  If it falls back to\n+the `rev-list` check, `--exclude-promisor-objects` is added so\n+that the traversal does not trigger on-demand fetches.\n+\n+Consider the following graph after a fetch, where all reference\n+tips point directly to commits.  For simplicity, only local\n+references appear on the already-connected side; alternate refs\n+play the same role.  N3 is a merge commit:\n+\n+            /-------------L2\n+           /\n+    C1---B1---C2---B2-----L1\n+          \\         \\\n+           N1        N3---T2\n+            \\       /\n+             N2-----------T1\n+\n+    L1, L2:         local refs\n+    T1, T2:         incoming tips (new refs)\n+    N1, N2, N3:     incoming commits (N3 is a merge)\n+    B1, B2:         boundary commits (already connected)\n+    C1, C2:         already connected (but not boundary)\n+\n+The incoming set is the commits reachable from the incoming\n+tips but not from the already-connected side.  Boundary commits\n+are the already-connected commits at the edge of that set.  Here\n+B1 is an ancestor of B2, which happens when incoming branches\n+fork at different depths in the existing history.\n+\n+The `--not --all` arguments establish the trust boundary: the\n+incoming set is the commits reachable from the new tips but not\n+from any existing reference.\n+\n+The traversal then walks the trees of the boundary commits and\n+marks those objects uninteresting.  Trees of incoming commits\n+are verified against these marks, stopping at objects already\n+covered.  The cost of marking boundary trees uninteresting\n+dominates when the boundary commits have large trees, even when\n+the incoming changes are small.\n+\n+Deepening fetches\n+~~~~~~~~~~~~~~~~~\n+\n+For deepening fetches (where the shallow boundary moves), the\n+full check omits `--not --all`.  There is no existing-reference\n+boundary at which the walk can stop.  Instead, traversal follows\n+the effective shallow boundary supplied for the deepened\n+repository.  The new content may be below the old shallow\n+boundary even when the tips themselves have not changed.\n+\n+Non-commit tips\n+~~~~~~~~~~~~~~~\n+\n+When a new reference points to a non-commit object, such as a\n+tag, tree, or blob, that object is not part of the commit walk.\n+These non-commit tips are verified during the tree and blob\n+traversal that follows the commit walk within the same rev-list\n+invocation.\n-- \ngitgitgadget\n\n\n"},{"id":"554752","messageId":"211794c9252181fa0c31fcd968379c189d001a8c.1791806877.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","subject":"[PATCH v3 2/4] p5413: add perf tests for connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-12T12:07:55Z","receivedAt":"2026-10-12T12:07:55Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd benchmarks for the rev-list connectivity check.  Each fixture\nvaries one dimension: total tree size, incoming commit count, or\nfiles changed per commit.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n ...enerate-repo-p5413-connectivity-check.perl | 44 ++++++++++\n t/perf/p5413-connectivity-check.sh            | 85 +++++++++++++++++++\n 2 files changed, 129 insertions(+)\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n\ndiff --git a/t/perf/generate-repo-p5413-connectivity-check.perl b/t/perf/generate-repo-p5413-connectivity-check.perl\nnew file mode 100644\nindex 0000000000..9546996280\n--- /dev/null\n+++ b/t/perf/generate-repo-p5413-connectivity-check.perl\n@@ -0,0 +1,44 @@\n+#!/usr/bin/perl\n+#\n+# Generate a fast-import stream for p5412 connectivity check benchmarks.\n+#\n+# Usage: generate-repo-p5412-connectivity-check.perl\n+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n+#\n+# Creates one initial commit with dirs*files_per_dir files, then\n+# <commits> additional commits each modifying <files_per_commit>\n+# files in directories chosen round-robin from 1..<hot_dirs>.\n+\n+use strict;\n+use warnings;\n+\n+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n+$hot = $nd if !$hot || $hot > $nd;\n+$fpc = 1   if !$fpc;\n+\n+sub data {\n+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n+}\n+\n+# Initial tree: one commit with nd*nf files.\n+printf \"commit refs/heads/main\\n\";\n+printf \"committer perf <perf\\@test.com> now\\n\";\n+data(\"initial\");\n+for my $d (1..$nd) {\n+\tfor my $f (1..$nf) {\n+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n+\t}\n+}\n+\n+# Subsequent commits (auto-chained by fast-import).\n+for my $i (1..$nc) {\n+\tprintf \"commit refs/heads/main\\n\";\n+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n+\tdata(sprintf \"change-%03d\", $i);\n+\tfor my $j (0..$fpc-1) {\n+\t\tmy $d = (($i + $j) % $hot) + 1;\n+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n+\t}\n+}\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..3fabc82a87\n--- /dev/null\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+\n+test_description='performance of connectivity check\n+\n+Benchmark the rev-list connectivity check, avoiding pack transfer\n+noise.\n+\n+Each repository has a flat tree of many directories with 100 files\n+in each.  Three axes are scaled independently: tree size, commit\n+count, and files changed per commit.'\n+\n+. ./perf-lib.sh\n+\n+test_perf_fresh_repo\n+\n+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5413-connectivity-check.perl\"\n+\n+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n+# $5=files_per_commit (optional, default=1)\n+test_perf_conn () {\n+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n+\tlocal total=$(($nd * $nf))\n+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n+\tlocal label=\"${total} files, ${nc} commits\"\n+\tif test \"$hot\" -lt \"$nd\"\n+\tthen\n+\t\tlabel=\"$label (${hot} hot dirs)\"\n+\tfi\n+\tif test \"$fpc\" -gt 1\n+\tthen\n+\t\tlabel=\"$label (${fpc} files/commit)\"\n+\tfi\n+\n+\ttest_expect_success \"setup $label\" '\n+\t\tif test -d '\"$name\"'\n+\t\tthen\n+\t\t\ttrue\n+\t\telse\n+\t\t\tgit init '\"$name\"' &&\n+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n+\t\t\t(\n+\t\t\t\tcd '\"$name\"' &&\n+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n+\t\t\t\tgit update-ref refs/heads/main \\\n+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n+\t\t\t\tgit repack -ad &&\n+\t\t\t\tgit config gc.auto 0\n+\t\t\t)\n+\t\tfi\n+\t'\n+\n+\ttest_perf \"$label\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+}\n+\n+# Scaling tree size (10 commits, 10 files/commit).\n+test_perf_conn   50 100 10   50 10\n+test_perf_conn  500 100 10  500 10\n+test_perf_conn 2000 100 10 2000 10\n+test_perf_conn 8000 100 10 8000 10\n+\n+# Scaling commit count (200K files, 10 files/commit).\n+test_perf_conn 2000 100    1 2000 10\n+test_perf_conn 2000 100   10 2000 10\n+test_perf_conn 2000 100  100 2000 10\n+test_perf_conn 2000 100   500 2000 10\n+test_perf_conn 2000 100  3000 2000 10\n+test_perf_conn 2000 100  5000 2000 10\n+test_perf_conn 2000 100 10000 2000 10\n+\n+# Scaling files per commit (200K files, 10 commits).\n+test_perf_conn 2000 100 10 2000   1\n+test_perf_conn 2000 100 10 2000  10\n+test_perf_conn 2000 100 10 2000  100\n+test_perf_conn 2000 100 10 2000  500\n+test_perf_conn 2000 100 10 2000 1000\n+test_perf_conn 2000 100 10 2000 2000\n+\n+test_done\n-- \ngitgitgadget\n\n\n"},{"id":"554753","messageId":"cdf28f1df6cb36b006f367de0e18e23e3aa8d992.1791806877.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","subject":"[PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-12T12:07:56Z","receivedAt":"2026-10-12T12:07:56Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd the transfer.connectivityCheck configuration option, accepting\n\"full\" (default) and \"incremental\".  When set to \"incremental\",\ncheck_connected() passes --verify-trees-incremental to its rev-list\nsubprocess.\n\nThe rev-list flag is accepted but has no effect yet; the incremental\nverification will be implemented in the next commit.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc  | 20 ++++++++++++++++++++\n Documentation/rev-list-options.adoc |  6 ++++++\n builtin/rev-list.c                  |  4 ++++\n connected.c                         | 24 ++++++++++++++++++++++++\n 4 files changed, 54 insertions(+)\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex f1ce50f4a6..b9939d3bde 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -1,3 +1,23 @@\n+transfer.connectivityCheck::\n+\tChoose which algorithm to use for the connectivity check\n+\tperformed during object transfer operations such as\n+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n+\tThe connectivity check verifies that all objects reachable\n+\tfrom the incoming tips are available locally or, in a partial\n+\tclone, promised by a promisor remote.\n+\tThe variants are as follows:\n++\n+--\n+`full` (default);;\n+\tWalk the full object closure of the boundary commits.\n+`incremental`;;\n+\tVerify incoming commits by diffing their trees against parent\n+\ttrees, recursively descending only into entries that differ.\n+\tThe largest benefits occur when incoming commits change a\n+\tsmall fraction of a large tree closure.\n+\tFalls back to `full` for deepening fetches.\n+--\n+\n transfer.credentialsInUrl::\n \tA configured URL can contain plaintext credentials in the form\n \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex bd9f345690..35c4bf882f 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1102,6 +1102,12 @@ It is an error to use `--missing-only` without `--missing=print` or\n \tstronger than `--missing=allow-promisor` because it limits the\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n+\n+`--verify-trees-incremental`::\n+\t(For internal use only.)  Verify tree connectivity\n+\tincrementally by comparing each commit's tree against its\n+\tparent trees.  Used by `check_connected()` when\n+\t`transfer.connectivityCheck` is set to `incremental`.\n endif::git-rev-list[]\n \n `--no-walk[=(sorted|unsorted)]`::\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 0faa833fac..9a76bb9c9c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -769,6 +769,8 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n+\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n+\t\t\trepo->fetch_if_missing = 0;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n \t\t} else if (!strcmp(arg, \"--missing-only\")) {\n@@ -848,6 +850,8 @@ int cmd_rev_list(int argc,\n \n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n \t\t\tcontinue; /* already handled above */\n+\t\tif (!strcmp(arg, \"--verify-trees-incremental\"))\n+\t\t\tcontinue; /* already handled above */\n \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n \t\t\tcontinue; /* already handled above */\n \ndiff --git a/connected.c b/connected.c\nindex 929b9bd28d..e3dcd8e2b0 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,6 +1,7 @@\n #define USE_THE_REPOSITORY_VARIABLE\n \n #include \"git-compat-util.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n@@ -67,6 +68,26 @@ static int check_connected_promisor(oid_iterate_fn fn,\n \treturn 1;\n }\n \n+static int incremental_check_applicable(struct check_connected_options *opt)\n+{\n+\tconst char *algorithm = NULL;\n+\n+\tif (repo_config_get_string_tmp(the_repository,\n+\t\t\t\t       \"transfer.connectivitycheck\",\n+\t\t\t\t       &algorithm))\n+\t\treturn 0;\n+\tif (!strcasecmp(algorithm, \"full\"))\n+\t\treturn 0;\n+\tif (strcasecmp(algorithm, \"incremental\"))\n+\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n+\t\t    algorithm);\n+\n+\tif (opt->is_deepening_fetch)\n+\t\treturn 0;\n+\n+\treturn 1;\n+}\n+\n /*\n  * If we feed all the commits we want to verify to this command\n  *\n@@ -133,6 +154,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n+\tif (incremental_check_applicable(opt))\n+\t\tstrvec_push(&rev_list.args,\n+\t\t\t    \"--verify-trees-incremental\");\n \n \trev_list.git_cmd = 1;\n \tif (opt->env)\n-- \ngitgitgadget\n\n\n"},{"id":"554754","messageId":"efaada0b6a8ffdb4d1c01d7f34a0d6596a99c149.1791806877.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","subject":"[PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-12T12:07:57Z","receivedAt":"2026-10-12T12:07:57Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe full connectivity check uses rev-list to find commits\nreachable from the incoming tips but not from the\nalready-connected side, then walks their object closure.  Commit\ntraversal stops at the connectivity boundary, but trees and blobs\nreachable from that boundary still need to be walked so they can\nbe marked uninteresting, allocating a struct object for each one.\nOn repositories where the boundary commits have large trees, the\nconnectivity check for small incoming changes visits and tracks\nmore objects than needed.\n\nAdd an alternative connectivity check that verifies incoming\ncommits incrementally against their parents.\n\nThe verifier processes incoming commits with ancestors first and\ncompares each new tree against its trusted parent trees.  Entries\nalready seen on the trusted side are skipped by OID, so unchanged\nsubtrees need not remain at the same path to be recognized.\nChanged subtrees are recursively compared against same-path parent\nsubtrees, and new subtrees without a comparison base are verified\nfrom scratch.\n\nSee Documentation/technical/connectivity-check.adoc for the trust\ninvariants and detailed algorithm.\n\nThe incremental check runs as an internal\n--verify-trees-incremental mode in rev-list.  After the normal\nrevision walk identifies the incoming commits, the verifier\nconsumes them before the usual object traversal; any pending\nnon-commit tips are still handled by the existing traversal.\n\nPartial-clone semantics are preserved: objects promised by a\npromisor remote are accepted as connected, and on-demand fetching\nis prevented by excluding promisor objects from traversal.  The\nnew mode is selected by transfer.connectivityCheck=incremental,\nwith full remaining the default.  Deepening fetches fall back to\nthe full check because they do not have the normal\nexisting-reference boundary, and traversal instead follows the\neffective shallow boundary.\n\np5413 results (median of 3), scaling one dimension at a time.\nEach fixture is a repository with a flat tree of many\ndirectories containing 100 files each.  The incoming set is a\nchain of commits on top of the existing history, with each\ncommit changing files in different directories round-robin.\nThe three axes vary the total repository tree size, the number\nof incoming commits, and the number of files changed per\nincoming commit.\n\nScaling tree size (10 incoming commits, 10 files/commit):\n\n    files    full  incr.  full/incr\n      5K    0.01s  0.01s    1.0x\n     50K    0.04s  0.01s    4.0x\n    200K    0.15s  0.01s   15.0x\n    800K    0.61s  0.03s   20.3x\n\nThe full mode must walk the trees of boundary commits, which\ngrows with overall tree size.  Incremental still scans the\nroot trees, but avoids descending into unchanged subtrees,\nso it grows much more slowly with repository size.\n\nScaling incoming commit count (200K files, 10 files/commit):\n\n    commits    full  incr.  full/incr\n          1    0.14s  0.01s   14.0x\n         10    0.15s  0.01s   15.0x\n        100    0.19s  0.05s    3.8x\n        500    0.32s  0.27s    1.2x\n       3000    1.23s  1.52s    0.8x\n       5000    1.88s  2.43s    0.8x\n      10000    3.72s  5.36s    0.7x\n\nWith many commits the per-commit overhead of scanning both\nthe new and parent root trees accumulates and incremental\nbecomes slower.  Breakeven is between 500 and 3000 commits\nand the ratio stabilizes near 0.7x for this fixture.\n\nScaling files per incoming commit (200K files, 10 commits):\n\n    files/commit    full  incr.  full/incr\n               1    0.14s  0.01s   14.0x\n              10    0.15s  0.01s   15.0x\n             100    0.16s  0.04s    4.0x\n             500    0.23s  0.14s    1.6x\n            1000    0.34s  0.28s    1.2x\n            2000    0.70s  0.63s    1.1x\n\nBreakeven is around 1000 files/commit.  At 2000 files/commit\n(every directory touched), incremental remains slightly faster;\nbypassing the object cache for tree reads likely helps here.\n\nFor small repositories both modes are fast enough that the\ndifference is difficult to measure reliably.\n\nSelected peak RSS measurements from the same fixtures:\n\n    case                        full    incr.  ratio\n    200K files, 10 commits      31 MB    12 MB   0.4x\n    800K files, 10 commits     110 MB    26 MB   0.2x\n    200K files, 5000 commits   155 MB   151 MB   1.0x\n\nIncremental uses substantially less memory when it can prune\nmost of the boundary tree walk.  In the long-history case the\ntwo modes visit similar object sets and memory converges.\n\nThe regression cases in the CPU benchmarks are in wall-clock\ntime rather than memory, primarily from scanning some trees\nmore than once.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n .../technical/connectivity-check.adoc         | 145 ++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  15 +\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n t/perf/p5413-connectivity-check.sh            |  15 +-\n t/t5413-connectivity-check.sh                 | 680 ++++++++++++++++++\n tree-verify.c                                 | 316 ++++++++\n tree-verify.h                                 |  15 +\n 9 files changed, 1185 insertions(+), 4 deletions(-)\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nindex abef02debd..a1d2e9709c 100644\n--- a/Documentation/technical/connectivity-check.adoc\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -108,3 +108,148 @@ tag, tree, or blob, that object is not part of the commit walk.\n These non-commit tips are verified during the tree and blob\n traversal that follows the commit walk within the same rev-list\n invocation.\n+\n+Incremental connectivity check\n+------------------------------\n+\n+The incremental mode, selected by\n+`transfer.connectivityCheck=incremental`, avoids traversing the\n+full tree walk of the boundary commits.  Instead, it verifies\n+each incoming commit's tree against the already-trusted trees of\n+its parent commits.\n+\n+The same `--not --all` boundary as the full check identifies the\n+incoming commits.  The incremental mode reuses this boundary but\n+replaces what happens afterwards: rather than walking boundary\n+trees to mark objects uninteresting, it compares each incoming\n+commit's tree against its parents' trees, processing commits in\n+topological order (ancestors first).\n+\n+Trust model\n+~~~~~~~~~~~\n+\n+A tree is trusted when its transitive object closure is known to\n+be connected.\n+\n+Every parent of an incoming commit is either on the\n+already-connected side of the `--not --all` boundary (a boundary\n+parent whose tree is trusted by definition) or is an earlier\n+incoming commit that was already verified due to the topological\n+processing order.  In both cases the parent's tree is trusted, so\n+all parent trees can be unconditionally used as comparison bases.\n+\n+Once an incoming commit's tree has been verified, it is trusted\n+and can serve as a comparison base for later descendants.  This\n+gives an inductive correctness argument: the base case is the\n+trust boundary established by `--not --all`, and the inductive\n+step is that verifying a commit extends the trusted set.\n+\n+Tree states\n+~~~~~~~~~~~\n+\n+The verifier tracks tree OIDs in three states:\n+\n+untrusted::\n+\tThe tree has not yet been established as connected.  This\n+\tis the implicit state of an OID not present in the state\n+\tmap.\n+\n+trusted::\n+\tThe tree is known to have a connected transitive closure,\n+\tbut its direct entries have not yet been published into the\n+\tverifier's trusted object sets.\n+\n+expanded::\n+\tThe tree is trusted and its direct non-gitlink entries\n+\thave also been published into the trusted object sets.\n+\n+State transitions are monotonic: a tree may move from untrusted\n+to trusted to expanded, but never backwards.  An untrusted tree\n+that is successfully verified goes directly to expanded.\n+\n+Blobs require only trusted/untrusted state: a blob becomes\n+trusted when it is found in a trusted tree or when its existence\n+and type have been verified directly.\n+\n+The trusted/expanded distinction is an optimization.  An expanded\n+parent does not need to be reread merely to publish entries that\n+are already trusted, though it may still be read when same-path\n+subtree bases are needed for recursive comparison.\n+\n+Algorithm\n+~~~~~~~~~\n+\n+At a high level:\n+\n+    verify(commits):\n+        sort topologically (ancestors first)\n+        for each commit:\n+            mark parent root trees as trusted\n+            verify_tree(commit.tree, parent root trees)\n+\n+    verify_tree(tree, base_trees):\n+        if tree already trusted: return\n+        read tree, collect entries not already trusted\n+        for each available base tree:\n+            publish its entries as trusted\n+            record same-path subtrees as bases\n+        for each collected entry:\n+            if now trusted: skip\n+            if blob: verify blob connectivity and type\n+            if tree: verify_tree(entry, its recorded bases)\n+        mark tree as expanded\n+\n+The important ordering within `verify_tree` is that entries from\n+the new tree are collected before the base trees are scanned, but\n+are processed only afterwards.  Trust learned from any base can\n+therefore eliminate work before recursive verification begins.\n+\n+Same-path parent subtrees are passed down as comparison bases\n+during recursive descent.  If no comparison base is available,\n+the new subtree is verified from scratch.\n+\n+Example\n+~~~~~~~\n+\n+Consider a commit that changes one file under `lib/` and moves an\n+unchanged subtree from `src/` to `dev/`:\n+\n+    Parent tree              New tree\n+    +-- src/   (aaa)         +-- dev/   (aaa)\n+    +-- lib/   (bbb)         +-- lib/   (ccc)\n+         +-- foo.c (ddd)          +-- foo.c (ddd)\n+         +-- bar.c (eee)          +-- bar.c (fff)\n+\n+Scanning the parent makes `aaa` trusted even though it moved from\n+`src/` to `dev/`, so that subtree is skipped.  The changed `ccc`\n+subtree is compared against its same-path parent `bbb`; scanning\n+`bbb` makes `ddd` and `eee` trusted, leaving only the new `fff`\n+blob to be checked.\n+\n+This illustrates two properties:\n+\n+* Trust is OID-based rather than path-based.  An unchanged subtree\n+  is recognized after a move.\n+\n+* Same-path parent subtrees provide recursive comparison bases.\n+  These bases improve pruning efficiency but are not required for\n+  correctness; a new subtree can always be verified from scratch.\n+\n+Multiple parents\n+~~~~~~~~~~~~~~~~\n+\n+For a merge commit, root trees from all parents are comparison\n+bases.  When several parents contain a same-path subtree, each\n+matching subtree is collected as a recursive comparison base.\n+\n+Entries published from any trusted parent become globally trusted,\n+so a matching tree or blob entry present in any parent can be\n+skipped while verifying the merge tree.\n+\n+Missing comparison bases\n+~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+A promised base tree may not be locally available for comparison.\n+In that case the verifier skips that base and verifies the new\n+subtree without it.  The missing comparison can reduce pruning but\n+does not affect correctness.\ndiff --git a/Makefile b/Makefile\nindex a96be506b5..fa42e295e2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1339,6 +1339,7 @@ LIB_OBJS += trailer.o\n LIB_OBJS += transport-helper.o\n LIB_OBJS += transport.o\n LIB_OBJS += tree-diff.o\n+LIB_OBJS += tree-verify.o\n LIB_OBJS += tree-walk.o\n LIB_OBJS += tree.o\n LIB_OBJS += unpack-trees.o\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 9a76bb9c9c..ae5012ec25 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -28,6 +28,7 @@\n #include \"commit-reach.h\"\n #include \"quote.h\"\n #include \"strbuf.h\"\n+#include \"tree-verify.h\"\n \n struct rev_list_info {\n \tstruct rev_info *revs;\n@@ -727,6 +728,7 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n+\tint verify_trees_incremental = 0;\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -770,6 +772,7 @@ int cmd_rev_list(int argc,\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n \t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n+\t\t\tverify_trees_incremental = 1;\n \t\t\trepo->fetch_if_missing = 0;\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n@@ -973,6 +976,18 @@ int cmd_rev_list(int argc,\n \n \tprepare_maximal_independent(&revs);\n \n+\tif (verify_trees_incremental) {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *new_commits = NULL;\n+\n+\t\twhile ((commit = get_revision(&revs)) != NULL)\n+\t\t\tcommit_list_insert(commit, &new_commits);\n+\n+\t\tverify_commits_incremental(repo, &new_commits,\n+\t\t\t\t\t   revs.exclude_promisor_objects);\n+\t\tcommit_list_free(new_commits);\n+\t}\n+\n \tif (revs.tree_objects)\n \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n \ndiff --git a/meson.build b/meson.build\nindex 0a95d90d21..d94dedc26c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -562,6 +562,7 @@ libgit_sources = [\n   'transport-helper.c',\n   'transport.c',\n   'tree-diff.c',\n+  'tree-verify.c',\n   'tree-walk.c',\n   'tree.c',\n   'unpack-trees.c',\ndiff --git a/t/meson.build b/t/meson.build\nindex f65eb04684..b23b4c90b0 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -654,6 +654,7 @@ integration_tests = [\n   't5410-receive-pack.sh',\n   't5411-proc-receive-hook.sh',\n   't5412-receive-report-hook.sh',\n+  't5413-connectivity-check.sh',\n   't5500-fetch-pack.sh',\n   't5501-fetch-push-alternates.sh',\n   't5502-quickfetch.sh',\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nindex 3fabc82a87..ed195089ef 100755\n--- a/t/perf/p5413-connectivity-check.sh\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -1,9 +1,9 @@\n #!/bin/sh\n \n-test_description='performance of connectivity check\n+test_description='performance of connectivity check modes\n \n-Benchmark the rev-list connectivity check, avoiding pack transfer\n-noise.\n+Compare the default and incremental rev-list connectivity modes\n+directly, avoiding pack transfer noise.\n \n Each repository has a flat tree of many directories with 100 files\n in each.  Three axes are scaled independently: tree size, commit\n@@ -51,12 +51,19 @@ test_perf_conn () {\n \t\tfi\n \t'\n \n-\ttest_perf \"$label\" '\n+\ttest_perf \"$label (full)\" '\n \t\tcat '\"${name}\"'_new |\n \t\tgit -C '\"$name\"' rev-list \\\n \t\t\t--objects --stdin --not --all --quiet \\\n \t\t\t--exclude-promisor-objects\n \t'\n+\n+\ttest_perf \"$label (incremental)\" '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list --verify-trees-incremental \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n }\n \n # Scaling tree size (10 commits, 10 files/commit).\ndiff --git a/t/t5413-connectivity-check.sh b/t/t5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..1c84cfe452\n--- /dev/null\n+++ b/t/t5413-connectivity-check.sh\n@@ -0,0 +1,680 @@\n+#!/bin/sh\n+\n+test_description='connectivity check (transfer.connectivityCheck)'\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_oid_cache <<-\\EOF\n+missing sha1:0000000000000000000000000000000000000001\n+missing sha256:0000000000000000000000000000000000000000000000000000000000000001\n+EOF\n+\n+set_connectivity_check () {\n+\tgit -C \"$1\" config transfer.connectivityCheck \"$2\"\n+}\n+\n+# Run a connectivity check via rev-list directly.  Uses $mode\n+# (set by the enclosing for-loop) to choose full or incremental.\n+check_connected () {\n+\tflags= &&\n+\tif test \"$mode\" = incremental\n+\tthen\n+\t\tflags=--verify-trees-incremental\n+\tfi &&\n+\tprintf '%s\\n' \"$@\" |\n+\tgit rev-list $flags \\\n+\t\t--objects --stdin --not --all --quiet \\\n+\t\t--exclude-promisor-objects\n+}\n+\n+# Run git with a temporary index, leaving the real index untouched.\n+tmpgit () {\n+\tGIT_INDEX_FILE=.git/tmp-idx git \"$@\"\n+}\n+\n+# Create a commit with one file changed, without modifying HEAD,\n+# index, or worktree.  Prints the new commit OID on stdout.\n+# Usage: commit_with_change <parent> <path> <content>\n+commit_with_change () {\n+\tnew_blob=$(echo \"$3\" | git hash-object -w --stdin) &&\n+\ttmpgit read-tree \"$1\" &&\n+\ttmpgit update-index --replace \\\n+\t\t--cacheinfo \"100644,$new_blob,$2\" &&\n+\tnew_tree=$(tmpgit write-tree) &&\n+\trm -f .git/tmp-idx &&\n+\tgit commit-tree \"$new_tree\" -p \"$1\" -m \"modify $2\"\n+}\n+\n+# Check OIDs and optionally verify trace2 counts.\n+# Usage: check_connected_trace <trace-file> <trees> <blobs> <oid>...\n+# An empty string for <trees> or <blobs> skips that assertion.\n+check_connected_trace () {\n+\ttrace_file=$1 trees=$2 blobs=$3 &&\n+\tshift 3 &&\n+\ttest_env GIT_TRACE2_EVENT=\"$(pwd)/$trace_file\" \\\n+\t\tcheck_connected \"$@\" &&\n+\tif test \"$mode\" != incremental\n+\tthen\n+\t\treturn\n+\tfi &&\n+\tif test -n \"$trees\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity trees_loaded \"$trees\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi &&\n+\tif test -n \"$blobs\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity blobs_checked \"$blobs\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi\n+}\n+\n+# Shared setup: a repo with several root-level files and nested dirs.\n+# The unchanged/ subtree (10 dirs x 10 files = 100 blobs, 11 trees)\n+# acts as a canary: any test asserting small tree/blob counts would\n+# fail dramatically if incremental accidentally walked into it.\n+\n+test_expect_success 'setup main repo' '\n+\tgit init main-repo &&\n+\t(\n+\t\tcd main-repo &&\n+\t\tfor i in $(test_seq 1 5)\n+\t\tdo\n+\t\t\techo \"file $i\" >\"file-$i.txt\" || return 1\n+\t\tdone &&\n+\t\tgit add file-*.txt &&\n+\t\tgit commit -m \"initial\" &&\n+\n+\t\tmkdir -p a/b/c &&\n+\t\techo deep >a/b/c/deep.txt &&\n+\t\techo other >a/other.txt &&\n+\t\tgit add a/b/c/deep.txt a/other.txt &&\n+\t\tgit commit -m \"add nested dirs\" &&\n+\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\td=\"unchanged/dir-$i\" &&\n+\t\t\tmkdir -p \"$d\" &&\n+\t\t\tfor j in $(test_seq 1 10)\n+\t\t\tdo\n+\t\t\t\techo \"$i $j\" >\"$d/file-$j.txt\" || return 1\n+\t\t\tdone\n+\t\tdone &&\n+\t\tgit add unchanged/ &&\n+\t\tgit commit -m \"add unchanged canary subtree\"\n+\t)\n+'\n+\n+test_expect_success 'setup replacement object repo' '\n+\tgit init replace-test &&\n+\t(\n+\t\tcd replace-test &&\n+\n+\t\ttest_commit --no-tag original file.txt &&\n+\t\toriginal=$(git rev-parse HEAD) &&\n+\t\torig_blob=$(git rev-parse HEAD:file.txt) &&\n+\n+\t\t# Orphan replacement commit with a different tree\n+\t\treplacement_tree=$(echo replaced | git hash-object -w --stdin |\n+\t\t\txargs -I{} git mktree <<-EOF\n+\t\t\t100644 blob {}\tfile.txt\n+\t\t\tEOF\n+\t\t) &&\n+\t\treplacement=$(git commit-tree -m \"replacement\" \\\n+\t\t\t\"$replacement_tree\") &&\n+\n+\t\tgit replace \"$original\" \"$replacement\" &&\n+\n+\t\t# Remove the original blob so only the replacement\n+\t\t# tree is complete.\n+\t\trm .git/objects/$(test_oid_to_path \"$orig_blob\") &&\n+\n+\t\t# Drop branch and HEAD so --not --all does not\n+\t\t# exclude the original commit.\n+\t\tgit update-ref -d refs/heads/main &&\n+\t\tgit update-ref -d HEAD &&\n+\n+\t\techo \"$original\" >.git/test-oid\n+\t)\n+'\n+\n+missing_oid=$(test_oid missing)\n+original_oid=$(cat replace-test/.git/test-oid)\n+\n+for mode in full incremental\n+do\n+\n+# Corruption detection: craft broken object graphs and verify detection.\n+# All tests use main-repo without modifying its refs or worktree.\n+\n+test_expect_success \"$mode: rejects commit with missing blob\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects commit with missing subtree\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"40000 tree ${missing_oid}\\tdir\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"bad tree object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct tree tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"hello\" | git hash-object -w --stdin) &&\n+\t\tcheck_connected \"$blob_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects missing direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$missing_oid\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects blob OID reused as tree entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\tblob_oid=$(git rev-parse HEAD:file-1.txt) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"40000 subdir\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a tree\" err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects tree OID reused as blob entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\ttree_oid=$(git rev-parse HEAD:a) &&\n+\t\tbin_oid=$(echo \"$tree_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"100644 fakefile\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a blob\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: checks multiple tips\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"tip-a\") &&\n+\t\tc2=$(commit_with_change HEAD file-2.txt \"tip-b\") &&\n+\t\tgit tag -a -m \"tagged\" multi-tag \"$c1\" &&\n+\t\ttag_oid=$(git rev-parse multi-tag) &&\n+\t\tgit tag -d multi-tag &&\n+\t\tcheck_connected \"$c2\" \"$tag_oid\"\n+\t)\n+'\n+\n+# Tree-diff optimization: verify trace2 counts.\n+\n+test_expect_success \"$mode: handles root commit (no parents)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"root-content\" | git hash-object -w --stdin) &&\n+\t\ttree_oid=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$blob_oid\" | git mktree) &&\n+\t\troot_oid=$(git commit-tree \"$tree_oid\" -m \"root\") &&\n+\n+\t\t# No parent trees, so the full tree is verified.\n+\t\t# 1 tree loaded (root), 1 blob checked.\n+\t\tcheck_connected_trace trace-root.txt 1 1 \"$root_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles single file change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD file-1.txt \"changed\") &&\n+\n+\t\t# 2 trees loaded (new root + parent root), 1 blob checked.\n+\t\tcheck_connected_trace trace-flat.txt 2 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles nested change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD a/b/c/deep.txt \"deep-changed\") &&\n+\t\t# 4 new trees + 4 parent trees = 8 loaded, 1 blob checked.\n+\t\tcheck_connected_trace trace-nested.txt 8 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles change-then-revert\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"revert-tmp\") &&\n+\t\tc2=$(commit_with_change \"$c1\" file-1.txt \"file 1\") &&\n+\t\tc3=$(commit_with_change \"$c2\" file-1.txt \"revert-final\") &&\n+\n+\t\t# c1: new root + parent root = 2 loads.  c2: root matches\n+\t\t# HEAD (already trusted), skipped.  c3: new root + parent\n+\t\t# already expanded = 1 load.  Total: 3 trees, 2 blobs.\n+\t\tcheck_connected_trace trace-revert.txt 3 2 \"$c3\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles subtree moved to another path\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tmoved_tree=$(git ls-tree HEAD |\n+\t\t\tsed \"s/\ta$/\tmoved/\" |\n+\t\t\tgit mktree) &&\n+\t\tmoved=$(git commit-tree \"$moved_tree\" -p HEAD -m move) &&\n+\t\t# New root + parent root scanned, but the moved subtree\n+\t\t# (same OID) is trusted and not descended into.\n+\t\tcheck_connected_trace trace-move.txt 2 0 \"$moved\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles multi-parent merge\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tleft=$(commit_with_change HEAD file-1.txt left) &&\n+\t\tright=$(commit_with_change HEAD file-2.txt right) &&\n+\t\tgit update-ref refs/heads/left \"$left\" &&\n+\t\tgit update-ref refs/heads/right \"$right\" &&\n+\t\tleft_blob=$(git rev-parse \"$left:file-1.txt\") &&\n+\t\tright_blob=$(git rev-parse \"$right:file-2.txt\") &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$left_blob,file-1.txt\" &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$right_blob,file-2.txt\" &&\n+\t\tmerge_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tmerge=$(git commit-tree \"$merge_tree\" \\\n+\t\t\t-p \"$left\" -p \"$right\" -m merge) &&\n+\t\t# Both parent root trees are scanned as bases, so\n+\t\t# blobs from each parent are trusted without ODB checks.\n+\t\tcheck_connected_trace trace-merge.txt 3 0 \"$merge\" &&\n+\t\tgit update-ref -d refs/heads/left &&\n+\t\tgit update-ref -d refs/heads/right\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles merge with incoming and boundary parents\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# parent1 is incoming (not a ref), HEAD is boundary.\n+\t\tparent1=$(commit_with_change HEAD file-1.txt merge-inc) &&\n+\t\tmerge=$(git commit-tree \\\n+\t\t\t\"$(git rev-parse \"$parent1^{tree}\")\" \\\n+\t\t\t-p \"$parent1\" -p HEAD -m merge-mixed) &&\n+\t\t# parent1 verified first (topo order): 2 trees, 1 blob.\n+\t\t# merge tree = parent1 tree (already trusted): 0 extra.\n+\t\tcheck_connected_trace trace-merge-inc.txt 2 1 \"$merge\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles gitlink entries (submodules)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"160000,$missing_oid,my-submodule\" &&\n+\t\tgitlink_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tgitlink_commit=$(git commit-tree \"$gitlink_tree\" -p HEAD \\\n+\t\t\t-m \"add gitlink\") &&\n+\n+\t\t# Gitlink entries are skipped -- the missing submodule\n+\t\t# commit OID does not cause a failure.\n+\t\tcheck_connected_trace trace-gitlink.txt 2 0 \"$gitlink_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles file-to-directory transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"foo\" is a blob at root.\n+\t\tblob_a=$(echo \"file-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,foo\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add foo as file\") &&\n+\n+\t\t# Child: \"foo\" becomes a directory (foo/bar.txt).\n+\t\tblob_b=$(echo \"dir-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove foo &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,foo/bar.txt\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"foo: file to directory\") &&\n+\t\tcheck_connected_trace trace-f2d.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles directory-to-file transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"bar/baz.txt\" exists (bar is a directory).\n+\t\tblob_a=$(echo \"nested\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,bar/baz.txt\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add bar as directory\") &&\n+\n+\t\t# Child: \"bar\" becomes a plain file.\n+\t\tblob_b=$(echo \"flat\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove bar/baz.txt &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,bar\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"bar: directory to file\") &&\n+\t\tcheck_connected_trace trace-d2f.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+# Replacement objects.\n+\n+test_expect_success \"$mode: accepts with replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tcheck_connected \"$original_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects without replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tGIT_NO_REPLACE_OBJECTS=1 &&\n+\t\texport GIT_NO_REPLACE_OBJECTS &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$original_oid\" 2>err &&\n+\t\ttest_grep \"missing blob object\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised blob\" '\n+\ttest_when_finished \"rm -rf prom-src prom-server.git prom-client\" &&\n+\tgit init prom-src &&\n+\ttest_commit -C prom-src --no-tag base file.txt original &&\n+\ttest_commit -C prom-src --no-tag \"add file2\" file2.txt extra &&\n+\tgit clone --bare prom-src prom-server.git &&\n+\tgit -C prom-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/prom-server.git\" prom-client &&\n+\t(\n+\t\tcd prom-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file2.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tnewname.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: does not lazy-fetch promised blob\" '\n+\ttest_when_finished \"rm -rf nofetch-src nofetch-server.git nofetch-client\" &&\n+\tgit init nofetch-src &&\n+\ttest_commit -C nofetch-src --no-tag base file.txt content &&\n+\tgit clone --bare nofetch-src nofetch-server.git &&\n+\tgit -C nofetch-server.git config uploadpack.allowfilter true &&\n+\tgit -C nofetch-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/nofetch-server.git\" nofetch-client &&\n+\t(\n+\t\tcd nofetch-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-m \"root commit with promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised tree\" '\n+\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n+\tgit init prom-tree-src &&\n+\tmkdir -p prom-tree-src/a/b &&\n+\ttest_commit -C prom-tree-src --no-tag \"nested dirs\" a/b/file.txt deep &&\n+\tgit clone --bare prom-tree-src prom-tree-server.git &&\n+\tgit -C prom-tree-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-tree-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-tree-server.git\" prom-tree-client &&\n+\t(\n+\t\tcd prom-tree-client &&\n+\t\tpromised_tree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\" &&\n+\t\tnew_tree=$(printf \"40000 tree %s\\trenamed\\n\" \\\n+\t\t\t\"$promised_tree\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised tree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies new subtree when parent subtree is promised\" '\n+\ttest_when_finished \"rm -rf prom-base-src prom-base-server.git prom-base-client\" &&\n+\tgit init prom-base-src &&\n+\tmkdir -p prom-base-src/a &&\n+\ttest_commit -C prom-base-src --no-tag \"base\" a/file.txt deep &&\n+\tgit clone --bare prom-base-src prom-base-server.git &&\n+\tgit -C prom-base-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-base-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-base-server.git\" prom-base-client &&\n+\t(\n+\t\tcd prom-base-client &&\n+\t\tparent_subtree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\" &&\n+\t\tnew_blob=$(echo \"local-content\" | git hash-object -w --stdin) &&\n+\t\tnew_subtree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$new_blob\" | git mktree) &&\n+\t\tnew_root=$(printf \"40000 tree %s\\ta\\n\" \\\n+\t\t\t\"$new_subtree\" | git mktree) &&\n+\t\tnew_commit=$(git commit-tree \"$new_root\" \\\n+\t\t\t-p HEAD -m \"replace promised subtree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies local commit in partial clone\" '\n+\ttest_when_finished \"rm -rf pc-src pc-server.git pc-client\" &&\n+\tgit init pc-src &&\n+\ttest_commit -C pc-src --no-tag base file.txt &&\n+\tgit clone --bare pc-src pc-server.git &&\n+\tgit -C pc-server.git config uploadpack.allowfilter true &&\n+\tgit -C pc-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/pc-server.git\" pc-client &&\n+\t(\n+\t\tcd pc-client &&\n+\t\tlocal_commit=$(commit_with_change HEAD file.txt local-content) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: respects shallow boundary\" '\n+\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n+\tgit init shallow-src &&\n+\ttest_commit -C shallow-src --no-tag base file content-1 &&\n+\tmkdir shallow-src/sub &&\n+\ttest_commit -C shallow-src --no-tag change sub/other content-2 &&\n+\tgit clone --depth=1 \"file://$(pwd)/shallow-src\" shallow &&\n+\t(\n+\t\tcd shallow &&\n+\t\ttip=$(git rev-parse HEAD) &&\n+\t\tgit for-each-ref --format=\"delete %(refname)\" |\n+\t\t\tgit update-ref --no-deref --stdin &&\n+\t\tcheck_connected \"$tip\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: deepening fetch succeeds\" '\n+\ttest_when_finished \"rm -rf deepen-src deepen-server.git deepen-client\" &&\n+\tgit init deepen-src &&\n+\ttest_commit -C deepen-src --no-tag c1 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c2 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c3 file.txt &&\n+\tgit clone --bare deepen-src deepen-server.git &&\n+\tgit clone --depth=1 \"file://$(pwd)/deepen-server.git\" deepen-client &&\n+\tset_connectivity_check deepen-client $mode &&\n+\ttest -f deepen-client/.git/shallow &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/deepen-trace.txt\" \\\n+\t\tgit -C deepen-client fetch --deepen=2 origin main &&\n+\t# Incremental falls back to full for deepening fetches,\n+\t# so the trees_loaded event should not appear.\n+\ttest_grep ! trees_loaded deepen-trace.txt\n+'\n+\n+test_expect_success \"$mode: malformed tree detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\techo abc >malformed-tree &&\n+\t\tmalformed_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tmalformed-tree) &&\n+\t\tmalformed_commit=$(git commit-tree \"$malformed_tree\" \\\n+\t\t\t-p HEAD -m malformed) &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$malformed_commit\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: mid-tree corruption detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# Build a tree with one valid entry followed by garbage.\n+\t\tblob_oid=$(echo \"valid\" | git hash-object -w --stdin) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\t\tprintf \"100644 good\\0${bin_oid}GARBAGE\" >corrupt-mid-tree &&\n+\t\tcorrupt_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tcorrupt-mid-tree) &&\n+\t\tcorrupt_commit=$(git commit-tree \"$corrupt_tree\" \\\n+\t\t\t-p HEAD -m \"mid-tree corruption\") &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$corrupt_commit\" 2>err &&\n+\t\ttest_grep \"too-short tree object\" err\n+\t)\n+'\n+\n+done\n+\n+# Algorithm selection.\n+\n+test_expect_success 'invalid transfer.connectivityCheck is rejected' '\n+\ttest_when_finished \"rm -rf invalid-cfg-src invalid-cfg-dst\" &&\n+\tgit init invalid-cfg-src &&\n+\ttest_commit -C invalid-cfg-src --no-tag base file.txt &&\n+\tgit clone invalid-cfg-src invalid-cfg-dst &&\n+\ttest_commit -C invalid-cfg-src --no-tag update file.txt updated &&\n+\tgit -C invalid-cfg-dst config transfer.connectivityCheck bogus &&\n+\ttest_must_fail git -C invalid-cfg-dst fetch origin main 2>err &&\n+\ttest_grep \"unknown transfer.connectivityCheck\" err\n+'\n+\n+test_expect_success 'push uses incremental when configured' '\n+\ttest_when_finished \"rm -rf int-src int-dst.git\" &&\n+\tgit init int-src &&\n+\ttest_commit -C int-src --no-tag base file.txt &&\n+\tgit clone --bare int-src int-dst.git &&\n+\ttest_commit -C int-src --no-tag update file.txt updated &&\n+\tset_connectivity_check int-dst.git incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/push-trace.txt\" \\\n+\t\tgit -C int-src push ../int-dst.git main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<push-trace.txt\n+'\n+\n+test_expect_success 'fetch uses incremental when configured' '\n+\ttest_when_finished \"rm -rf fetch-src fetch-dst\" &&\n+\tgit init fetch-src &&\n+\ttest_commit -C fetch-src --no-tag base file.txt &&\n+\tgit clone fetch-src fetch-dst &&\n+\ttest_commit -C fetch-src --no-tag update file.txt updated &&\n+\tset_connectivity_check fetch-dst incremental &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/fetch-trace.txt\" \\\n+\t\tgit -C fetch-dst fetch origin main &&\n+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n+\t\t<fetch-trace.txt\n+'\n+\n+test_expect_success 'clone respects transfer.connectivityCheck' '\n+\ttest_when_finished \"rm -rf clone-src clone-dst\" &&\n+\tgit init clone-src &&\n+\ttest_commit -C clone-src --no-tag base file.txt &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/clone-trace.txt\" \\\n+\t\tgit -c transfer.connectivityCheck=incremental \\\n+\t\tclone --no-local clone-src clone-dst &&\n+\ttest_trace2_data_singular connectivity trees_loaded 0 \\\n+\t\t<clone-trace.txt\n+'\n+\n+test_done\ndiff --git a/tree-verify.c b/tree-verify.c\nnew file mode 100644\nindex 0000000000..b381a50102\n--- /dev/null\n+++ b/tree-verify.c\n@@ -0,0 +1,316 @@\n+#include \"git-compat-util.h\"\n+#include \"commit.h\"\n+#include \"gettext.h\"\n+#include \"hex.h\"\n+#include \"khash.h\"\n+#include \"object.h\"\n+#include \"odb.h\"\n+#include \"oid-array.h\"\n+#include \"oidset.h\"\n+#include \"tree.h\"\n+#include \"tree-walk.h\"\n+#include \"tree-verify.h\"\n+#include \"packfile.h\"\n+#include \"trace2.h\"\n+\n+enum tree_state {\n+\tTREE_UNTRUSTED = 0,\n+\tTREE_TRUSTED   = 1,\n+\tTREE_EXPANDED  = 2,\n+};\n+\n+KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n+\t   oidhash_by_value, oideq_by_value)\n+\n+static enum tree_state tree_map_get(kh_oid_tree_t *m,\n+\t\t\t\t    const struct object_id *oid)\n+{\n+\tkhint_t pos = kh_get_oid_tree(m, *oid);\n+\tif (pos == kh_end(m))\n+\t\treturn TREE_UNTRUSTED;\n+\treturn kh_val(m, pos);\n+}\n+\n+static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n+\t\t\t enum tree_state state)\n+{\n+\tint added;\n+\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n+\tif (added)\n+\t\tkh_val(m, pos) = state;\n+\telse if (state > kh_val(m, pos))\n+\t\tkh_val(m, pos) = state;\n+}\n+\n+struct work_item {\n+\tstruct name_entry entry;\n+\tstruct oid_array parent_trees;\n+};\n+\n+struct verify_state {\n+\tkh_oid_tree_t *trees;\n+\tstruct oidset trusted_blobs;\n+\tint trees_loaded;\n+\tint blobs_checked;\n+\tint exclude_promisor_objects;\n+};\n+\n+/*\n+ * Merge-walk the work list against one base tree entry, recording\n+ * same-path parent subtrees as recursive comparison bases.\n+ * Returns the updated work-list cursor.\n+ */\n+static size_t collect_subtree_bases(struct work_item *work, size_t nr_work,\n+\t\t\t\t    size_t wi, const struct name_entry *entry)\n+{\n+\twhile (wi < nr_work) {\n+\t\tint cmp = base_name_compare(\n+\t\t\twork[wi].entry.path, work[wi].entry.pathlen,\n+\t\t\twork[wi].entry.mode,\n+\t\t\tentry->path, entry->pathlen,\n+\t\t\tentry->mode);\n+\t\tif (cmp > 0)\n+\t\t\tbreak;\n+\t\tif (cmp < 0) {\n+\t\t\twi++;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (S_ISDIR(work[wi].entry.mode) &&\n+\t\t    S_ISDIR(entry->mode))\n+\t\t\toid_array_append(&work[wi].parent_trees,\n+\t\t\t\t\t &entry->oid);\n+\t\treturn wi + 1;\n+\t}\n+\treturn wi;\n+}\n+\n+static void verify_blob(struct repository *repo,\n+\t\t\tconst struct object_id *oid,\n+\t\t\tstruct verify_state *vs)\n+{\n+\tint type;\n+\n+\tif (oidset_contains(&vs->trusted_blobs, oid))\n+\t\treturn;\n+\n+\tvs->blobs_checked++;\n+\ttype = odb_read_object_info(repo->objects, oid, NULL);\n+\tif (type == OBJ_BLOB) {\n+\t\toidset_insert(&vs->trusted_blobs, oid);\n+\t\treturn;\n+\t}\n+\tif (type >= 0)\n+\t\tdie(_(\"object %s is a %s, not a %s\"),\n+\t\t    oid_to_hex(oid), type_name(type), \"blob\");\n+\tif (vs->exclude_promisor_objects &&\n+\t    is_promisor_object(repo, oid))\n+\t\treturn;\n+\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n+}\n+\n+static void *read_tree_object(struct object_database *odb,\n+\t\t\t      const struct object_id *oid,\n+\t\t\t      size_t *sizep)\n+{\n+\tenum object_type type;\n+\tvoid *buf = odb_read_object(odb, oid, &type, sizep);\n+\n+\tif (buf && type != OBJ_TREE) {\n+\t\tfree(buf);\n+\t\tdie(_(\"object %s is a %s, not a %s\"),\n+\t\t    oid_to_hex(oid), type_name(type), \"tree\");\n+\t}\n+\treturn buf;\n+}\n+\n+static void verify_tree(struct repository *repo,\n+\t\t\tconst struct object_id *new_tree_oid,\n+\t\t\tconst struct oid_array *base_trees,\n+\t\t\tstruct verify_state *vs, int depth)\n+{\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+\tstruct work_item *work = NULL;\n+\tsize_t nr_work = 0, alloc_work = 0;\n+\tint need_subtree_bases = 0;\n+\tsize_t i, tree_size;\n+\tvoid *tree_buf;\n+\n+\tif (depth > repo->settings.max_allowed_tree_depth)\n+\t\tdie(_(\"exceeded maximum allowed tree depth\"));\n+\n+\tif (tree_map_get(vs->trees, new_tree_oid) >= TREE_TRUSTED)\n+\t\treturn;\n+\n+\ttree_buf = read_tree_object(repo->objects, new_tree_oid, &tree_size);\n+\tif (!tree_buf) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, new_tree_oid))\n+\t\t\treturn;\n+\t\tdie(_(\"bad tree object %s\"),\n+\t\t    oid_to_hex(new_tree_oid));\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tinit_tree_desc(&desc, new_tree_oid, tree_buf, tree_size);\n+\n+\twhile (tree_entry(&desc, &entry)) {\n+\t\tif (S_ISGITLINK(entry.mode))\n+\t\t\tcontinue;\n+\t\tif (S_ISDIR(entry.mode)) {\n+\t\t\tif (tree_map_get(vs->trees, &entry.oid) >= TREE_TRUSTED)\n+\t\t\t\tcontinue;\n+\t\t\tneed_subtree_bases = 1;\n+\t\t} else {\n+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry.oid))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\tALLOC_GROW(work, nr_work + 1, alloc_work);\n+\t\tmemset(&work[nr_work], 0, sizeof(work[nr_work]));\n+\t\twork[nr_work].entry = entry;\n+\t\tnr_work++;\n+\t}\n+\n+\tif (!nr_work) {\n+\t\tfree(tree_buf);\n+\t\tgoto done;\n+\t}\n+\n+\tfor (i = 0; base_trees && i < base_trees->nr; i++) {\n+\t\tconst struct object_id *base_oid = &base_trees->oid[i];\n+\t\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n+\t\tstruct tree_desc base_desc;\n+\t\tstruct name_entry scan_entry;\n+\t\tsize_t wi = 0, base_size;\n+\t\tvoid *base_buf;\n+\n+\t\tif (expanded && !need_subtree_bases)\n+\t\t\tcontinue;\n+\n+\t\tbase_buf = read_tree_object(repo->objects, base_oid,\n+\t\t\t\t\t    &base_size);\n+\t\tif (!base_buf) {\n+\t\t\tif (vs->exclude_promisor_objects &&\n+\t\t\t    is_promisor_object(repo, base_oid))\n+\t\t\t\tcontinue;\n+\t\t\tdie(_(\"bad tree object %s\"),\n+\t\t\t    oid_to_hex(base_oid));\n+\t\t}\n+\n+\t\tvs->trees_loaded++;\n+\t\tinit_tree_desc(&base_desc, base_oid, base_buf, base_size);\n+\n+\t\twhile (tree_entry(&base_desc, &scan_entry)) {\n+\t\t\tif (S_ISGITLINK(scan_entry.mode))\n+\t\t\t\tcontinue;\n+\n+\t\t\tif (need_subtree_bases)\n+\t\t\t\twi = collect_subtree_bases(work, nr_work,\n+\t\t\t\t\t\t\t   wi, &scan_entry);\n+\n+\t\t\tif (!expanded) {\n+\t\t\t\tif (S_ISDIR(scan_entry.mode))\n+\t\t\t\t\ttree_map_add(vs->trees,\n+\t\t\t\t\t\t     &scan_entry.oid,\n+\t\t\t\t\t\t     TREE_TRUSTED);\n+\t\t\t\telse\n+\t\t\t\t\toidset_insert(&vs->trusted_blobs,\n+\t\t\t\t\t\t      &scan_entry.oid);\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!expanded)\n+\t\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n+\n+\t\tfree(base_buf);\n+\t}\n+\n+\tfor (i = 0; i < nr_work; i++) {\n+\t\tif (S_ISDIR(work[i].entry.mode))\n+\t\t\tverify_tree(repo, &work[i].entry.oid,\n+\t\t\t\t    &work[i].parent_trees, vs,\n+\t\t\t\t    depth + 1);\n+\t\telse\n+\t\t\tverify_blob(repo, &work[i].entry.oid, vs);\n+\t}\n+\n+\tfree(tree_buf);\n+\n+done:\n+\ttree_map_add(vs->trees, new_tree_oid, TREE_EXPANDED);\n+\tfor (i = 0; i < nr_work; i++)\n+\t\toid_array_clear(&work[i].parent_trees);\n+\tfree(work);\n+}\n+\n+static void verify_commit_tree(struct repository *repo,\n+\t\t\t       struct commit *commit,\n+\t\t\t       struct verify_state *vs)\n+{\n+\tstruct oid_array base_trees = OID_ARRAY_INIT;\n+\tconst struct object_id *tree_oid;\n+\tstruct commit_list *p;\n+\n+\t/*\n+\t * Parent trees are trusted: boundary parents are already\n+\t * connected, and earlier incoming parents were verified\n+\t * first due to the topological processing order.\n+\t */\n+\tfor (p = commit->parents; p; p = p->next) {\n+\t\tparse_commit_or_die(p->item);\n+\t\ttree_oid = get_commit_tree_oid(p->item);\n+\t\tif (!tree_oid)\n+\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n+\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n+\t\toid_array_append(&base_trees, tree_oid);\n+\t}\n+\n+\ttree_oid = get_commit_tree_oid(commit);\n+\tif (!tree_oid)\n+\t\tdie(_(\"unable to load root tree for commit %s\"),\n+\t\t    oid_to_hex(&commit->object.oid));\n+\tverify_tree(repo, tree_oid, &base_trees, vs, 0);\n+\toid_array_clear(&base_trees);\n+}\n+\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects)\n+{\n+\tstruct verify_state vs = { 0 };\n+\tstruct commit_list *iter;\n+\tunsigned nr_before;\n+\n+\tif (repo->fetch_if_missing)\n+\t\tBUG(\"verify_commits_incremental must not be called \"\n+\t\t    \"with fetch_if_missing set\");\n+\n+\tvs.trees = kh_init_oid_tree();\n+\tvs.exclude_promisor_objects = exclude_promisor_objects;\n+\n+\t/*\n+\t * Ancestors must be verified before descendants so that parent\n+\t * trees can be trusted without re-verification.  Sort explicitly\n+\t * rather than relying on the caller's ordering.\n+\t *\n+\t * sort_in_topological_order() silently drops cycle members,\n+\t * so explicitly check if the size has changed.\n+\t */\n+\tnr_before = commit_list_count(*commits);\n+\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n+\tif (commit_list_count(*commits) < nr_before)\n+\t\tdie(_(\"cycle detected in incoming commit graph\"));\n+\n+\t*commits = commit_list_reverse(*commits);\n+\n+\tfor (iter = *commits; iter; iter = iter->next)\n+\t\tverify_commit_tree(repo, iter->item, &vs);\n+\n+\tkh_destroy_oid_tree(vs.trees);\n+\toidset_clear(&vs.trusted_blobs);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"trees_loaded\", vs.trees_loaded);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"blobs_checked\", vs.blobs_checked);\n+}\ndiff --git a/tree-verify.h b/tree-verify.h\nnew file mode 100644\nindex 0000000000..6aadadff70\n--- /dev/null\n+++ b/tree-verify.h\n@@ -0,0 +1,15 @@\n+#ifndef TREE_VERIFY_H\n+#define TREE_VERIFY_H\n+\n+struct commit_list;\n+struct repository;\n+\n+/*\n+ * Verify trees of commits incrementally against their parents.\n+ * Dies on verification failure.\n+ */\n+void verify_commits_incremental(struct repository *repo,\n+\t\t\t\tstruct commit_list **commits,\n+\t\t\t\tint exclude_promisor_objects);\n+\n+#endif /* TREE_VERIFY_H */\n-- \ngitgitgadget\n\n"},{"id":"554794","messageId":"xmqq7bjm7o0r.fsf@gitster.g","threadId":"66326","inReplyTo":"cdf28f1df6cb36b006f367de0e18e23e3aa8d992.1791806877.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-12T18:49:08Z","receivedAt":"2026-10-12T18:49:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> diff --git a/connected.c b/connected.c\n> index 929b9bd28d..e3dcd8e2b0 100644\n> --- a/connected.c\n> +++ b/connected.c\n> @@ -1,6 +1,7 @@\n>  #define USE_THE_REPOSITORY_VARIABLE\n>  \n>  #include \"git-compat-util.h\"\n> +#include \"config.h\"\n>  #include \"gettext.h\"\n>  #include \"hex.h\"\n>  #include \"odb.h\"\n> @@ -67,6 +68,26 @@ static int check_connected_promisor(oid_iterate_fn fn,\n>  \treturn 1;\n>  }\n>  \n> +static int incremental_check_applicable(struct check_connected_options *opt)\n> +{\n> +\tconst char *algorithm = NULL;\n> +\n> +\tif (repo_config_get_string_tmp(the_repository,\n> +\t\t\t\t       \"transfer.connectivitycheck\",\n> +\t\t\t\t       &algorithm))\n> +\t\treturn 0;\n\nThe file does declare USE_THE_REPOSITORY_VARIABLE but I wonder if we\ncan easily reduce the dependency to the_repository by adding \"struct\nrepository *repo\" pointer to the check_connected_options structure\nand using it here and passing it to check_connected_promisor() which\nis a file-scope static nearby.\n\n> +\tif (!strcasecmp(algorithm, \"full\"))\n> +\t\treturn 0;\n> +\tif (strcasecmp(algorithm, \"incremental\"))\n> +\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n> +\t\t    algorithm);\n> +\n> +\tif (opt->is_deepening_fetch)\n> +\t\treturn 0;\n> +\n> +\treturn 1;\n> +}\n> +\n>  /*\n>   * If we feed all the commits we want to verify to this command\n>   *\n> @@ -133,6 +154,9 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n>  \tif (opt->progress)\n>  \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n>  \t\t\t     _(\"Checking connectivity\"));\n> +\tif (incremental_check_applicable(opt))\n> +\t\tstrvec_push(&rev_list.args,\n> +\t\t\t    \"--verify-trees-incremental\");\n>  \n>  \trev_list.git_cmd = 1;\n>  \tif (opt->env)\n\n"},{"id":"554795","messageId":"xmqqzewi695w.fsf@gitster.g","threadId":"66326","inReplyTo":"pull.2211.v3.git.1791806877.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/4] connected: add incremental connectivity check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-12T18:55:23Z","receivedAt":"2026-10-12T18:55:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> I'll just mention a short summary here, to avoid repeating what's already in\n> the commit message. The incremental mode is faster than the full mode when\n> there are few commits to verify and when the active object tree is large. In\n> the happy case, the work tracks the changed paths and their comparison trees\n> rather than the full reachable object closure, which substantially reduces\n> the dependence on total repository size.\n\nThis is exciting.\n\n> There are also regression cases in the synthetic fixtures. With long\n> incoming histories, the extra parent-tree scans accumulate; in the synthetic\n> fixture incremental is about 1.5x slower at 10000 commits. Per-commit\n> ...\n> A safety net for this regression could be to dynamically disable the\n> incremental mode if the number of incoming commits is too large, but this is\n> left out of the initial version to avoid overly speculative code.\n\nHmm, ... OK.\n\n> Deepening fetches currently fall back to the full check because the full\n> check omits --not --all for deepening -- there is no existing-reference\n> boundary at which the walk can stop. An incremental approach is possible\n> here too -- using the old shallow roots as the trusted boundary and walking\n> the deepened ancestry forward -- but that is a separate change and left for\n> future work. Deepening is also less common than regular fetch and\n> receive-pack, where the speedup matters most.\n\nOK.\n\n"},{"id":"554838","messageId":"CAL71e4MVf6QcFsAdUnW3XGw_g-OLdUZFyopkELVLsCqvM=Qinw@mail.gmail.com","threadId":"66326","inReplyTo":"xmqq7bjm7o0r.fsf@gitster.g","subject":"Re: [PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-13T07:41:24Z","receivedAt":"2026-10-13T07:41:24Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 12 Oct 2026 at 20:49, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> > +static int incremental_check_applicable(struct check_connected_options *opt)\n> > +{\n> > +     const char *algorithm = NULL;\n> > +\n> > +     if (repo_config_get_string_tmp(the_repository,\n> > +                                    \"transfer.connectivitycheck\",\n> > +                                    &algorithm))\n> > +             return 0;\n>\n> The file does declare USE_THE_REPOSITORY_VARIABLE but I wonder if we\n> can easily reduce the dependency to the_repository by adding \"struct\n> repository *repo\" pointer to the check_connected_options structure\n> and using it here and passing it to check_connected_promisor() which\n> is a file-scope static nearby.\n\nYes, I could add the repo pointer to the structure if needed.\nI initally just considered adding it as a function parameter\n(which would not be a large benefit, except pushing out the\nrepository usage to the outer layers at least) but adding\nit to the struct would perhaps be cleaner.\n\nI could also add a slightly larger patch to fix\nthe entire file and remove USE_THE_REPOSITORY_VARIABLE.\nI don't think it would be overly messy.\n\nI will play around a bit with both these ideas and see\nwhere it ends up for the next reroll.\n\nThanks,\nKristofer\n\n"},{"id":"554839","messageId":"CAL71e4OUW4rYqxaGgcrsy0aqjRSdWsz8ghtNRGQVYd9Gkz4NgQ@mail.gmail.com","threadId":"66326","inReplyTo":"xmqqzewi695w.fsf@gitster.g","subject":"Re: [PATCH v3 0/4] connected: add incremental connectivity check","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-13T07:54:38Z","receivedAt":"2026-10-13T07:54:38Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 12 Oct 2026 at 20:55, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> > I'll just mention a short summary here, to avoid repeating what's already in\n> > the commit message. The incremental mode is faster than the full mode when\n> > there are few commits to verify and when the active object tree is large. In\n> > the happy case, the work tracks the changed paths and their comparison trees\n> > rather than the full reachable object closure, which substantially reduces\n> > the dependence on total repository size.\n>\n> This is exciting.\n\nI think so too, especially since I think it simplifies the\nfollow-up where the scaling with number of refs can also\nbe addressed and I hope to write up some more concrete ideas\nfor that in the not too distant future.\n\n> > There are also regression cases in the synthetic fixtures. With long\n> > incoming histories, the extra parent-tree scans accumulate; in the synthetic\n> > fixture incremental is about 1.5x slower at 10000 commits. Per-commit\n> > ...\n> > A safety net for this regression could be to dynamically disable the\n> > incremental mode if the number of incoming commits is too large, but this is\n> > left out of the initial version to avoid overly speculative code.\n>\n> Hmm, ... OK.\n\nThis is where I would need feedback from you and the mailing list.\nI have been reasoning about this internally and decided to simply\nsubmit the smallest approach as a starting point.\n\nNow with the config flag in place to gate usage as well as having\nthe worst-case regression bounded in how bad it can be\n(2x more time, not worse in memory) I was thinking it may not need\nfurther protection.  I was also not sure how common it would be to\nget pushes with so many commits that it becomes an issue, and that\npart is fundamentally hard to know for git as a general tool.\n\nThat said, I'd be happy to introduce some extra safety net here,\nperhaps add a fallback if there are more than N commits to verify.\n(Hardcoded based on synthetic benchmarks? Configurable?)\n\nOne partly annoying thing here may be that if we add fallback\nfor the bad case, it forces us to keep both code paths and thus\nlose the potential for removing code.  My dream scenario would\nbe if there was a single implementation of the connectivity\ncheck that performs well enough in all scenarios, but maybe\nthat will come anyway eventually and I should not speculate\ntoo much.\n\nWhat would be your preferred approach here?\n\nThanks,\nKristofer\n\n"},{"id":"554858","messageId":"xmqqse291zog.fsf@gitster.g","threadId":"66326","inReplyTo":"CAL71e4MVf6QcFsAdUnW3XGw_g-OLdUZFyopkELVLsCqvM=Qinw@mail.gmail.com","subject":"Re: [PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-13T13:46:07Z","receivedAt":"2026-10-13T13:46:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> I could also add a slightly larger patch to fix\n> the entire file and remove USE_THE_REPOSITORY_VARIABLE.\n> I don't think it would be overly messy.\n>\n> I will play around a bit with both these ideas and see\n> where it ends up for the next reroll.\n\nAlthough I wouldn't mind if we decide to leave it outside the scope\nof the current topic in the end, it is worth considering when we see\na patch adding new use of the global.\n\nThanks.\n\n"},{"id":"555101","messageId":"atB42M2GJ3NitXdj@pks.im","threadId":"66326","inReplyTo":"19ea8cb534f0aaabdf21f358364ad32295b6bfc8.1791806877.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 1/4] Documentation: describe connectivity checking","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-15T06:55:20Z","receivedAt":"2026-10-15T06:55:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 12, 2026 at 12:07:54PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson <krka@spotify.com>\n> \n> Add Documentation/technical/connectivity-check.adoc describing\n> the connectivity invariant and the full connectivity check.\n> \n> Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n> ---\n>  .../technical/connectivity-check.adoc         | 110 ++++++++++++++++++\n>  1 file changed, 110 insertions(+)\n>  create mode 100644 Documentation/technical/connectivity-check.adoc\n\nShouldn't both the Makefile (`TECH_DOCS`) and Meson (`articles`) be\nadapted to build this?\n\n> diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> new file mode 100644\n> index 0000000000..abef02debd\n> --- /dev/null\n> +++ b/Documentation/technical/connectivity-check.adoc\n[snip]\n> +Full connectivity check\n> +-----------------------\n> +\n> +The full connectivity check runs a `rev-list` subprocess, feeding\n> +the new reference tips via stdin.  A normal invocation is roughly:\n> +\n> +    git rev-list --objects --stdin --not --all --quiet\n> +        --alternate-refs [--exclude-promisor-objects]\n> +\n> +When promisor remotes are configured, a fast path based on\n> +promisor packfiles is attempted first.  If it falls back to\n> +the `rev-list` check, `--exclude-promisor-objects` is added so\n> +that the traversal does not trigger on-demand fetches.\n\nWould it make sense to document that fast path here, too?\n\n> +Consider the following graph after a fetch, where all reference\n> +tips point directly to commits.  For simplicity, only local\n> +references appear on the already-connected side; alternate refs\n> +play the same role.  N3 is a merge commit:\n> +\n> +            /-------------L2\n> +           /\n> +    C1---B1---C2---B2-----L1\n> +          \\         \\\n> +           N1        N3---T2\n> +            \\       /\n> +             N2-----------T1\n> +\n> +    L1, L2:         local refs\n> +    T1, T2:         incoming tips (new refs)\n> +    N1, N2, N3:     incoming commits (N3 is a merge)\n> +    B1, B2:         boundary commits (already connected)\n> +    C1, C2:         already connected (but not boundary)\n> +\n> +The incoming set is the commits reachable from the incoming\n> +tips but not from the already-connected side.  Boundary commits\n> +are the already-connected commits at the edge of that set.  Here\n> +B1 is an ancestor of B2, which happens when incoming branches\n> +fork at different depths in the existing history.\n> +\n> +The `--not --all` arguments establish the trust boundary: the\n> +incoming set is the commits reachable from the new tips but not\n> +from any existing reference.\n> +\n> +The traversal then walks the trees of the boundary commits and\n> +marks those objects uninteresting.  Trees of incoming commits\n> +are verified against these marks, stopping at objects already\n\nNit: \"verified against these marks\" reads a tiny bit awkward. How about\n\"Trees of incoming commits are then walked, stopping at objects already\nmarked as uninteresting\".\n\n> +covered.  The cost of marking boundary trees uninteresting\n> +dominates when the boundary commits have large trees, even when\n> +the incoming changes are small.\n\nOverall I think this part is a lot better now, thanks.\n\nPatrick\n\n"},{"id":"555102","messageId":"atB468yrVqsp9Eg6@pks.im","threadId":"66326","inReplyTo":"211794c9252181fa0c31fcd968379c189d001a8c.1791806877.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 2/4] p5413: add perf tests for connectivity check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-15T06:55:39Z","receivedAt":"2026-10-15T06:55:39Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 12, 2026 at 12:07:55PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson <krka@spotify.com>\n> diff --git a/t/perf/generate-repo-p5413-connectivity-check.perl b/t/perf/generate-repo-p5413-connectivity-check.perl\n> new file mode 100644\n> index 0000000000..9546996280\n> --- /dev/null\n> +++ b/t/perf/generate-repo-p5413-connectivity-check.perl\n\nAll of the benchmarks will need to be marked with the PERL prerequisite.\n\n> @@ -0,0 +1,44 @@\n> +#!/usr/bin/perl\n> +#\n> +# Generate a fast-import stream for p5412 connectivity check benchmarks.\n> +#\n> +# Usage: generate-repo-p5412-connectivity-check.perl\n> +#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n\nThese still say p5412, but it's p5413.\n\nPatrick\n\n"},{"id":"555103","messageId":"atB48HLgZr3ssOfx@pks.im","threadId":"66326","inReplyTo":"efaada0b6a8ffdb4d1c01d7f34a0d6596a99c149.1791806877.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-15T06:55:44Z","receivedAt":"2026-10-15T06:55:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 12, 2026 at 12:07:57PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson <krka@spotify.com>\n[snip]\n> Scaling tree size (10 incoming commits, 10 files/commit):\n> \n>     files    full  incr.  full/incr\n>       5K    0.01s  0.01s    1.0x\n>      50K    0.04s  0.01s    4.0x\n>     200K    0.15s  0.01s   15.0x\n>     800K    0.61s  0.03s   20.3x\n> \n> The full mode must walk the trees of boundary commits, which\n> grows with overall tree size.  Incremental still scans the\n> root trees, but avoids descending into unchanged subtrees,\n> so it grows much more slowly with repository size.\n\nOkay, so this is getting better the larger the trees.\n\n> Scaling incoming commit count (200K files, 10 files/commit):\n> \n>     commits    full  incr.  full/incr\n>           1    0.14s  0.01s   14.0x\n>          10    0.15s  0.01s   15.0x\n>         100    0.19s  0.05s    3.8x\n>         500    0.32s  0.27s    1.2x\n>        3000    1.23s  1.52s    0.8x\n>        5000    1.88s  2.43s    0.8x\n>       10000    3.72s  5.36s    0.7x\n> \n> With many commits the per-commit overhead of scanning both\n> the new and parent root trees accumulates and incremental\n> becomes slower.  Breakeven is between 500 and 3000 commits\n> and the ratio stabilizes near 0.7x for this fixture.\n\nAnd worse the more commits we have. So the new mode is only really\napplicable in scenarios where you know what your typical workload looks\nlike. The big question is of course whether we can have the best of both\nworlds, where Git knows to switch between the two modes once it crosses\nthe threshold.\n\n> diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> index abef02debd..a1d2e9709c 100644\n> --- a/Documentation/technical/connectivity-check.adoc\n> +++ b/Documentation/technical/connectivity-check.adoc\n> +Algorithm\n> +~~~~~~~~~\n> +\n> +At a high level:\n> +\n> +    verify(commits):\n> +        sort topologically (ancestors first)\n> +        for each commit:\n> +            mark parent root trees as trusted\n> +            verify_tree(commit.tree, parent root trees)\n\nSo based on this, `verify()` already knows which commits it will be\nchecking for connectivity. So wouldn't it be possible to fall back to\nthe original algortihm here in case we see more than N commits?\n\n> diff --git a/tree-verify.c b/tree-verify.c\n> new file mode 100644\n> index 0000000000..b381a50102\n> --- /dev/null\n> +++ b/tree-verify.c\n> @@ -0,0 +1,316 @@\n> +#include \"git-compat-util.h\"\n> +#include \"commit.h\"\n> +#include \"gettext.h\"\n> +#include \"hex.h\"\n> +#include \"khash.h\"\n> +#include \"object.h\"\n> +#include \"odb.h\"\n> +#include \"oid-array.h\"\n> +#include \"oidset.h\"\n> +#include \"tree.h\"\n> +#include \"tree-walk.h\"\n> +#include \"tree-verify.h\"\n> +#include \"packfile.h\"\n> +#include \"trace2.h\"\n> +\n> +enum tree_state {\n> +\tTREE_UNTRUSTED = 0,\n> +\tTREE_TRUSTED   = 1,\n> +\tTREE_EXPANDED  = 2,\n> +};\n> +\n> +KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n\nThe values are `enum tree_state`s, right? Why don't we use that type\nhere?\n\n> +\t   oidhash_by_value, oideq_by_value)\n> +\n> +static enum tree_state tree_map_get(kh_oid_tree_t *m,\n> +\t\t\t\t    const struct object_id *oid)\n> +{\n> +\tkhint_t pos = kh_get_oid_tree(m, *oid);\n> +\tif (pos == kh_end(m))\n> +\t\treturn TREE_UNTRUSTED;\n> +\treturn kh_val(m, pos);\n> +}\n> +\n> +static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n> +\t\t\t enum tree_state state)\n> +{\n> +\tint added;\n> +\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n> +\tif (added)\n> +\t\tkh_val(m, pos) = state;\n> +\telse if (state > kh_val(m, pos))\n> +\t\tkh_val(m, pos) = state;\n> +}\n> +\n> +struct work_item {\n> +\tstruct name_entry entry;\n> +\tstruct oid_array parent_trees;\n> +};\n> +\n> +struct verify_state {\n> +\tkh_oid_tree_t *trees;\n> +\tstruct oidset trusted_blobs;\n> +\tint trees_loaded;\n> +\tint blobs_checked;\n\nThese probably should be `size_t`.\n\n> +\tint exclude_promisor_objects;\n> +};\n> +\n> +/*\n> + * Merge-walk the work list against one base tree entry, recording\n> + * same-path parent subtrees as recursive comparison bases.\n> + * Returns the updated work-list cursor.\n> + */\n> +static size_t collect_subtree_bases(struct work_item *work, size_t nr_work,\n> +\t\t\t\t    size_t wi, const struct name_entry *entry)\n> +{\n> +\twhile (wi < nr_work) {\n> +\t\tint cmp = base_name_compare(\n> +\t\t\twork[wi].entry.path, work[wi].entry.pathlen,\n> +\t\t\twork[wi].entry.mode,\n> +\t\t\tentry->path, entry->pathlen,\n> +\t\t\tentry->mode);\n> +\t\tif (cmp > 0)\n> +\t\t\tbreak;\n> +\t\tif (cmp < 0) {\n> +\t\t\twi++;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\t\tif (S_ISDIR(work[wi].entry.mode) &&\n> +\t\t    S_ISDIR(entry->mode))\n> +\t\t\toid_array_append(&work[wi].parent_trees,\n> +\t\t\t\t\t &entry->oid);\n> +\t\treturn wi + 1;\n> +\t}\n> +\treturn wi;\n> +}\n> +\n> +static void verify_blob(struct repository *repo,\n> +\t\t\tconst struct object_id *oid,\n> +\t\t\tstruct verify_state *vs)\n> +{\n> +\tint type;\n> +\n> +\tif (oidset_contains(&vs->trusted_blobs, oid))\n> +\t\treturn;\n> +\n> +\tvs->blobs_checked++;\n> +\ttype = odb_read_object_info(repo->objects, oid, NULL);\n> +\tif (type == OBJ_BLOB) {\n> +\t\toidset_insert(&vs->trusted_blobs, oid);\n> +\t\treturn;\n> +\t}\n> +\tif (type >= 0)\n> +\t\tdie(_(\"object %s is a %s, not a %s\"),\n> +\t\t    oid_to_hex(oid), type_name(type), \"blob\");\n\nI'd love not having all these calls to `die()`, as this logic is\nessentially part of a library, and dying inside a library is a bad\nsmell. We do this everywhere else in Git, too, but we're slowly getting\nrid of this as part of our libification efforts.\n\n> +\tif (vs->exclude_promisor_objects &&\n> +\t    is_promisor_object(repo, oid))\n> +\t\treturn;\n> +\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n\nMaybe the object is missing, but it may just as well be corrupt. So this\nshould probably say \"unable to read\".\n\n> +}\n> +\n> +static void *read_tree_object(struct object_database *odb,\n> +\t\t\t      const struct object_id *oid,\n> +\t\t\t      size_t *sizep)\n> +{\n> +\tenum object_type type;\n> +\tvoid *buf = odb_read_object(odb, oid, &type, sizep);\n> +\n> +\tif (buf && type != OBJ_TREE) {\n> +\t\tfree(buf);\n> +\t\tdie(_(\"object %s is a %s, not a %s\"),\n> +\t\t    oid_to_hex(oid), type_name(type), \"tree\");\n> +\t}\n> +\treturn buf;\n> +}\n> +\n> +static void verify_tree(struct repository *repo,\n> +\t\t\tconst struct object_id *new_tree_oid,\n> +\t\t\tconst struct oid_array *base_trees,\n\nI think `base_trees` is a bit of a confusing term, as I think those\nalways are the parent's trees, right? Should we maybe name it\naccordingly?\n\n> +\t\t\tstruct verify_state *vs, int depth)\n> +{\n> +\tstruct tree_desc desc;\n> +\tstruct name_entry entry;\n> +\tstruct work_item *work = NULL;\n> +\tsize_t nr_work = 0, alloc_work = 0;\n> +\tint need_subtree_bases = 0;\n\nNit: this should be a `bool`.\n\n> +\tsize_t i, tree_size;\n> +\tvoid *tree_buf;\n> +\n> +\tif (depth > repo->settings.max_allowed_tree_depth)\n> +\t\tdie(_(\"exceeded maximum allowed tree depth\"));\n> +\n> +\tif (tree_map_get(vs->trees, new_tree_oid) >= TREE_TRUSTED)\n> +\t\treturn;\n> +\n> +\ttree_buf = read_tree_object(repo->objects, new_tree_oid, &tree_size);\n> +\tif (!tree_buf) {\n> +\t\tif (vs->exclude_promisor_objects &&\n> +\t\t    is_promisor_object(repo, new_tree_oid))\n> +\t\t\treturn;\n> +\t\tdie(_(\"bad tree object %s\"),\n> +\t\t    oid_to_hex(new_tree_oid));\n> +\t}\n> +\n> +\tvs->trees_loaded++;\n> +\tinit_tree_desc(&desc, new_tree_oid, tree_buf, tree_size);\n> +\n> +\twhile (tree_entry(&desc, &entry)) {\n> +\t\tif (S_ISGITLINK(entry.mode))\n> +\t\t\tcontinue;\n> +\t\tif (S_ISDIR(entry.mode)) {\n> +\t\t\tif (tree_map_get(vs->trees, &entry.oid) >= TREE_TRUSTED)\n> +\t\t\t\tcontinue;\n> +\t\t\tneed_subtree_bases = 1;\n> +\t\t} else {\n> +\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry.oid))\n> +\t\t\t\tcontinue;\n> +\t\t}\n> +\t\tALLOC_GROW(work, nr_work + 1, alloc_work);\n> +\t\tmemset(&work[nr_work], 0, sizeof(work[nr_work]));\n> +\t\twork[nr_work].entry = entry;\n> +\t\tnr_work++;\n> +\t}\n\nOkay, so first we gather all the tree entries that need verification\nbecaues we don't trust them yet.\n\n> +\n> +\tif (!nr_work) {\n> +\t\tfree(tree_buf);\n> +\t\tgoto done;\n> +\t}\n> +\n> +\tfor (i = 0; base_trees && i < base_trees->nr; i++) {\n> +\t\tconst struct object_id *base_oid = &base_trees->oid[i];\n> +\t\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n> +\t\tstruct tree_desc base_desc;\n> +\t\tstruct name_entry scan_entry;\n> +\t\tsize_t wi = 0, base_size;\n> +\t\tvoid *base_buf;\n> +\n> +\t\tif (expanded && !need_subtree_bases)\n> +\t\t\tcontinue;\n> +\n> +\t\tbase_buf = read_tree_object(repo->objects, base_oid,\n> +\t\t\t\t\t    &base_size);\n> +\t\tif (!base_buf) {\n> +\t\t\tif (vs->exclude_promisor_objects &&\n> +\t\t\t    is_promisor_object(repo, base_oid))\n> +\t\t\t\tcontinue;\n> +\t\t\tdie(_(\"bad tree object %s\"),\n> +\t\t\t    oid_to_hex(base_oid));\n> +\t\t}\n> +\n> +\t\tvs->trees_loaded++;\n> +\t\tinit_tree_desc(&base_desc, base_oid, base_buf, base_size);\n> +\n> +\t\twhile (tree_entry(&base_desc, &scan_entry)) {\n> +\t\t\tif (S_ISGITLINK(scan_entry.mode))\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tif (need_subtree_bases)\n> +\t\t\t\twi = collect_subtree_bases(work, nr_work,\n> +\t\t\t\t\t\t\t   wi, &scan_entry);\n> +\n> +\t\t\tif (!expanded) {\n> +\t\t\t\tif (S_ISDIR(scan_entry.mode))\n> +\t\t\t\t\ttree_map_add(vs->trees,\n> +\t\t\t\t\t\t     &scan_entry.oid,\n> +\t\t\t\t\t\t     TREE_TRUSTED);\n> +\t\t\t\telse\n> +\t\t\t\t\toidset_insert(&vs->trusted_blobs,\n> +\t\t\t\t\t\t      &scan_entry.oid);\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tif (!expanded)\n> +\t\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n> +\n> +\t\tfree(base_buf);\n> +\t}\n\nThen we expand all of our parent trees to exand the trust boundary. I\nthink this part is rather non-obvious though, and could really use some\ncode comments to explain what's even going on here.\n\n> +\tfor (i = 0; i < nr_work; i++) {\n> +\t\tif (S_ISDIR(work[i].entry.mode))\n> +\t\t\tverify_tree(repo, &work[i].entry.oid,\n> +\t\t\t\t    &work[i].parent_trees, vs,\n> +\t\t\t\t    depth + 1);\n\nAnd here we then recurse. The recursion isn't unbounded, but we may go\nup to 2048 layers deep here. We already do this recursively in other\nplaces, but I do wonder whether we can maybe avoid the recursion and\ninstead use a work queue so that we don't bust the stack.\n\n> +\t\telse\n> +\t\t\tverify_blob(repo, &work[i].entry.oid, vs);\n> +\t}\n> +\n> +\tfree(tree_buf);\n> +\n> +done:\n> +\ttree_map_add(vs->trees, new_tree_oid, TREE_EXPANDED);\n> +\tfor (i = 0; i < nr_work; i++)\n> +\t\toid_array_clear(&work[i].parent_trees);\n> +\tfree(work);\n> +}\n> +\n> +static void verify_commit_tree(struct repository *repo,\n> +\t\t\t       struct commit *commit,\n> +\t\t\t       struct verify_state *vs)\n> +{\n> +\tstruct oid_array base_trees = OID_ARRAY_INIT;\n> +\tconst struct object_id *tree_oid;\n> +\tstruct commit_list *p;\n> +\n> +\t/*\n> +\t * Parent trees are trusted: boundary parents are already\n> +\t * connected, and earlier incoming parents were verified\n> +\t * first due to the topological processing order.\n> +\t */\n\nOkay. Parents are trusted because we go from oldest to newest in\ntopological order.\n\n> +\tfor (p = commit->parents; p; p = p->next) {\n> +\t\tparse_commit_or_die(p->item);\n\nThis part here may die in case the repository is a partial clone with\nmissing commits because we enforce that `fetch_if_missing` is disabled.\n\n> +\t\ttree_oid = get_commit_tree_oid(p->item);\n> +\t\tif (!tree_oid)\n> +\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n> +\t\t\t    oid_to_hex(&p->item->object.oid));\n> +\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n> +\t\toid_array_append(&base_trees, tree_oid);\n\nAnd we only mark those trees as TRUSTED, not EXPANDED, as we didn't read\nthe trees themselves.\n\n> +\t}\n> +\n> +\ttree_oid = get_commit_tree_oid(commit);\n> +\tif (!tree_oid)\n> +\t\tdie(_(\"unable to load root tree for commit %s\"),\n> +\t\t    oid_to_hex(&commit->object.oid));\n> +\tverify_tree(repo, tree_oid, &base_trees, vs, 0);\n\nAnd this function here recursively verifies the root tree itself.\n\n> +\toid_array_clear(&base_trees);\n> +}\n> +\n> +void verify_commits_incremental(struct repository *repo,\n> +\t\t\t\tstruct commit_list **commits,\n> +\t\t\t\tint exclude_promisor_objects)\n> +{\n> +\tstruct verify_state vs = { 0 };\n> +\tstruct commit_list *iter;\n> +\tunsigned nr_before;\n> +\n> +\tif (repo->fetch_if_missing)\n> +\t\tBUG(\"verify_commits_incremental must not be called \"\n> +\t\t    \"with fetch_if_missing set\");\n> +\n> +\tvs.trees = kh_init_oid_tree();\n> +\tvs.exclude_promisor_objects = exclude_promisor_objects;\n> +\n> +\t/*\n> +\t * Ancestors must be verified before descendants so that parent\n> +\t * trees can be trusted without re-verification.  Sort explicitly\n> +\t * rather than relying on the caller's ordering.\n> +\t *\n> +\t * sort_in_topological_order() silently drops cycle members,\n> +\t * so explicitly check if the size has changed.\n> +\t */\n> +\tnr_before = commit_list_count(*commits);\n> +\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n> +\tif (commit_list_count(*commits) < nr_before)\n> +\t\tdie(_(\"cycle detected in incoming commit graph\"));\n> +\n> +\t*commits = commit_list_reverse(*commits);\n> +\n> +\tfor (iter = *commits; iter; iter = iter->next)\n> +\t\tverify_commit_tree(repo, iter->item, &vs);\n> +\n> +\tkh_destroy_oid_tree(vs.trees);\n> +\toidset_clear(&vs.trusted_blobs);\n> +\ttrace2_data_intmax(\"connectivity\", repo,\n> +\t\t\t   \"trees_loaded\", vs.trees_loaded);\n> +\ttrace2_data_intmax(\"connectivity\", repo,\n> +\t\t\t   \"blobs_checked\", vs.blobs_checked);\n> +}\n\nThanks!\n\nPatrick\n\n"},{"id":"555104","messageId":"atB49SAaLKS3lfGE@pks.im","threadId":"66326","inReplyTo":"cdf28f1df6cb36b006f367de0e18e23e3aa8d992.1791806877.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-15T06:55:49Z","receivedAt":"2026-10-15T06:55:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 12, 2026 at 12:07:56PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson <krka@spotify.com>\n> \n> Add the transfer.connectivityCheck configuration option, accepting\n> \"full\" (default) and \"incremental\".  When set to \"incremental\",\n> check_connected() passes --verify-trees-incremental to its rev-list\n> subprocess.\n> \n> The rev-list flag is accepted but has no effect yet; the incremental\n> verification will be implemented in the next commit.\n\nI think this step is a tiny bit awkward. My suggestion was rather aiming\nto introduce the configuration option, but initially only accept \"full\"\nfor the current behaviour. The \"--verify-trees-incremental\" flag could\nthen be implemented in the next commit, and the final commit would then\ntie all of this together by introducing the \"incremental\" value.\n\nPatrick\n\n"},{"id":"555121","messageId":"CAL71e4O0vG9SPz9gNP_vjp-1ftSZ9g8RkXRvvpTRaA5+Q1pwsg@mail.gmail.com","threadId":"66326","inReplyTo":"atB42M2GJ3NitXdj@pks.im","subject":"Re: [PATCH v3 1/4] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-15T09:12:50Z","receivedAt":"2026-10-15T09:12:50Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 15 Oct 2026 at 08:55, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> On Mon, Oct 12, 2026 at 12:07:54PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> > From: Kristofer Karlsson <krka@spotify.com>\n> >\n> > Add Documentation/technical/connectivity-check.adoc describing\n> > the connectivity invariant and the full connectivity check.\n> >\n> > Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n> > ---\n> >  .../technical/connectivity-check.adoc         | 110 ++++++++++++++++++\n> >  1 file changed, 110 insertions(+)\n> >  create mode 100644 Documentation/technical/connectivity-check.adoc\n>\n> Shouldn't both the Makefile (`TECH_DOCS`) and Meson (`articles`) be\n> adapted to build this?\n\nAh yes, will fix for next reroll.\n\n>\n> > diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> > new file mode 100644\n> > index 0000000000..abef02debd\n> > --- /dev/null\n> > +++ b/Documentation/technical/connectivity-check.adoc\n> [snip]\n> > +Full connectivity check\n> > +-----------------------\n> > +\n> > +The full connectivity check runs a `rev-list` subprocess, feeding\n> > +the new reference tips via stdin.  A normal invocation is roughly:\n> > +\n> > +    git rev-list --objects --stdin --not --all --quiet\n> > +        --alternate-refs [--exclude-promisor-objects]\n> > +\n> > +When promisor remotes are configured, a fast path based on\n> > +promisor packfiles is attempted first.  If it falls back to\n> > +the `rev-list` check, `--exclude-promisor-objects` is added so\n> > +that the traversal does not trigger on-demand fetches.\n>\n> Would it make sense to document that fast path here, too?\n\nHm perhaps.  Or I can remove it instead, I think it depends on\nhow detailed we want this to be.  I will take a deeper look at\nthis.\n\n> > +The traversal then walks the trees of the boundary commits and\n> > +marks those objects uninteresting.  Trees of incoming commits\n> > +are verified against these marks, stopping at objects already\n>\n> Nit: \"verified against these marks\" reads a tiny bit awkward. How about\n> \"Trees of incoming commits are then walked, stopping at objects already\n> marked as uninteresting\".\n\nAgreed, I think that would be easier to read. Will fix.\n\nThanks,\nKristofer\n\n"},{"id":"555122","messageId":"CAL71e4OJGW5uLax2nAZ=XPgbDwXEe1vE-6pf=L_Ka9MQo6NrUg@mail.gmail.com","threadId":"66326","inReplyTo":"atB468yrVqsp9Eg6@pks.im","subject":"Re: [PATCH v3 2/4] p5413: add perf tests for connectivity check","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-15T09:14:48Z","receivedAt":"2026-10-15T09:14:48Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 15 Oct 2026 at 08:55, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> All of the benchmarks will need to be marked with the PERL prerequisite.\n\nAh good catch, will add.\n\n>\n> > @@ -0,0 +1,44 @@\n> > +#!/usr/bin/perl\n> > +#\n> > +# Generate a fast-import stream for p5412 connectivity check benchmarks.\n> > +#\n> > +# Usage: generate-repo-p5412-connectivity-check.perl\n> > +#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n>\n> These still say p5412, but it's p5413.\n\nOops, I fixed the file collision but forgot to go back and\nclean up text references, will fix.\n\nThanks,\nKristofer\n\n"},{"id":"555126","messageId":"CAL71e4Meuv1KXTX9HeQzmyVdmtYDCrd4M6VNcRRt5HZ-DKzbQA@mail.gmail.com","threadId":"66326","inReplyTo":"atB48HLgZr3ssOfx@pks.im","subject":"Re: [PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-15T09:51:46Z","receivedAt":"2026-10-15T09:51:46Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 15 Oct 2026 at 08:55, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> On Mon, Oct 12, 2026 at 12:07:57PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> > From: Kristofer Karlsson <krka@spotify.com>\n> [snip]\n> > Scaling tree size (10 incoming commits, 10 files/commit):\n> >\n> >     files    full  incr.  full/incr\n> >       5K    0.01s  0.01s    1.0x\n> >      50K    0.04s  0.01s    4.0x\n> >     200K    0.15s  0.01s   15.0x\n> >     800K    0.61s  0.03s   20.3x\n> >\n> > The full mode must walk the trees of boundary commits, which\n> > grows with overall tree size.  Incremental still scans the\n> > root trees, but avoids descending into unchanged subtrees,\n> > so it grows much more slowly with repository size.\n>\n> Okay, so this is getting better the larger the trees.\n>\n> > Scaling incoming commit count (200K files, 10 files/commit):\n> >\n> >     commits    full  incr.  full/incr\n> >           1    0.14s  0.01s   14.0x\n> >          10    0.15s  0.01s   15.0x\n> >         100    0.19s  0.05s    3.8x\n> >         500    0.32s  0.27s    1.2x\n> >        3000    1.23s  1.52s    0.8x\n> >        5000    1.88s  2.43s    0.8x\n> >       10000    3.72s  5.36s    0.7x\n> >\n> > With many commits the per-commit overhead of scanning both\n> > the new and parent root trees accumulates and incremental\n> > becomes slower.  Breakeven is between 500 and 3000 commits\n> > and the ratio stabilizes near 0.7x for this fixture.\n>\n> And worse the more commits we have. So the new mode is only really\n> applicable in scenarios where you know what your typical workload looks\n> like. The big question is of course whether we can have the best of both\n> worlds, where Git knows to switch between the two modes once it crosses\n> the threshold.\n\nI think this is where it gets tricky to reason about.  The win\nfrom from larger trees pays off without bound -- the larger\ntrees you have, the larger the win.\n\nScaling with commits on the other hand we have ceiling of how\nmuch it can regress.  I am thinking that could be reasonable\nthat anyone who enables the config flag for this mode could\nsimply accept that it would likely speed things up in most\ncases but in other cases they could see a ~1.5x regression.\n\nI am hoping that this tradeoff would then be acceptable,\notherwise we would indeed need to add some way of switching,\nthough I don't think there's a single good indicator value\nfor when it's better to switch.\n\nI could add a new config flag like\ntransfer.connectivityCheckIncrementalMaxCommits to control it\nperhaps.\n\n>\n> > diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> > index abef02debd..a1d2e9709c 100644\n> > --- a/Documentation/technical/connectivity-check.adoc\n> > +++ b/Documentation/technical/connectivity-check.adoc\n> > +Algorithm\n> > +~~~~~~~~~\n> > +\n> > +At a high level:\n> > +\n> > +    verify(commits):\n> > +        sort topologically (ancestors first)\n> > +        for each commit:\n> > +            mark parent root trees as trusted\n> > +            verify_tree(commit.tree, parent root trees)\n>\n> So based on this, `verify()` already knows which commits it will be\n> checking for connectivity. So wouldn't it be possible to fall back to\n> the original algortihm here in case we see more than N commits?\n\nYes, but which N? I think it would depend on the shape of the\nrepository too -- size of object tree, merge-heavy vs linear.\n\n> > +enum tree_state {\n> > +     TREE_UNTRUSTED = 0,\n> > +     TREE_TRUSTED   = 1,\n> > +     TREE_EXPANDED  = 2,\n> > +};\n> > +\n> > +KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n>\n> The values are `enum tree_state`s, right? Why don't we use that type\n> here?\n\nI tried that first, but the natural size of enum is larger than unsigned char,\nso this reduces the memory usage.\n\n> > +struct verify_state {\n> > +     kh_oid_tree_t *trees;\n> > +     struct oidset trusted_blobs;\n> > +     int trees_loaded;\n> > +     int blobs_checked;\n>\n> These probably should be `size_t`.\n\nSure, I can change that.  Or intmax_t since we'll need to cast\nto that later anyway for the trace2 output?\n\n> > +     if (type >= 0)\n> > +             die(_(\"object %s is a %s, not a %s\"),\n> > +                 oid_to_hex(oid), type_name(type), \"blob\");\n>\n> I'd love not having all these calls to `die()`, as this logic is\n> essentially part of a library, and dying inside a library is a bad\n> smell. We do this everywhere else in Git, too, but we're slowly getting\n> rid of this as part of our libification efforts.\n\nI think this would be a bigger refactor in general --\nthe existing connectivity check also dies as the way to\nreport problems.  This works since it's running as a subprocess.\n\nIf this is something we want to change, I think it would be\ngood to do that refactoring separately and make the connectivity\ncheck subprocess only die() at the very edge, and making all of\nthe internal library code support not dying.  My first prototype\ntried to do this but the amount of changes to internal code got\nout of hand, so this version was the simplest I could make it.\n\nBut I'm happy to take a step back and revisit that approach if\nthis is a blocker.\n\n>\n> > +     if (vs->exclude_promisor_objects &&\n> > +         is_promisor_object(repo, oid))\n> > +             return;\n> > +     die(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n>\n> Maybe the object is missing, but it may just as well be corrupt. So this\n> should probably say \"unable to read\".\n\nHmm yes, perhaps.  I will need to dig into that more.\n\n>\n> > +}\n> > +\n> > +static void *read_tree_object(struct object_database *odb,\n> > +                           const struct object_id *oid,\n> > +                           size_t *sizep)\n> > +{\n> > +     enum object_type type;\n> > +     void *buf = odb_read_object(odb, oid, &type, sizep);\n> > +\n> > +     if (buf && type != OBJ_TREE) {\n> > +             free(buf);\n> > +             die(_(\"object %s is a %s, not a %s\"),\n> > +                 oid_to_hex(oid), type_name(type), \"tree\");\n> > +     }\n> > +     return buf;\n> > +}\n> > +\n> > +static void verify_tree(struct repository *repo,\n> > +                     const struct object_id *new_tree_oid,\n> > +                     const struct oid_array *base_trees,\n>\n> I think `base_trees` is a bit of a confusing term, as I think those\n> always are the parent's trees, right? Should we maybe name it\n> accordingly?\n\nTechnically you could use any base, as long as it's trusted.\nThat said, the trees here _do_ come from parent commits\n(not necessarily direct from the parent commit, it goes\ndeeper down into the object tree).\n\nI could rename it, though I think it would leak semantics that\nverify_tree() does not require.\n\n\n> > +     int need_subtree_bases = 0;\n>\n> Nit: this should be a `bool`.\n\nTrue, will fix.  I think I got confused because the code base\nuse both styles in different places.\n\n> > +\n> > +     if (!nr_work) {\n> > +             free(tree_buf);\n> > +             goto done;\n> > +     }\n> > +\n> > +     for (i = 0; base_trees && i < base_trees->nr; i++) {\n> > +             const struct object_id *base_oid = &base_trees->oid[i];\n> > +             int expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n> > +             struct tree_desc base_desc;\n> > +             struct name_entry scan_entry;\n> > +             size_t wi = 0, base_size;\n> > +             void *base_buf;\n> > +\n> > +             if (expanded && !need_subtree_bases)\n> > +                     continue;\n> > +\n> > +             base_buf = read_tree_object(repo->objects, base_oid,\n> > +                                         &base_size);\n> > +             if (!base_buf) {\n> > +                     if (vs->exclude_promisor_objects &&\n> > +                         is_promisor_object(repo, base_oid))\n> > +                             continue;\n> > +                     die(_(\"bad tree object %s\"),\n> > +                         oid_to_hex(base_oid));\n> > +             }\n> > +\n> > +             vs->trees_loaded++;\n> > +             init_tree_desc(&base_desc, base_oid, base_buf, base_size);\n> > +\n> > +             while (tree_entry(&base_desc, &scan_entry)) {\n> > +                     if (S_ISGITLINK(scan_entry.mode))\n> > +                             continue;\n> > +\n> > +                     if (need_subtree_bases)\n> > +                             wi = collect_subtree_bases(work, nr_work,\n> > +                                                        wi, &scan_entry);\n> > +\n> > +                     if (!expanded) {\n> > +                             if (S_ISDIR(scan_entry.mode))\n> > +                                     tree_map_add(vs->trees,\n> > +                                                  &scan_entry.oid,\n> > +                                                  TREE_TRUSTED);\n> > +                             else\n> > +                                     oidset_insert(&vs->trusted_blobs,\n> > +                                                   &scan_entry.oid);\n> > +                     }\n> > +             }\n> > +\n> > +             if (!expanded)\n> > +                     tree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n> > +\n> > +             free(base_buf);\n> > +     }\n>\n> Then we expand all of our parent trees to exand the trust boundary. I\n> think this part is rather non-obvious though, and could really use some\n> code comments to explain what's even going on here.\n\nYes, this is the trickiest part of the code.  I tried to add\ncomments here in a local draft but it became messy and did not\nhelp enough.  I will make a new attempt at it though.\n\n>\n> > +     for (i = 0; i < nr_work; i++) {\n> > +             if (S_ISDIR(work[i].entry.mode))\n> > +                     verify_tree(repo, &work[i].entry.oid,\n> > +                                 &work[i].parent_trees, vs,\n> > +                                 depth + 1);\n>\n> And here we then recurse. The recursion isn't unbounded, but we may go\n> up to 2048 layers deep here. We already do this recursively in other\n> places, but I do wonder whether we can maybe avoid the recursion and\n> instead use a work queue so that we don't bust the stack.\n\nHm, initially I was thinking it would get very messy with a\nmanual stack or queue here, but maybe it would not be so bad.\nI will investigate if I can rework it nicely, but otherwise I\nthink this version is reasonable, and aligns with other\ntree-walking code.\n\n> > +     for (p = commit->parents; p; p = p->next) {\n> > +             parse_commit_or_die(p->item);\n>\n> This part here may die in case the repository is a partial clone with\n> missing commits because we enforce that `fetch_if_missing` is disabled.\n\nI think it would be unreachable, since we first collect the\ncommits via the regular boundary walk.  But now I started to\ndoubt myself, I will try to re-validate this with a test that\nputs the boundary exactly at such a problematic commit.\n\nThanks for the detailed review!\nKristofer\n\n"},{"id":"555127","messageId":"CAL71e4O6jccjXbnNjJfZjO9d_t9kVi2PKhD5Tdd_6aUnfYQ4rw@mail.gmail.com","threadId":"66326","inReplyTo":"atB49SAaLKS3lfGE@pks.im","subject":"Re: [PATCH v3 3/4] connected: add transfer.connectivityCheck config and rev-list flag","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-15T09:53:16Z","receivedAt":"2026-10-15T09:53:16Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 15 Oct 2026 at 08:55, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> > The rev-list flag is accepted but has no effect yet; the incremental\n> > verification will be implemented in the next commit.\n>\n> I think this step is a tiny bit awkward. My suggestion was rather aiming\n> to introduce the configuration option, but initially only accept \"full\"\n> for the current behaviour. The \"--verify-trees-incremental\" flag could\n> then be implemented in the next commit, and the final commit would then\n> tie all of this together by introducing the \"incremental\" value.\n\nHm yes, I can try to do that.\n\nThanks,\nKristofer\n\n"},{"id":"555138","messageId":"atC-GVBnQrfr3cQ0@pks.im","threadId":"66326","inReplyTo":"CAL71e4Meuv1KXTX9HeQzmyVdmtYDCrd4M6VNcRRt5HZ-DKzbQA@mail.gmail.com","subject":"Re: [PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-15T11:50:49Z","receivedAt":"2026-10-15T11:50:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Oct 15, 2026 at 11:51:46AM +0200, Kristofer Karlsson wrote:\n> On Thu, 15 Oct 2026 at 08:55, Patrick Steinhardt <ps@pks.im> wrote:\n> > On Mon, Oct 12, 2026 at 12:07:57PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> > > From: Kristofer Karlsson <krka@spotify.com>\n[snip]\n> > > Scaling incoming commit count (200K files, 10 files/commit):\n> > >\n> > >     commits    full  incr.  full/incr\n> > >           1    0.14s  0.01s   14.0x\n> > >          10    0.15s  0.01s   15.0x\n> > >         100    0.19s  0.05s    3.8x\n> > >         500    0.32s  0.27s    1.2x\n> > >        3000    1.23s  1.52s    0.8x\n> > >        5000    1.88s  2.43s    0.8x\n> > >       10000    3.72s  5.36s    0.7x\n> > >\n> > > With many commits the per-commit overhead of scanning both\n> > > the new and parent root trees accumulates and incremental\n> > > becomes slower.  Breakeven is between 500 and 3000 commits\n> > > and the ratio stabilizes near 0.7x for this fixture.\n> >\n> > And worse the more commits we have. So the new mode is only really\n> > applicable in scenarios where you know what your typical workload looks\n> > like. The big question is of course whether we can have the best of both\n> > worlds, where Git knows to switch between the two modes once it crosses\n> > the threshold.\n> \n> I think this is where it gets tricky to reason about.  The win\n> from from larger trees pays off without bound -- the larger\n> trees you have, the larger the win.\n> \n> Scaling with commits on the other hand we have ceiling of how\n> much it can regress.  I am thinking that could be reasonable\n> that anyone who enables the config flag for this mode could\n> simply accept that it would likely speed things up in most\n> cases but in other cases they could see a ~1.5x regression.\n> \n> I am hoping that this tradeoff would then be acceptable,\n> otherwise we would indeed need to add some way of switching,\n> though I don't think there's a single good indicator value\n> for when it's better to switch.\n> \n> I could add a new config flag like\n> transfer.connectivityCheckIncrementalMaxCommits to control it\n> perhaps.\n\nYeah, making this configurable may be reasonable. The way I understand\nit, the new connectivity check is most likely to be usable for smaller,\nincremental pushes. And based on the above table you have, we see that\nup to 1k commits the new algorithm is faster. Sure, it's an artificial\nbenchmark, but maybe this is a good enough cutoff for now?\n\n> > > diff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\n> > > index abef02debd..a1d2e9709c 100644\n> > > --- a/Documentation/technical/connectivity-check.adoc\n> > > +++ b/Documentation/technical/connectivity-check.adoc\n[snip]\n> > > +struct verify_state {\n> > > +     kh_oid_tree_t *trees;\n> > > +     struct oidset trusted_blobs;\n> > > +     int trees_loaded;\n> > > +     int blobs_checked;\n> >\n> > These probably should be `size_t`.\n> \n> Sure, I can change that.  Or intmax_t since we'll need to cast\n> to that later anyway for the trace2 output?\n\nHm, okay. Not a fan of using signed integers as counters, but so be it.\n\n> > > +     if (type >= 0)\n> > > +             die(_(\"object %s is a %s, not a %s\"),\n> > > +                 oid_to_hex(oid), type_name(type), \"blob\");\n> >\n> > I'd love not having all these calls to `die()`, as this logic is\n> > essentially part of a library, and dying inside a library is a bad\n> > smell. We do this everywhere else in Git, too, but we're slowly getting\n> > rid of this as part of our libification efforts.\n> \n> I think this would be a bigger refactor in general --\n> the existing connectivity check also dies as the way to\n> report problems.  This works since it's running as a subprocess.\n> \n> If this is something we want to change, I think it would be\n> good to do that refactoring separately and make the connectivity\n> check subprocess only die() at the very edge, and making all of\n> the internal library code support not dying.  My first prototype\n> tried to do this but the amount of changes to internal code got\n> out of hand, so this version was the simplest I could make it.\n> \n> But I'm happy to take a step back and revisit that approach if\n> this is a blocker.\n\nI mostly aim at not making _new_ library code die anymore. In any case,\nI don't think it's a blocker, but I nowadays consider this to be a code\nsmell.\n\nIt would be great if we eventually deny-listed `die()` completely for\nanything compiled into libgit.a.\n\nPatrick\n\n"},{"id":"555233","messageId":"CAL71e4NA3SH8V_haxJFeYCBmKk8+96j7ckxLDZA7+XMNcVfdKg@mail.gmail.com","threadId":"66326","inReplyTo":"atC-GVBnQrfr3cQ0@pks.im","subject":"Re: [PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-16T11:50:21Z","receivedAt":"2026-10-16T11:50:21Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 15 Oct 2026 at 13:50, Patrick Steinhardt <ps@pks.im> wrote:\n> >\n> > I think this is where it gets tricky to reason about.  The win\n> > from from larger trees pays off without bound -- the larger\n> > trees you have, the larger the win.\n> >\n> > Scaling with commits on the other hand we have ceiling of how\n> > much it can regress.  I am thinking that could be reasonable\n> > that anyone who enables the config flag for this mode could\n> > simply accept that it would likely speed things up in most\n> > cases but in other cases they could see a ~1.5x regression.\n> >\n> > I am hoping that this tradeoff would then be acceptable,\n> > otherwise we would indeed need to add some way of switching,\n> > though I don't think there's a single good indicator value\n> > for when it's better to switch.\n> >\n> > I could add a new config flag like\n> > transfer.connectivityCheckIncrementalMaxCommits to control it\n> > perhaps.\n>\n> Yeah, making this configurable may be reasonable. The way I understand\n> it, the new connectivity check is most likely to be usable for smaller,\n> incremental pushes. And based on the above table you have, we see that\n> up to 1k commits the new algorithm is faster. Sure, it's an artificial\n> benchmark, but maybe this is a good enough cutoff for now?\n\nI looked into this a bit more and I have two main notes:\n\nFirst of all, I benchmarked multiple repos, both with small\nobject trees (such as git.git), medium (linux.git) and some\nreally big ones (proprietary).\n\n    repo          files   commits   full    incr.  full/incr\n    internal A    552K       1000   1.69s   0.66s    2.6x\n    internal B    331K       1000   1.48s   0.57s    2.6x\n    linux          95K        609   0.34s   0.12s    2.8x\n    linux          95K      13965   1.86s   1.90s    1.0x\n    git.git        4.9K       356   0.09s   0.13s    0.7x\n    git.git        4.9K       976   0.23s   0.31s    0.7x\n\nSo for large and medium repos the cutoff of 1000 is\nconservative. Incremental mode is faster but the full mode\nis also good enough here.\n\nFor small repos like git, the incremental mode is slightly\nslower, but since the number of files is so small, both modes\nare still fast enough that it does not matter.\n\n(I could not get an exact 1000 commit test synthetically by\nlooking back in the history for all repos, since it depends\na bit on the shape of the graph of commits.)\n\nThus, I think having a cutoff of 1000 is indeed a valid\nstrategy here.\n\nSecond of all (and with the previous in mind), I think I would\nwant to change the configuration into three modes:\n\n  - full (default): work as it does today.\n  - incremental:    only use the incremental mode\n  - auto:           decide on full or incremental depending\n                    number of commits to process.\n\nThis would make things safe to roll out and start gaining more\nreal life confidence before potentially changing the default\nto auto instead.\n\nWe could also make the cutoff configurable, but I am not sure\nthat is needed in practice.\n\n> > > These probably should be `size_t`.\n> >\n> > Sure, I can change that.  Or intmax_t since we'll need to cast\n> > to that later anyway for the trace2 output?\n>\n> Hm, okay. Not a fan of using signed integers as counters, but so be it.\n\nNo worries, I will switch to size_t instead.\n\n>\n> > > > +     if (type >= 0)\n> > > > +             die(_(\"object %s is a %s, not a %s\"),\n> > > > +                 oid_to_hex(oid), type_name(type), \"blob\");\n> > >\n> > > I'd love not having all these calls to `die()`, as this logic is\n> > > essentially part of a library, and dying inside a library is a bad\n> > > smell. We do this everywhere else in Git, too, but we're slowly getting\n> > > rid of this as part of our libification efforts.\n> >\n> > I think this would be a bigger refactor in general --\n> > the existing connectivity check also dies as the way to\n> > report problems.  This works since it's running as a subprocess.\n> >\n> > If this is something we want to change, I think it would be\n> > good to do that refactoring separately and make the connectivity\n> > check subprocess only die() at the very edge, and making all of\n> > the internal library code support not dying.  My first prototype\n> > tried to do this but the amount of changes to internal code got\n> > out of hand, so this version was the simplest I could make it.\n> >\n> > But I'm happy to take a step back and revisit that approach if\n> > this is a blocker.\n>\n> I mostly aim at not making _new_ library code die anymore. In any case,\n> I don't think it's a blocker, but I nowadays consider this to be a code\n> smell.\n\nI have spent some more time thinking about this and I have\nchanged my mind -- I think I was too stuck in the idea of\nall-or-nothing here, but I agree I can make this work in a\nmixed way -- not introducing any new die() but accept the\ncases where it already dies.\n\nI will include this in the reroll (likely next week, don't\nwant to rush things).\n\nThanks,\nKristofer\n\n"},{"id":"555393","messageId":"atXHKkGztmg8QFYt@pks.im","threadId":"66326","inReplyTo":"CAL71e4NA3SH8V_haxJFeYCBmKk8+96j7ckxLDZA7+XMNcVfdKg@mail.gmail.com","subject":"Re: [PATCH v3 4/4] connected: add incremental connectivity check via rev-list","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-19T07:30:50Z","receivedAt":"2026-10-19T07:30:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Oct 16, 2026 at 01:50:21PM +0200, Kristofer Karlsson wrote:\n> On Thu, 15 Oct 2026 at 13:50, Patrick Steinhardt <ps@pks.im> wrote:\n> > >\n> > > I think this is where it gets tricky to reason about.  The win\n> > > from from larger trees pays off without bound -- the larger\n> > > trees you have, the larger the win.\n> > >\n> > > Scaling with commits on the other hand we have ceiling of how\n> > > much it can regress.  I am thinking that could be reasonable\n> > > that anyone who enables the config flag for this mode could\n> > > simply accept that it would likely speed things up in most\n> > > cases but in other cases they could see a ~1.5x regression.\n> > >\n> > > I am hoping that this tradeoff would then be acceptable,\n> > > otherwise we would indeed need to add some way of switching,\n> > > though I don't think there's a single good indicator value\n> > > for when it's better to switch.\n> > >\n> > > I could add a new config flag like\n> > > transfer.connectivityCheckIncrementalMaxCommits to control it\n> > > perhaps.\n> >\n> > Yeah, making this configurable may be reasonable. The way I understand\n> > it, the new connectivity check is most likely to be usable for smaller,\n> > incremental pushes. And based on the above table you have, we see that\n> > up to 1k commits the new algorithm is faster. Sure, it's an artificial\n> > benchmark, but maybe this is a good enough cutoff for now?\n> \n> I looked into this a bit more and I have two main notes:\n> \n> First of all, I benchmarked multiple repos, both with small\n> object trees (such as git.git), medium (linux.git) and some\n> really big ones (proprietary).\n> \n>     repo          files   commits   full    incr.  full/incr\n>     internal A    552K       1000   1.69s   0.66s    2.6x\n>     internal B    331K       1000   1.48s   0.57s    2.6x\n>     linux          95K        609   0.34s   0.12s    2.8x\n>     linux          95K      13965   1.86s   1.90s    1.0x\n>     git.git        4.9K       356   0.09s   0.13s    0.7x\n>     git.git        4.9K       976   0.23s   0.31s    0.7x\n> \n> So for large and medium repos the cutoff of 1000 is\n> conservative. Incremental mode is faster but the full mode\n> is also good enough here.\n> \n> For small repos like git, the incremental mode is slightly\n> slower, but since the number of files is so small, both modes\n> are still fast enough that it does not matter.\n> \n> (I could not get an exact 1000 commit test synthetically by\n> looking back in the history for all repos, since it depends\n> a bit on the shape of the graph of commits.)\n> \n> Thus, I think having a cutoff of 1000 is indeed a valid\n> strategy here.\n\nYeah, thanks for evaluating this. Heuristics are of course going to go\noff rails at times, but I agree that 1000 seems like a good enough first\niteration.\n\n> Second of all (and with the previous in mind), I think I would\n> want to change the configuration into three modes:\n> \n>   - full (default): work as it does today.\n>   - incremental:    only use the incremental mode\n>   - auto:           decide on full or incremental depending\n>                     number of commits to process.\n> \n> This would make things safe to roll out and start gaining more\n> real life confidence before potentially changing the default\n> to auto instead.\n> \n> We could also make the cutoff configurable, but I am not sure\n> that is needed in practice.\n\nHmm. As the heuristic can easily go wrong I think it would be a good\nidea to make it configurable. So I'd propose `auto[=<cutoff>]`. If left\nout we default to 1000, otherwise we pick whatever the user has\nspecified.\n\n[snip]\n> > > > > +     if (type >= 0)\n> > > > > +             die(_(\"object %s is a %s, not a %s\"),\n> > > > > +                 oid_to_hex(oid), type_name(type), \"blob\");\n> > > >\n> > > > I'd love not having all these calls to `die()`, as this logic is\n> > > > essentially part of a library, and dying inside a library is a bad\n> > > > smell. We do this everywhere else in Git, too, but we're slowly getting\n> > > > rid of this as part of our libification efforts.\n> > >\n> > > I think this would be a bigger refactor in general --\n> > > the existing connectivity check also dies as the way to\n> > > report problems.  This works since it's running as a subprocess.\n> > >\n> > > If this is something we want to change, I think it would be\n> > > good to do that refactoring separately and make the connectivity\n> > > check subprocess only die() at the very edge, and making all of\n> > > the internal library code support not dying.  My first prototype\n> > > tried to do this but the amount of changes to internal code got\n> > > out of hand, so this version was the simplest I could make it.\n> > >\n> > > But I'm happy to take a step back and revisit that approach if\n> > > this is a blocker.\n> >\n> > I mostly aim at not making _new_ library code die anymore. In any case,\n> > I don't think it's a blocker, but I nowadays consider this to be a code\n> > smell.\n> \n> I have spent some more time thinking about this and I have\n> changed my mind -- I think I was too stuck in the idea of\n> all-or-nothing here, but I agree I can make this work in a\n> mixed way -- not introducing any new die() but accept the\n> cases where it already dies.\n> \n> I will include this in the reroll (likely next week, don't\n> want to rush things).\n\nThanks, I appreciate it when folks take their time :)\n\nPatrick\n\n"},{"id":"555715","messageId":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH v4 0/6] connected: add incremental connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:18Z","receivedAt":"2026-10-21T11:22:18Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"This series adds an incremental mode for the connectivity check, gated\nbehind transfer.connectivityCheck=incremental or \"auto\" (no expected changes\nunless you opt in).\n\nThe intent is to solve the problem of the connectivity check slowing down as\nthe number of reachable objects from the boundary grows.\n\nIt relates to the RFC I sent out earlier:\n\n[RFC] check_connected: toward incoming-proportional cost\nhttps://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/\n\n\n\nDesign\n======\n\nThe verifier runs inside the same rev-list subprocess that check_connected()\nalready spawns, triggered by a new internal option --connectivity-check=.\nAfter get_revision() collects the incoming commits, the verifier processes\nthem in topological order (ancestors before descendants).\n\nThe idea is to keep a set of trusted objects, shared across the incoming\ncommits, that grows over time. We visit the new/untrusted commit trees and\ndo a comparison walk over the trees of their parents. Entries discovered on\nthe trusted parent side are remembered as trusted, which lets later\nverification skip matching objects and avoid descending into unchanged\nsubtrees.\n\nMore algorithmic details are in\nDocumentation/technical/connectivity-check.adoc.\n\n\nBenchmarks\n==========\n\nI'll just mention a short summary here, to avoid repeating what's already in\nthe commit messages (5/6 and 6/6). The incremental mode is faster than the\nfull mode when there are few commits to verify and when the active object\ntree is large. In the happy case, the work tracks the changed paths and\ntheir comparison trees rather than the full reachable object closure, which\nsubstantially reduces the dependence on total repository size. I've seen\nspeedups of up to about 18x in the synthetic perf tests.\n\nRunning against a large real-world repo (3.4 GB boundary closure), the\nnumbers are more dramatic. These were measured on v1 and not re-run since;\nthe real-repo numbers for v4 are in 6/6. All timings use rev-list directly\nwith --not HEADN, isolating the tree verification cost from the\nboundary-finding cost:\n\ncommits    full  incr.  speedup   full RSS  incr. RSS\n      1    1.9s  0.01s    190x      3.4 GB     14 MB\n     10    1.9s  0.04s     48x      3.4 GB    125 MB\n    100    1.9s  0.37s      5x      3.4 GB    1.1 GB\n\n\nThe full mode takes ~1.9s regardless of commit count because it is dominated\nby walking the boundary closure. Incremental scales with the number of\nincoming commits and the paths they touch. Memory follows the same pattern:\nincremental uses a fraction of the full mode's RSS for small pushes,\nconverging only when many commits are verified.\n\nThere are also regression cases in the synthetic fixtures. With long\nincoming histories, the extra parent-tree scans accumulate; in the synthetic\nfixture incremental is about 1.5x slower at 10000 commits. Per-commit\nchanges have less impact than expected: even when every directory is\ntouched, incremental remains competitive.\n\nIn the cases I have tested the regression has remained modest, and it shows\nup with long incoming histories, while the target case (small pushes to\nlarge repositories) sees the largest speedup. That is what the \"auto\" mode\nis for: it uses the incremental check for small numbers of incoming commits\nand the full check otherwise, with a cutoff that can be configured.\n\nDeepening fetches currently fall back to the full check because the full\ncheck omits --not --all for deepening -- there is no existing-reference\nboundary at which the walk can stop. An incremental approach is possible\nhere too -- using the old shallow roots as the trusted boundary and walking\nthe deepened ancestry forward -- but that is a separate change and left for\nfuture work. Deepening is also less common than regular fetch and\nreceive-pack, where the speedup matters most.\n\n\nTest coverage\n=============\n\nMost correctness cases in t5413-connectivity-check.sh are run in both full\nand incremental modes to check semantic equivalence. Selected cases\nadditionally assert trace2 tree/blob counts for the incremental mode, to\nverify that unchanged portions of the object graph are actually skipped. The\ntests cover:\n\n * Corruption detection: missing blobs, missing trees, type mismatches,\n   malformed trees (unparseable, mid-tree corruption), and corrupt blobs\n   (incremental only, since the full check does not read blobs)\n * Tree optimization: trace2 assertions confirm unchanged subtrees are\n   skipped, subtree moves, merge parent boundaries\n * Root commits (no parents -- verifies full tree closure)\n * Partial clones: missing promised blobs, missing promised trees, missing\n   promised parent commits, verification of local commits\n * Replacement objects (with and without GIT_NO_REPLACE_OBJECTS)\n * Shallow boundaries\n * Deepening fetches (falls back to full check)\n * Algorithm selection: clones and pushes into an empty repository use the\n   full check, \"auto\" and its cutoff, and unknown config values\n * Tree depth: the incremental check honors core.maxTreeDepth (t6700)\n * Integration: real push, fetch, and clone\n\nMost tests call git rev-list directly with the appropriate flags;\nintegration tests exercise the full check_connected() path through push,\nfetch, and clone.\n\n\nAlternatives considered\n=======================\n\nMy first prototype ran the verifier in-process inside connected.c. This\nrequired a second rev-list subprocess just for boundary finding, _nofetch\nvariants of several object-reading functions to prevent lazy fetches in\npartial clones, explicit shallow-file plumbing, and careful avoidance of\ndie() in all code paths reachable from the verifier. The result worked but\nwas fragile and touched many files.\n\nMoving the verifier into the rev-list subprocess eliminated all of those\nproblems: in partial clones the existing --exclude-promisor-objects handling\nalready disables lazy fetching, a die() in existing helpers only ends the\nsubprocess, shallow and replacement semantics are established before the\nverifier runs, and error routing comes for free via stderr.\n\nSo while I liked the idea of doing the check in-process, making that work\nended up being a lot more complex.\n\n\nNext steps\n==========\n\nThis series only addresses tree verification; the other significant cost is\nfinding the commit boundary, especially for repos with many refs. I already\nhave some prototypes for optimizing that too, and if this ends up landing,\nthat would be something I would start polishing up.\n\n\nChanges since v3\n================\n\n * Split, along the lines Patrick suggested, into three steps: the config\n   option and the internal --connectivity-check= rev-list option with only\n   \"full\" (4/6), the incremental check (5/6), and the \"auto\" mode (6/6),\n   since choosing between the two is subtle enough to review on its own.\n\n * The internal rev-list option --verify-trees-incremental is replaced by\n   --connectivity-check=, which check_connected() now always passes.\n   rev-list dies on an unknown mode, since only check_connected() passes it.\n\n * An unknown transfer.connectivityCheck value now falls back to full with a\n   warning instead of dying, so config written for a newer Git does not\n   break fetches.\n\n * \"auto\" takes an optional cutoff, \"auto=\", as Patrick suggested,\n   defaulting to 1000.\n\n * Clones and pushes into an empty repository use the full check even with\n   incremental or auto, since there is no boundary to compare against (see\n   5/6). check_connected() now decides the deepening fallback before reading\n   the config.\n\n * New patch 3/6 drops the dependency on the_repository in connected.c, as\n   Junio suggested. The repository is passed to check_connected() as a\n   parameter rather than in check_connected_options, like other functions\n   that take a repository.\n\n * Use parse_commit_gently() instead of dying on a parent that cannot be\n   parsed, which Patrick pointed out could happen in a partial clone with\n   missing commits. As in the revision walk, a missing parent that is\n   promised is skipped; added a test for that. More generally, the verifier\n   adds no new die() calls: it returns errors, and rev-list dies once when\n   the check fails.\n\n * An object that cannot be read is now reported as \"missing\" only when it\n   does not exist, and as \"unable to read\" otherwise, e.g. for a corrupt\n   blob.\n\n * Renamed variables to parent_trees / trusted_bases to make the trust\n   direction clearer.\n\n * size_t for counts and sizes.\n\n * verify_tree() still recurses, bounded by core.maxTreeDepth like the other\n   tree walkers. Its frame was made smaller by moving the tree descriptor\n   scratch space into the shared state: with gcc -O2 its frame is 272 bytes,\n   against 240 for process_tree(), so at the default depth of 2048 it needs\n   about the same stack as \"rev-list --objects\". The listing of untrusted\n   entries and the scan of the base trees are split out into helpers,\n   without changing the logic. New t6700 tests check that the incremental\n   check (given a boundary, so that it is not replaced by the full check)\n   honors core.maxTreeDepth and fails gracefully at the default limit.\n   \n   I tried a work queue as suggested, but it got messy, and it would be\n   stricter than the code we already rely on: the full check's\n   process_tree() recurses over trees in the same way, and be20128bfa (add\n   core.maxTreeDepth) settled on bounding tree recursion rather than\n   removing it. If we want an iterative walk, I think it belongs in a\n   separate series that also covers process_tree().\n\n * Also addressed from v3 review: the technical doc is listed in the\n   Makefile and in meson.build, the perf tests require PERL, the leftover\n   p5412 is now p5413, \"verified against these marks\" is reworded, int flags\n   are bool, the tree state map's byte-sized values are explained in a\n   comment, and the parent tree expansion in tree-verify.c has comments\n   explaining it.\n\n * Documentation: reworded the technical doc, including a shorter\n   description of the promisor fast path; transfer.connectivityCheck\n   documents the fallbacks to full; and the rev-list option description is\n   trimmed to an internal-use note.\n\n * The --connectivity-check parsing in rev-list moved into a small helper,\n   with the mode and cutoff in a struct local to cmd_rev_list() instead of\n   globals.\n\n * Tests are faster and simpler: the auto cutoff is tested with \"auto=\"\n   instead of building long histories, and the mode selection tests share\n   two small helpers.\n\n * p5413 passes the mode explicitly, and the tables in 5/6 were re-measured;\n   they now report the best of 3 runs, which is what the perf framework\n   reports, rather than \"median of 3\".\n\n\nChanges since v2\n================\n\nAddresses Patrick's review (thanks!).\n\n * Split from 2 commits to 4: documentation, perf test baseline, config\n   plumbing + rev-list flag, then the incremental implementation.\n * Documentation rewrites: added unreachable-objects caveat, removed code\n   references to reduce staleness risk, replaced three-phase list with\n   cost-focused prose, rewrote trust model to explicitly describe boundary\n   mechanism and inductive argument, and several smaller clarifications.\n * Extracted tree_oid variable in verify_commit_tree() to avoid calling\n   get_commit_tree_oid(commit) twice.\n * Rebased onto latest master to resolve a meson.build conflict (t5412 was\n   taken by another topic). Renumbered test and perf files to t5413/p5413.\n\n\nChanges since v1\n================\n\n * Guard against get_commit_tree_oid() returning NULL in\n   verify_commit_tree(), dying with a clear message instead of a NULL\n   dereference. (Thanks Junio for catching this)\n * Set fetch_if_missing = 0 in the rev-list early option parse when\n   --verify-trees-incremental is seen, and add a BUG() assertion in\n   verify_commits_incremental() to catch any future caller that forgets.\n   This makes the lazy-fetch protection explicit rather than relying solely\n   on --exclude-promisor-objects having set it earlier. (Thanks again to\n   Junio)\n * Add a test that verifies promised blobs are not lazy-fetched during the\n   incremental check.\n * Reuse the existing 3-argument type-mismatch format string to avoid adding\n   new strings for localization.\n\nThanks, Kristofer\n\nKristofer Karlsson (6):\n  Documentation: describe connectivity checking\n  p5413: add perf tests for connectivity check\n  connected: drop dependency on `the_repository`\n  connected: add transfer.connectivityCheck config\n  connected: add incremental connectivity check via rev-list\n  connected: add \"auto\" connectivity check mode\n\n Documentation/Makefile                        |   1 +\n Documentation/config/transfer.adoc            |  30 +\n Documentation/rev-list-options.adoc           |   5 +\n .../technical/connectivity-check.adoc         | 259 ++++++\n Documentation/technical/meson.build           |   1 +\n Makefile                                      |   1 +\n builtin/clone.c                               |   2 +-\n builtin/fetch.c                               |   5 +-\n builtin/receive-pack.c                        |  12 +-\n builtin/rev-list.c                            |  77 ++\n bundle.c                                      |   2 +-\n connected.c                                   |  67 +-\n connected.h                                   |   4 +-\n fetch-pack.c                                  |   3 +-\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5413-connectivity-check.perl |  44 +\n t/perf/p5413-connectivity-check.sh            |  92 ++\n t/t5413-connectivity-check.sh                 | 795 ++++++++++++++++++\n t/t6700-tree-depth.sh                         |  17 +\n tree-verify.c                                 | 427 ++++++++++\n tree-verify.h                                 |  15 +\n 22 files changed, 1840 insertions(+), 21 deletions(-)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\n\nbase-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2211%2Fspkrka%2Ftree-diff-connectivity-v1-clean-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2211/spkrka/tree-diff-connectivity-v1-clean-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/2211\n\nRange-diff vs v3:\n\n 1:  19ea8cb534 ! 1:  e25824c21d Documentation: describe connectivity checking\n     @@ Commit message\n      \n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n     + ## Documentation/Makefile ##\n     +@@ Documentation/Makefile: TECH_DOCS += technical/bitmap-format\n     + TECH_DOCS += technical/build-systems\n     + TECH_DOCS += technical/bundle-uri\n     + TECH_DOCS += technical/commit-graph\n     ++TECH_DOCS += technical/connectivity-check\n     + TECH_DOCS += technical/directory-rename-detection\n     + TECH_DOCS += technical/hash-function-transition\n     + TECH_DOCS += technical/large-object-promisors\n     +\n       ## Documentation/technical/connectivity-check.adoc (new) ##\n      @@\n      +Connectivity checking\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +    git rev-list --objects --stdin --not --all --quiet\n      +        --alternate-refs [--exclude-promisor-objects]\n      +\n     -+When promisor remotes are configured, a fast path based on\n     -+promisor packfiles is attempted first.  If it falls back to\n     -+the `rev-list` check, `--exclude-promisor-objects` is added so\n     -+that the traversal does not trigger on-demand fetches.\n     ++In a partial clone, objects received from a promisor remote do\n     ++not need to be checked: anything they reference is promised by\n     ++that remote and can be fetched on demand.  So if every new tip is\n     ++found in a promisor packfile, `rev-list` is not run at all.\n     ++Otherwise it runs with `--exclude-promisor-objects`, so that the\n     ++traversal stops at promisor objects instead of fetching them.\n      +\n      +Consider the following graph after a fetch, where all reference\n      +tips point directly to commits.  For simplicity, only local\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +\n      +The traversal then walks the trees of the boundary commits and\n      +marks those objects uninteresting.  Trees of incoming commits\n     -+are verified against these marks, stopping at objects already\n     -+covered.  The cost of marking boundary trees uninteresting\n     ++are then walked, stopping at objects already marked as\n     ++uninteresting.  The cost of marking boundary trees uninteresting\n      +dominates when the boundary commits have large trees, even when\n      +the incoming changes are small.\n      +\n     @@ Documentation/technical/connectivity-check.adoc (new)\n      +These non-commit tips are verified during the tree and blob\n      +traversal that follows the commit walk within the same rev-list\n      +invocation.\n     +\n     + ## Documentation/technical/meson.build ##\n     +@@ Documentation/technical/meson.build: articles = [\n     +   'build-systems.adoc',\n     +   'bundle-uri.adoc',\n     +   'commit-graph.adoc',\n     ++  'connectivity-check.adoc',\n     +   'directory-rename-detection.adoc',\n     +   'hash-function-transition.adoc',\n     +   'large-object-promisors.adoc',\n 2:  211794c925 ! 2:  427b7315a5 p5413: add perf tests for connectivity check\n     @@ t/perf/generate-repo-p5413-connectivity-check.perl (new)\n      @@\n      +#!/usr/bin/perl\n      +#\n     -+# Generate a fast-import stream for p5412 connectivity check benchmarks.\n     ++# Generate a fast-import stream for p5413 connectivity check benchmarks.\n      +#\n     -+# Usage: generate-repo-p5412-connectivity-check.perl\n     ++# Usage: generate-repo-p5413-connectivity-check.perl\n      +#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n      +#\n      +# Creates one initial commit with dirs*files_per_dir files, then\n     @@ t/perf/p5413-connectivity-check.sh (new)\n      +\t\tlabel=\"$label (${fpc} files/commit)\"\n      +\tfi\n      +\n     -+\ttest_expect_success \"setup $label\" '\n     ++\ttest_expect_success PERL \"setup $label\" '\n      +\t\tif test -d '\"$name\"'\n      +\t\tthen\n      +\t\t\ttrue\n     @@ t/perf/p5413-connectivity-check.sh (new)\n      +\t\tfi\n      +\t'\n      +\n     -+\ttest_perf \"$label\" '\n     ++\ttest_perf \"$label\" --prereq PERL '\n      +\t\tcat '\"${name}\"'_new |\n      +\t\tgit -C '\"$name\"' rev-list \\\n      +\t\t\t--objects --stdin --not --all --quiet \\\n -:  ---------- > 3:  3a7a1fb953 connected: drop dependency on `the_repository`\n 3:  cdf28f1df6 ! 4:  87dbb1f055 connected: add transfer.connectivityCheck config and rev-list flag\n     @@ Metadata\n      Author: Kristofer Karlsson <krka@spotify.com>\n      \n       ## Commit message ##\n     -    connected: add transfer.connectivityCheck config and rev-list flag\n     +    connected: add transfer.connectivityCheck config\n      \n     -    Add the transfer.connectivityCheck configuration option, accepting\n     -    \"full\" (default) and \"incremental\".  When set to \"incremental\",\n     -    check_connected() passes --verify-trees-incremental to its rev-list\n     -    subprocess.\n     +    Add the transfer.connectivityCheck configuration option to choose\n     +    the algorithm used for the connectivity check.  For now the only\n     +    accepted value is \"full\", which is the existing behavior and the\n     +    default.  Unknown values fall back to \"full\" with a warning, so\n     +    that configuration written for a newer version of Git does not\n     +    break fetches and pushes; \"full\" is always a correct check.\n      \n     -    The rev-list flag is accepted but has no effect yet; the incremental\n     -    verification will be implemented in the next commit.\n     +    check_connected() now always tells its rev-list subprocess which\n     +    algorithm to use with an internal --connectivity-check=<mode>\n     +    option, which so far only accepts \"full\".  Passing the mode\n     +    explicitly means rev-list knows it is running a connectivity\n     +    check, so later optimizations can be limited to that case.\n      \n     +    This prepares for an incremental check added in the next commit.\n     +\n     +    Suggested-by: Patrick Steinhardt <ps@pks.im>\n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n       ## Documentation/config/transfer.adoc ##\n     @@ Documentation/config/transfer.adoc\n      +--\n      +`full` (default);;\n      +\tWalk the full object closure of the boundary commits.\n     -+`incremental`;;\n     -+\tVerify incoming commits by diffing their trees against parent\n     -+\ttrees, recursively descending only into entries that differ.\n     -+\tThe largest benefits occur when incoming commits change a\n     -+\tsmall fraction of a large tree closure.\n     -+\tFalls back to `full` for deepening fetches.\n      +--\n     +++\n     ++Unknown values are treated as `full`, with a warning.\n      +\n       transfer.credentialsInUrl::\n       \tA configured URL can contain plaintext credentials in the form\n     @@ Documentation/rev-list-options.adoc: It is an error to use `--missing-only` with\n       \ttraversal, rather than just silencing errors about missing\n       \tobjects.\n      +\n     -+`--verify-trees-incremental`::\n     -+\t(For internal use only.)  Verify tree connectivity\n     -+\tincrementally by comparing each commit's tree against its\n     -+\tparent trees.  Used by `check_connected()` when\n     -+\t`transfer.connectivityCheck` is set to `incremental`.\n     ++`--connectivity-check=full`::\n     ++\t(For internal use only.)  Used by `check_connected()` to\n     ++\tselect the connectivity check algorithm; see\n     ++\t`transfer.connectivityCheck` in linkgit:git-config[1].\n       endif::git-rev-list[]\n       \n       `--no-walk[=(sorted|unsorted)]`::\n      \n       ## builtin/rev-list.c ##\n     +@@\n     + #include \"quote.h\"\n     + #include \"strbuf.h\"\n     + \n     ++enum connectivity_check_mode {\n     ++\tCONNECTIVITY_CHECK_FULL,\n     ++};\n     ++\n     ++struct connectivity_check {\n     ++\tenum connectivity_check_mode mode;\n     ++};\n     ++\n     ++static void parse_connectivity_check(struct repository *repo UNUSED,\n     ++\t\t\t\t     const char *arg,\n     ++\t\t\t\t     struct connectivity_check *check)\n     ++{\n     ++\tif (!strcmp(arg, \"full\"))\n     ++\t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n     ++\telse\n     ++\t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n     ++}\n     ++\n     + struct rev_list_info {\n     + \tstruct rev_info *revs;\n     + \tint flags;\n     +@@ builtin/rev-list.c: int cmd_rev_list(int argc,\n     + \tint bisect_find_all = 0;\n     + \tint use_bitmap_index = 0;\n     + \tint filter_provided_objects = 0;\n     ++\tstruct connectivity_check connectivity_check = { 0 };\n     + \tconst char *show_progress = NULL;\n     + \tint ret = 0;\n     + \n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n       \t\t\trepo->fetch_if_missing = 0;\n       \t\t\trevs.exclude_promisor_objects = 1;\n     -+\t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n     -+\t\t\trepo->fetch_if_missing = 0;\n     ++\t\t} else if (skip_prefix(arg, \"--connectivity-check=\", &arg)) {\n     ++\t\t\tparse_connectivity_check(repo, arg, &connectivity_check);\n       \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n       \t\t\tparse_missing_action_value(repo, arg);\n       \t\t} else if (!strcmp(arg, \"--missing-only\")) {\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \n       \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n       \t\t\tcontinue; /* already handled above */\n     -+\t\tif (!strcmp(arg, \"--verify-trees-incremental\"))\n     ++\t\tif (starts_with(arg, \"--connectivity-check=\"))\n      +\t\t\tcontinue; /* already handled above */\n       \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n       \t\t\tcontinue; /* already handled above */\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n      \n       ## connected.c ##\n      @@\n     - #define USE_THE_REPOSITORY_VARIABLE\n     - \n       #include \"git-compat-util.h\"\n      +#include \"config.h\"\n       #include \"gettext.h\"\n       #include \"hex.h\"\n       #include \"odb.h\"\n     -@@ connected.c: static int check_connected_promisor(oid_iterate_fn fn,\n     +@@ connected.c: static int check_connected_promisor(struct repository *repo,\n       \treturn 1;\n       }\n       \n     -+static int incremental_check_applicable(struct check_connected_options *opt)\n     ++/*\n     ++ * Fills in the --connectivity-check mode to pass to rev-list.\n     ++ */\n     ++static void connectivity_check_mode(struct repository *repo,\n     ++\t\t\t\t    struct strbuf *mode)\n      +{\n      +\tconst char *algorithm = NULL;\n      +\n     -+\tif (repo_config_get_string_tmp(the_repository,\n     ++\tif (repo_config_get_string_tmp(repo,\n      +\t\t\t\t       \"transfer.connectivitycheck\",\n      +\t\t\t\t       &algorithm))\n     -+\t\treturn 0;\n     ++\t\talgorithm = \"full\";\n      +\tif (!strcasecmp(algorithm, \"full\"))\n     -+\t\treturn 0;\n     -+\tif (strcasecmp(algorithm, \"incremental\"))\n     -+\t\tdie(_(\"unknown transfer.connectivityCheck algorithm '%s'\"),\n     -+\t\t    algorithm);\n     -+\n     -+\tif (opt->is_deepening_fetch)\n     -+\t\treturn 0;\n     -+\n     -+\treturn 1;\n     ++\t\tstrbuf_addstr(mode, \"full\");\n     ++\telse {\n     ++\t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n     ++\t\t\t  \"using 'full'\"), algorithm);\n     ++\t\tstrbuf_addstr(mode, \"full\");\n     ++\t}\n      +}\n      +\n       /*\n        * If we feed all the commits we want to verify to this command\n        *\n     -@@ connected.c: int check_connected(oid_iterate_fn fn, void *cb_data,\n     +@@ connected.c: int check_connected(struct repository *repo,\n     + \tstruct child_process rev_list = CHILD_PROCESS_INIT;\n     + \tFILE *rev_list_in;\n     + \tstruct check_connected_options defaults = CHECK_CONNECTED_INIT;\n     ++\tstruct strbuf mode = STRBUF_INIT;\n     + \tconst struct object_id *oid;\n     + \tint err = 0;\n     + \tstruct packed_git *new_pack = NULL;\n     +@@ connected.c: int check_connected(struct repository *repo,\n       \tif (opt->progress)\n       \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n       \t\t\t     _(\"Checking connectivity\"));\n     -+\tif (incremental_check_applicable(opt))\n     -+\t\tstrvec_push(&rev_list.args,\n     -+\t\t\t    \"--verify-trees-incremental\");\n     ++\tconnectivity_check_mode(repo, &mode);\n     ++\tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n     ++\tstrbuf_release(&mode);\n       \n       \trev_list.git_cmd = 1;\n       \tif (opt->env)\n 4:  efaada0b6a ! 5:  d76dae801b connected: add incremental connectivity check via rev-list\n     @@ Commit message\n          See Documentation/technical/connectivity-check.adoc for the trust\n          invariants and detailed algorithm.\n      \n     -    The incremental check runs as an internal\n     -    --verify-trees-incremental mode in rev-list.  After the normal\n     +    The incremental check runs inside rev-list, selected by the\n     +    internal --connectivity-check option.  After the normal\n          revision walk identifies the incoming commits, the verifier\n          consumes them before the usual object traversal; any pending\n          non-commit tips are still handled by the existing traversal.\n      \n          Partial-clone semantics are preserved: objects promised by a\n          promisor remote are accepted as connected, and on-demand fetching\n     -    is prevented by excluding promisor objects from traversal.  The\n     -    new mode is selected by transfer.connectivityCheck=incremental,\n     -    with full remaining the default.  Deepening fetches fall back to\n     -    the full check because they do not have the normal\n     +    is prevented by excluding promisor objects from traversal.\n     +\n     +    A new transfer.connectivityCheck value \"incremental\" selects the\n     +    new mode, with \"full\" remaining the default.  Deepening fetches\n     +    fall back to the full check because they do not have the normal\n          existing-reference boundary, and traversal instead follows the\n          effective shallow boundary.\n      \n     -    p5413 results (median of 3), scaling one dimension at a time.\n     +    Clones and pushes into an empty repository also use the full\n     +    check.  Without negative revisions there is no trusted boundary,\n     +    and the incremental check would have to verify all of history\n     +    from scratch like the full check does, with extra per-commit\n     +    overhead on top.  On git.git, checking 70K commits without a\n     +    boundary takes 5.3s with the full check and 9.2s with the\n     +    incremental one.\n     +\n     +    p5413 results (best of 3), scaling one dimension at a time.\n          Each fixture is a repository with a flat tree of many\n          directories containing 100 files each.  The incoming set is a\n          chain of commits on top of the existing history, with each\n     @@ Commit message\n              files    full  incr.  full/incr\n                5K    0.01s  0.01s    1.0x\n               50K    0.04s  0.01s    4.0x\n     -        200K    0.15s  0.01s   15.0x\n     -        800K    0.61s  0.03s   20.3x\n     +        200K    0.13s  0.01s   13.0x\n     +        800K    0.54s  0.03s   18.0x\n      \n          The full mode must walk the trees of boundary commits, which\n          grows with overall tree size.  Incremental still scans the\n     @@ Commit message\n          Scaling incoming commit count (200K files, 10 files/commit):\n      \n              commits    full  incr.  full/incr\n     -              1    0.14s  0.01s   14.0x\n     -             10    0.15s  0.01s   15.0x\n     -            100    0.19s  0.05s    3.8x\n     -            500    0.32s  0.27s    1.2x\n     -           3000    1.23s  1.52s    0.8x\n     -           5000    1.88s  2.43s    0.8x\n     -          10000    3.72s  5.36s    0.7x\n     +              1    0.13s  0.01s   13.0x\n     +             10    0.13s  0.01s   13.0x\n     +            100    0.16s  0.06s    2.7x\n     +            500    0.29s  0.25s    1.2x\n     +           3000    1.14s  1.44s    0.8x\n     +           5000    1.87s  2.39s    0.8x\n     +          10000    3.45s  5.13s    0.7x\n      \n          With many commits the per-commit overhead of scanning both\n          the new and parent root trees accumulates and incremental\n     @@ Commit message\n          Scaling files per incoming commit (200K files, 10 commits):\n      \n              files/commit    full  incr.  full/incr\n     -                   1    0.14s  0.01s   14.0x\n     -                  10    0.15s  0.01s   15.0x\n     -                 100    0.16s  0.04s    4.0x\n     -                 500    0.23s  0.14s    1.6x\n     -                1000    0.34s  0.28s    1.2x\n     -                2000    0.70s  0.63s    1.1x\n     -\n     -    Breakeven is around 1000 files/commit.  At 2000 files/commit\n     -    (every directory touched), incremental remains slightly faster;\n     -    bypassing the object cache for tree reads likely helps here.\n     +                   1    0.13s  0.01s   13.0x\n     +                  10    0.13s  0.01s   13.0x\n     +                 100    0.15s  0.03s    5.0x\n     +                 500    0.23s  0.13s    1.8x\n     +                1000    0.34s  0.27s    1.3x\n     +                2000    0.65s  0.56s    1.2x\n      \n     -    For small repositories both modes are fast enough that the\n     -    difference is difficult to measure reliably.\n     +    The advantage shrinks as more files change per commit, but\n     +    incremental stays slightly faster even at 2000 files/commit,\n     +    where every directory is touched.\n      \n          Selected peak RSS measurements from the same fixtures:\n      \n     @@ Commit message\n              800K files, 10 commits     110 MB    26 MB   0.2x\n              200K files, 5000 commits   155 MB   151 MB   1.0x\n      \n     -    Incremental uses substantially less memory when it can prune\n     -    most of the boundary tree walk.  In the long-history case the\n     -    two modes visit similar object sets and memory converges.\n     +    Incremental uses less memory when it can prune enough of the\n     +    boundary tree walk.  In the long-history case the two modes\n     +    visit similar object sets and memory converges.\n      \n          The regression cases in the CPU benchmarks are in wall-clock\n          time rather than memory, primarily from scanning some trees\n     -    more than once.\n     +    more than once: a commit's tree is read as a candidate when the\n     +    commit is verified, and again as a trusted parent tree when its\n     +    children are verified.\n      \n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n     + ## Documentation/config/transfer.adoc ##\n     +@@ Documentation/config/transfer.adoc: transfer.connectivityCheck::\n     + --\n     + `full` (default);;\n     + \tWalk the full object closure of the boundary commits.\n     ++`incremental`;;\n     ++\tVerify incoming commits by diffing their trees against parent\n     ++\ttrees, recursively descending only into entries that differ.\n     ++\tThe largest benefits occur when incoming commits change a\n     ++\tsmall fraction of a large tree closure, and it can be slower\n     ++\tthan `full` when there are many incoming commits.\n     ++\tFalls back to `full` for clones, pushes into an empty\n     ++\trepository and deepening fetches.\n     + --\n     + +\n     + Unknown values are treated as `full`, with a warning.\n     +\n     + ## Documentation/rev-list-options.adoc ##\n     +@@ Documentation/rev-list-options.adoc: It is an error to use `--missing-only` without `--missing=print` or\n     + \ttraversal, rather than just silencing errors about missing\n     + \tobjects.\n     + \n     +-`--connectivity-check=full`::\n     ++`--connectivity-check=(full|incremental)`::\n     + \t(For internal use only.)  Used by `check_connected()` to\n     + \tselect the connectivity check algorithm; see\n     + \t`transfer.connectivityCheck` in linkgit:git-config[1].\n     +\n       ## Documentation/technical/connectivity-check.adoc ##\n      @@ Documentation/technical/connectivity-check.adoc: tag, tree, or blob, that object is not part of the commit walk.\n       These non-commit tips are verified during the tree and blob\n     @@ Documentation/technical/connectivity-check.adoc: tag, tree, or blob, that object\n      +replaces what happens afterwards: rather than walking boundary\n      +trees to mark objects uninteresting, it compares each incoming\n      +commit's tree against its parents' trees, processing commits in\n     -+topological order (ancestors first).\n     ++topological order (ancestors first).  This all happens inside\n     ++the same `rev-list` invocation, and the promisor fast path above\n     ++still runs first.\n      +\n      +Trust model\n      +~~~~~~~~~~~\n     @@ builtin/rev-list.c\n       #include \"strbuf.h\"\n      +#include \"tree-verify.h\"\n       \n     - struct rev_list_info {\n     - \tstruct rev_info *revs;\n     -@@ builtin/rev-list.c: int cmd_rev_list(int argc,\n     - \tint bisect_find_all = 0;\n     - \tint use_bitmap_index = 0;\n     - \tint filter_provided_objects = 0;\n     -+\tint verify_trees_incremental = 0;\n     - \tconst char *show_progress = NULL;\n     - \tint ret = 0;\n     + enum connectivity_check_mode {\n     + \tCONNECTIVITY_CHECK_FULL,\n     ++\tCONNECTIVITY_CHECK_INCREMENTAL,\n     + };\n       \n     -@@ builtin/rev-list.c: int cmd_rev_list(int argc,\n     - \t\t\trepo->fetch_if_missing = 0;\n     - \t\t\trevs.exclude_promisor_objects = 1;\n     - \t\t} else if (!strcmp(arg, \"--verify-trees-incremental\")) {\n     -+\t\t\tverify_trees_incremental = 1;\n     - \t\t\trepo->fetch_if_missing = 0;\n     - \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n     - \t\t\tparse_missing_action_value(repo, arg);\n     + struct connectivity_check {\n     + \tenum connectivity_check_mode mode;\n     + };\n     + \n     +-static void parse_connectivity_check(struct repository *repo UNUSED,\n     ++static void parse_connectivity_check(struct repository *repo,\n     + \t\t\t\t     const char *arg,\n     + \t\t\t\t     struct connectivity_check *check)\n     + {\n     + \tif (!strcmp(arg, \"full\"))\n     + \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n     ++\telse if (!strcmp(arg, \"incremental\"))\n     ++\t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n     + \telse\n     + \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n     ++\n     ++\tif (check->mode != CONNECTIVITY_CHECK_FULL)\n     ++\t\trepo->fetch_if_missing = 0;\n     + }\n     + \n     + struct rev_list_info {\n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \n       \tprepare_maximal_independent(&revs);\n       \n     -+\tif (verify_trees_incremental) {\n     ++\t/*\n     ++\t * Without a boundary of negative revisions, as in a clone or a\n     ++\t * push into an empty repository, every incoming commit is new.\n     ++\t * The incremental check would still work, but it would verify\n     ++\t * the whole object graph like the full check, with extra\n     ++\t * per-commit overhead on top, so use the full check.\n     ++\t */\n     ++\tif (!revs.limited)\n     ++\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n     ++\n     ++\tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n      +\t\tstruct commit *commit;\n      +\t\tstruct commit_list *new_commits = NULL;\n      +\n      +\t\twhile ((commit = get_revision(&revs)) != NULL)\n      +\t\t\tcommit_list_insert(commit, &new_commits);\n      +\n     -+\t\tverify_commits_incremental(repo, &new_commits,\n     -+\t\t\t\t\t   revs.exclude_promisor_objects);\n     ++\t\tif (verify_commits_incremental(repo, &new_commits,\n     ++\t\t\t\t\t       revs.exclude_promisor_objects))\n     ++\t\t\tdie(_(\"connectivity check failed\"));\n      +\t\tcommit_list_free(new_commits);\n      +\t}\n      +\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n       \n      \n     + ## connected.c ##\n     +@@ connected.c: static int check_connected_promisor(struct repository *repo,\n     +  * Fills in the --connectivity-check mode to pass to rev-list.\n     +  */\n     + static void connectivity_check_mode(struct repository *repo,\n     ++\t\t\t\t    struct check_connected_options *opt,\n     + \t\t\t\t    struct strbuf *mode)\n     + {\n     + \tconst char *algorithm = NULL;\n     + \n     ++\t/*\n     ++\t * Deepening fetches do not have the usual boundary of\n     ++\t * existing references, so use the full check for them.\n     ++\t */\n     ++\tif (opt->is_deepening_fetch) {\n     ++\t\tstrbuf_addstr(mode, \"full\");\n     ++\t\treturn;\n     ++\t}\n     ++\n     + \tif (repo_config_get_string_tmp(repo,\n     + \t\t\t\t       \"transfer.connectivitycheck\",\n     + \t\t\t\t       &algorithm))\n     + \t\talgorithm = \"full\";\n     + \tif (!strcasecmp(algorithm, \"full\"))\n     + \t\tstrbuf_addstr(mode, \"full\");\n     ++\telse if (!strcasecmp(algorithm, \"incremental\"))\n     ++\t\tstrbuf_addstr(mode, \"incremental\");\n     + \telse {\n     + \t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n     + \t\t\t  \"using 'full'\"), algorithm);\n     +@@ connected.c: int check_connected(struct repository *repo,\n     + \tif (opt->progress)\n     + \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n     + \t\t\t     _(\"Checking connectivity\"));\n     +-\tconnectivity_check_mode(repo, &mode);\n     ++\tconnectivity_check_mode(repo, opt, &mode);\n     + \tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n     + \tstrbuf_release(&mode);\n     + \n     +\n       ## meson.build ##\n      @@ meson.build: libgit_sources = [\n         'transport-helper.c',\n     @@ t/perf/p5413-connectivity-check.sh: test_perf_conn () {\n       \t\tfi\n       \t'\n       \n     --\ttest_perf \"$label\" '\n     -+\ttest_perf \"$label (full)\" '\n     +-\ttest_perf \"$label\" --prereq PERL '\n     ++\ttest_perf \"$label (full)\" --prereq PERL '\n       \t\tcat '\"${name}\"'_new |\n     - \t\tgit -C '\"$name\"' rev-list \\\n     - \t\t\t--objects --stdin --not --all --quiet \\\n     - \t\t\t--exclude-promisor-objects\n     - \t'\n     -+\n     -+\ttest_perf \"$label (incremental)\" '\n     -+\t\tcat '\"${name}\"'_new |\n     -+\t\tgit -C '\"$name\"' rev-list --verify-trees-incremental \\\n     +-\t\tgit -C '\"$name\"' rev-list \\\n     ++\t\tgit -C '\"$name\"' rev-list --connectivity-check=full \\\n      +\t\t\t--objects --stdin --not --all --quiet \\\n      +\t\t\t--exclude-promisor-objects\n      +\t'\n     - }\n     - \n     - # Scaling tree size (10 commits, 10 files/commit).\n     ++\n     ++\ttest_perf \"$label (incremental)\" --prereq PERL '\n     ++\t\tcat '\"${name}\"'_new |\n     ++\t\tgit -C '\"$name\"' rev-list --connectivity-check=incremental \\\n     + \t\t\t--objects --stdin --not --all --quiet \\\n     + \t\t\t--exclude-promisor-objects\n     + \t'\n      \n       ## t/t5413-connectivity-check.sh (new) ##\n      @@\n     @@ t/t5413-connectivity-check.sh (new)\n      +# Run a connectivity check via rev-list directly.  Uses $mode\n      +# (set by the enclosing for-loop) to choose full or incremental.\n      +check_connected () {\n     -+\tflags= &&\n     -+\tif test \"$mode\" = incremental\n     -+\tthen\n     -+\t\tflags=--verify-trees-incremental\n     -+\tfi &&\n      +\tprintf '%s\\n' \"$@\" |\n     -+\tgit rev-list $flags \\\n     ++\tgit rev-list --connectivity-check=\"$mode\" \\\n      +\t\t--objects --stdin --not --all --quiet \\\n      +\t\t--exclude-promisor-objects\n      +}\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t\tgit update-ref -d refs/heads/main &&\n      +\t\tgit update-ref -d HEAD &&\n      +\n     -+\t\techo \"$original\" >.git/test-oid\n     ++\t\techo \"$original\" >.git/test-oid &&\n     ++\t\techo \"$orig_blob\" >.git/test-blob\n      +\t)\n      +'\n      +\n      +missing_oid=$(test_oid missing)\n      +original_oid=$(cat replace-test/.git/test-oid)\n     ++orig_blob_oid=$(cat replace-test/.git/test-blob)\n      +\n      +for mode in full incremental\n      +do\n      +\n     ++# The full traversal reports a missing tree as \"bad tree object\".\n     ++case \"$mode\" in\n     ++full)\n     ++\tmissing_tree=\"bad tree object\"\n     ++\t;;\n     ++incremental)\n     ++\tmissing_tree=\"missing tree object\"\n     ++\t;;\n     ++esac\n     ++\n      +# Corruption detection: craft broken object graphs and verify detection.\n      +# All tests use main-repo without modifying its refs or worktree.\n      +\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t\t\tgit mktree --missing) &&\n      +\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n      +\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n     -+\t\ttest_grep \"missing blob object\" err\n     ++\t\ttest_grep \"missing blob object.*$missing_oid\" err\n      +\t)\n      +'\n      +\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t\t\tgit mktree --missing) &&\n      +\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n      +\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n     -+\t\ttest_grep \"bad tree object\" err\n     ++\t\ttest_grep \"$missing_tree $missing_oid\" err\n      +\t)\n      +'\n      +\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n      +\t\t\tgit mktree --missing) &&\n      +\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n     -+\t\ttest_grep \"missing blob object\" err\n     ++\t\ttest_grep \"missing blob object.*$missing_oid\" err\n      +\t)\n      +'\n      +\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t\texport GIT_NO_REPLACE_OBJECTS &&\n      +\t\ttest_expect_code 128 check_connected \\\n      +\t\t\t\"$original_oid\" 2>err &&\n     -+\t\ttest_grep \"missing blob object\" err\n     ++\t\ttest_grep \"missing blob object.*$orig_blob_oid\" err\n      +\t)\n      +'\n      +\n     @@ t/t5413-connectivity-check.sh (new)\n      +\t)\n      +'\n      +\n     ++test_expect_success \"$mode: tolerates missing promisor parent commit\" '\n     ++\ttest_when_finished \"rm -rf mp-src mp-server.git mp-client\" &&\n     ++\tgit init mp-src &&\n     ++\ttest_commit -C mp-src --no-tag base file.txt &&\n     ++\tgit clone --bare mp-src mp-server.git &&\n     ++\tgit -C mp-server.git config uploadpack.allowfilter true &&\n     ++\tgit -C mp-server.git config uploadpack.allowanysha1inwant true &&\n     ++\tgit clone --filter=blob:none \\\n     ++\t\t\"file://$(pwd)/mp-server.git\" mp-client &&\n     ++\ttest_commit -C mp-src --no-tag c1 file.txt &&\n     ++\ttest_commit -C mp-src --no-tag c2 file.txt &&\n     ++\tc1=$(git -C mp-src rev-parse HEAD~1) &&\n     ++\tc2=$(git -C mp-src rev-parse HEAD) &&\n     ++\t(\n     ++\t\tcd mp-client &&\n     ++\t\t# Add a promisor pack holding only c2, so its parent c1\n     ++\t\t# is missing but promised (as after a lazy fetch that\n     ++\t\t# omits dependents).\n     ++\t\techo \"$c2\" |\n     ++\t\tgit -C ../mp-src pack-objects --stdout >c2.pack &&\n     ++\t\tgit index-pack --promisor --stdin <c2.pack &&\n     ++\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n     ++\t\t\tgit cat-file -e \"$c1\" &&\n     ++\t\ttree=$(git rev-parse HEAD^{tree}) &&\n     ++\t\tlocal_commit=$(git hash-object -t commit -w --stdin <<-EOF\n     ++\t\ttree $tree\n     ++\t\tparent $c1\n     ++\t\tauthor A U Thor <author@example.com> 1112912053 -0700\n     ++\t\tcommitter C O Mitter <committer@example.com> 1112912053 -0700\n     ++\n     ++\t\ton top of a missing promisor commit\n     ++\t\tEOF\n     ++\t\t) &&\n     ++\t\tcheck_connected \"$local_commit\"\n     ++\t)\n     ++'\n     ++\n      +test_expect_success \"$mode: respects shallow boundary\" '\n      +\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n      +\tgit init shallow-src &&\n     @@ t/t5413-connectivity-check.sh (new)\n      +\n      +done\n      +\n     ++# The full check only checks that blobs exist, but the incremental\n     ++# check reads their headers and reports a corrupt blob.\n     ++test_expect_success 'incremental: rejects corrupt blob' '\n     ++\ttest_when_finished \"rm -rf corrupt-repo\" &&\n     ++\tgit init corrupt-repo &&\n     ++\t(\n     ++\t\tcd corrupt-repo &&\n     ++\t\ttest_commit base &&\n     ++\t\tblob=$(echo corrupt | git hash-object -w --stdin) &&\n     ++\t\ttree=$(printf \"100644 blob $blob\\tfile\\n\" | git mktree) &&\n     ++\t\tcommit=$(git commit-tree \"$tree\" -p HEAD -m corrupt) &&\n     ++\t\tpath=.git/objects/$(test_oid_to_path \"$blob\") &&\n     ++\t\tchmod +w \"$path\" &&\n     ++\t\techo garbage >\"$path\" &&\n     ++\t\tmode=incremental &&\n     ++\t\ttest_expect_code 128 check_connected \"$commit\" 2>err &&\n     ++\t\ttest_grep \"unable to read blob object $blob\" err\n     ++\t)\n     ++'\n     ++\n      +# Algorithm selection.\n      +\n     -+test_expect_success 'invalid transfer.connectivityCheck is rejected' '\n     -+\ttest_when_finished \"rm -rf invalid-cfg-src invalid-cfg-dst\" &&\n     -+\tgit init invalid-cfg-src &&\n     -+\ttest_commit -C invalid-cfg-src --no-tag base file.txt &&\n     -+\tgit clone invalid-cfg-src invalid-cfg-dst &&\n     -+\ttest_commit -C invalid-cfg-src --no-tag update file.txt updated &&\n     -+\tgit -C invalid-cfg-dst config transfer.connectivityCheck bogus &&\n     -+\ttest_must_fail git -C invalid-cfg-dst fetch origin main 2>err &&\n     -+\ttest_grep \"unknown transfer.connectivityCheck\" err\n     ++# Run a command with its trace2 events in \"trace\" and its stderr\n     ++# in \"err\".\n     ++run_traced () {\n     ++\trm -f trace &&\n     ++\tGIT_TRACE2_EVENT=\"$(pwd)/trace\" \"$@\" 2>err\n     ++}\n     ++\n     ++# Check which connectivity check the last run_traced used.\n     ++expect_mode () {\n     ++\tcase \"$1\" in\n     ++\tfull)\n     ++\t\ttest_grep ! trees_loaded trace\n     ++\t\t;;\n     ++\tincremental)\n     ++\t\ttest_grep trees_loaded trace\n     ++\t\t;;\n     ++\t*)\n     ++\t\tBUG \"unknown mode: $1\"\n     ++\t\t;;\n     ++\tesac\n     ++}\n     ++\n     ++test_expect_success 'setup source repository' '\n     ++\tgit init src &&\n     ++\ttest_commit -C src base file.txt &&\n     ++\ttest_commit -C src --no-tag update file.txt updated\n     ++'\n     ++\n     ++test_expect_success 'unknown transfer.connectivityCheck falls back to full' '\n     ++\ttest_when_finished \"rm -rf dst\" &&\n     ++\tgit init --bare dst &&\n     ++\tgit -C dst fetch ../src base:refs/heads/main &&\n     ++\tfor value in bogus\n     ++\tdo\n     ++\t\tset_connectivity_check dst \"$value\" &&\n     ++\t\trun_traced git -C dst fetch ../src main &&\n     ++\t\texpect_mode full &&\n     ++\t\ttest_grep \"unknown transfer.connectivityCheck algorithm .$value., using .full.\" err ||\n     ++\t\treturn 1\n     ++\tdone\n      +'\n      +\n      +test_expect_success 'push uses incremental when configured' '\n     -+\ttest_when_finished \"rm -rf int-src int-dst.git\" &&\n     -+\tgit init int-src &&\n     -+\ttest_commit -C int-src --no-tag base file.txt &&\n     -+\tgit clone --bare int-src int-dst.git &&\n     -+\ttest_commit -C int-src --no-tag update file.txt updated &&\n     -+\tset_connectivity_check int-dst.git incremental &&\n     -+\tGIT_TRACE2_EVENT=\"$(pwd)/push-trace.txt\" \\\n     -+\t\tgit -C int-src push ../int-dst.git main &&\n     -+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n     -+\t\t<push-trace.txt\n     ++\ttest_when_finished \"rm -rf dst\" &&\n     ++\tgit init --bare dst &&\n     ++\tgit -C src push ../dst base:refs/heads/main &&\n     ++\tset_connectivity_check dst incremental &&\n     ++\trun_traced git -C src push ../dst main &&\n     ++\texpect_mode incremental\n      +'\n      +\n      +test_expect_success 'fetch uses incremental when configured' '\n     -+\ttest_when_finished \"rm -rf fetch-src fetch-dst\" &&\n     -+\tgit init fetch-src &&\n     -+\ttest_commit -C fetch-src --no-tag base file.txt &&\n     -+\tgit clone fetch-src fetch-dst &&\n     -+\ttest_commit -C fetch-src --no-tag update file.txt updated &&\n     -+\tset_connectivity_check fetch-dst incremental &&\n     -+\tGIT_TRACE2_EVENT=\"$(pwd)/fetch-trace.txt\" \\\n     -+\t\tgit -C fetch-dst fetch origin main &&\n     -+\ttest_trace2_data_singular connectivity trees_loaded 2 \\\n     -+\t\t<fetch-trace.txt\n     ++\ttest_when_finished \"rm -rf dst\" &&\n     ++\tgit init --bare dst &&\n     ++\tgit -C dst fetch ../src base:refs/heads/main &&\n     ++\tset_connectivity_check dst incremental &&\n     ++\trun_traced git -C dst fetch ../src main &&\n     ++\texpect_mode incremental\n      +'\n      +\n     -+test_expect_success 'clone respects transfer.connectivityCheck' '\n     -+\ttest_when_finished \"rm -rf clone-src clone-dst\" &&\n     -+\tgit init clone-src &&\n     -+\ttest_commit -C clone-src --no-tag base file.txt &&\n     -+\tGIT_TRACE2_EVENT=\"$(pwd)/clone-trace.txt\" \\\n     -+\t\tgit -c transfer.connectivityCheck=incremental \\\n     -+\t\tclone --no-local clone-src clone-dst &&\n     -+\ttest_trace2_data_singular connectivity trees_loaded 0 \\\n     -+\t\t<clone-trace.txt\n     -+'\n     ++for mode in incremental\n     ++do\n     ++\ttest_expect_success \"push with $mode into an empty repository uses full\" '\n     ++\t\ttest_when_finished \"rm -rf dst\" &&\n     ++\t\tgit init --bare dst &&\n     ++\t\tset_connectivity_check dst $mode &&\n     ++\t\trun_traced git -C src push ../dst main &&\n     ++\t\texpect_mode full\n     ++\t'\n     ++\n     ++\ttest_expect_success \"clone with $mode uses full\" '\n     ++\t\ttest_when_finished \"rm -rf dst\" &&\n     ++\t\trun_traced git -c transfer.connectivityCheck=$mode \\\n     ++\t\t\tclone --no-local src dst &&\n     ++\t\texpect_mode full\n     ++\t'\n     ++done\n      +\n      +test_done\n      \n     + ## t/t6700-tree-depth.sh ##\n     +@@ t/t6700-tree-depth.sh: test_expect_success 'default limit for rev-list fails gracefully' '\n     + \ttest_must_fail git rev-list --objects big >/dev/null\n     + '\n     + \n     ++# The incremental check needs a boundary; without negative revisions\n     ++# rev-list falls back to the full check.\n     ++test_expect_success 'limit recursion of incremental check' '\n     ++\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n     ++\tgit $small_ok rev-list --connectivity-check=incremental \\\n     ++\t\t--objects small --not $boundary >/dev/null &&\n     ++\ttest_must_fail git $small_no rev-list \\\n     ++\t\t--connectivity-check=incremental --objects small \\\n     ++\t\t--not $boundary >/dev/null\n     ++'\n     ++\n     ++test_expect_success 'default limit for incremental check fails gracefully' '\n     ++\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n     ++\ttest_must_fail git rev-list --connectivity-check=incremental \\\n     ++\t\t--objects big --not $boundary >/dev/null\n     ++'\n     ++\n     + test_expect_success 'limit recursion of diff-tree -r' '\n     + \tgit $small_ok diff-tree -r $EMPTY_TREE small &&\n     + \ttest_must_fail git $small_no diff-tree -r $EMPTY_TREE small\n     +\n       ## tree-verify.c (new) ##\n      @@\n      +#include \"git-compat-util.h\"\n     @@ tree-verify.c (new)\n      +\tTREE_EXPANDED  = 2,\n      +};\n      +\n     ++/* Values are enum tree_state, stored as a byte to keep the map small. */\n      +KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n      +\t   oidhash_by_value, oideq_by_value)\n      +\n     @@ tree-verify.c (new)\n      +\tstruct oid_array parent_trees;\n      +};\n      +\n     ++/* The entries of a tree that still need to be verified. */\n     ++struct work_list {\n     ++\tstruct work_item *items;\n     ++\tsize_t nr, alloc;\n     ++\tbool has_subtrees;\n     ++};\n     ++\n     ++static void work_list_clear(struct work_list *list)\n     ++{\n     ++\tfor (size_t i = 0; i < list->nr; i++)\n     ++\t\toid_array_clear(&list->items[i].parent_trees);\n     ++\tfree(list->items);\n     ++}\n     ++\n      +struct verify_state {\n      +\tkh_oid_tree_t *trees;\n      +\tstruct oidset trusted_blobs;\n     -+\tint trees_loaded;\n     -+\tint blobs_checked;\n     ++\tsize_t trees_loaded;\n     ++\tsize_t blobs_checked;\n      +\tint exclude_promisor_objects;\n     ++\n     ++\t/*\n     ++\t * Scratch space for reading one tree at a time.  It lives here\n     ++\t * rather than on the stack to keep the frame of the recursive\n     ++\t * verify_tree() small.\n     ++\t */\n     ++\tstruct tree_desc desc;\n     ++\tstruct name_entry entry;\n      +};\n      +\n      +/*\n     @@ tree-verify.c (new)\n      + * same-path parent subtrees as recursive comparison bases.\n      + * Returns the updated work-list cursor.\n      + */\n     -+static size_t collect_subtree_bases(struct work_item *work, size_t nr_work,\n     -+\t\t\t\t    size_t wi, const struct name_entry *entry)\n     ++static size_t collect_subtree_bases(struct work_list *list, size_t wi,\n     ++\t\t\t\t    const struct name_entry *entry)\n      +{\n     -+\twhile (wi < nr_work) {\n     -+\t\tint cmp = base_name_compare(\n     -+\t\t\twork[wi].entry.path, work[wi].entry.pathlen,\n     -+\t\t\twork[wi].entry.mode,\n     -+\t\t\tentry->path, entry->pathlen,\n     -+\t\t\tentry->mode);\n     ++\twhile (wi < list->nr) {\n     ++\t\tstruct work_item *item = &list->items[wi];\n     ++\t\tint cmp = base_name_compare(item->entry.path,\n     ++\t\t\t\t\t    item->entry.pathlen,\n     ++\t\t\t\t\t    item->entry.mode,\n     ++\t\t\t\t\t    entry->path, entry->pathlen,\n     ++\t\t\t\t\t    entry->mode);\n      +\t\tif (cmp > 0)\n      +\t\t\tbreak;\n      +\t\tif (cmp < 0) {\n      +\t\t\twi++;\n      +\t\t\tcontinue;\n      +\t\t}\n     -+\t\tif (S_ISDIR(work[wi].entry.mode) &&\n     -+\t\t    S_ISDIR(entry->mode))\n     -+\t\t\toid_array_append(&work[wi].parent_trees,\n     -+\t\t\t\t\t &entry->oid);\n     ++\t\tif (S_ISDIR(item->entry.mode) && S_ISDIR(entry->mode))\n     ++\t\t\toid_array_append(&item->parent_trees, &entry->oid);\n      +\t\treturn wi + 1;\n      +\t}\n      +\treturn wi;\n      +}\n      +\n     -+static void verify_blob(struct repository *repo,\n     -+\t\t\tconst struct object_id *oid,\n     -+\t\t\tstruct verify_state *vs)\n     ++/*\n     ++ * Report an object that could not be read, telling a missing object\n     ++ * apart from one that exists but cannot be read, e.g. because it is\n     ++ * corrupt.\n     ++ */\n     ++static int unreadable_object_error(struct repository *repo,\n     ++\t\t\t\t   const struct object_id *oid,\n     ++\t\t\t\t   const char *type)\n     ++{\n     ++\tif (!odb_has_object(repo->objects, oid, 0))\n     ++\t\treturn error(_(\"missing %s object %s\"), type, oid_to_hex(oid));\n     ++\treturn error(_(\"unable to read %s object %s\"), type, oid_to_hex(oid));\n     ++}\n     ++\n     ++static int verify_blob(struct repository *repo,\n     ++\t\t       const struct object_id *oid,\n     ++\t\t       struct verify_state *vs)\n      +{\n      +\tint type;\n      +\n      +\tif (oidset_contains(&vs->trusted_blobs, oid))\n     -+\t\treturn;\n     ++\t\treturn 0;\n      +\n      +\tvs->blobs_checked++;\n      +\ttype = odb_read_object_info(repo->objects, oid, NULL);\n      +\tif (type == OBJ_BLOB) {\n      +\t\toidset_insert(&vs->trusted_blobs, oid);\n     -+\t\treturn;\n     ++\t\treturn 0;\n      +\t}\n      +\tif (type >= 0)\n     -+\t\tdie(_(\"object %s is a %s, not a %s\"),\n     -+\t\t    oid_to_hex(oid), type_name(type), \"blob\");\n     ++\t\treturn error(_(\"object %s is a %s, not a %s\"),\n     ++\t\t\t     oid_to_hex(oid), type_name(type), \"blob\");\n      +\tif (vs->exclude_promisor_objects &&\n      +\t    is_promisor_object(repo, oid))\n     -+\t\treturn;\n     -+\tdie(_(\"missing blob object '%s'\"), oid_to_hex(oid));\n     ++\t\treturn 0;\n     ++\treturn unreadable_object_error(repo, oid, \"blob\");\n      +}\n      +\n     -+static void *read_tree_object(struct object_database *odb,\n     -+\t\t\t      const struct object_id *oid,\n     -+\t\t\t      size_t *sizep)\n     ++/*\n     ++ * Read a tree object into *bufp.  Returns -1 if the object exists\n     ++ * but is not a tree.  Otherwise returns 0, leaving *bufp NULL if the\n     ++ * object could not be read.\n     ++ */\n     ++static int read_tree_object(struct object_database *odb,\n     ++\t\t\t    const struct object_id *oid,\n     ++\t\t\t    void **bufp, size_t *sizep)\n      +{\n      +\tenum object_type type;\n     -+\tvoid *buf = odb_read_object(odb, oid, &type, sizep);\n      +\n     -+\tif (buf && type != OBJ_TREE) {\n     -+\t\tfree(buf);\n     -+\t\tdie(_(\"object %s is a %s, not a %s\"),\n     -+\t\t    oid_to_hex(oid), type_name(type), \"tree\");\n     ++\t*bufp = odb_read_object(odb, oid, &type, sizep);\n     ++\tif (*bufp && type != OBJ_TREE) {\n     ++\t\tFREE_AND_NULL(*bufp);\n     ++\t\treturn error(_(\"object %s is a %s, not a %s\"),\n     ++\t\t\t     oid_to_hex(oid), type_name(type), \"tree\");\n      +\t}\n     -+\treturn buf;\n     ++\treturn 0;\n      +}\n      +\n     -+static void verify_tree(struct repository *repo,\n     -+\t\t\tconst struct object_id *new_tree_oid,\n     -+\t\t\tconst struct oid_array *base_trees,\n     -+\t\t\tstruct verify_state *vs, int depth)\n     ++/* List the entries of a tree that are not yet known to be trusted. */\n     ++static void list_untrusted_entries(struct verify_state *vs,\n     ++\t\t\t\t   const struct object_id *tree_oid,\n     ++\t\t\t\t   void *tree_buf, size_t tree_size,\n     ++\t\t\t\t   struct work_list *list)\n      +{\n     -+\tstruct tree_desc desc;\n     -+\tstruct name_entry entry;\n     -+\tstruct work_item *work = NULL;\n     -+\tsize_t nr_work = 0, alloc_work = 0;\n     -+\tint need_subtree_bases = 0;\n     -+\tsize_t i, tree_size;\n     -+\tvoid *tree_buf;\n     -+\n     -+\tif (depth > repo->settings.max_allowed_tree_depth)\n     -+\t\tdie(_(\"exceeded maximum allowed tree depth\"));\n     -+\n     -+\tif (tree_map_get(vs->trees, new_tree_oid) >= TREE_TRUSTED)\n     -+\t\treturn;\n     -+\n     -+\ttree_buf = read_tree_object(repo->objects, new_tree_oid, &tree_size);\n     -+\tif (!tree_buf) {\n     -+\t\tif (vs->exclude_promisor_objects &&\n     -+\t\t    is_promisor_object(repo, new_tree_oid))\n     -+\t\t\treturn;\n     -+\t\tdie(_(\"bad tree object %s\"),\n     -+\t\t    oid_to_hex(new_tree_oid));\n     -+\t}\n     -+\n     -+\tvs->trees_loaded++;\n     -+\tinit_tree_desc(&desc, new_tree_oid, tree_buf, tree_size);\n     ++\tstruct name_entry *entry = &vs->entry;\n      +\n     -+\twhile (tree_entry(&desc, &entry)) {\n     -+\t\tif (S_ISGITLINK(entry.mode))\n     ++\tinit_tree_desc(&vs->desc, tree_oid, tree_buf, tree_size);\n     ++\twhile (tree_entry(&vs->desc, entry)) {\n     ++\t\tif (S_ISGITLINK(entry->mode))\n      +\t\t\tcontinue;\n     -+\t\tif (S_ISDIR(entry.mode)) {\n     -+\t\t\tif (tree_map_get(vs->trees, &entry.oid) >= TREE_TRUSTED)\n     ++\t\tif (S_ISDIR(entry->mode)) {\n     ++\t\t\tif (tree_map_get(vs->trees, &entry->oid) >=\n     ++\t\t\t    TREE_TRUSTED)\n      +\t\t\t\tcontinue;\n     -+\t\t\tneed_subtree_bases = 1;\n     ++\t\t\tlist->has_subtrees = true;\n      +\t\t} else {\n     -+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry.oid))\n     ++\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry->oid))\n      +\t\t\t\tcontinue;\n      +\t\t}\n     -+\t\tALLOC_GROW(work, nr_work + 1, alloc_work);\n     -+\t\tmemset(&work[nr_work], 0, sizeof(work[nr_work]));\n     -+\t\twork[nr_work].entry = entry;\n     -+\t\tnr_work++;\n     ++\t\tALLOC_GROW(list->items, list->nr + 1, list->alloc);\n     ++\t\tmemset(&list->items[list->nr], 0, sizeof(list->items[0]));\n     ++\t\tlist->items[list->nr++].entry = *entry;\n      +\t}\n     ++}\n     ++\n     ++/*\n     ++ * Scan one trusted base tree: expand it if that has not happened yet,\n     ++ * and collect its same-path subtrees as bases for untrusted subtrees.\n     ++ */\n     ++static int scan_base_tree(struct repository *repo, struct verify_state *vs,\n     ++\t\t\t  const struct object_id *base_oid,\n     ++\t\t\t  struct work_list *list)\n     ++{\n     ++\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n     ++\tstruct name_entry *entry = &vs->entry;\n     ++\tsize_t wi = 0, base_size;\n     ++\tvoid *base_buf;\n     ++\n     ++\tif (expanded && !list->has_subtrees)\n     ++\t\treturn 0;\n      +\n     -+\tif (!nr_work) {\n     -+\t\tfree(tree_buf);\n     -+\t\tgoto done;\n     ++\tif (read_tree_object(repo->objects, base_oid, &base_buf, &base_size))\n     ++\t\treturn -1;\n     ++\tif (!base_buf) {\n     ++\t\tif (vs->exclude_promisor_objects &&\n     ++\t\t    is_promisor_object(repo, base_oid))\n     ++\t\t\treturn 0;\n     ++\t\treturn unreadable_object_error(repo, base_oid, \"tree\");\n      +\t}\n      +\n     -+\tfor (i = 0; base_trees && i < base_trees->nr; i++) {\n     -+\t\tconst struct object_id *base_oid = &base_trees->oid[i];\n     -+\t\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n     -+\t\tstruct tree_desc base_desc;\n     -+\t\tstruct name_entry scan_entry;\n     -+\t\tsize_t wi = 0, base_size;\n     -+\t\tvoid *base_buf;\n     ++\tvs->trees_loaded++;\n     ++\tinit_tree_desc(&vs->desc, base_oid, base_buf, base_size);\n      +\n     -+\t\tif (expanded && !need_subtree_bases)\n     ++\twhile (tree_entry(&vs->desc, entry)) {\n     ++\t\tif (S_ISGITLINK(entry->mode))\n      +\t\t\tcontinue;\n      +\n     -+\t\tbase_buf = read_tree_object(repo->objects, base_oid,\n     -+\t\t\t\t\t    &base_size);\n     -+\t\tif (!base_buf) {\n     -+\t\t\tif (vs->exclude_promisor_objects &&\n     -+\t\t\t    is_promisor_object(repo, base_oid))\n     -+\t\t\t\tcontinue;\n     -+\t\t\tdie(_(\"bad tree object %s\"),\n     -+\t\t\t    oid_to_hex(base_oid));\n     ++\t\t/* Same-path subtree as a base for an untrusted one. */\n     ++\t\tif (list->has_subtrees)\n     ++\t\t\twi = collect_subtree_bases(list, wi, entry);\n     ++\n     ++\t\t/* Expand: the base is trusted, so are its entries. */\n     ++\t\tif (!expanded) {\n     ++\t\t\tif (S_ISDIR(entry->mode))\n     ++\t\t\t\ttree_map_add(vs->trees, &entry->oid,\n     ++\t\t\t\t\t     TREE_TRUSTED);\n     ++\t\t\telse\n     ++\t\t\t\toidset_insert(&vs->trusted_blobs,\n     ++\t\t\t\t\t      &entry->oid);\n      +\t\t}\n     ++\t}\n      +\n     -+\t\tvs->trees_loaded++;\n     -+\t\tinit_tree_desc(&base_desc, base_oid, base_buf, base_size);\n     ++\tif (!expanded)\n     ++\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n      +\n     -+\t\twhile (tree_entry(&base_desc, &scan_entry)) {\n     -+\t\t\tif (S_ISGITLINK(scan_entry.mode))\n     -+\t\t\t\tcontinue;\n     ++\tfree(base_buf);\n     ++\treturn 0;\n     ++}\n      +\n     -+\t\t\tif (need_subtree_bases)\n     -+\t\t\t\twi = collect_subtree_bases(work, nr_work,\n     -+\t\t\t\t\t\t\t   wi, &scan_entry);\n     -+\n     -+\t\t\tif (!expanded) {\n     -+\t\t\t\tif (S_ISDIR(scan_entry.mode))\n     -+\t\t\t\t\ttree_map_add(vs->trees,\n     -+\t\t\t\t\t\t     &scan_entry.oid,\n     -+\t\t\t\t\t\t     TREE_TRUSTED);\n     -+\t\t\t\telse\n     -+\t\t\t\t\toidset_insert(&vs->trusted_blobs,\n     -+\t\t\t\t\t\t      &scan_entry.oid);\n     -+\t\t\t}\n     -+\t\t}\n     ++/*\n     ++ * Verify a tree against the trusted base trees, loading as little as\n     ++ * possible:\n     ++ *\n     ++ * First list the entries that are not yet known to be trusted.  When\n     ++ * the parent trees have been expanded before, only the changed entries\n     ++ * remain.\n     ++ *\n     ++ * Then scan the trusted base trees (the same-path trees of the parent\n     ++ * commits).  A base that has not been expanded yet is read so that its\n     ++ * entries can be marked trusted; this often shows that listed entries\n     ++ * are trusted after all.  An already expanded base is still read when\n     ++ * there are untrusted subtrees, because its subtree at the same path is\n     ++ * what lets the untrusted subtree skip its own unchanged entries in\n     ++ * turn.\n     ++ *\n     ++ * Finally check that the remaining blobs exist, and recurse into the\n     ++ * remaining subtrees with the same-path subtrees as their bases.\n     ++ */\n     ++static int verify_tree(struct repository *repo,\n     ++\t\t       const struct object_id *tree_oid,\n     ++\t\t       const struct oid_array *trusted_bases,\n     ++\t\t       struct verify_state *vs, int depth)\n     ++{\n     ++\tstruct work_list list = { 0 };\n     ++\tsize_t i, tree_size;\n     ++\tvoid *tree_buf;\n     ++\tint ret = 0;\n     ++\n     ++\t/*\n     ++\t * Enforce core.maxTreeDepth like process_tree() in list-objects.c,\n     ++\t * so that the full and incremental checks reject the same trees.\n     ++\t */\n     ++\tif (depth > repo->settings.max_allowed_tree_depth)\n     ++\t\treturn error(_(\"exceeded maximum allowed tree depth\"));\n     ++\n     ++\tif (tree_map_get(vs->trees, tree_oid) >= TREE_TRUSTED)\n     ++\t\treturn 0;\n      +\n     -+\t\tif (!expanded)\n     -+\t\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n     ++\tif (read_tree_object(repo->objects, tree_oid, &tree_buf, &tree_size))\n     ++\t\treturn -1;\n     ++\tif (!tree_buf) {\n     ++\t\t/*\n     ++\t\t * A missing tree that is promised can be fetched on demand,\n     ++\t\t * so it is not an error, as in the full check.\n     ++\t\t */\n     ++\t\tif (vs->exclude_promisor_objects &&\n     ++\t\t    is_promisor_object(repo, tree_oid))\n     ++\t\t\treturn 0;\n     ++\t\treturn unreadable_object_error(repo, tree_oid, \"tree\");\n     ++\t}\n     ++\n     ++\tvs->trees_loaded++;\n     ++\tlist_untrusted_entries(vs, tree_oid, tree_buf, tree_size, &list);\n      +\n     -+\t\tfree(base_buf);\n     ++\t/* Scan the base trees, unless nothing is left to verify. */\n     ++\tfor (i = 0; list.nr && i < trusted_bases->nr; i++) {\n     ++\t\tret = scan_base_tree(repo, vs, &trusted_bases->oid[i], &list);\n     ++\t\tif (ret)\n     ++\t\t\tgoto done;\n      +\t}\n      +\n     -+\tfor (i = 0; i < nr_work; i++) {\n     -+\t\tif (S_ISDIR(work[i].entry.mode))\n     -+\t\t\tverify_tree(repo, &work[i].entry.oid,\n     -+\t\t\t\t    &work[i].parent_trees, vs,\n     -+\t\t\t\t    depth + 1);\n     ++\t/*\n     ++\t * Check the remaining entries in tree order.  Verifying a subtree\n     ++\t * can show that later entries are trusted, which then makes them\n     ++\t * cheap to skip.\n     ++\t */\n     ++\tfor (i = 0; i < list.nr && !ret; i++) {\n     ++\t\tstruct work_item *item = &list.items[i];\n     ++\n     ++\t\tif (S_ISDIR(item->entry.mode))\n     ++\t\t\tret = verify_tree(repo, &item->entry.oid,\n     ++\t\t\t\t\t  &item->parent_trees, vs, depth + 1);\n      +\t\telse\n     -+\t\t\tverify_blob(repo, &work[i].entry.oid, vs);\n     ++\t\t\tret = verify_blob(repo, &item->entry.oid, vs);\n      +\t}\n     -+\n     -+\tfree(tree_buf);\n     ++\tif (!ret)\n     ++\t\ttree_map_add(vs->trees, tree_oid, TREE_EXPANDED);\n      +\n      +done:\n     -+\ttree_map_add(vs->trees, new_tree_oid, TREE_EXPANDED);\n     -+\tfor (i = 0; i < nr_work; i++)\n     -+\t\toid_array_clear(&work[i].parent_trees);\n     -+\tfree(work);\n     ++\tfree(tree_buf);\n     ++\twork_list_clear(&list);\n     ++\treturn ret;\n      +}\n      +\n     -+static void verify_commit_tree(struct repository *repo,\n     -+\t\t\t       struct commit *commit,\n     -+\t\t\t       struct verify_state *vs)\n     ++static int verify_commit_tree(struct repository *repo,\n     ++\t\t\t      struct commit *commit,\n     ++\t\t\t      struct verify_state *vs)\n      +{\n     -+\tstruct oid_array base_trees = OID_ARRAY_INIT;\n     ++\tstruct oid_array parent_trees = OID_ARRAY_INIT;\n      +\tconst struct object_id *tree_oid;\n      +\tstruct commit_list *p;\n     ++\tint ret;\n      +\n      +\t/*\n      +\t * Parent trees are trusted: boundary parents are already\n      +\t * connected, and earlier incoming parents were verified\n      +\t * first due to the topological processing order.\n     ++\t *\n     ++\t * The revision walk skips missing parents that are promised\n     ++\t * (see process_parents()), so do the same here.  Such a parent\n     ++\t * contributes no base, which only means less of the new tree\n     ++\t * can be skipped.\n      +\t */\n      +\tfor (p = commit->parents; p; p = p->next) {\n     -+\t\tparse_commit_or_die(p->item);\n     ++\t\tif (repo_parse_commit_gently(repo, p->item, 1) < 0) {\n     ++\t\t\tif (vs->exclude_promisor_objects &&\n     ++\t\t\t    is_promisor_object(repo, &p->item->object.oid))\n     ++\t\t\t\tcontinue;\n     ++\t\t\tret = error(_(\"unable to parse commit %s\"),\n     ++\t\t\t\t    oid_to_hex(&p->item->object.oid));\n     ++\t\t\tgoto out;\n     ++\t\t}\n      +\t\ttree_oid = get_commit_tree_oid(p->item);\n     -+\t\tif (!tree_oid)\n     -+\t\t\tdie(_(\"unable to load root tree for commit %s\"),\n     -+\t\t\t    oid_to_hex(&p->item->object.oid));\n     ++\t\tif (!tree_oid) {\n     ++\t\t\tret = error(_(\"unable to load root tree for commit %s\"),\n     ++\t\t\t\t    oid_to_hex(&p->item->object.oid));\n     ++\t\t\tgoto out;\n     ++\t\t}\n      +\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n     -+\t\toid_array_append(&base_trees, tree_oid);\n     ++\t\toid_array_append(&parent_trees, tree_oid);\n      +\t}\n      +\n      +\ttree_oid = get_commit_tree_oid(commit);\n     -+\tif (!tree_oid)\n     -+\t\tdie(_(\"unable to load root tree for commit %s\"),\n     -+\t\t    oid_to_hex(&commit->object.oid));\n     -+\tverify_tree(repo, tree_oid, &base_trees, vs, 0);\n     -+\toid_array_clear(&base_trees);\n     ++\tif (!tree_oid) {\n     ++\t\tret = error(_(\"unable to load root tree for commit %s\"),\n     ++\t\t\t    oid_to_hex(&commit->object.oid));\n     ++\t\tgoto out;\n     ++\t}\n     ++\tret = verify_tree(repo, tree_oid, &parent_trees, vs, 0);\n     ++out:\n     ++\toid_array_clear(&parent_trees);\n     ++\treturn ret;\n      +}\n      +\n     -+void verify_commits_incremental(struct repository *repo,\n     -+\t\t\t\tstruct commit_list **commits,\n     -+\t\t\t\tint exclude_promisor_objects)\n     ++int verify_commits_incremental(struct repository *repo,\n     ++\t\t\t       struct commit_list **commits,\n     ++\t\t\t       int exclude_promisor_objects)\n      +{\n      +\tstruct verify_state vs = { 0 };\n      +\tstruct commit_list *iter;\n      +\tunsigned nr_before;\n     ++\tint ret = 0;\n      +\n      +\tif (repo->fetch_if_missing)\n      +\t\tBUG(\"verify_commits_incremental must not be called \"\n     @@ tree-verify.c (new)\n      +\t */\n      +\tnr_before = commit_list_count(*commits);\n      +\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n     -+\tif (commit_list_count(*commits) < nr_before)\n     -+\t\tdie(_(\"cycle detected in incoming commit graph\"));\n     ++\tif (commit_list_count(*commits) < nr_before) {\n     ++\t\tret = error(_(\"cycle detected in incoming commit graph\"));\n     ++\t\tgoto out;\n     ++\t}\n      +\n      +\t*commits = commit_list_reverse(*commits);\n      +\n     -+\tfor (iter = *commits; iter; iter = iter->next)\n     -+\t\tverify_commit_tree(repo, iter->item, &vs);\n     ++\tfor (iter = *commits; iter && !ret; iter = iter->next)\n     ++\t\tret = verify_commit_tree(repo, iter->item, &vs);\n      +\n     ++out:\n      +\tkh_destroy_oid_tree(vs.trees);\n      +\toidset_clear(&vs.trusted_blobs);\n      +\ttrace2_data_intmax(\"connectivity\", repo,\n     -+\t\t\t   \"trees_loaded\", vs.trees_loaded);\n     ++\t\t\t   \"trees_loaded\", (intmax_t)vs.trees_loaded);\n      +\ttrace2_data_intmax(\"connectivity\", repo,\n     -+\t\t\t   \"blobs_checked\", vs.blobs_checked);\n     ++\t\t\t   \"blobs_checked\", (intmax_t)vs.blobs_checked);\n     ++\treturn ret;\n      +}\n      \n       ## tree-verify.h (new) ##\n     @@ tree-verify.h (new)\n      +\n      +/*\n      + * Verify trees of commits incrementally against their parents.\n     -+ * Dies on verification failure.\n     ++ * Returns 0 on success, or reports an error and returns -1.\n      + */\n     -+void verify_commits_incremental(struct repository *repo,\n     -+\t\t\t\tstruct commit_list **commits,\n     -+\t\t\t\tint exclude_promisor_objects);\n     ++int verify_commits_incremental(struct repository *repo,\n     ++\t\t\t       struct commit_list **commits,\n     ++\t\t\t       int exclude_promisor_objects);\n      +\n      +#endif /* TREE_VERIFY_H */\n -:  ---------- > 6:  f6031967e6 connected: add \"auto\" connectivity check mode\n\n-- \ngitgitgadget\n\n"},{"id":"555716","messageId":"e25824c21d0dbc6161da81cae84cd755fc0e6fec.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 1/6] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:19Z","receivedAt":"2026-10-21T11:22:19Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd Documentation/technical/connectivity-check.adoc describing\nthe connectivity invariant and the full connectivity check.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/Makefile                        |   1 +\n .../technical/connectivity-check.adoc         | 112 ++++++++++++++++++\n Documentation/technical/meson.build           |   1 +\n 3 files changed, 114 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex f8dea4b395..1747f4a00b 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -122,6 +122,7 @@ TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/build-systems\n TECH_DOCS += technical/bundle-uri\n TECH_DOCS += technical/commit-graph\n+TECH_DOCS += technical/connectivity-check\n TECH_DOCS += technical/directory-rename-detection\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/large-object-promisors\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nnew file mode 100644\nindex 0000000000..a112d5be3e\n--- /dev/null\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -0,0 +1,112 @@\n+Connectivity checking\n+=====================\n+\n+After receiving new objects via fetch, push (receive-pack), clone,\n+or bundle, Git verifies that the new reference tips do not leave\n+the repository in a state where reachable objects are missing.\n+This verification is called the connectivity check.\n+\n+Connectivity invariant\n+----------------------\n+\n+A repository is connected when every object reachable from its\n+references is available locally (with exceptions noted below).\n+The reverse does not hold: objects that exist in the object\n+database but are not reachable from any reference may be\n+incomplete or lack a connected closure.\n+\n+The connectivity check maintains this invariant when references\n+are updated.  It trusts that all objects reachable from existing\n+reference tips are already fully connected and verifies that the\n+new reference tips do not introduce references to unavailable\n+objects.  Verification is permitted to stop when it reaches\n+objects already reachable from existing references, since their\n+closure is already connected.  These trusted references include\n+local references and references from alternate object stores.\n+\n+Without this check, a truncated or corrupted transfer could leave\n+a repository in a state where later history walks encounter\n+missing objects.\n+\n+Exceptions\n+~~~~~~~~~~\n+\n+Gitlink entries (submodule references) are excluded from\n+connectivity checking.  Their target objects belong to a separate\n+repository.\n+\n+In partial clones, objects promised by a promisor remote are\n+accepted as connected without requiring local existence.  The\n+check excludes promisor objects from traversal so that it does\n+not trigger on-demand fetches for them.\n+\n+Full connectivity check\n+-----------------------\n+\n+The full connectivity check runs a `rev-list` subprocess, feeding\n+the new reference tips via stdin.  A normal invocation is roughly:\n+\n+    git rev-list --objects --stdin --not --all --quiet\n+        --alternate-refs [--exclude-promisor-objects]\n+\n+In a partial clone, objects received from a promisor remote do\n+not need to be checked: anything they reference is promised by\n+that remote and can be fetched on demand.  So if every new tip is\n+found in a promisor packfile, `rev-list` is not run at all.\n+Otherwise it runs with `--exclude-promisor-objects`, so that the\n+traversal stops at promisor objects instead of fetching them.\n+\n+Consider the following graph after a fetch, where all reference\n+tips point directly to commits.  For simplicity, only local\n+references appear on the already-connected side; alternate refs\n+play the same role.  N3 is a merge commit:\n+\n+            /-------------L2\n+           /\n+    C1---B1---C2---B2-----L1\n+          \\         \\\n+           N1        N3---T2\n+            \\       /\n+             N2-----------T1\n+\n+    L1, L2:         local refs\n+    T1, T2:         incoming tips (new refs)\n+    N1, N2, N3:     incoming commits (N3 is a merge)\n+    B1, B2:         boundary commits (already connected)\n+    C1, C2:         already connected (but not boundary)\n+\n+The incoming set is the commits reachable from the incoming\n+tips but not from the already-connected side.  Boundary commits\n+are the already-connected commits at the edge of that set.  Here\n+B1 is an ancestor of B2, which happens when incoming branches\n+fork at different depths in the existing history.\n+\n+The `--not --all` arguments establish the trust boundary: the\n+incoming set is the commits reachable from the new tips but not\n+from any existing reference.\n+\n+The traversal then walks the trees of the boundary commits and\n+marks those objects uninteresting.  Trees of incoming commits\n+are then walked, stopping at objects already marked as\n+uninteresting.  The cost of marking boundary trees uninteresting\n+dominates when the boundary commits have large trees, even when\n+the incoming changes are small.\n+\n+Deepening fetches\n+~~~~~~~~~~~~~~~~~\n+\n+For deepening fetches (where the shallow boundary moves), the\n+full check omits `--not --all`.  There is no existing-reference\n+boundary at which the walk can stop.  Instead, traversal follows\n+the effective shallow boundary supplied for the deepened\n+repository.  The new content may be below the old shallow\n+boundary even when the tips themselves have not changed.\n+\n+Non-commit tips\n+~~~~~~~~~~~~~~~\n+\n+When a new reference points to a non-commit object, such as a\n+tag, tree, or blob, that object is not part of the commit walk.\n+These non-commit tips are verified during the tree and blob\n+traversal that follows the commit walk within the same rev-list\n+invocation.\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex 9ce11d5e48..c439a7a072 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -11,6 +11,7 @@ articles = [\n   'build-systems.adoc',\n   'bundle-uri.adoc',\n   'commit-graph.adoc',\n+  'connectivity-check.adoc',\n   'directory-rename-detection.adoc',\n   'hash-function-transition.adoc',\n   'large-object-promisors.adoc',\n-- \ngitgitgadget\n\n\n"},{"id":"555717","messageId":"427b7315a5167f4b5546a08c851a4aa222fcc2f6.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 2/6] p5413: add perf tests for connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:20Z","receivedAt":"2026-10-21T11:22:20Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd benchmarks for the rev-list connectivity check.  Each fixture\nvaries one dimension: total tree size, incoming commit count, or\nfiles changed per commit.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n ...enerate-repo-p5413-connectivity-check.perl | 44 ++++++++++\n t/perf/p5413-connectivity-check.sh            | 85 +++++++++++++++++++\n 2 files changed, 129 insertions(+)\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n\ndiff --git a/t/perf/generate-repo-p5413-connectivity-check.perl b/t/perf/generate-repo-p5413-connectivity-check.perl\nnew file mode 100644\nindex 0000000000..bfa25f60fc\n--- /dev/null\n+++ b/t/perf/generate-repo-p5413-connectivity-check.perl\n@@ -0,0 +1,44 @@\n+#!/usr/bin/perl\n+#\n+# Generate a fast-import stream for p5413 connectivity check benchmarks.\n+#\n+# Usage: generate-repo-p5413-connectivity-check.perl\n+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n+#\n+# Creates one initial commit with dirs*files_per_dir files, then\n+# <commits> additional commits each modifying <files_per_commit>\n+# files in directories chosen round-robin from 1..<hot_dirs>.\n+\n+use strict;\n+use warnings;\n+\n+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n+$hot = $nd if !$hot || $hot > $nd;\n+$fpc = 1   if !$fpc;\n+\n+sub data {\n+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n+}\n+\n+# Initial tree: one commit with nd*nf files.\n+printf \"commit refs/heads/main\\n\";\n+printf \"committer perf <perf\\@test.com> now\\n\";\n+data(\"initial\");\n+for my $d (1..$nd) {\n+\tfor my $f (1..$nf) {\n+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n+\t}\n+}\n+\n+# Subsequent commits (auto-chained by fast-import).\n+for my $i (1..$nc) {\n+\tprintf \"commit refs/heads/main\\n\";\n+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n+\tdata(sprintf \"change-%03d\", $i);\n+\tfor my $j (0..$fpc-1) {\n+\t\tmy $d = (($i + $j) % $hot) + 1;\n+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n+\t}\n+}\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..9e968d0106\n--- /dev/null\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+\n+test_description='performance of connectivity check\n+\n+Benchmark the rev-list connectivity check, avoiding pack transfer\n+noise.\n+\n+Each repository has a flat tree of many directories with 100 files\n+in each.  Three axes are scaled independently: tree size, commit\n+count, and files changed per commit.'\n+\n+. ./perf-lib.sh\n+\n+test_perf_fresh_repo\n+\n+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5413-connectivity-check.perl\"\n+\n+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n+# $5=files_per_commit (optional, default=1)\n+test_perf_conn () {\n+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n+\tlocal total=$(($nd * $nf))\n+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n+\tlocal label=\"${total} files, ${nc} commits\"\n+\tif test \"$hot\" -lt \"$nd\"\n+\tthen\n+\t\tlabel=\"$label (${hot} hot dirs)\"\n+\tfi\n+\tif test \"$fpc\" -gt 1\n+\tthen\n+\t\tlabel=\"$label (${fpc} files/commit)\"\n+\tfi\n+\n+\ttest_expect_success PERL \"setup $label\" '\n+\t\tif test -d '\"$name\"'\n+\t\tthen\n+\t\t\ttrue\n+\t\telse\n+\t\t\tgit init '\"$name\"' &&\n+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n+\t\t\t(\n+\t\t\t\tcd '\"$name\"' &&\n+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n+\t\t\t\tgit update-ref refs/heads/main \\\n+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n+\t\t\t\tgit repack -ad &&\n+\t\t\t\tgit config gc.auto 0\n+\t\t\t)\n+\t\tfi\n+\t'\n+\n+\ttest_perf \"$label\" --prereq PERL '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+}\n+\n+# Scaling tree size (10 commits, 10 files/commit).\n+test_perf_conn   50 100 10   50 10\n+test_perf_conn  500 100 10  500 10\n+test_perf_conn 2000 100 10 2000 10\n+test_perf_conn 8000 100 10 8000 10\n+\n+# Scaling commit count (200K files, 10 files/commit).\n+test_perf_conn 2000 100    1 2000 10\n+test_perf_conn 2000 100   10 2000 10\n+test_perf_conn 2000 100  100 2000 10\n+test_perf_conn 2000 100   500 2000 10\n+test_perf_conn 2000 100  3000 2000 10\n+test_perf_conn 2000 100  5000 2000 10\n+test_perf_conn 2000 100 10000 2000 10\n+\n+# Scaling files per commit (200K files, 10 commits).\n+test_perf_conn 2000 100 10 2000   1\n+test_perf_conn 2000 100 10 2000  10\n+test_perf_conn 2000 100 10 2000  100\n+test_perf_conn 2000 100 10 2000  500\n+test_perf_conn 2000 100 10 2000 1000\n+test_perf_conn 2000 100 10 2000 2000\n+\n+test_done\n-- \ngitgitgadget\n\n\n"},{"id":"555718","messageId":"3a7a1fb95320ed7226d00b5ff759cf88dedf5de0.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 3/6] connected: drop dependency on `the_repository`","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:21Z","receivedAt":"2026-10-21T11:22:21Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe next commits will need to use the repository to read configuration\nin check_connected(). Rather than accessing `the_repository`, refactor\nthe code to instead require the caller to pass in a repository so that\nwe can remove `USE_THE_REPOSITORY_VARIABLE`.\n\nSuggested-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n builtin/clone.c        |  2 +-\n builtin/fetch.c        |  5 +++--\n builtin/receive-pack.c | 12 ++++++++----\n bundle.c               |  2 +-\n connected.c            | 22 +++++++++++-----------\n connected.h            |  4 +++-\n fetch-pack.c           |  3 ++-\n 7 files changed, 29 insertions(+), 21 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 44037dd428..1532361ea0 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -572,7 +572,7 @@ static void update_remote_refs(const struct ref *refs,\n \t\topt.transport = transport;\n \t\topt.progress = transport->progress;\n \n-\t\tif (check_connected(iterate_ref_map, &rm, &opt))\n+\t\tif (check_connected(the_repository, iterate_ref_map, &rm, &opt))\n \t\t\tdie(_(\"remote did not send all necessary objects\"));\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex b2decc6cfd..0ef7fe92f1 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1250,7 +1250,8 @@ static int store_updated_refs(struct display_state *display_state,\n \n \t\topt.exclude_hidden_refs_section = \"fetch\";\n \t\trm = ref_map;\n-\t\tif (check_connected(iterate_ref_map, &rm, &opt)) {\n+\t\tif (check_connected(the_repository, iterate_ref_map,\n+\t\t\t\t    &rm, &opt)) {\n \t\t\trc = error(_(\"%s did not send all necessary objects\"),\n \t\t\t\t   display_state->url);\n \t\t\tgoto abort;\n@@ -1422,7 +1423,7 @@ static int check_exist_and_connected(struct ref *ref_map)\n \n \topt.quiet = 1;\n \topt.exclude_hidden_refs_section = \"fetch\";\n-\treturn check_connected(iterate_ref_map, &rm, &opt);\n+\treturn check_connected(the_repository, iterate_ref_map, &rm, &opt);\n }\n \n static int fetch_and_consume_refs(struct display_state *display_state,\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 1d5b050beb..4f8f58950a 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -1385,7 +1385,8 @@ static int update_shallow_ref(struct command *cmd, struct shallow_info *si)\n \t\t\toid_array_append(&extra, &si->shallow->oid[i]);\n \n \tsetup_alternate_shallow(&shallow_lock, &opt.shallow_file, &extra);\n-\tif (check_connected(command_singleton_iterator, cmd, &opt)) {\n+\tif (check_connected(the_repository, command_singleton_iterator,\n+\t\t\t    cmd, &opt)) {\n \t\trollback_shallow_file(the_repository, &shallow_lock);\n \t\toid_array_clear(&extra);\n \t\treturn -1;\n@@ -1827,8 +1828,9 @@ static void set_connectivity_errors(struct command *commands,\n \t\todb_transaction_env(transaction, &env);\n \t\topt.env = env.v;\n \n-\t\tif (!check_connected(command_singleton_iterator, &singleton,\n-\t\t\t\t     &opt))\n+\t\tif (!check_connected(the_repository,\n+\t\t\t\t     command_singleton_iterator,\n+\t\t\t\t     &singleton, &opt))\n \t\t\tcontinue;\n \n \t\tcmd->error_string = \"missing necessary objects\";\n@@ -2090,7 +2092,9 @@ static void execute_commands(struct command *commands,\n \t\topt.env = env.v;\n \t\topt.exclude_hidden_refs_section = \"receive\";\n \n-\t\tif (check_connected(iterate_receive_command_list, &data, &opt))\n+\t\tif (check_connected(the_repository,\n+\t\t\t\t    iterate_receive_command_list,\n+\t\t\t\t    &data, &opt))\n \t\t\tset_connectivity_errors(commands, si, transaction);\n \n \t\tif (use_sideband)\ndiff --git a/bundle.c b/bundle.c\nindex f55a521b2a..0a68e856f5 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -259,7 +259,7 @@ int verify_bundle(struct repository *r,\n \tif (ret)\n \t\tgoto cleanup;\n \n-\tif ((ret = check_connected(iterate_ref_map, &iter, &opts)))\n+\tif ((ret = check_connected(r, iterate_ref_map, &iter, &opts)))\n \t\terror(_(\"some prerequisite commits exist in the object store, \"\n \t\t\t\"but are not connected to the repository's history\"));\n \ndiff --git a/connected.c b/connected.c\nindex 929b9bd28d..34045f1032 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,5 +1,3 @@\n-#define USE_THE_REPOSITORY_VARIABLE\n-\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n@@ -35,23 +33,24 @@ static int promised_object_cb(const struct object_id *oid UNUSED,\n  * objects in non-promisor packs, in which case we'll have to fall back to the\n  * rev-list-based connectivity checks. Returns a negative error code on error.\n  */\n-static int check_connected_promisor(oid_iterate_fn fn,\n+static int check_connected_promisor(struct repository *repo,\n+\t\t\t\t    oid_iterate_fn fn,\n \t\t\t\t    void *cb_data,\n \t\t\t\t    const struct object_id **oid)\n {\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = ODB_FOR_EACH_OBJECT_PROMISOR_ONLY,\n-\t\t.prefix_hex_len = the_repository->hash_algo->hexsz,\n+\t\t.prefix_hex_len = repo->hash_algo->hexsz,\n \t};\n \tint err;\n \n-\todb_reprepare(the_repository->objects);\n+\todb_reprepare(repo->objects);\n \tdo {\n \t\tbool found = false;\n \n \t\topts.prefix = *oid;\n \n-\t\terr = odb_for_each_object_ext(the_repository->objects, NULL,\n+\t\terr = odb_for_each_object_ext(repo->objects, NULL,\n \t\t\t\t\t      promised_object_cb, &found, &opts);\n \t\tif (err < 0)\n \t\t\treturn err;\n@@ -78,7 +77,8 @@ static int check_connected_promisor(oid_iterate_fn fn,\n  *\n  * Returns 0 if everything is connected, non-zero otherwise.\n  */\n-int check_connected(oid_iterate_fn fn, void *cb_data,\n+int check_connected(struct repository *repo,\n+\t\t    oid_iterate_fn fn, void *cb_data,\n \t\t    struct check_connected_options *opt)\n {\n \tstruct child_process rev_list = CHILD_PROCESS_INIT;\n@@ -101,8 +101,8 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \t\treturn err;\n \t}\n \n-\tif (repo_has_promisor_remote(the_repository)) {\n-\t\terr = check_connected_promisor(fn, cb_data, &oid);\n+\tif (repo_has_promisor_remote(repo)) {\n+\t\terr = check_connected_promisor(repo, fn, cb_data, &oid);\n \t\tif (err) {\n \t\t\tif (opt->err_fd)\n \t\t\t\tclose(opt->err_fd);\n@@ -119,7 +119,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \tstrvec_push(&rev_list.args,\"rev-list\");\n \tstrvec_push(&rev_list.args, \"--objects\");\n \tstrvec_push(&rev_list.args, \"--stdin\");\n-\tif (repo_has_promisor_remote(the_repository))\n+\tif (repo_has_promisor_remote(repo))\n \t\tstrvec_push(&rev_list.args, \"--exclude-promisor-objects\");\n \tif (!opt->is_deepening_fetch) {\n \t\tstrvec_push(&rev_list.args, \"--not\");\n@@ -160,7 +160,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \t\tstrbuf_add(&idx_file, transport->pack_lockfiles.items[0].string,\n \t\t\t   base_len);\n \t\tstrbuf_addstr(&idx_file, \".idx\");\n-\t\tnew_pack = add_packed_git(the_repository, idx_file.buf,\n+\t\tnew_pack = add_packed_git(repo, idx_file.buf,\n \t\t\t\t\t  idx_file.len, 1);\n \t\tstrbuf_release(&idx_file);\n \t}\ndiff --git a/connected.h b/connected.h\nindex 16b2c84f2e..0624965baa 100644\n--- a/connected.h\n+++ b/connected.h\n@@ -2,6 +2,7 @@\n #define CONNECTED_H\n \n struct object_id;\n+struct repository;\n struct transport;\n \n /*\n@@ -66,7 +67,8 @@ struct check_connected_options {\n  *\n  * If \"opt\" is NULL, behaves as if CHECK_CONNECTED_INIT was passed.\n  */\n-int check_connected(oid_iterate_fn fn, void *cb_data,\n+int check_connected(struct repository *repo,\n+\t\t    oid_iterate_fn fn, void *cb_data,\n \t\t    struct check_connected_options *opt);\n \n #endif /* CONNECTED_H */\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 30a4dd6119..2cf8f8d653 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -2186,7 +2186,8 @@ struct ref *fetch_pack(struct fetch_pack_args *args,\n \t\topt.shallow_file = alternate_shallow_file;\n \t\tif (args->deepen)\n \t\t\topt.is_deepening_fetch = 1;\n-\t\tif (check_connected(iterate_ref_map, &iterator, &opt)) {\n+\t\tif (check_connected(the_repository, iterate_ref_map,\n+\t\t\t\t    &iterator, &opt)) {\n \t\t\terror(_(\"remote did not send all necessary objects\"));\n \t\t\tfree_refs(ref_cpy);\n \t\t\tref_cpy = NULL;\n-- \ngitgitgadget\n\n\n"},{"id":"555719","messageId":"87dbb1f055ded59c640b46af043d647119fe6550.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 4/6] connected: add transfer.connectivityCheck config","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:22Z","receivedAt":"2026-10-21T11:22:22Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd the transfer.connectivityCheck configuration option to choose\nthe algorithm used for the connectivity check.  For now the only\naccepted value is \"full\", which is the existing behavior and the\ndefault.  Unknown values fall back to \"full\" with a warning, so\nthat configuration written for a newer version of Git does not\nbreak fetches and pushes; \"full\" is always a correct check.\n\ncheck_connected() now always tells its rev-list subprocess which\nalgorithm to use with an internal --connectivity-check=<mode>\noption, which so far only accepts \"full\".  Passing the mode\nexplicitly means rev-list knows it is running a connectivity\ncheck, so later optimizations can be limited to that case.\n\nThis prepares for an incremental check added in the next commit.\n\nSuggested-by: Patrick Steinhardt <ps@pks.im>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc  | 16 ++++++++++++++++\n Documentation/rev-list-options.adoc |  5 +++++\n builtin/rev-list.c                  | 23 +++++++++++++++++++++++\n connected.c                         | 26 ++++++++++++++++++++++++++\n 4 files changed, 70 insertions(+)\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex f1ce50f4a6..811456f1bf 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -1,3 +1,19 @@\n+transfer.connectivityCheck::\n+\tChoose which algorithm to use for the connectivity check\n+\tperformed during object transfer operations such as\n+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n+\tThe connectivity check verifies that all objects reachable\n+\tfrom the incoming tips are available locally or, in a partial\n+\tclone, promised by a promisor remote.\n+\tThe variants are as follows:\n++\n+--\n+`full` (default);;\n+\tWalk the full object closure of the boundary commits.\n+--\n++\n+Unknown values are treated as `full`, with a warning.\n+\n transfer.credentialsInUrl::\n \tA configured URL can contain plaintext credentials in the form\n \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex bd9f345690..3979a142a5 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1102,6 +1102,11 @@ It is an error to use `--missing-only` without `--missing=print` or\n \tstronger than `--missing=allow-promisor` because it limits the\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n+\n+`--connectivity-check=full`::\n+\t(For internal use only.)  Used by `check_connected()` to\n+\tselect the connectivity check algorithm; see\n+\t`transfer.connectivityCheck` in linkgit:git-config[1].\n endif::git-rev-list[]\n \n `--no-walk[=(sorted|unsorted)]`::\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 0faa833fac..810eaa15e3 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -29,6 +29,24 @@\n #include \"quote.h\"\n #include \"strbuf.h\"\n \n+enum connectivity_check_mode {\n+\tCONNECTIVITY_CHECK_FULL,\n+};\n+\n+struct connectivity_check {\n+\tenum connectivity_check_mode mode;\n+};\n+\n+static void parse_connectivity_check(struct repository *repo UNUSED,\n+\t\t\t\t     const char *arg,\n+\t\t\t\t     struct connectivity_check *check)\n+{\n+\tif (!strcmp(arg, \"full\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n+\telse\n+\t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n+}\n+\n struct rev_list_info {\n \tstruct rev_info *revs;\n \tint flags;\n@@ -727,6 +745,7 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n+\tstruct connectivity_check connectivity_check = { 0 };\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -769,6 +788,8 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n+\t\t} else if (skip_prefix(arg, \"--connectivity-check=\", &arg)) {\n+\t\t\tparse_connectivity_check(repo, arg, &connectivity_check);\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n \t\t} else if (!strcmp(arg, \"--missing-only\")) {\n@@ -848,6 +869,8 @@ int cmd_rev_list(int argc,\n \n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n \t\t\tcontinue; /* already handled above */\n+\t\tif (starts_with(arg, \"--connectivity-check=\"))\n+\t\t\tcontinue; /* already handled above */\n \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n \t\t\tcontinue; /* already handled above */\n \ndiff --git a/connected.c b/connected.c\nindex 34045f1032..0147b7d3b3 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,4 +1,5 @@\n #include \"git-compat-util.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n@@ -66,6 +67,27 @@ static int check_connected_promisor(struct repository *repo,\n \treturn 1;\n }\n \n+/*\n+ * Fills in the --connectivity-check mode to pass to rev-list.\n+ */\n+static void connectivity_check_mode(struct repository *repo,\n+\t\t\t\t    struct strbuf *mode)\n+{\n+\tconst char *algorithm = NULL;\n+\n+\tif (repo_config_get_string_tmp(repo,\n+\t\t\t\t       \"transfer.connectivitycheck\",\n+\t\t\t\t       &algorithm))\n+\t\talgorithm = \"full\";\n+\tif (!strcasecmp(algorithm, \"full\"))\n+\t\tstrbuf_addstr(mode, \"full\");\n+\telse {\n+\t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n+\t\t\t  \"using 'full'\"), algorithm);\n+\t\tstrbuf_addstr(mode, \"full\");\n+\t}\n+}\n+\n /*\n  * If we feed all the commits we want to verify to this command\n  *\n@@ -84,6 +106,7 @@ int check_connected(struct repository *repo,\n \tstruct child_process rev_list = CHILD_PROCESS_INIT;\n \tFILE *rev_list_in;\n \tstruct check_connected_options defaults = CHECK_CONNECTED_INIT;\n+\tstruct strbuf mode = STRBUF_INIT;\n \tconst struct object_id *oid;\n \tint err = 0;\n \tstruct packed_git *new_pack = NULL;\n@@ -133,6 +156,9 @@ int check_connected(struct repository *repo,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n+\tconnectivity_check_mode(repo, &mode);\n+\tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n+\tstrbuf_release(&mode);\n \n \trev_list.git_cmd = 1;\n \tif (opt->env)\n-- \ngitgitgadget\n\n\n"},{"id":"555720","messageId":"d76dae801b3303ef4271f71c7b3999c10cac8556.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:23Z","receivedAt":"2026-10-21T11:22:23Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe full connectivity check uses rev-list to find commits\nreachable from the incoming tips but not from the\nalready-connected side, then walks their object closure.  Commit\ntraversal stops at the connectivity boundary, but trees and blobs\nreachable from that boundary still need to be walked so they can\nbe marked uninteresting, allocating a struct object for each one.\nOn repositories where the boundary commits have large trees, the\nconnectivity check for small incoming changes visits and tracks\nmore objects than needed.\n\nAdd an alternative connectivity check that verifies incoming\ncommits incrementally against their parents.\n\nThe verifier processes incoming commits with ancestors first and\ncompares each new tree against its trusted parent trees.  Entries\nalready seen on the trusted side are skipped by OID, so unchanged\nsubtrees need not remain at the same path to be recognized.\nChanged subtrees are recursively compared against same-path parent\nsubtrees, and new subtrees without a comparison base are verified\nfrom scratch.\n\nSee Documentation/technical/connectivity-check.adoc for the trust\ninvariants and detailed algorithm.\n\nThe incremental check runs inside rev-list, selected by the\ninternal --connectivity-check option.  After the normal\nrevision walk identifies the incoming commits, the verifier\nconsumes them before the usual object traversal; any pending\nnon-commit tips are still handled by the existing traversal.\n\nPartial-clone semantics are preserved: objects promised by a\npromisor remote are accepted as connected, and on-demand fetching\nis prevented by excluding promisor objects from traversal.\n\nA new transfer.connectivityCheck value \"incremental\" selects the\nnew mode, with \"full\" remaining the default.  Deepening fetches\nfall back to the full check because they do not have the normal\nexisting-reference boundary, and traversal instead follows the\neffective shallow boundary.\n\nClones and pushes into an empty repository also use the full\ncheck.  Without negative revisions there is no trusted boundary,\nand the incremental check would have to verify all of history\nfrom scratch like the full check does, with extra per-commit\noverhead on top.  On git.git, checking 70K commits without a\nboundary takes 5.3s with the full check and 9.2s with the\nincremental one.\n\np5413 results (best of 3), scaling one dimension at a time.\nEach fixture is a repository with a flat tree of many\ndirectories containing 100 files each.  The incoming set is a\nchain of commits on top of the existing history, with each\ncommit changing files in different directories round-robin.\nThe three axes vary the total repository tree size, the number\nof incoming commits, and the number of files changed per\nincoming commit.\n\nScaling tree size (10 incoming commits, 10 files/commit):\n\n    files    full  incr.  full/incr\n      5K    0.01s  0.01s    1.0x\n     50K    0.04s  0.01s    4.0x\n    200K    0.13s  0.01s   13.0x\n    800K    0.54s  0.03s   18.0x\n\nThe full mode must walk the trees of boundary commits, which\ngrows with overall tree size.  Incremental still scans the\nroot trees, but avoids descending into unchanged subtrees,\nso it grows much more slowly with repository size.\n\nScaling incoming commit count (200K files, 10 files/commit):\n\n    commits    full  incr.  full/incr\n          1    0.13s  0.01s   13.0x\n         10    0.13s  0.01s   13.0x\n        100    0.16s  0.06s    2.7x\n        500    0.29s  0.25s    1.2x\n       3000    1.14s  1.44s    0.8x\n       5000    1.87s  2.39s    0.8x\n      10000    3.45s  5.13s    0.7x\n\nWith many commits the per-commit overhead of scanning both\nthe new and parent root trees accumulates and incremental\nbecomes slower.  Breakeven is between 500 and 3000 commits\nand the ratio stabilizes near 0.7x for this fixture.\n\nScaling files per incoming commit (200K files, 10 commits):\n\n    files/commit    full  incr.  full/incr\n               1    0.13s  0.01s   13.0x\n              10    0.13s  0.01s   13.0x\n             100    0.15s  0.03s    5.0x\n             500    0.23s  0.13s    1.8x\n            1000    0.34s  0.27s    1.3x\n            2000    0.65s  0.56s    1.2x\n\nThe advantage shrinks as more files change per commit, but\nincremental stays slightly faster even at 2000 files/commit,\nwhere every directory is touched.\n\nSelected peak RSS measurements from the same fixtures:\n\n    case                        full    incr.  ratio\n    200K files, 10 commits      31 MB    12 MB   0.4x\n    800K files, 10 commits     110 MB    26 MB   0.2x\n    200K files, 5000 commits   155 MB   151 MB   1.0x\n\nIncremental uses less memory when it can prune enough of the\nboundary tree walk.  In the long-history case the two modes\nvisit similar object sets and memory converges.\n\nThe regression cases in the CPU benchmarks are in wall-clock\ntime rather than memory, primarily from scanning some trees\nmore than once: a commit's tree is read as a candidate when the\ncommit is verified, and again as a trusted parent tree when its\nchildren are verified.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc            |   8 +\n Documentation/rev-list-options.adoc           |   2 +-\n .../technical/connectivity-check.adoc         | 147 ++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  32 +-\n connected.c                                   |  14 +-\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n t/perf/p5413-connectivity-check.sh            |  17 +-\n t/t5413-connectivity-check.sh                 | 774 ++++++++++++++++++\n t/t6700-tree-depth.sh                         |  17 +\n tree-verify.c                                 | 427 ++++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1448 insertions(+), 8 deletions(-)\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex 811456f1bf..cba7597a58 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -10,6 +10,14 @@ transfer.connectivityCheck::\n --\n `full` (default);;\n \tWalk the full object closure of the boundary commits.\n+`incremental`;;\n+\tVerify incoming commits by diffing their trees against parent\n+\ttrees, recursively descending only into entries that differ.\n+\tThe largest benefits occur when incoming commits change a\n+\tsmall fraction of a large tree closure, and it can be slower\n+\tthan `full` when there are many incoming commits.\n+\tFalls back to `full` for clones, pushes into an empty\n+\trepository and deepening fetches.\n --\n +\n Unknown values are treated as `full`, with a warning.\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 3979a142a5..2023ad5f75 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1103,7 +1103,7 @@ It is an error to use `--missing-only` without `--missing=print` or\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n \n-`--connectivity-check=full`::\n+`--connectivity-check=(full|incremental)`::\n \t(For internal use only.)  Used by `check_connected()` to\n \tselect the connectivity check algorithm; see\n \t`transfer.connectivityCheck` in linkgit:git-config[1].\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nindex a112d5be3e..8b7d619d4c 100644\n--- a/Documentation/technical/connectivity-check.adoc\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -110,3 +110,150 @@ tag, tree, or blob, that object is not part of the commit walk.\n These non-commit tips are verified during the tree and blob\n traversal that follows the commit walk within the same rev-list\n invocation.\n+\n+Incremental connectivity check\n+------------------------------\n+\n+The incremental mode, selected by\n+`transfer.connectivityCheck=incremental`, avoids traversing the\n+full tree walk of the boundary commits.  Instead, it verifies\n+each incoming commit's tree against the already-trusted trees of\n+its parent commits.\n+\n+The same `--not --all` boundary as the full check identifies the\n+incoming commits.  The incremental mode reuses this boundary but\n+replaces what happens afterwards: rather than walking boundary\n+trees to mark objects uninteresting, it compares each incoming\n+commit's tree against its parents' trees, processing commits in\n+topological order (ancestors first).  This all happens inside\n+the same `rev-list` invocation, and the promisor fast path above\n+still runs first.\n+\n+Trust model\n+~~~~~~~~~~~\n+\n+A tree is trusted when its transitive object closure is known to\n+be connected.\n+\n+Every parent of an incoming commit is either on the\n+already-connected side of the `--not --all` boundary (a boundary\n+parent whose tree is trusted by definition) or is an earlier\n+incoming commit that was already verified due to the topological\n+processing order.  In both cases the parent's tree is trusted, so\n+all parent trees can be unconditionally used as comparison bases.\n+\n+Once an incoming commit's tree has been verified, it is trusted\n+and can serve as a comparison base for later descendants.  This\n+gives an inductive correctness argument: the base case is the\n+trust boundary established by `--not --all`, and the inductive\n+step is that verifying a commit extends the trusted set.\n+\n+Tree states\n+~~~~~~~~~~~\n+\n+The verifier tracks tree OIDs in three states:\n+\n+untrusted::\n+\tThe tree has not yet been established as connected.  This\n+\tis the implicit state of an OID not present in the state\n+\tmap.\n+\n+trusted::\n+\tThe tree is known to have a connected transitive closure,\n+\tbut its direct entries have not yet been published into the\n+\tverifier's trusted object sets.\n+\n+expanded::\n+\tThe tree is trusted and its direct non-gitlink entries\n+\thave also been published into the trusted object sets.\n+\n+State transitions are monotonic: a tree may move from untrusted\n+to trusted to expanded, but never backwards.  An untrusted tree\n+that is successfully verified goes directly to expanded.\n+\n+Blobs require only trusted/untrusted state: a blob becomes\n+trusted when it is found in a trusted tree or when its existence\n+and type have been verified directly.\n+\n+The trusted/expanded distinction is an optimization.  An expanded\n+parent does not need to be reread merely to publish entries that\n+are already trusted, though it may still be read when same-path\n+subtree bases are needed for recursive comparison.\n+\n+Algorithm\n+~~~~~~~~~\n+\n+At a high level:\n+\n+    verify(commits):\n+        sort topologically (ancestors first)\n+        for each commit:\n+            mark parent root trees as trusted\n+            verify_tree(commit.tree, parent root trees)\n+\n+    verify_tree(tree, base_trees):\n+        if tree already trusted: return\n+        read tree, collect entries not already trusted\n+        for each available base tree:\n+            publish its entries as trusted\n+            record same-path subtrees as bases\n+        for each collected entry:\n+            if now trusted: skip\n+            if blob: verify blob connectivity and type\n+            if tree: verify_tree(entry, its recorded bases)\n+        mark tree as expanded\n+\n+The important ordering within `verify_tree` is that entries from\n+the new tree are collected before the base trees are scanned, but\n+are processed only afterwards.  Trust learned from any base can\n+therefore eliminate work before recursive verification begins.\n+\n+Same-path parent subtrees are passed down as comparison bases\n+during recursive descent.  If no comparison base is available,\n+the new subtree is verified from scratch.\n+\n+Example\n+~~~~~~~\n+\n+Consider a commit that changes one file under `lib/` and moves an\n+unchanged subtree from `src/` to `dev/`:\n+\n+    Parent tree              New tree\n+    +-- src/   (aaa)         +-- dev/   (aaa)\n+    +-- lib/   (bbb)         +-- lib/   (ccc)\n+         +-- foo.c (ddd)          +-- foo.c (ddd)\n+         +-- bar.c (eee)          +-- bar.c (fff)\n+\n+Scanning the parent makes `aaa` trusted even though it moved from\n+`src/` to `dev/`, so that subtree is skipped.  The changed `ccc`\n+subtree is compared against its same-path parent `bbb`; scanning\n+`bbb` makes `ddd` and `eee` trusted, leaving only the new `fff`\n+blob to be checked.\n+\n+This illustrates two properties:\n+\n+* Trust is OID-based rather than path-based.  An unchanged subtree\n+  is recognized after a move.\n+\n+* Same-path parent subtrees provide recursive comparison bases.\n+  These bases improve pruning efficiency but are not required for\n+  correctness; a new subtree can always be verified from scratch.\n+\n+Multiple parents\n+~~~~~~~~~~~~~~~~\n+\n+For a merge commit, root trees from all parents are comparison\n+bases.  When several parents contain a same-path subtree, each\n+matching subtree is collected as a recursive comparison base.\n+\n+Entries published from any trusted parent become globally trusted,\n+so a matching tree or blob entry present in any parent can be\n+skipped while verifying the merge tree.\n+\n+Missing comparison bases\n+~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+A promised base tree may not be locally available for comparison.\n+In that case the verifier skips that base and verifies the new\n+subtree without it.  The missing comparison can reduce pruning but\n+does not affect correctness.\ndiff --git a/Makefile b/Makefile\nindex a96be506b5..fa42e295e2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1339,6 +1339,7 @@ LIB_OBJS += trailer.o\n LIB_OBJS += transport-helper.o\n LIB_OBJS += transport.o\n LIB_OBJS += tree-diff.o\n+LIB_OBJS += tree-verify.o\n LIB_OBJS += tree-walk.o\n LIB_OBJS += tree.o\n LIB_OBJS += unpack-trees.o\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 810eaa15e3..2b428c0c84 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -28,23 +28,30 @@\n #include \"commit-reach.h\"\n #include \"quote.h\"\n #include \"strbuf.h\"\n+#include \"tree-verify.h\"\n \n enum connectivity_check_mode {\n \tCONNECTIVITY_CHECK_FULL,\n+\tCONNECTIVITY_CHECK_INCREMENTAL,\n };\n \n struct connectivity_check {\n \tenum connectivity_check_mode mode;\n };\n \n-static void parse_connectivity_check(struct repository *repo UNUSED,\n+static void parse_connectivity_check(struct repository *repo,\n \t\t\t\t     const char *arg,\n \t\t\t\t     struct connectivity_check *check)\n {\n \tif (!strcmp(arg, \"full\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n+\telse if (!strcmp(arg, \"incremental\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n \telse\n \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n+\n+\tif (check->mode != CONNECTIVITY_CHECK_FULL)\n+\t\trepo->fetch_if_missing = 0;\n }\n \n struct rev_list_info {\n@@ -992,6 +999,29 @@ int cmd_rev_list(int argc,\n \n \tprepare_maximal_independent(&revs);\n \n+\t/*\n+\t * Without a boundary of negative revisions, as in a clone or a\n+\t * push into an empty repository, every incoming commit is new.\n+\t * The incremental check would still work, but it would verify\n+\t * the whole object graph like the full check, with extra\n+\t * per-commit overhead on top, so use the full check.\n+\t */\n+\tif (!revs.limited)\n+\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\n+\tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *new_commits = NULL;\n+\n+\t\twhile ((commit = get_revision(&revs)) != NULL)\n+\t\t\tcommit_list_insert(commit, &new_commits);\n+\n+\t\tif (verify_commits_incremental(repo, &new_commits,\n+\t\t\t\t\t       revs.exclude_promisor_objects))\n+\t\t\tdie(_(\"connectivity check failed\"));\n+\t\tcommit_list_free(new_commits);\n+\t}\n+\n \tif (revs.tree_objects)\n \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n \ndiff --git a/connected.c b/connected.c\nindex 0147b7d3b3..c032987410 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -71,16 +71,28 @@ static int check_connected_promisor(struct repository *repo,\n  * Fills in the --connectivity-check mode to pass to rev-list.\n  */\n static void connectivity_check_mode(struct repository *repo,\n+\t\t\t\t    struct check_connected_options *opt,\n \t\t\t\t    struct strbuf *mode)\n {\n \tconst char *algorithm = NULL;\n \n+\t/*\n+\t * Deepening fetches do not have the usual boundary of\n+\t * existing references, so use the full check for them.\n+\t */\n+\tif (opt->is_deepening_fetch) {\n+\t\tstrbuf_addstr(mode, \"full\");\n+\t\treturn;\n+\t}\n+\n \tif (repo_config_get_string_tmp(repo,\n \t\t\t\t       \"transfer.connectivitycheck\",\n \t\t\t\t       &algorithm))\n \t\talgorithm = \"full\";\n \tif (!strcasecmp(algorithm, \"full\"))\n \t\tstrbuf_addstr(mode, \"full\");\n+\telse if (!strcasecmp(algorithm, \"incremental\"))\n+\t\tstrbuf_addstr(mode, \"incremental\");\n \telse {\n \t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n \t\t\t  \"using 'full'\"), algorithm);\n@@ -156,7 +168,7 @@ int check_connected(struct repository *repo,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n-\tconnectivity_check_mode(repo, &mode);\n+\tconnectivity_check_mode(repo, opt, &mode);\n \tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n \tstrbuf_release(&mode);\n \ndiff --git a/meson.build b/meson.build\nindex 0a95d90d21..d94dedc26c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -562,6 +562,7 @@ libgit_sources = [\n   'transport-helper.c',\n   'transport.c',\n   'tree-diff.c',\n+  'tree-verify.c',\n   'tree-walk.c',\n   'tree.c',\n   'unpack-trees.c',\ndiff --git a/t/meson.build b/t/meson.build\nindex f65eb04684..b23b4c90b0 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -654,6 +654,7 @@ integration_tests = [\n   't5410-receive-pack.sh',\n   't5411-proc-receive-hook.sh',\n   't5412-receive-report-hook.sh',\n+  't5413-connectivity-check.sh',\n   't5500-fetch-pack.sh',\n   't5501-fetch-push-alternates.sh',\n   't5502-quickfetch.sh',\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nindex 9e968d0106..250140a79a 100755\n--- a/t/perf/p5413-connectivity-check.sh\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -1,9 +1,9 @@\n #!/bin/sh\n \n-test_description='performance of connectivity check\n+test_description='performance of connectivity check modes\n \n-Benchmark the rev-list connectivity check, avoiding pack transfer\n-noise.\n+Compare the default and incremental rev-list connectivity modes\n+directly, avoiding pack transfer noise.\n \n Each repository has a flat tree of many directories with 100 files\n in each.  Three axes are scaled independently: tree size, commit\n@@ -51,9 +51,16 @@ test_perf_conn () {\n \t\tfi\n \t'\n \n-\ttest_perf \"$label\" --prereq PERL '\n+\ttest_perf \"$label (full)\" --prereq PERL '\n \t\tcat '\"${name}\"'_new |\n-\t\tgit -C '\"$name\"' rev-list \\\n+\t\tgit -C '\"$name\"' rev-list --connectivity-check=full \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+\n+\ttest_perf \"$label (incremental)\" --prereq PERL '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list --connectivity-check=incremental \\\n \t\t\t--objects --stdin --not --all --quiet \\\n \t\t\t--exclude-promisor-objects\n \t'\ndiff --git a/t/t5413-connectivity-check.sh b/t/t5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..109ad17cff\n--- /dev/null\n+++ b/t/t5413-connectivity-check.sh\n@@ -0,0 +1,774 @@\n+#!/bin/sh\n+\n+test_description='connectivity check (transfer.connectivityCheck)'\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_oid_cache <<-\\EOF\n+missing sha1:0000000000000000000000000000000000000001\n+missing sha256:0000000000000000000000000000000000000000000000000000000000000001\n+EOF\n+\n+set_connectivity_check () {\n+\tgit -C \"$1\" config transfer.connectivityCheck \"$2\"\n+}\n+\n+# Run a connectivity check via rev-list directly.  Uses $mode\n+# (set by the enclosing for-loop) to choose full or incremental.\n+check_connected () {\n+\tprintf '%s\\n' \"$@\" |\n+\tgit rev-list --connectivity-check=\"$mode\" \\\n+\t\t--objects --stdin --not --all --quiet \\\n+\t\t--exclude-promisor-objects\n+}\n+\n+# Run git with a temporary index, leaving the real index untouched.\n+tmpgit () {\n+\tGIT_INDEX_FILE=.git/tmp-idx git \"$@\"\n+}\n+\n+# Create a commit with one file changed, without modifying HEAD,\n+# index, or worktree.  Prints the new commit OID on stdout.\n+# Usage: commit_with_change <parent> <path> <content>\n+commit_with_change () {\n+\tnew_blob=$(echo \"$3\" | git hash-object -w --stdin) &&\n+\ttmpgit read-tree \"$1\" &&\n+\ttmpgit update-index --replace \\\n+\t\t--cacheinfo \"100644,$new_blob,$2\" &&\n+\tnew_tree=$(tmpgit write-tree) &&\n+\trm -f .git/tmp-idx &&\n+\tgit commit-tree \"$new_tree\" -p \"$1\" -m \"modify $2\"\n+}\n+\n+# Check OIDs and optionally verify trace2 counts.\n+# Usage: check_connected_trace <trace-file> <trees> <blobs> <oid>...\n+# An empty string for <trees> or <blobs> skips that assertion.\n+check_connected_trace () {\n+\ttrace_file=$1 trees=$2 blobs=$3 &&\n+\tshift 3 &&\n+\ttest_env GIT_TRACE2_EVENT=\"$(pwd)/$trace_file\" \\\n+\t\tcheck_connected \"$@\" &&\n+\tif test \"$mode\" != incremental\n+\tthen\n+\t\treturn\n+\tfi &&\n+\tif test -n \"$trees\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity trees_loaded \"$trees\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi &&\n+\tif test -n \"$blobs\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity blobs_checked \"$blobs\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi\n+}\n+\n+# Shared setup: a repo with several root-level files and nested dirs.\n+# The unchanged/ subtree (10 dirs x 10 files = 100 blobs, 11 trees)\n+# acts as a canary: any test asserting small tree/blob counts would\n+# fail dramatically if incremental accidentally walked into it.\n+\n+test_expect_success 'setup main repo' '\n+\tgit init main-repo &&\n+\t(\n+\t\tcd main-repo &&\n+\t\tfor i in $(test_seq 1 5)\n+\t\tdo\n+\t\t\techo \"file $i\" >\"file-$i.txt\" || return 1\n+\t\tdone &&\n+\t\tgit add file-*.txt &&\n+\t\tgit commit -m \"initial\" &&\n+\n+\t\tmkdir -p a/b/c &&\n+\t\techo deep >a/b/c/deep.txt &&\n+\t\techo other >a/other.txt &&\n+\t\tgit add a/b/c/deep.txt a/other.txt &&\n+\t\tgit commit -m \"add nested dirs\" &&\n+\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\td=\"unchanged/dir-$i\" &&\n+\t\t\tmkdir -p \"$d\" &&\n+\t\t\tfor j in $(test_seq 1 10)\n+\t\t\tdo\n+\t\t\t\techo \"$i $j\" >\"$d/file-$j.txt\" || return 1\n+\t\t\tdone\n+\t\tdone &&\n+\t\tgit add unchanged/ &&\n+\t\tgit commit -m \"add unchanged canary subtree\"\n+\t)\n+'\n+\n+test_expect_success 'setup replacement object repo' '\n+\tgit init replace-test &&\n+\t(\n+\t\tcd replace-test &&\n+\n+\t\ttest_commit --no-tag original file.txt &&\n+\t\toriginal=$(git rev-parse HEAD) &&\n+\t\torig_blob=$(git rev-parse HEAD:file.txt) &&\n+\n+\t\t# Orphan replacement commit with a different tree\n+\t\treplacement_tree=$(echo replaced | git hash-object -w --stdin |\n+\t\t\txargs -I{} git mktree <<-EOF\n+\t\t\t100644 blob {}\tfile.txt\n+\t\t\tEOF\n+\t\t) &&\n+\t\treplacement=$(git commit-tree -m \"replacement\" \\\n+\t\t\t\"$replacement_tree\") &&\n+\n+\t\tgit replace \"$original\" \"$replacement\" &&\n+\n+\t\t# Remove the original blob so only the replacement\n+\t\t# tree is complete.\n+\t\trm .git/objects/$(test_oid_to_path \"$orig_blob\") &&\n+\n+\t\t# Drop branch and HEAD so --not --all does not\n+\t\t# exclude the original commit.\n+\t\tgit update-ref -d refs/heads/main &&\n+\t\tgit update-ref -d HEAD &&\n+\n+\t\techo \"$original\" >.git/test-oid &&\n+\t\techo \"$orig_blob\" >.git/test-blob\n+\t)\n+'\n+\n+missing_oid=$(test_oid missing)\n+original_oid=$(cat replace-test/.git/test-oid)\n+orig_blob_oid=$(cat replace-test/.git/test-blob)\n+\n+for mode in full incremental\n+do\n+\n+# The full traversal reports a missing tree as \"bad tree object\".\n+case \"$mode\" in\n+full)\n+\tmissing_tree=\"bad tree object\"\n+\t;;\n+incremental)\n+\tmissing_tree=\"missing tree object\"\n+\t;;\n+esac\n+\n+# Corruption detection: craft broken object graphs and verify detection.\n+# All tests use main-repo without modifying its refs or worktree.\n+\n+test_expect_success \"$mode: rejects commit with missing blob\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects commit with missing subtree\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"40000 tree ${missing_oid}\\tdir\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"$missing_tree $missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct tree tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"hello\" | git hash-object -w --stdin) &&\n+\t\tcheck_connected \"$blob_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects missing direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$missing_oid\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects blob OID reused as tree entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\tblob_oid=$(git rev-parse HEAD:file-1.txt) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"40000 subdir\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a tree\" err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects tree OID reused as blob entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\ttree_oid=$(git rev-parse HEAD:a) &&\n+\t\tbin_oid=$(echo \"$tree_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"100644 fakefile\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a blob\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: checks multiple tips\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"tip-a\") &&\n+\t\tc2=$(commit_with_change HEAD file-2.txt \"tip-b\") &&\n+\t\tgit tag -a -m \"tagged\" multi-tag \"$c1\" &&\n+\t\ttag_oid=$(git rev-parse multi-tag) &&\n+\t\tgit tag -d multi-tag &&\n+\t\tcheck_connected \"$c2\" \"$tag_oid\"\n+\t)\n+'\n+\n+# Tree-diff optimization: verify trace2 counts.\n+\n+test_expect_success \"$mode: handles root commit (no parents)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"root-content\" | git hash-object -w --stdin) &&\n+\t\ttree_oid=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$blob_oid\" | git mktree) &&\n+\t\troot_oid=$(git commit-tree \"$tree_oid\" -m \"root\") &&\n+\n+\t\t# No parent trees, so the full tree is verified.\n+\t\t# 1 tree loaded (root), 1 blob checked.\n+\t\tcheck_connected_trace trace-root.txt 1 1 \"$root_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles single file change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD file-1.txt \"changed\") &&\n+\n+\t\t# 2 trees loaded (new root + parent root), 1 blob checked.\n+\t\tcheck_connected_trace trace-flat.txt 2 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles nested change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD a/b/c/deep.txt \"deep-changed\") &&\n+\t\t# 4 new trees + 4 parent trees = 8 loaded, 1 blob checked.\n+\t\tcheck_connected_trace trace-nested.txt 8 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles change-then-revert\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"revert-tmp\") &&\n+\t\tc2=$(commit_with_change \"$c1\" file-1.txt \"file 1\") &&\n+\t\tc3=$(commit_with_change \"$c2\" file-1.txt \"revert-final\") &&\n+\n+\t\t# c1: new root + parent root = 2 loads.  c2: root matches\n+\t\t# HEAD (already trusted), skipped.  c3: new root + parent\n+\t\t# already expanded = 1 load.  Total: 3 trees, 2 blobs.\n+\t\tcheck_connected_trace trace-revert.txt 3 2 \"$c3\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles subtree moved to another path\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tmoved_tree=$(git ls-tree HEAD |\n+\t\t\tsed \"s/\ta$/\tmoved/\" |\n+\t\t\tgit mktree) &&\n+\t\tmoved=$(git commit-tree \"$moved_tree\" -p HEAD -m move) &&\n+\t\t# New root + parent root scanned, but the moved subtree\n+\t\t# (same OID) is trusted and not descended into.\n+\t\tcheck_connected_trace trace-move.txt 2 0 \"$moved\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles multi-parent merge\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tleft=$(commit_with_change HEAD file-1.txt left) &&\n+\t\tright=$(commit_with_change HEAD file-2.txt right) &&\n+\t\tgit update-ref refs/heads/left \"$left\" &&\n+\t\tgit update-ref refs/heads/right \"$right\" &&\n+\t\tleft_blob=$(git rev-parse \"$left:file-1.txt\") &&\n+\t\tright_blob=$(git rev-parse \"$right:file-2.txt\") &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$left_blob,file-1.txt\" &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$right_blob,file-2.txt\" &&\n+\t\tmerge_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tmerge=$(git commit-tree \"$merge_tree\" \\\n+\t\t\t-p \"$left\" -p \"$right\" -m merge) &&\n+\t\t# Both parent root trees are scanned as bases, so\n+\t\t# blobs from each parent are trusted without ODB checks.\n+\t\tcheck_connected_trace trace-merge.txt 3 0 \"$merge\" &&\n+\t\tgit update-ref -d refs/heads/left &&\n+\t\tgit update-ref -d refs/heads/right\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles merge with incoming and boundary parents\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# parent1 is incoming (not a ref), HEAD is boundary.\n+\t\tparent1=$(commit_with_change HEAD file-1.txt merge-inc) &&\n+\t\tmerge=$(git commit-tree \\\n+\t\t\t\"$(git rev-parse \"$parent1^{tree}\")\" \\\n+\t\t\t-p \"$parent1\" -p HEAD -m merge-mixed) &&\n+\t\t# parent1 verified first (topo order): 2 trees, 1 blob.\n+\t\t# merge tree = parent1 tree (already trusted): 0 extra.\n+\t\tcheck_connected_trace trace-merge-inc.txt 2 1 \"$merge\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles gitlink entries (submodules)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"160000,$missing_oid,my-submodule\" &&\n+\t\tgitlink_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tgitlink_commit=$(git commit-tree \"$gitlink_tree\" -p HEAD \\\n+\t\t\t-m \"add gitlink\") &&\n+\n+\t\t# Gitlink entries are skipped -- the missing submodule\n+\t\t# commit OID does not cause a failure.\n+\t\tcheck_connected_trace trace-gitlink.txt 2 0 \"$gitlink_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles file-to-directory transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"foo\" is a blob at root.\n+\t\tblob_a=$(echo \"file-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,foo\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add foo as file\") &&\n+\n+\t\t# Child: \"foo\" becomes a directory (foo/bar.txt).\n+\t\tblob_b=$(echo \"dir-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove foo &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,foo/bar.txt\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"foo: file to directory\") &&\n+\t\tcheck_connected_trace trace-f2d.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles directory-to-file transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"bar/baz.txt\" exists (bar is a directory).\n+\t\tblob_a=$(echo \"nested\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,bar/baz.txt\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add bar as directory\") &&\n+\n+\t\t# Child: \"bar\" becomes a plain file.\n+\t\tblob_b=$(echo \"flat\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove bar/baz.txt &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,bar\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"bar: directory to file\") &&\n+\t\tcheck_connected_trace trace-d2f.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+# Replacement objects.\n+\n+test_expect_success \"$mode: accepts with replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tcheck_connected \"$original_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects without replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tGIT_NO_REPLACE_OBJECTS=1 &&\n+\t\texport GIT_NO_REPLACE_OBJECTS &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$original_oid\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$orig_blob_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised blob\" '\n+\ttest_when_finished \"rm -rf prom-src prom-server.git prom-client\" &&\n+\tgit init prom-src &&\n+\ttest_commit -C prom-src --no-tag base file.txt original &&\n+\ttest_commit -C prom-src --no-tag \"add file2\" file2.txt extra &&\n+\tgit clone --bare prom-src prom-server.git &&\n+\tgit -C prom-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/prom-server.git\" prom-client &&\n+\t(\n+\t\tcd prom-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file2.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tnewname.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: does not lazy-fetch promised blob\" '\n+\ttest_when_finished \"rm -rf nofetch-src nofetch-server.git nofetch-client\" &&\n+\tgit init nofetch-src &&\n+\ttest_commit -C nofetch-src --no-tag base file.txt content &&\n+\tgit clone --bare nofetch-src nofetch-server.git &&\n+\tgit -C nofetch-server.git config uploadpack.allowfilter true &&\n+\tgit -C nofetch-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/nofetch-server.git\" nofetch-client &&\n+\t(\n+\t\tcd nofetch-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-m \"root commit with promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised tree\" '\n+\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n+\tgit init prom-tree-src &&\n+\tmkdir -p prom-tree-src/a/b &&\n+\ttest_commit -C prom-tree-src --no-tag \"nested dirs\" a/b/file.txt deep &&\n+\tgit clone --bare prom-tree-src prom-tree-server.git &&\n+\tgit -C prom-tree-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-tree-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-tree-server.git\" prom-tree-client &&\n+\t(\n+\t\tcd prom-tree-client &&\n+\t\tpromised_tree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\" &&\n+\t\tnew_tree=$(printf \"40000 tree %s\\trenamed\\n\" \\\n+\t\t\t\"$promised_tree\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised tree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies new subtree when parent subtree is promised\" '\n+\ttest_when_finished \"rm -rf prom-base-src prom-base-server.git prom-base-client\" &&\n+\tgit init prom-base-src &&\n+\tmkdir -p prom-base-src/a &&\n+\ttest_commit -C prom-base-src --no-tag \"base\" a/file.txt deep &&\n+\tgit clone --bare prom-base-src prom-base-server.git &&\n+\tgit -C prom-base-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-base-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-base-server.git\" prom-base-client &&\n+\t(\n+\t\tcd prom-base-client &&\n+\t\tparent_subtree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\" &&\n+\t\tnew_blob=$(echo \"local-content\" | git hash-object -w --stdin) &&\n+\t\tnew_subtree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$new_blob\" | git mktree) &&\n+\t\tnew_root=$(printf \"40000 tree %s\\ta\\n\" \\\n+\t\t\t\"$new_subtree\" | git mktree) &&\n+\t\tnew_commit=$(git commit-tree \"$new_root\" \\\n+\t\t\t-p HEAD -m \"replace promised subtree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies local commit in partial clone\" '\n+\ttest_when_finished \"rm -rf pc-src pc-server.git pc-client\" &&\n+\tgit init pc-src &&\n+\ttest_commit -C pc-src --no-tag base file.txt &&\n+\tgit clone --bare pc-src pc-server.git &&\n+\tgit -C pc-server.git config uploadpack.allowfilter true &&\n+\tgit -C pc-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/pc-server.git\" pc-client &&\n+\t(\n+\t\tcd pc-client &&\n+\t\tlocal_commit=$(commit_with_change HEAD file.txt local-content) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: tolerates missing promisor parent commit\" '\n+\ttest_when_finished \"rm -rf mp-src mp-server.git mp-client\" &&\n+\tgit init mp-src &&\n+\ttest_commit -C mp-src --no-tag base file.txt &&\n+\tgit clone --bare mp-src mp-server.git &&\n+\tgit -C mp-server.git config uploadpack.allowfilter true &&\n+\tgit -C mp-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/mp-server.git\" mp-client &&\n+\ttest_commit -C mp-src --no-tag c1 file.txt &&\n+\ttest_commit -C mp-src --no-tag c2 file.txt &&\n+\tc1=$(git -C mp-src rev-parse HEAD~1) &&\n+\tc2=$(git -C mp-src rev-parse HEAD) &&\n+\t(\n+\t\tcd mp-client &&\n+\t\t# Add a promisor pack holding only c2, so its parent c1\n+\t\t# is missing but promised (as after a lazy fetch that\n+\t\t# omits dependents).\n+\t\techo \"$c2\" |\n+\t\tgit -C ../mp-src pack-objects --stdout >c2.pack &&\n+\t\tgit index-pack --promisor --stdin <c2.pack &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$c1\" &&\n+\t\ttree=$(git rev-parse HEAD^{tree}) &&\n+\t\tlocal_commit=$(git hash-object -t commit -w --stdin <<-EOF\n+\t\ttree $tree\n+\t\tparent $c1\n+\t\tauthor A U Thor <author@example.com> 1112912053 -0700\n+\t\tcommitter C O Mitter <committer@example.com> 1112912053 -0700\n+\n+\t\ton top of a missing promisor commit\n+\t\tEOF\n+\t\t) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: respects shallow boundary\" '\n+\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n+\tgit init shallow-src &&\n+\ttest_commit -C shallow-src --no-tag base file content-1 &&\n+\tmkdir shallow-src/sub &&\n+\ttest_commit -C shallow-src --no-tag change sub/other content-2 &&\n+\tgit clone --depth=1 \"file://$(pwd)/shallow-src\" shallow &&\n+\t(\n+\t\tcd shallow &&\n+\t\ttip=$(git rev-parse HEAD) &&\n+\t\tgit for-each-ref --format=\"delete %(refname)\" |\n+\t\t\tgit update-ref --no-deref --stdin &&\n+\t\tcheck_connected \"$tip\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: deepening fetch succeeds\" '\n+\ttest_when_finished \"rm -rf deepen-src deepen-server.git deepen-client\" &&\n+\tgit init deepen-src &&\n+\ttest_commit -C deepen-src --no-tag c1 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c2 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c3 file.txt &&\n+\tgit clone --bare deepen-src deepen-server.git &&\n+\tgit clone --depth=1 \"file://$(pwd)/deepen-server.git\" deepen-client &&\n+\tset_connectivity_check deepen-client $mode &&\n+\ttest -f deepen-client/.git/shallow &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/deepen-trace.txt\" \\\n+\t\tgit -C deepen-client fetch --deepen=2 origin main &&\n+\t# Incremental falls back to full for deepening fetches,\n+\t# so the trees_loaded event should not appear.\n+\ttest_grep ! trees_loaded deepen-trace.txt\n+'\n+\n+test_expect_success \"$mode: malformed tree detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\techo abc >malformed-tree &&\n+\t\tmalformed_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tmalformed-tree) &&\n+\t\tmalformed_commit=$(git commit-tree \"$malformed_tree\" \\\n+\t\t\t-p HEAD -m malformed) &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$malformed_commit\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: mid-tree corruption detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# Build a tree with one valid entry followed by garbage.\n+\t\tblob_oid=$(echo \"valid\" | git hash-object -w --stdin) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\t\tprintf \"100644 good\\0${bin_oid}GARBAGE\" >corrupt-mid-tree &&\n+\t\tcorrupt_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tcorrupt-mid-tree) &&\n+\t\tcorrupt_commit=$(git commit-tree \"$corrupt_tree\" \\\n+\t\t\t-p HEAD -m \"mid-tree corruption\") &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$corrupt_commit\" 2>err &&\n+\t\ttest_grep \"too-short tree object\" err\n+\t)\n+'\n+\n+done\n+\n+# The full check only checks that blobs exist, but the incremental\n+# check reads their headers and reports a corrupt blob.\n+test_expect_success 'incremental: rejects corrupt blob' '\n+\ttest_when_finished \"rm -rf corrupt-repo\" &&\n+\tgit init corrupt-repo &&\n+\t(\n+\t\tcd corrupt-repo &&\n+\t\ttest_commit base &&\n+\t\tblob=$(echo corrupt | git hash-object -w --stdin) &&\n+\t\ttree=$(printf \"100644 blob $blob\\tfile\\n\" | git mktree) &&\n+\t\tcommit=$(git commit-tree \"$tree\" -p HEAD -m corrupt) &&\n+\t\tpath=.git/objects/$(test_oid_to_path \"$blob\") &&\n+\t\tchmod +w \"$path\" &&\n+\t\techo garbage >\"$path\" &&\n+\t\tmode=incremental &&\n+\t\ttest_expect_code 128 check_connected \"$commit\" 2>err &&\n+\t\ttest_grep \"unable to read blob object $blob\" err\n+\t)\n+'\n+\n+# Algorithm selection.\n+\n+# Run a command with its trace2 events in \"trace\" and its stderr\n+# in \"err\".\n+run_traced () {\n+\trm -f trace &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace\" \"$@\" 2>err\n+}\n+\n+# Check which connectivity check the last run_traced used.\n+expect_mode () {\n+\tcase \"$1\" in\n+\tfull)\n+\t\ttest_grep ! trees_loaded trace\n+\t\t;;\n+\tincremental)\n+\t\ttest_grep trees_loaded trace\n+\t\t;;\n+\t*)\n+\t\tBUG \"unknown mode: $1\"\n+\t\t;;\n+\tesac\n+}\n+\n+test_expect_success 'setup source repository' '\n+\tgit init src &&\n+\ttest_commit -C src base file.txt &&\n+\ttest_commit -C src --no-tag update file.txt updated\n+'\n+\n+test_expect_success 'unknown transfer.connectivityCheck falls back to full' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C dst fetch ../src base:refs/heads/main &&\n+\tfor value in bogus\n+\tdo\n+\t\tset_connectivity_check dst \"$value\" &&\n+\t\trun_traced git -C dst fetch ../src main &&\n+\t\texpect_mode full &&\n+\t\ttest_grep \"unknown transfer.connectivityCheck algorithm .$value., using .full.\" err ||\n+\t\treturn 1\n+\tdone\n+'\n+\n+test_expect_success 'push uses incremental when configured' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C src push ../dst base:refs/heads/main &&\n+\tset_connectivity_check dst incremental &&\n+\trun_traced git -C src push ../dst main &&\n+\texpect_mode incremental\n+'\n+\n+test_expect_success 'fetch uses incremental when configured' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C dst fetch ../src base:refs/heads/main &&\n+\tset_connectivity_check dst incremental &&\n+\trun_traced git -C dst fetch ../src main &&\n+\texpect_mode incremental\n+'\n+\n+for mode in incremental\n+do\n+\ttest_expect_success \"push with $mode into an empty repository uses full\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\tgit init --bare dst &&\n+\t\tset_connectivity_check dst $mode &&\n+\t\trun_traced git -C src push ../dst main &&\n+\t\texpect_mode full\n+\t'\n+\n+\ttest_expect_success \"clone with $mode uses full\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\trun_traced git -c transfer.connectivityCheck=$mode \\\n+\t\t\tclone --no-local src dst &&\n+\t\texpect_mode full\n+\t'\n+done\n+\n+test_done\ndiff --git a/t/t6700-tree-depth.sh b/t/t6700-tree-depth.sh\nindex 0f6a2ad9b5..edb31e3914 100755\n--- a/t/t6700-tree-depth.sh\n+++ b/t/t6700-tree-depth.sh\n@@ -82,6 +82,23 @@ test_expect_success 'default limit for rev-list fails gracefully' '\n \ttest_must_fail git rev-list --objects big >/dev/null\n '\n \n+# The incremental check needs a boundary; without negative revisions\n+# rev-list falls back to the full check.\n+test_expect_success 'limit recursion of incremental check' '\n+\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n+\tgit $small_ok rev-list --connectivity-check=incremental \\\n+\t\t--objects small --not $boundary >/dev/null &&\n+\ttest_must_fail git $small_no rev-list \\\n+\t\t--connectivity-check=incremental --objects small \\\n+\t\t--not $boundary >/dev/null\n+'\n+\n+test_expect_success 'default limit for incremental check fails gracefully' '\n+\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n+\ttest_must_fail git rev-list --connectivity-check=incremental \\\n+\t\t--objects big --not $boundary >/dev/null\n+'\n+\n test_expect_success 'limit recursion of diff-tree -r' '\n \tgit $small_ok diff-tree -r $EMPTY_TREE small &&\n \ttest_must_fail git $small_no diff-tree -r $EMPTY_TREE small\ndiff --git a/tree-verify.c b/tree-verify.c\nnew file mode 100644\nindex 0000000000..6126e99853\n--- /dev/null\n+++ b/tree-verify.c\n@@ -0,0 +1,427 @@\n+#include \"git-compat-util.h\"\n+#include \"commit.h\"\n+#include \"gettext.h\"\n+#include \"hex.h\"\n+#include \"khash.h\"\n+#include \"object.h\"\n+#include \"odb.h\"\n+#include \"oid-array.h\"\n+#include \"oidset.h\"\n+#include \"tree.h\"\n+#include \"tree-walk.h\"\n+#include \"tree-verify.h\"\n+#include \"packfile.h\"\n+#include \"trace2.h\"\n+\n+enum tree_state {\n+\tTREE_UNTRUSTED = 0,\n+\tTREE_TRUSTED   = 1,\n+\tTREE_EXPANDED  = 2,\n+};\n+\n+/* Values are enum tree_state, stored as a byte to keep the map small. */\n+KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n+\t   oidhash_by_value, oideq_by_value)\n+\n+static enum tree_state tree_map_get(kh_oid_tree_t *m,\n+\t\t\t\t    const struct object_id *oid)\n+{\n+\tkhint_t pos = kh_get_oid_tree(m, *oid);\n+\tif (pos == kh_end(m))\n+\t\treturn TREE_UNTRUSTED;\n+\treturn kh_val(m, pos);\n+}\n+\n+static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n+\t\t\t enum tree_state state)\n+{\n+\tint added;\n+\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n+\tif (added)\n+\t\tkh_val(m, pos) = state;\n+\telse if (state > kh_val(m, pos))\n+\t\tkh_val(m, pos) = state;\n+}\n+\n+struct work_item {\n+\tstruct name_entry entry;\n+\tstruct oid_array parent_trees;\n+};\n+\n+/* The entries of a tree that still need to be verified. */\n+struct work_list {\n+\tstruct work_item *items;\n+\tsize_t nr, alloc;\n+\tbool has_subtrees;\n+};\n+\n+static void work_list_clear(struct work_list *list)\n+{\n+\tfor (size_t i = 0; i < list->nr; i++)\n+\t\toid_array_clear(&list->items[i].parent_trees);\n+\tfree(list->items);\n+}\n+\n+struct verify_state {\n+\tkh_oid_tree_t *trees;\n+\tstruct oidset trusted_blobs;\n+\tsize_t trees_loaded;\n+\tsize_t blobs_checked;\n+\tint exclude_promisor_objects;\n+\n+\t/*\n+\t * Scratch space for reading one tree at a time.  It lives here\n+\t * rather than on the stack to keep the frame of the recursive\n+\t * verify_tree() small.\n+\t */\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+};\n+\n+/*\n+ * Merge-walk the work list against one base tree entry, recording\n+ * same-path parent subtrees as recursive comparison bases.\n+ * Returns the updated work-list cursor.\n+ */\n+static size_t collect_subtree_bases(struct work_list *list, size_t wi,\n+\t\t\t\t    const struct name_entry *entry)\n+{\n+\twhile (wi < list->nr) {\n+\t\tstruct work_item *item = &list->items[wi];\n+\t\tint cmp = base_name_compare(item->entry.path,\n+\t\t\t\t\t    item->entry.pathlen,\n+\t\t\t\t\t    item->entry.mode,\n+\t\t\t\t\t    entry->path, entry->pathlen,\n+\t\t\t\t\t    entry->mode);\n+\t\tif (cmp > 0)\n+\t\t\tbreak;\n+\t\tif (cmp < 0) {\n+\t\t\twi++;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (S_ISDIR(item->entry.mode) && S_ISDIR(entry->mode))\n+\t\t\toid_array_append(&item->parent_trees, &entry->oid);\n+\t\treturn wi + 1;\n+\t}\n+\treturn wi;\n+}\n+\n+/*\n+ * Report an object that could not be read, telling a missing object\n+ * apart from one that exists but cannot be read, e.g. because it is\n+ * corrupt.\n+ */\n+static int unreadable_object_error(struct repository *repo,\n+\t\t\t\t   const struct object_id *oid,\n+\t\t\t\t   const char *type)\n+{\n+\tif (!odb_has_object(repo->objects, oid, 0))\n+\t\treturn error(_(\"missing %s object %s\"), type, oid_to_hex(oid));\n+\treturn error(_(\"unable to read %s object %s\"), type, oid_to_hex(oid));\n+}\n+\n+static int verify_blob(struct repository *repo,\n+\t\t       const struct object_id *oid,\n+\t\t       struct verify_state *vs)\n+{\n+\tint type;\n+\n+\tif (oidset_contains(&vs->trusted_blobs, oid))\n+\t\treturn 0;\n+\n+\tvs->blobs_checked++;\n+\ttype = odb_read_object_info(repo->objects, oid, NULL);\n+\tif (type == OBJ_BLOB) {\n+\t\toidset_insert(&vs->trusted_blobs, oid);\n+\t\treturn 0;\n+\t}\n+\tif (type >= 0)\n+\t\treturn error(_(\"object %s is a %s, not a %s\"),\n+\t\t\t     oid_to_hex(oid), type_name(type), \"blob\");\n+\tif (vs->exclude_promisor_objects &&\n+\t    is_promisor_object(repo, oid))\n+\t\treturn 0;\n+\treturn unreadable_object_error(repo, oid, \"blob\");\n+}\n+\n+/*\n+ * Read a tree object into *bufp.  Returns -1 if the object exists\n+ * but is not a tree.  Otherwise returns 0, leaving *bufp NULL if the\n+ * object could not be read.\n+ */\n+static int read_tree_object(struct object_database *odb,\n+\t\t\t    const struct object_id *oid,\n+\t\t\t    void **bufp, size_t *sizep)\n+{\n+\tenum object_type type;\n+\n+\t*bufp = odb_read_object(odb, oid, &type, sizep);\n+\tif (*bufp && type != OBJ_TREE) {\n+\t\tFREE_AND_NULL(*bufp);\n+\t\treturn error(_(\"object %s is a %s, not a %s\"),\n+\t\t\t     oid_to_hex(oid), type_name(type), \"tree\");\n+\t}\n+\treturn 0;\n+}\n+\n+/* List the entries of a tree that are not yet known to be trusted. */\n+static void list_untrusted_entries(struct verify_state *vs,\n+\t\t\t\t   const struct object_id *tree_oid,\n+\t\t\t\t   void *tree_buf, size_t tree_size,\n+\t\t\t\t   struct work_list *list)\n+{\n+\tstruct name_entry *entry = &vs->entry;\n+\n+\tinit_tree_desc(&vs->desc, tree_oid, tree_buf, tree_size);\n+\twhile (tree_entry(&vs->desc, entry)) {\n+\t\tif (S_ISGITLINK(entry->mode))\n+\t\t\tcontinue;\n+\t\tif (S_ISDIR(entry->mode)) {\n+\t\t\tif (tree_map_get(vs->trees, &entry->oid) >=\n+\t\t\t    TREE_TRUSTED)\n+\t\t\t\tcontinue;\n+\t\t\tlist->has_subtrees = true;\n+\t\t} else {\n+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry->oid))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\tALLOC_GROW(list->items, list->nr + 1, list->alloc);\n+\t\tmemset(&list->items[list->nr], 0, sizeof(list->items[0]));\n+\t\tlist->items[list->nr++].entry = *entry;\n+\t}\n+}\n+\n+/*\n+ * Scan one trusted base tree: expand it if that has not happened yet,\n+ * and collect its same-path subtrees as bases for untrusted subtrees.\n+ */\n+static int scan_base_tree(struct repository *repo, struct verify_state *vs,\n+\t\t\t  const struct object_id *base_oid,\n+\t\t\t  struct work_list *list)\n+{\n+\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n+\tstruct name_entry *entry = &vs->entry;\n+\tsize_t wi = 0, base_size;\n+\tvoid *base_buf;\n+\n+\tif (expanded && !list->has_subtrees)\n+\t\treturn 0;\n+\n+\tif (read_tree_object(repo->objects, base_oid, &base_buf, &base_size))\n+\t\treturn -1;\n+\tif (!base_buf) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, base_oid))\n+\t\t\treturn 0;\n+\t\treturn unreadable_object_error(repo, base_oid, \"tree\");\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tinit_tree_desc(&vs->desc, base_oid, base_buf, base_size);\n+\n+\twhile (tree_entry(&vs->desc, entry)) {\n+\t\tif (S_ISGITLINK(entry->mode))\n+\t\t\tcontinue;\n+\n+\t\t/* Same-path subtree as a base for an untrusted one. */\n+\t\tif (list->has_subtrees)\n+\t\t\twi = collect_subtree_bases(list, wi, entry);\n+\n+\t\t/* Expand: the base is trusted, so are its entries. */\n+\t\tif (!expanded) {\n+\t\t\tif (S_ISDIR(entry->mode))\n+\t\t\t\ttree_map_add(vs->trees, &entry->oid,\n+\t\t\t\t\t     TREE_TRUSTED);\n+\t\t\telse\n+\t\t\t\toidset_insert(&vs->trusted_blobs,\n+\t\t\t\t\t      &entry->oid);\n+\t\t}\n+\t}\n+\n+\tif (!expanded)\n+\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n+\n+\tfree(base_buf);\n+\treturn 0;\n+}\n+\n+/*\n+ * Verify a tree against the trusted base trees, loading as little as\n+ * possible:\n+ *\n+ * First list the entries that are not yet known to be trusted.  When\n+ * the parent trees have been expanded before, only the changed entries\n+ * remain.\n+ *\n+ * Then scan the trusted base trees (the same-path trees of the parent\n+ * commits).  A base that has not been expanded yet is read so that its\n+ * entries can be marked trusted; this often shows that listed entries\n+ * are trusted after all.  An already expanded base is still read when\n+ * there are untrusted subtrees, because its subtree at the same path is\n+ * what lets the untrusted subtree skip its own unchanged entries in\n+ * turn.\n+ *\n+ * Finally check that the remaining blobs exist, and recurse into the\n+ * remaining subtrees with the same-path subtrees as their bases.\n+ */\n+static int verify_tree(struct repository *repo,\n+\t\t       const struct object_id *tree_oid,\n+\t\t       const struct oid_array *trusted_bases,\n+\t\t       struct verify_state *vs, int depth)\n+{\n+\tstruct work_list list = { 0 };\n+\tsize_t i, tree_size;\n+\tvoid *tree_buf;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Enforce core.maxTreeDepth like process_tree() in list-objects.c,\n+\t * so that the full and incremental checks reject the same trees.\n+\t */\n+\tif (depth > repo->settings.max_allowed_tree_depth)\n+\t\treturn error(_(\"exceeded maximum allowed tree depth\"));\n+\n+\tif (tree_map_get(vs->trees, tree_oid) >= TREE_TRUSTED)\n+\t\treturn 0;\n+\n+\tif (read_tree_object(repo->objects, tree_oid, &tree_buf, &tree_size))\n+\t\treturn -1;\n+\tif (!tree_buf) {\n+\t\t/*\n+\t\t * A missing tree that is promised can be fetched on demand,\n+\t\t * so it is not an error, as in the full check.\n+\t\t */\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, tree_oid))\n+\t\t\treturn 0;\n+\t\treturn unreadable_object_error(repo, tree_oid, \"tree\");\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tlist_untrusted_entries(vs, tree_oid, tree_buf, tree_size, &list);\n+\n+\t/* Scan the base trees, unless nothing is left to verify. */\n+\tfor (i = 0; list.nr && i < trusted_bases->nr; i++) {\n+\t\tret = scan_base_tree(repo, vs, &trusted_bases->oid[i], &list);\n+\t\tif (ret)\n+\t\t\tgoto done;\n+\t}\n+\n+\t/*\n+\t * Check the remaining entries in tree order.  Verifying a subtree\n+\t * can show that later entries are trusted, which then makes them\n+\t * cheap to skip.\n+\t */\n+\tfor (i = 0; i < list.nr && !ret; i++) {\n+\t\tstruct work_item *item = &list.items[i];\n+\n+\t\tif (S_ISDIR(item->entry.mode))\n+\t\t\tret = verify_tree(repo, &item->entry.oid,\n+\t\t\t\t\t  &item->parent_trees, vs, depth + 1);\n+\t\telse\n+\t\t\tret = verify_blob(repo, &item->entry.oid, vs);\n+\t}\n+\tif (!ret)\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_EXPANDED);\n+\n+done:\n+\tfree(tree_buf);\n+\twork_list_clear(&list);\n+\treturn ret;\n+}\n+\n+static int verify_commit_tree(struct repository *repo,\n+\t\t\t      struct commit *commit,\n+\t\t\t      struct verify_state *vs)\n+{\n+\tstruct oid_array parent_trees = OID_ARRAY_INIT;\n+\tconst struct object_id *tree_oid;\n+\tstruct commit_list *p;\n+\tint ret;\n+\n+\t/*\n+\t * Parent trees are trusted: boundary parents are already\n+\t * connected, and earlier incoming parents were verified\n+\t * first due to the topological processing order.\n+\t *\n+\t * The revision walk skips missing parents that are promised\n+\t * (see process_parents()), so do the same here.  Such a parent\n+\t * contributes no base, which only means less of the new tree\n+\t * can be skipped.\n+\t */\n+\tfor (p = commit->parents; p; p = p->next) {\n+\t\tif (repo_parse_commit_gently(repo, p->item, 1) < 0) {\n+\t\t\tif (vs->exclude_promisor_objects &&\n+\t\t\t    is_promisor_object(repo, &p->item->object.oid))\n+\t\t\t\tcontinue;\n+\t\t\tret = error(_(\"unable to parse commit %s\"),\n+\t\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\t\tgoto out;\n+\t\t}\n+\t\ttree_oid = get_commit_tree_oid(p->item);\n+\t\tif (!tree_oid) {\n+\t\t\tret = error(_(\"unable to load root tree for commit %s\"),\n+\t\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\t\tgoto out;\n+\t\t}\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n+\t\toid_array_append(&parent_trees, tree_oid);\n+\t}\n+\n+\ttree_oid = get_commit_tree_oid(commit);\n+\tif (!tree_oid) {\n+\t\tret = error(_(\"unable to load root tree for commit %s\"),\n+\t\t\t    oid_to_hex(&commit->object.oid));\n+\t\tgoto out;\n+\t}\n+\tret = verify_tree(repo, tree_oid, &parent_trees, vs, 0);\n+out:\n+\toid_array_clear(&parent_trees);\n+\treturn ret;\n+}\n+\n+int verify_commits_incremental(struct repository *repo,\n+\t\t\t       struct commit_list **commits,\n+\t\t\t       int exclude_promisor_objects)\n+{\n+\tstruct verify_state vs = { 0 };\n+\tstruct commit_list *iter;\n+\tunsigned nr_before;\n+\tint ret = 0;\n+\n+\tif (repo->fetch_if_missing)\n+\t\tBUG(\"verify_commits_incremental must not be called \"\n+\t\t    \"with fetch_if_missing set\");\n+\n+\tvs.trees = kh_init_oid_tree();\n+\tvs.exclude_promisor_objects = exclude_promisor_objects;\n+\n+\t/*\n+\t * Ancestors must be verified before descendants so that parent\n+\t * trees can be trusted without re-verification.  Sort explicitly\n+\t * rather than relying on the caller's ordering.\n+\t *\n+\t * sort_in_topological_order() silently drops cycle members,\n+\t * so explicitly check if the size has changed.\n+\t */\n+\tnr_before = commit_list_count(*commits);\n+\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n+\tif (commit_list_count(*commits) < nr_before) {\n+\t\tret = error(_(\"cycle detected in incoming commit graph\"));\n+\t\tgoto out;\n+\t}\n+\n+\t*commits = commit_list_reverse(*commits);\n+\n+\tfor (iter = *commits; iter && !ret; iter = iter->next)\n+\t\tret = verify_commit_tree(repo, iter->item, &vs);\n+\n+out:\n+\tkh_destroy_oid_tree(vs.trees);\n+\toidset_clear(&vs.trusted_blobs);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"trees_loaded\", (intmax_t)vs.trees_loaded);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"blobs_checked\", (intmax_t)vs.blobs_checked);\n+\treturn ret;\n+}\ndiff --git a/tree-verify.h b/tree-verify.h\nnew file mode 100644\nindex 0000000000..ac7092da1c\n--- /dev/null\n+++ b/tree-verify.h\n@@ -0,0 +1,15 @@\n+#ifndef TREE_VERIFY_H\n+#define TREE_VERIFY_H\n+\n+struct commit_list;\n+struct repository;\n+\n+/*\n+ * Verify trees of commits incrementally against their parents.\n+ * Returns 0 on success, or reports an error and returns -1.\n+ */\n+int verify_commits_incremental(struct repository *repo,\n+\t\t\t       struct commit_list **commits,\n+\t\t\t       int exclude_promisor_objects);\n+\n+#endif /* TREE_VERIFY_H */\n-- \ngitgitgadget\n\n\n"},{"id":"555721","messageId":"f6031967e6995060ebdf3ccc8cc280b69c0aa3fe.1792581744.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v4.git.1792581744.gitgitgadget@gmail.com","subject":"[PATCH v4 6/6] connected: add \"auto\" connectivity check mode","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-21T11:22:24Z","receivedAt":"2026-10-21T11:22:24Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe incremental connectivity check pays a per-commit cost.  For\nevery incoming commit it reads the new trees along the changed\npaths, and the trees at the same paths in its parents to compare\nagainst.  A parent tree is often read again even though an earlier\ncommit already read it: on git.git, about a third of the tree reads\nfor 2000 incoming commits are such re-reads.  The full check\ninstead pays a mostly fixed cost for walking the boundary trees,\nso with many incoming commits the incremental check becomes the\nslower one (see the previous commit).  Users who mostly\nreceive small pushes but occasionally a large one should not have\nto choose between the two.\n\nAdd an \"auto\" value for transfer.connectivityCheck that picks the\nmode based on the incoming history.  When the revision walk is\nlimited, which is the case whenever the receiving repository has\nreferences, the incoming commit list is already computed by\nlimit_list(), so counting it is cheap.  \"auto\" then uses the\nincremental check for up to 1000 incoming commits, and the full\ncheck above that.\n\nThe cutoff is a fixed constant based on the p5413 fixture in the\nprevious commit, where breakeven is between 500 and 3000 commits.\nIt is conservative for repositories with large trees, where the\nfixed cost of the full check is higher.  Measured on real\nrepositories (rev-list against a single boundary commit, mean\nof at least 10 runs):\n\n    repository   files  commits    full  incr.  full/incr\n    internal A    556K     1000   1.71s  0.61s    2.8x\n    internal A    556K     4000   2.78s  2.83s    1.0x\n    internal B    288K     1000   1.78s  0.88s    2.0x\n    linux          95K      598   0.33s  0.12s    2.7x\n    linux          95K     1728   0.56s  0.35s    1.6x\n    git           4.9K      984   0.22s  0.27s    0.8x\n\nWith small trees incremental can be slower within the cutoff,\nbut both modes are fast enough there (0.22s vs 0.27s for git\nabove) that the difference does not matter in practice.\n\nNo single cutoff is optimal for every repository, so it can be set\nwith \"auto=<cutoff>\"; plain \"auto\" uses 1000.  A malformed cutoff\nis treated like any other unknown value: it falls back to \"full\"\nwith a warning.\n\nSuggested-by: Patrick Steinhardt <ps@pks.im>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc  |  6 ++++++\n Documentation/rev-list-options.adoc |  2 +-\n builtin/rev-list.c                  | 28 ++++++++++++++++++++++++++--\n connected.c                         |  9 ++++++++-\n t/t5413-connectivity-check.sh       | 25 +++++++++++++++++++++++--\n 5 files changed, 64 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex cba7597a58..f77b25ff8b 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -18,6 +18,12 @@ transfer.connectivityCheck::\n \tthan `full` when there are many incoming commits.\n \tFalls back to `full` for clones, pushes into an empty\n \trepository and deepening fetches.\n+`auto`;;\n+`auto=<cutoff>`;;\n+\tChoose between `full` and `incremental` with a heuristic.\n+\tCurrently `incremental` is used when there are at most\n+\t_<cutoff>_ incoming commits (1000 by default), and `full`\n+\totherwise.  The heuristic may change in future versions.\n --\n +\n Unknown values are treated as `full`, with a warning.\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 2023ad5f75..285fb3809f 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1103,7 +1103,7 @@ It is an error to use `--missing-only` without `--missing=print` or\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n \n-`--connectivity-check=(full|incremental)`::\n+`--connectivity-check=(full|incremental|auto[=<cutoff>])`::\n \t(For internal use only.)  Used by `check_connected()` to\n \tselect the connectivity check algorithm; see\n \t`transfer.connectivityCheck` in linkgit:git-config[1].\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 2b428c0c84..85efc0fac8 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -29,24 +29,36 @@\n #include \"quote.h\"\n #include \"strbuf.h\"\n #include \"tree-verify.h\"\n+#include \"parse.h\"\n+\n+#define INCREMENTAL_MAX_COMMITS 1000\n \n enum connectivity_check_mode {\n \tCONNECTIVITY_CHECK_FULL,\n \tCONNECTIVITY_CHECK_INCREMENTAL,\n+\tCONNECTIVITY_CHECK_AUTO,\n };\n \n struct connectivity_check {\n \tenum connectivity_check_mode mode;\n+\tunsigned long max_commits;\n };\n \n static void parse_connectivity_check(struct repository *repo,\n \t\t\t\t     const char *arg,\n \t\t\t\t     struct connectivity_check *check)\n {\n+\tconst char *cutoff;\n+\n \tif (!strcmp(arg, \"full\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n \telse if (!strcmp(arg, \"incremental\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n+\telse if (!strcmp(arg, \"auto\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_AUTO;\n+\telse if (skip_prefix(arg, \"auto=\", &cutoff) &&\n+\t\t git_parse_ulong(cutoff, &check->max_commits))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_AUTO;\n \telse\n \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n \n@@ -752,7 +764,9 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n-\tstruct connectivity_check connectivity_check = { 0 };\n+\tstruct connectivity_check connectivity_check = {\n+\t\t.max_commits = INCREMENTAL_MAX_COMMITS,\n+\t};\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -1005,9 +1019,19 @@ int cmd_rev_list(int argc,\n \t * The incremental check would still work, but it would verify\n \t * the whole object graph like the full check, with extra\n \t * per-commit overhead on top, so use the full check.\n+\t *\n+\t * Otherwise the walk is limited, and revs.commits already holds\n+\t * all incoming commits, so counting them for \"auto\" is cheap.\n \t */\n-\tif (!revs.limited)\n+\tif (!revs.limited) {\n \t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\t} else if (connectivity_check.mode == CONNECTIVITY_CHECK_AUTO) {\n+\t\tif (commit_list_count(revs.commits) <=\n+\t\t    connectivity_check.max_commits)\n+\t\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_INCREMENTAL;\n+\t\telse\n+\t\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\t}\n \n \tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n \t\tstruct commit *commit;\ndiff --git a/connected.c b/connected.c\nindex c032987410..8ceaa45644 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -3,6 +3,7 @@\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"connected.h\"\n@@ -74,7 +75,8 @@ static void connectivity_check_mode(struct repository *repo,\n \t\t\t\t    struct check_connected_options *opt,\n \t\t\t\t    struct strbuf *mode)\n {\n-\tconst char *algorithm = NULL;\n+\tconst char *algorithm = NULL, *cutoff;\n+\tunsigned long max_commits;\n \n \t/*\n \t * Deepening fetches do not have the usual boundary of\n@@ -93,6 +95,11 @@ static void connectivity_check_mode(struct repository *repo,\n \t\tstrbuf_addstr(mode, \"full\");\n \telse if (!strcasecmp(algorithm, \"incremental\"))\n \t\tstrbuf_addstr(mode, \"incremental\");\n+\telse if (!strcasecmp(algorithm, \"auto\"))\n+\t\tstrbuf_addstr(mode, \"auto\");\n+\telse if (skip_iprefix(algorithm, \"auto=\", &cutoff) &&\n+\t\t git_parse_ulong(cutoff, &max_commits))\n+\t\tstrbuf_addf(mode, \"auto=%lu\", max_commits);\n \telse {\n \t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n \t\t\t  \"using 'full'\"), algorithm);\ndiff --git a/t/t5413-connectivity-check.sh b/t/t5413-connectivity-check.sh\nindex 109ad17cff..60edf99b63 100755\n--- a/t/t5413-connectivity-check.sh\n+++ b/t/t5413-connectivity-check.sh\n@@ -725,7 +725,7 @@ test_expect_success 'unknown transfer.connectivityCheck falls back to full' '\n \ttest_when_finished \"rm -rf dst\" &&\n \tgit init --bare dst &&\n \tgit -C dst fetch ../src base:refs/heads/main &&\n-\tfor value in bogus\n+\tfor value in bogus auto= auto=x\n \tdo\n \t\tset_connectivity_check dst \"$value\" &&\n \t\trun_traced git -C dst fetch ../src main &&\n@@ -753,7 +753,7 @@ test_expect_success 'fetch uses incremental when configured' '\n \texpect_mode incremental\n '\n \n-for mode in incremental\n+for mode in incremental auto\n do\n \ttest_expect_success \"push with $mode into an empty repository uses full\" '\n \t\ttest_when_finished \"rm -rf dst\" &&\n@@ -771,4 +771,25 @@ do\n \t'\n done\n \n+for value in auto auto=1\n+do\n+\ttest_expect_success \"push with $value uses incremental up to the cutoff\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\tgit init --bare dst &&\n+\t\tgit -C src push ../dst base:refs/heads/main &&\n+\t\tset_connectivity_check dst $value &&\n+\t\trun_traced git -C src push ../dst main &&\n+\t\texpect_mode incremental\n+\t'\n+done\n+\n+test_expect_success 'push with auto uses full above the cutoff' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C src push ../dst base:refs/heads/main &&\n+\tset_connectivity_check dst auto=0 &&\n+\trun_traced git -C src push ../dst main &&\n+\texpect_mode full\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"555785","messageId":"xmqqecdin6xr.fsf@gitster.g","threadId":"66326","inReplyTo":"d76dae801b3303ef4271f71c7b3999c10cac8556.1792581744.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-21T20:18:40Z","receivedAt":"2026-10-21T20:18:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> -static void parse_connectivity_check(struct repository *repo UNUSED,\n> +static void parse_connectivity_check(struct repository *repo,\n>  \t\t\t\t     const char *arg,\n>  \t\t\t\t     struct connectivity_check *check)\n>  {\n>  \tif (!strcmp(arg, \"full\"))\n>  \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n> +\telse if (!strcmp(arg, \"incremental\"))\n> +\t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n>  \telse\n>  \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n> +\n> +\tif (check->mode != CONNECTIVITY_CHECK_FULL)\n> +\t\trepo->fetch_if_missing = 0;\n>  }\n\nThis assumes that everybody that is different from full want to\navoid lazy fetching.  It may be true right now as you have only full\nand incremental, but it somehow feels like a trap waiting to bite\nfuture developers.\n\n> @@ -992,6 +999,29 @@ int cmd_rev_list(int argc,\n>  \n>  \tprepare_maximal_independent(&revs);\n>  \n> +\t/*\n> +\t * Without a boundary of negative revisions, as in a clone or a\n> +\t * push into an empty repository, every incoming commit is new.\n> +\t * The incremental check would still work, but it would verify\n> +\t * the whole object graph like the full check, with extra\n> +\t * per-commit overhead on top, so use the full check.\n> +\t */\n> +\tif (!revs.limited)\n> +\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n> +\n> +\tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n> +\t\tstruct commit *commit;\n> +\t\tstruct commit_list *new_commits = NULL;\n> +\n> +\t\twhile ((commit = get_revision(&revs)) != NULL)\n> +\t\t\tcommit_list_insert(commit, &new_commits);\n> +\n> +\t\tif (verify_commits_incremental(repo, &new_commits,\n> +\t\t\t\t\t       revs.exclude_promisor_objects))\n> +\t\t\tdie(_(\"connectivity check failed\"));\n> +\t\tcommit_list_free(new_commits);\n> +\t}\n> +\n\nWe only call verify_commits_incremental() when check.mode says INCREMENTAL,\nwhich makes sense.\n\n> +static int verify_blob(struct repository *repo,\n> +\t\t       const struct object_id *oid,\n> +\t\t       struct verify_state *vs)\n> +{\n> +\tint type;\n> +\n> +\tif (oidset_contains(&vs->trusted_blobs, oid))\n> +\t\treturn 0;\n> +\n> +\tvs->blobs_checked++;\n> +\ttype = odb_read_object_info(repo->objects, oid, NULL);\n\nI wonder if this needs to be _extended() kind that passes a flag bit\nto avoid fetching from promisors.  repo->fetch_if_missing that is\nset does prevent lazy fetching, but I wonder if it is better to be\nmore explicit.\n\n> +int verify_commits_incremental(struct repository *repo,\n> +\t\t\t       struct commit_list **commits,\n> +\t\t\t       int exclude_promisor_objects)\n> +{\n> +\tstruct verify_state vs = { 0 };\n> +\tstruct commit_list *iter;\n> +\tunsigned nr_before;\n> +\tint ret = 0;\n> +\n> +\tif (repo->fetch_if_missing)\n> +\t\tBUG(\"verify_commits_incremental must not be called \"\n> +\t\t    \"with fetch_if_missing set\");\n\nAnd we make sure that the caller flipped the bit before they decide\nto call us, which makes sense.\n\n\n"},{"id":"555829","messageId":"CAL71e4Pe_NRxeu9_+pnp=pBOk3SBNyePUbT5DDtDhd1bJpwJng@mail.gmail.com","threadId":"66326","inReplyTo":"xmqqecdin6xr.fsf@gitster.g","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-22T07:00:12Z","receivedAt":"2026-10-22T07:00:12Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Wed, 21 Oct 2026 at 22:18, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> > -static void parse_connectivity_check(struct repository *repo UNUSED,\n> > +static void parse_connectivity_check(struct repository *repo,\n> >                                    const char *arg,\n> >                                    struct connectivity_check *check)\n> >  {\n> >       if (!strcmp(arg, \"full\"))\n> >               check->mode = CONNECTIVITY_CHECK_FULL;\n> > +     else if (!strcmp(arg, \"incremental\"))\n> > +             check->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n> >       else\n> >               die(_(\"unknown --connectivity-check mode '%s'\"), arg);\n> > +\n> > +     if (check->mode != CONNECTIVITY_CHECK_FULL)\n> > +             repo->fetch_if_missing = 0;\n> >  }\n>\n> This assumes that everybody that is different from full want to\n> avoid lazy fetching.  It may be true right now as you have only full\n> and incremental, but it somehow feels like a trap waiting to bite\n> future developers.\n\nGood point, and I think I could simplify this to always set\nfetch_if_missing = 0 for any type of connectivity check, since\nno connectivity check should be allowed to trigger remote\ncalls.\n\nI think the reason this was not in place before was that\nrev-list did not know that the intent was a connectivity\ncheck but now by passing this new --connectivity-check\nit always does.\n\n> > +static int verify_blob(struct repository *repo,\n> > +                    const struct object_id *oid,\n> > +                    struct verify_state *vs)\n> > +{\n> > +     int type;\n> > +\n> > +     if (oidset_contains(&vs->trusted_blobs, oid))\n> > +             return 0;\n> > +\n> > +     vs->blobs_checked++;\n> > +     type = odb_read_object_info(repo->objects, oid, NULL);\n>\n> I wonder if this needs to be _extended() kind that passes a flag bit\n> to avoid fetching from promisors.  repo->fetch_if_missing that is\n> set does prevent lazy fetching, but I wonder if it is better to be\n> more explicit.\n\nYes, and an earlier internal prototype (that I never sent) did\nthis. Blob and tree reads are easy to switch to\nodb_read_object_info_extended() with OBJECT_INFO_SKIP_FETCH_OBJECT,\nbut parsing the parent commits goes through\nrepo_parse_commit_gently(), which has no such flag.  That would\nneed a no-fetch flag passed down through commit parsing, which I\nthink is not too bad on its own.\n\nBut even then, the revision walk around the verifier and the full\ncheck would still depend on fetch_if_missing, so the verifier alone\nwould not let us drop it.  I think it is better to refactor that all\nat once in a separate follow-up, since this series is already big\nenough.  However, if you would prefer to have it now already, I\ncould try to add a new commit on top for the next reroll.\n\n(My earlier prototype was more complex because it ran in-process\nrather than via rev-list, so it also had to avoid die(), which\ntriggered even more low level function changes.  That does not\napply here though, fortunately.)\n\nThanks for the review,\nKristofer\n\n"},{"id":"555834","messageId":"CAL71e4MepqPJOqUQxoNc_8Q6PaWZwQX45tGZ2z4+iqGnNCDAEw@mail.gmail.com","threadId":"66326","inReplyTo":"CAL71e4Pe_NRxeu9_+pnp=pBOk3SBNyePUbT5DDtDhd1bJpwJng@mail.gmail.com","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-22T08:35:41Z","receivedAt":"2026-10-22T08:35:41Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 22 Oct 2026 at 09:00, Kristofer Karlsson <krka@spotify.com> wrote:\n>\n> On Wed, 21 Oct 2026 at 22:18, Junio C Hamano <gitster@pobox.com> wrote:\n> >\n> > I wonder if this needs to be _extended() kind that passes a flag bit\n> > to avoid fetching from promisors.  repo->fetch_if_missing that is\n> > set does prevent lazy fetching, but I wonder if it is better to be\n> > more explicit.\n>\n> Yes, and an earlier internal prototype (that I never sent) did\n> this. Blob and tree reads are easy to switch to\n> odb_read_object_info_extended() with OBJECT_INFO_SKIP_FETCH_OBJECT,\n> but parsing the parent commits goes through\n> repo_parse_commit_gently(), which has no such flag.  That would\n> need a no-fetch flag passed down through commit parsing, which I\n> think is not too bad on its own.\n\nSorry, I think I was wrong about this part -- commit parsing\nnever fetches.  I think this means my initial problems were\npartly to die().  I will re-explore this and see if eliminating\nfetch_if_missing = 0 is less work than I first thought.\n\n"},{"id":"555872","messageId":"xmqq4iedkd2u.fsf@gitster.g","threadId":"66326","inReplyTo":"CAL71e4Pe_NRxeu9_+pnp=pBOk3SBNyePUbT5DDtDhd1bJpwJng@mail.gmail.com","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-22T14:46:33Z","receivedAt":"2026-10-22T14:46:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> Yes, and an earlier internal prototype (that I never sent) did\n> this. ...\n> But even then, the revision walk around the verifier and the full\n> check would still depend on fetch_if_missing, so the verifier alone\n> would not let us drop it.\n\nAs use of _extended there in new code will certainly be a way to\navoid making things worse, the above in distilled form should be in\nthe proposed commit log message to help future readers who may\nwonder why these new code paths are written to rely on the\nfetch_if_missing flag.\n\n> I think it is better to refactor that all\n> at once in a separate follow-up, ...\n\nYeah, I think it is a good idea to leave it outside the scope of\nthis one.\n\nThanks.\n\n"},{"id":"555877","messageId":"CAL71e4Ngsxv=JU_qzVGVyHPZiB2Qzvjn22NeVQAtZOdCr2vyXQ@mail.gmail.com","threadId":"66326","inReplyTo":"xmqq4iedkd2u.fsf@gitster.g","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-10-22T16:11:01Z","receivedAt":"2026-10-22T16:11:01Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Thu, 22 Oct 2026 at 16:46, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Kristofer Karlsson <krka@spotify.com> writes:\n>\n> > Yes, and an earlier internal prototype (that I never sent) did\n> > this. ...\n> > But even then, the revision walk around the verifier and the full\n> > check would still depend on fetch_if_missing, so the verifier alone\n> > would not let us drop it.\n>\n> As use of _extended there in new code will certainly be a way to\n> avoid making things worse, the above in distilled form should be in\n> the proposed commit log message to help future readers who may\n> wonder why these new code paths are written to rely on the\n> fetch_if_missing flag.\n\nAh, I was not aware fetch_if_missing is something we want to\nmove away from, but now I see it's mentioned in the docs.\n\nI looked into the code more after your pointers and I think it's\nindeed doable (and not too messy, just a bit more code) to\nuse odb_read_object_info_extended() instead and completely stop\ntouching repo->fetch_if_missing. It's still being touched by\nthe rest of the rev-list code, but at least this series no longer\nmakes it worse. The full check is unaffected, since\ncheck_connected() already passes --exclude-promisor-objects in a\npartial clone, which clears the flag as before.\n\nTo check it, I ran rev-list --connectivity-check=incremental by hand\nin a partial clone, deliberately omitting --exclude-promisor-objects,\non commits that reference a promised blob and a promised tree. Both\nwere reported as missing and neither was fetched. I did not turn\nthat into a test, since check_connected() never invokes rev-list\nthat way, and the existing tests all pass.\n\nThis ends up nicer than what I had -- I can also get rid of the\nBUG() guard for checking that flag state.\n\nSo, I think my plan is to squash this change into v5.\n\nDo you still want me to include this reasoning in the commit,\nor is that now unnecessary given that it would no longer touch\nfetch_if_missing?\n\nThanks, this really helped improve the patch,\nKristofer\n\n"},{"id":"556017","messageId":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.git.1789379276.gitgitgadget@gmail.com","subject":"[PATCH v5 0/6] connected: add incremental connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:53Z","receivedAt":"2026-10-23T13:11:53Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"This series adds an incremental mode for the connectivity check, gated\nbehind transfer.connectivityCheck=incremental or \"auto\" (no expected changes\nunless you opt in).\n\nThe intent is to solve the problem of the connectivity check slowing down as\nthe number of reachable objects from the boundary grows.\n\nIt relates to the RFC I sent out earlier:\n\n[RFC] check_connected: toward incoming-proportional cost\nhttps://lore.kernel.org/git/CAL71e4Nf=-zCrfN7ghEVGq11irajJhtdxYZgKe0Ycux0qs1ZvQ@mail.gmail.com/\n\n\n\nDesign\n======\n\nThe verifier runs inside the same rev-list subprocess that check_connected()\nalready spawns, triggered by a new internal option --connectivity-check=.\nAfter get_revision() collects the incoming commits, the verifier processes\nthem in topological order (ancestors before descendants).\n\nThe idea is to keep a set of trusted objects, shared across the incoming\ncommits, that grows over time. We visit the new/untrusted commit trees and\ndo a comparison walk over the trees of their parents. Entries discovered on\nthe trusted parent side are remembered as trusted, which lets later\nverification skip matching objects and avoid descending into unchanged\nsubtrees.\n\nMore algorithmic details are in\nDocumentation/technical/connectivity-check.adoc.\n\n\nBenchmarks\n==========\n\nI'll just mention a short summary here, to avoid repeating what's already in\nthe commit messages (5/6 and 6/6). The incremental mode is faster than the\nfull mode when there are few commits to verify and when the active object\ntree is large. In the happy case, the work tracks the changed paths and\ntheir comparison trees rather than the full reachable object closure, which\nsubstantially reduces the dependence on total repository size. I've seen\nspeedups of up to about 18x in the synthetic perf tests.\n\nRunning against a large real-world repo (3.4 GB boundary closure), the\nnumbers are more dramatic. These were measured on v1 and not re-run since;\nthe real-repo numbers for v4 are in 6/6. All timings use rev-list directly\nwith --not HEADN, isolating the tree verification cost from the\nboundary-finding cost:\n\ncommits    full  incr.  speedup   full RSS  incr. RSS\n      1    1.9s  0.01s    190x      3.4 GB     14 MB\n     10    1.9s  0.04s     48x      3.4 GB    125 MB\n    100    1.9s  0.37s      5x      3.4 GB    1.1 GB\n\n\nThe full mode takes ~1.9s regardless of commit count because it is dominated\nby walking the boundary closure. Incremental scales with the number of\nincoming commits and the paths they touch. Memory follows the same pattern:\nincremental uses a fraction of the full mode's RSS for small pushes,\nconverging only when many commits are verified.\n\nThere are also regression cases in the synthetic fixtures. With long\nincoming histories, the extra parent-tree scans accumulate; in the synthetic\nfixture incremental is about 1.5x slower at 10000 commits. Per-commit\nchanges have less impact than expected: even when every directory is\ntouched, incremental remains competitive.\n\nIn the cases I have tested the regression has remained modest, and it shows\nup with long incoming histories, while the target case (small pushes to\nlarge repositories) sees the largest speedup. That is what the \"auto\" mode\nis for: it uses the incremental check for small numbers of incoming commits\nand the full check otherwise, with a cutoff that can be configured.\n\nDeepening fetches currently fall back to the full check because the full\ncheck omits --not --all for deepening -- there is no existing-reference\nboundary at which the walk can stop. An incremental approach is possible\nhere too -- using the old shallow roots as the trusted boundary and walking\nthe deepened ancestry forward -- but that is a separate change and left for\nfuture work. Deepening is also less common than regular fetch and\nreceive-pack, where the speedup matters most.\n\n\nTest coverage\n=============\n\nMost correctness cases in t5413-connectivity-check.sh are run in both full\nand incremental modes to check semantic equivalence. Selected cases\nadditionally assert trace2 tree/blob counts for the incremental mode, to\nverify that unchanged portions of the object graph are actually skipped. The\ntests cover:\n\n * Corruption detection: missing blobs, missing trees, type mismatches,\n   malformed trees (unparseable, mid-tree corruption), and corrupt blobs\n   (incremental only, since the full check does not read blobs)\n * Tree optimization: trace2 assertions confirm unchanged subtrees are\n   skipped, subtree moves, merge parent boundaries\n * Root commits (no parents -- verifies full tree closure)\n * Partial clones: missing promised blobs, missing promised trees, missing\n   promised parent commits, verification of local commits\n * Replacement objects (with and without GIT_NO_REPLACE_OBJECTS)\n * Shallow boundaries\n * Deepening fetches (falls back to full check)\n * Algorithm selection: clones and pushes into an empty repository use the\n   full check, \"auto\" and its cutoff, and unknown config values\n * Tree depth: the incremental check honors core.maxTreeDepth (t6700)\n * Integration: real push, fetch, and clone\n\nMost tests call git rev-list directly with the appropriate flags;\nintegration tests exercise the full check_connected() path through push,\nfetch, and clone.\n\n\nAlternatives considered\n=======================\n\nMy first prototype ran the verifier in-process inside connected.c. This\nrequired a second rev-list subprocess just for boundary finding, _nofetch\nvariants of several object-reading functions to prevent lazy fetches in\npartial clones, explicit shallow-file plumbing, and careful avoidance of\ndie() in all code paths reachable from the verifier. The result worked but\nwas fragile and touched many files.\n\nMoving the verifier into the rev-list subprocess eliminated all of those\nproblems: in partial clones the existing --exclude-promisor-objects handling\nalready disables lazy fetching, a die() in existing helpers only ends the\nsubprocess, shallow and replacement semantics are established before the\nverifier runs, and error routing comes for free via stderr.\n\nSo while I liked the idea of doing the check in-process, making that work\nended up being a lot more complex.\n\n\nNext steps\n==========\n\nThis series only addresses tree verification; the other significant cost is\nfinding the commit boundary, especially for repos with many refs. I already\nhave some prototypes for optimizing that too, and if this ends up landing,\nthat would be something I would start polishing up.\n\n\nChanges since v4\n================\n\n * The series no longer touches repo->fetch_if_missing, which Junio pointed\n   out was a trap when cleared only for some modes. The incremental verifier\n   instead reads blobs and trees with odb_read_object_info_extended() and\n   OBJECT_INFO_SKIP_FETCH_OBJECT, so it never lazily fetches on its own;\n   commit parsing already never fetches. The BUG() guard on the flag is\n   gone, and parse_connectivity_check() no longer takes a repository. The\n   full check is unchanged: in a partial clone check_connected() passes\n   --exclude-promisor-objects as before.\n * A tree that cannot be read because it is corrupt is now reported as an\n   error, like a corrupt blob, instead of dying.\n\n\nChanges since v3\n================\n\n * Split, along the lines Patrick suggested, into three steps: the config\n   option and the internal --connectivity-check= rev-list option with only\n   \"full\" (4/6), the incremental check (5/6), and the \"auto\" mode (6/6),\n   since choosing between the two is subtle enough to review on its own.\n\n * The internal rev-list option --verify-trees-incremental is replaced by\n   --connectivity-check=, which check_connected() now always passes.\n   rev-list dies on an unknown mode, since only check_connected() passes it.\n\n * An unknown transfer.connectivityCheck value now falls back to full with a\n   warning instead of dying, so config written for a newer Git does not\n   break fetches.\n\n * \"auto\" takes an optional cutoff, \"auto=\", as Patrick suggested,\n   defaulting to 1000.\n\n * Clones and pushes into an empty repository use the full check even with\n   incremental or auto, since there is no boundary to compare against (see\n   5/6). check_connected() now decides the deepening fallback before reading\n   the config.\n\n * New patch 3/6 drops the dependency on the_repository in connected.c, as\n   Junio suggested. The repository is passed to check_connected() as a\n   parameter rather than in check_connected_options, like other functions\n   that take a repository.\n\n * Use parse_commit_gently() instead of dying on a parent that cannot be\n   parsed, which Patrick pointed out could happen in a partial clone with\n   missing commits. As in the revision walk, a missing parent that is\n   promised is skipped; added a test for that. More generally, the verifier\n   adds no new die() calls: it returns errors, and rev-list dies once when\n   the check fails.\n\n * An object that cannot be read is now reported as \"missing\" only when it\n   does not exist, and as \"unable to read\" otherwise, e.g. for a corrupt\n   blob.\n\n * Renamed variables to parent_trees / trusted_bases to make the trust\n   direction clearer.\n\n * size_t for counts and sizes.\n\n * verify_tree() still recurses, bounded by core.maxTreeDepth like the other\n   tree walkers. Its frame was made smaller by moving the tree descriptor\n   scratch space into the shared state: with gcc -O2 its frame is 272 bytes,\n   against 240 for process_tree(), so at the default depth of 2048 it needs\n   about the same stack as \"rev-list --objects\". The listing of untrusted\n   entries and the scan of the base trees are split out into helpers,\n   without changing the logic. New t6700 tests check that the incremental\n   check (given a boundary, so that it is not replaced by the full check)\n   honors core.maxTreeDepth and fails gracefully at the default limit.\n   \n   I tried a work queue as suggested, but it got messy, and it would be\n   stricter than the code we already rely on: the full check's\n   process_tree() recurses over trees in the same way, and be20128bfa (add\n   core.maxTreeDepth) settled on bounding tree recursion rather than\n   removing it. If we want an iterative walk, I think it belongs in a\n   separate series that also covers process_tree().\n\n * Also addressed from v3 review: the technical doc is listed in the\n   Makefile and in meson.build, the perf tests require PERL, the leftover\n   p5412 is now p5413, \"verified against these marks\" is reworded, int flags\n   are bool, the tree state map's byte-sized values are explained in a\n   comment, and the parent tree expansion in tree-verify.c has comments\n   explaining it.\n\n * Documentation: reworded the technical doc, including a shorter\n   description of the promisor fast path; transfer.connectivityCheck\n   documents the fallbacks to full; and the rev-list option description is\n   trimmed to an internal-use note.\n\n * The --connectivity-check parsing in rev-list moved into a small helper,\n   with the mode and cutoff in a struct local to cmd_rev_list() instead of\n   globals.\n\n * Tests are faster and simpler: the auto cutoff is tested with \"auto=\"\n   instead of building long histories, and the mode selection tests share\n   two small helpers.\n\n * p5413 passes the mode explicitly, and the tables in 5/6 were re-measured;\n   they now report the best of 3 runs, which is what the perf framework\n   reports, rather than \"median of 3\".\n\n\nChanges since v2\n================\n\nAddresses Patrick's review (thanks!).\n\n * Split from 2 commits to 4: documentation, perf test baseline, config\n   plumbing + rev-list flag, then the incremental implementation.\n * Documentation rewrites: added unreachable-objects caveat, removed code\n   references to reduce staleness risk, replaced three-phase list with\n   cost-focused prose, rewrote trust model to explicitly describe boundary\n   mechanism and inductive argument, and several smaller clarifications.\n * Extracted tree_oid variable in verify_commit_tree() to avoid calling\n   get_commit_tree_oid(commit) twice.\n * Rebased onto latest master to resolve a meson.build conflict (t5412 was\n   taken by another topic). Renumbered test and perf files to t5413/p5413.\n\n\nChanges since v1\n================\n\n * Guard against get_commit_tree_oid() returning NULL in\n   verify_commit_tree(), dying with a clear message instead of a NULL\n   dereference. (Thanks Junio for catching this)\n * Set fetch_if_missing = 0 in the rev-list early option parse when\n   --verify-trees-incremental is seen, and add a BUG() assertion in\n   verify_commits_incremental() to catch any future caller that forgets.\n   This makes the lazy-fetch protection explicit rather than relying solely\n   on --exclude-promisor-objects having set it earlier. (Thanks again to\n   Junio)\n * Add a test that verifies promised blobs are not lazy-fetched during the\n   incremental check.\n * Reuse the existing 3-argument type-mismatch format string to avoid adding\n   new strings for localization.\n\nThanks, Kristofer\n\ncc: Kristofer Karlsson krka@spotify.com cc: Patrick Steinhardt ps@pks.im\n\nKristofer Karlsson (6):\n  Documentation: describe connectivity checking\n  p5413: add perf tests for connectivity check\n  connected: drop dependency on `the_repository`\n  connected: add transfer.connectivityCheck config\n  connected: add incremental connectivity check via rev-list\n  connected: add \"auto\" connectivity check mode\n\n Documentation/Makefile                        |   1 +\n Documentation/config/transfer.adoc            |  30 +\n Documentation/rev-list-options.adoc           |   5 +\n .../technical/connectivity-check.adoc         | 259 ++++++\n Documentation/technical/meson.build           |   1 +\n Makefile                                      |   1 +\n builtin/clone.c                               |   2 +-\n builtin/fetch.c                               |   5 +-\n builtin/receive-pack.c                        |  12 +-\n builtin/rev-list.c                            |  73 ++\n bundle.c                                      |   2 +-\n connected.c                                   |  67 +-\n connected.h                                   |   4 +-\n fetch-pack.c                                  |   3 +-\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n ...enerate-repo-p5413-connectivity-check.perl |  44 +\n t/perf/p5413-connectivity-check.sh            |  92 ++\n t/t5413-connectivity-check.sh                 | 795 ++++++++++++++++++\n t/t6700-tree-depth.sh                         |  17 +\n tree-verify.c                                 | 433 ++++++++++\n tree-verify.h                                 |  15 +\n 22 files changed, 1842 insertions(+), 21 deletions(-)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\n\nbase-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2211%2Fspkrka%2Ftree-diff-connectivity-v1-clean-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2211/spkrka/tree-diff-connectivity-v1-clean-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/2211\n\nRange-diff vs v4:\n\n 1:  e25824c21d = 1:  e25824c21d Documentation: describe connectivity checking\n 2:  427b7315a5 = 2:  427b7315a5 p5413: add perf tests for connectivity check\n 3:  3a7a1fb953 = 3:  3a7a1fb953 connected: drop dependency on `the_repository`\n 4:  87dbb1f055 ! 4:  7ef572bc05 connected: add transfer.connectivityCheck config\n     @@ builtin/rev-list.c\n      +\tenum connectivity_check_mode mode;\n      +};\n      +\n     -+static void parse_connectivity_check(struct repository *repo UNUSED,\n     -+\t\t\t\t     const char *arg,\n     ++static void parse_connectivity_check(const char *arg,\n      +\t\t\t\t     struct connectivity_check *check)\n      +{\n      +\tif (!strcmp(arg, \"full\"))\n     @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \t\t\trepo->fetch_if_missing = 0;\n       \t\t\trevs.exclude_promisor_objects = 1;\n      +\t\t} else if (skip_prefix(arg, \"--connectivity-check=\", &arg)) {\n     -+\t\t\tparse_connectivity_check(repo, arg, &connectivity_check);\n     ++\t\t\tparse_connectivity_check(arg, &connectivity_check);\n       \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n       \t\t\tparse_missing_action_value(repo, arg);\n       \t\t} else if (!strcmp(arg, \"--missing-only\")) {\n 5:  d76dae801b ! 5:  9e84dd5c32 connected: add incremental connectivity check via rev-list\n     @@ builtin/rev-list.c\n       };\n       \n       struct connectivity_check {\n     - \tenum connectivity_check_mode mode;\n     - };\n     - \n     --static void parse_connectivity_check(struct repository *repo UNUSED,\n     -+static void parse_connectivity_check(struct repository *repo,\n     - \t\t\t\t     const char *arg,\n     - \t\t\t\t     struct connectivity_check *check)\n     +@@ builtin/rev-list.c: static void parse_connectivity_check(const char *arg,\n       {\n       \tif (!strcmp(arg, \"full\"))\n       \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n     @@ builtin/rev-list.c\n      +\t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n       \telse\n       \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n     -+\n     -+\tif (check->mode != CONNECTIVITY_CHECK_FULL)\n     -+\t\trepo->fetch_if_missing = 0;\n       }\n     - \n     - struct rev_list_info {\n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \n       \tprepare_maximal_independent(&revs);\n     @@ tree-verify.c (new)\n      +\t\t       const struct object_id *oid,\n      +\t\t       struct verify_state *vs)\n      +{\n     -+\tint type;\n     ++\tstruct object_info oi = OBJECT_INFO_INIT;\n     ++\tenum object_type type;\n      +\n      +\tif (oidset_contains(&vs->trusted_blobs, oid))\n      +\t\treturn 0;\n      +\n      +\tvs->blobs_checked++;\n     -+\ttype = odb_read_object_info(repo->objects, oid, NULL);\n     -+\tif (type == OBJ_BLOB) {\n     -+\t\toidset_insert(&vs->trusted_blobs, oid);\n     -+\t\treturn 0;\n     ++\toi.typep = &type;\n     ++\tif (odb_read_object_info_extended(repo->objects, oid, &oi,\n     ++\t\t\t\t\t  OBJECT_INFO_LOOKUP_REPLACE |\n     ++\t\t\t\t\t  OBJECT_INFO_SKIP_FETCH_OBJECT)) {\n     ++\t\tif (vs->exclude_promisor_objects &&\n     ++\t\t    is_promisor_object(repo, oid))\n     ++\t\t\treturn 0;\n     ++\t\treturn unreadable_object_error(repo, oid, \"blob\");\n      +\t}\n     -+\tif (type >= 0)\n     ++\tif (type != OBJ_BLOB)\n      +\t\treturn error(_(\"object %s is a %s, not a %s\"),\n      +\t\t\t     oid_to_hex(oid), type_name(type), \"blob\");\n     -+\tif (vs->exclude_promisor_objects &&\n     -+\t    is_promisor_object(repo, oid))\n     -+\t\treturn 0;\n     -+\treturn unreadable_object_error(repo, oid, \"blob\");\n     ++\toidset_insert(&vs->trusted_blobs, oid);\n     ++\treturn 0;\n      +}\n      +\n      +/*\n     @@ tree-verify.c (new)\n      +\t\t\t    const struct object_id *oid,\n      +\t\t\t    void **bufp, size_t *sizep)\n      +{\n     ++\tstruct object_info oi = OBJECT_INFO_INIT;\n      +\tenum object_type type;\n      +\n     -+\t*bufp = odb_read_object(odb, oid, &type, sizep);\n     ++\toi.typep = &type;\n     ++\toi.sizep = sizep;\n     ++\toi.contentp = bufp;\n     ++\tif (odb_read_object_info_extended(odb, oid, &oi,\n     ++\t\t\t\t\t  OBJECT_INFO_LOOKUP_REPLACE |\n     ++\t\t\t\t\t  OBJECT_INFO_SKIP_FETCH_OBJECT))\n     ++\t\t*bufp = NULL;\n      +\tif (*bufp && type != OBJ_TREE) {\n      +\t\tFREE_AND_NULL(*bufp);\n      +\t\treturn error(_(\"object %s is a %s, not a %s\"),\n     @@ tree-verify.c (new)\n      +\tunsigned nr_before;\n      +\tint ret = 0;\n      +\n     -+\tif (repo->fetch_if_missing)\n     -+\t\tBUG(\"verify_commits_incremental must not be called \"\n     -+\t\t    \"with fetch_if_missing set\");\n     -+\n      +\tvs.trees = kh_init_oid_tree();\n      +\tvs.exclude_promisor_objects = exclude_promisor_objects;\n      +\n 6:  f6031967e6 ! 6:  98f67caa04 connected: add \"auto\" connectivity check mode\n     @@ builtin/rev-list.c\n      +\tunsigned long max_commits;\n       };\n       \n     - static void parse_connectivity_check(struct repository *repo,\n     - \t\t\t\t     const char *arg,\n     + static void parse_connectivity_check(const char *arg,\n       \t\t\t\t     struct connectivity_check *check)\n       {\n      +\tconst char *cutoff;\n     @@ builtin/rev-list.c\n      +\t\tcheck->mode = CONNECTIVITY_CHECK_AUTO;\n       \telse\n       \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n     - \n     + }\n      @@ builtin/rev-list.c: int cmd_rev_list(int argc,\n       \tint bisect_find_all = 0;\n       \tint use_bitmap_index = 0;\n\n-- \ngitgitgadget\n\n"},{"id":"556018","messageId":"e25824c21d0dbc6161da81cae84cd755fc0e6fec.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 1/6] Documentation: describe connectivity checking","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:54Z","receivedAt":"2026-10-23T13:11:54Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd Documentation/technical/connectivity-check.adoc describing\nthe connectivity invariant and the full connectivity check.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/Makefile                        |   1 +\n .../technical/connectivity-check.adoc         | 112 ++++++++++++++++++\n Documentation/technical/meson.build           |   1 +\n 3 files changed, 114 insertions(+)\n create mode 100644 Documentation/technical/connectivity-check.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex f8dea4b395..1747f4a00b 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -122,6 +122,7 @@ TECH_DOCS += technical/bitmap-format\n TECH_DOCS += technical/build-systems\n TECH_DOCS += technical/bundle-uri\n TECH_DOCS += technical/commit-graph\n+TECH_DOCS += technical/connectivity-check\n TECH_DOCS += technical/directory-rename-detection\n TECH_DOCS += technical/hash-function-transition\n TECH_DOCS += technical/large-object-promisors\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nnew file mode 100644\nindex 0000000000..a112d5be3e\n--- /dev/null\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -0,0 +1,112 @@\n+Connectivity checking\n+=====================\n+\n+After receiving new objects via fetch, push (receive-pack), clone,\n+or bundle, Git verifies that the new reference tips do not leave\n+the repository in a state where reachable objects are missing.\n+This verification is called the connectivity check.\n+\n+Connectivity invariant\n+----------------------\n+\n+A repository is connected when every object reachable from its\n+references is available locally (with exceptions noted below).\n+The reverse does not hold: objects that exist in the object\n+database but are not reachable from any reference may be\n+incomplete or lack a connected closure.\n+\n+The connectivity check maintains this invariant when references\n+are updated.  It trusts that all objects reachable from existing\n+reference tips are already fully connected and verifies that the\n+new reference tips do not introduce references to unavailable\n+objects.  Verification is permitted to stop when it reaches\n+objects already reachable from existing references, since their\n+closure is already connected.  These trusted references include\n+local references and references from alternate object stores.\n+\n+Without this check, a truncated or corrupted transfer could leave\n+a repository in a state where later history walks encounter\n+missing objects.\n+\n+Exceptions\n+~~~~~~~~~~\n+\n+Gitlink entries (submodule references) are excluded from\n+connectivity checking.  Their target objects belong to a separate\n+repository.\n+\n+In partial clones, objects promised by a promisor remote are\n+accepted as connected without requiring local existence.  The\n+check excludes promisor objects from traversal so that it does\n+not trigger on-demand fetches for them.\n+\n+Full connectivity check\n+-----------------------\n+\n+The full connectivity check runs a `rev-list` subprocess, feeding\n+the new reference tips via stdin.  A normal invocation is roughly:\n+\n+    git rev-list --objects --stdin --not --all --quiet\n+        --alternate-refs [--exclude-promisor-objects]\n+\n+In a partial clone, objects received from a promisor remote do\n+not need to be checked: anything they reference is promised by\n+that remote and can be fetched on demand.  So if every new tip is\n+found in a promisor packfile, `rev-list` is not run at all.\n+Otherwise it runs with `--exclude-promisor-objects`, so that the\n+traversal stops at promisor objects instead of fetching them.\n+\n+Consider the following graph after a fetch, where all reference\n+tips point directly to commits.  For simplicity, only local\n+references appear on the already-connected side; alternate refs\n+play the same role.  N3 is a merge commit:\n+\n+            /-------------L2\n+           /\n+    C1---B1---C2---B2-----L1\n+          \\         \\\n+           N1        N3---T2\n+            \\       /\n+             N2-----------T1\n+\n+    L1, L2:         local refs\n+    T1, T2:         incoming tips (new refs)\n+    N1, N2, N3:     incoming commits (N3 is a merge)\n+    B1, B2:         boundary commits (already connected)\n+    C1, C2:         already connected (but not boundary)\n+\n+The incoming set is the commits reachable from the incoming\n+tips but not from the already-connected side.  Boundary commits\n+are the already-connected commits at the edge of that set.  Here\n+B1 is an ancestor of B2, which happens when incoming branches\n+fork at different depths in the existing history.\n+\n+The `--not --all` arguments establish the trust boundary: the\n+incoming set is the commits reachable from the new tips but not\n+from any existing reference.\n+\n+The traversal then walks the trees of the boundary commits and\n+marks those objects uninteresting.  Trees of incoming commits\n+are then walked, stopping at objects already marked as\n+uninteresting.  The cost of marking boundary trees uninteresting\n+dominates when the boundary commits have large trees, even when\n+the incoming changes are small.\n+\n+Deepening fetches\n+~~~~~~~~~~~~~~~~~\n+\n+For deepening fetches (where the shallow boundary moves), the\n+full check omits `--not --all`.  There is no existing-reference\n+boundary at which the walk can stop.  Instead, traversal follows\n+the effective shallow boundary supplied for the deepened\n+repository.  The new content may be below the old shallow\n+boundary even when the tips themselves have not changed.\n+\n+Non-commit tips\n+~~~~~~~~~~~~~~~\n+\n+When a new reference points to a non-commit object, such as a\n+tag, tree, or blob, that object is not part of the commit walk.\n+These non-commit tips are verified during the tree and blob\n+traversal that follows the commit walk within the same rev-list\n+invocation.\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex 9ce11d5e48..c439a7a072 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -11,6 +11,7 @@ articles = [\n   'build-systems.adoc',\n   'bundle-uri.adoc',\n   'commit-graph.adoc',\n+  'connectivity-check.adoc',\n   'directory-rename-detection.adoc',\n   'hash-function-transition.adoc',\n   'large-object-promisors.adoc',\n-- \ngitgitgadget\n\n\n"},{"id":"556019","messageId":"427b7315a5167f4b5546a08c851a4aa222fcc2f6.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 2/6] p5413: add perf tests for connectivity check","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:55Z","receivedAt":"2026-10-23T13:11:55Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd benchmarks for the rev-list connectivity check.  Each fixture\nvaries one dimension: total tree size, incoming commit count, or\nfiles changed per commit.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n ...enerate-repo-p5413-connectivity-check.perl | 44 ++++++++++\n t/perf/p5413-connectivity-check.sh            | 85 +++++++++++++++++++\n 2 files changed, 129 insertions(+)\n create mode 100644 t/perf/generate-repo-p5413-connectivity-check.perl\n create mode 100755 t/perf/p5413-connectivity-check.sh\n\ndiff --git a/t/perf/generate-repo-p5413-connectivity-check.perl b/t/perf/generate-repo-p5413-connectivity-check.perl\nnew file mode 100644\nindex 0000000000..bfa25f60fc\n--- /dev/null\n+++ b/t/perf/generate-repo-p5413-connectivity-check.perl\n@@ -0,0 +1,44 @@\n+#!/usr/bin/perl\n+#\n+# Generate a fast-import stream for p5413 connectivity check benchmarks.\n+#\n+# Usage: generate-repo-p5413-connectivity-check.perl\n+#            <dirs> <files_per_dir> <commits> [<hot_dirs>] [<files_per_commit>]\n+#\n+# Creates one initial commit with dirs*files_per_dir files, then\n+# <commits> additional commits each modifying <files_per_commit>\n+# files in directories chosen round-robin from 1..<hot_dirs>.\n+\n+use strict;\n+use warnings;\n+\n+my ($nd, $nf, $nc, $hot, $fpc) = @ARGV;\n+$hot = $nd if !$hot || $hot > $nd;\n+$fpc = 1   if !$fpc;\n+\n+sub data {\n+\tprintf \"data %d\\n%s\\n\", length($_[0]), $_[0];\n+}\n+\n+# Initial tree: one commit with nd*nf files.\n+printf \"commit refs/heads/main\\n\";\n+printf \"committer perf <perf\\@test.com> now\\n\";\n+data(\"initial\");\n+for my $d (1..$nd) {\n+\tfor my $f (1..$nf) {\n+\t\tprintf \"M 100644 inline d-%04d/f-%03d\\n\", $d, $f;\n+\t\tdata(sprintf \"%03d%03d\", $d, $f);\n+\t}\n+}\n+\n+# Subsequent commits (auto-chained by fast-import).\n+for my $i (1..$nc) {\n+\tprintf \"commit refs/heads/main\\n\";\n+\tprintf \"committer perf <perf\\@test.com> now\\n\";\n+\tdata(sprintf \"change-%03d\", $i);\n+\tfor my $j (0..$fpc-1) {\n+\t\tmy $d = (($i + $j) % $hot) + 1;\n+\t\tprintf \"M 100644 inline d-%04d/f-001\\n\", $d;\n+\t\tdata(sprintf \"c%d-%d\", $i, $j);\n+\t}\n+}\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..9e968d0106\n--- /dev/null\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+\n+test_description='performance of connectivity check\n+\n+Benchmark the rev-list connectivity check, avoiding pack transfer\n+noise.\n+\n+Each repository has a flat tree of many directories with 100 files\n+in each.  Three axes are scaled independently: tree size, commit\n+count, and files changed per commit.'\n+\n+. ./perf-lib.sh\n+\n+test_perf_fresh_repo\n+\n+generate=\"$TEST_DIRECTORY/perf/generate-repo-p5413-connectivity-check.perl\"\n+\n+# $1=dirs  $2=files_per_dir  $3=commits  $4=hot_dirs (optional, default=all)\n+# $5=files_per_commit (optional, default=1)\n+test_perf_conn () {\n+\tlocal nd=\"$1\" nf=\"$2\" nc=\"$3\" hot=\"${4:-$1}\" fpc=\"${5:-1}\"\n+\tlocal total=$(($nd * $nf))\n+\tlocal name=\"repo-${nd}d-${nf}f-${nc}c-${hot}h-${fpc}fpc\"\n+\tlocal label=\"${total} files, ${nc} commits\"\n+\tif test \"$hot\" -lt \"$nd\"\n+\tthen\n+\t\tlabel=\"$label (${hot} hot dirs)\"\n+\tfi\n+\tif test \"$fpc\" -gt 1\n+\tthen\n+\t\tlabel=\"$label (${fpc} files/commit)\"\n+\tfi\n+\n+\ttest_expect_success PERL \"setup $label\" '\n+\t\tif test -d '\"$name\"'\n+\t\tthen\n+\t\t\ttrue\n+\t\telse\n+\t\t\tgit init '\"$name\"' &&\n+\t\t\t\"$PERL_PATH\" '\"$generate\"' '\"$nd\"' '\"$nf\"' '\"$nc\"' '\"$hot\"' '\"$fpc\"' |\n+\t\t\tgit -C '\"$name\"' fast-import --date-format=now --quiet &&\n+\t\t\t(\n+\t\t\t\tcd '\"$name\"' &&\n+\t\t\t\tgit rev-parse main~'\"$nc\"' >../'\"${name}\"'_old &&\n+\t\t\t\tgit rev-parse main >../'\"${name}\"'_new &&\n+\t\t\t\tgit update-ref refs/heads/main \\\n+\t\t\t\t\t$(cat ../'\"${name}\"'_old) &&\n+\t\t\t\tgit repack -ad &&\n+\t\t\t\tgit config gc.auto 0\n+\t\t\t)\n+\t\tfi\n+\t'\n+\n+\ttest_perf \"$label\" --prereq PERL '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+}\n+\n+# Scaling tree size (10 commits, 10 files/commit).\n+test_perf_conn   50 100 10   50 10\n+test_perf_conn  500 100 10  500 10\n+test_perf_conn 2000 100 10 2000 10\n+test_perf_conn 8000 100 10 8000 10\n+\n+# Scaling commit count (200K files, 10 files/commit).\n+test_perf_conn 2000 100    1 2000 10\n+test_perf_conn 2000 100   10 2000 10\n+test_perf_conn 2000 100  100 2000 10\n+test_perf_conn 2000 100   500 2000 10\n+test_perf_conn 2000 100  3000 2000 10\n+test_perf_conn 2000 100  5000 2000 10\n+test_perf_conn 2000 100 10000 2000 10\n+\n+# Scaling files per commit (200K files, 10 commits).\n+test_perf_conn 2000 100 10 2000   1\n+test_perf_conn 2000 100 10 2000  10\n+test_perf_conn 2000 100 10 2000  100\n+test_perf_conn 2000 100 10 2000  500\n+test_perf_conn 2000 100 10 2000 1000\n+test_perf_conn 2000 100 10 2000 2000\n+\n+test_done\n-- \ngitgitgadget\n\n\n"},{"id":"556020","messageId":"3a7a1fb95320ed7226d00b5ff759cf88dedf5de0.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 3/6] connected: drop dependency on `the_repository`","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:56Z","receivedAt":"2026-10-23T13:11:56Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe next commits will need to use the repository to read configuration\nin check_connected(). Rather than accessing `the_repository`, refactor\nthe code to instead require the caller to pass in a repository so that\nwe can remove `USE_THE_REPOSITORY_VARIABLE`.\n\nSuggested-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n builtin/clone.c        |  2 +-\n builtin/fetch.c        |  5 +++--\n builtin/receive-pack.c | 12 ++++++++----\n bundle.c               |  2 +-\n connected.c            | 22 +++++++++++-----------\n connected.h            |  4 +++-\n fetch-pack.c           |  3 ++-\n 7 files changed, 29 insertions(+), 21 deletions(-)\n\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 44037dd428..1532361ea0 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -572,7 +572,7 @@ static void update_remote_refs(const struct ref *refs,\n \t\topt.transport = transport;\n \t\topt.progress = transport->progress;\n \n-\t\tif (check_connected(iterate_ref_map, &rm, &opt))\n+\t\tif (check_connected(the_repository, iterate_ref_map, &rm, &opt))\n \t\t\tdie(_(\"remote did not send all necessary objects\"));\n \t}\n \ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex b2decc6cfd..0ef7fe92f1 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -1250,7 +1250,8 @@ static int store_updated_refs(struct display_state *display_state,\n \n \t\topt.exclude_hidden_refs_section = \"fetch\";\n \t\trm = ref_map;\n-\t\tif (check_connected(iterate_ref_map, &rm, &opt)) {\n+\t\tif (check_connected(the_repository, iterate_ref_map,\n+\t\t\t\t    &rm, &opt)) {\n \t\t\trc = error(_(\"%s did not send all necessary objects\"),\n \t\t\t\t   display_state->url);\n \t\t\tgoto abort;\n@@ -1422,7 +1423,7 @@ static int check_exist_and_connected(struct ref *ref_map)\n \n \topt.quiet = 1;\n \topt.exclude_hidden_refs_section = \"fetch\";\n-\treturn check_connected(iterate_ref_map, &rm, &opt);\n+\treturn check_connected(the_repository, iterate_ref_map, &rm, &opt);\n }\n \n static int fetch_and_consume_refs(struct display_state *display_state,\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 1d5b050beb..4f8f58950a 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -1385,7 +1385,8 @@ static int update_shallow_ref(struct command *cmd, struct shallow_info *si)\n \t\t\toid_array_append(&extra, &si->shallow->oid[i]);\n \n \tsetup_alternate_shallow(&shallow_lock, &opt.shallow_file, &extra);\n-\tif (check_connected(command_singleton_iterator, cmd, &opt)) {\n+\tif (check_connected(the_repository, command_singleton_iterator,\n+\t\t\t    cmd, &opt)) {\n \t\trollback_shallow_file(the_repository, &shallow_lock);\n \t\toid_array_clear(&extra);\n \t\treturn -1;\n@@ -1827,8 +1828,9 @@ static void set_connectivity_errors(struct command *commands,\n \t\todb_transaction_env(transaction, &env);\n \t\topt.env = env.v;\n \n-\t\tif (!check_connected(command_singleton_iterator, &singleton,\n-\t\t\t\t     &opt))\n+\t\tif (!check_connected(the_repository,\n+\t\t\t\t     command_singleton_iterator,\n+\t\t\t\t     &singleton, &opt))\n \t\t\tcontinue;\n \n \t\tcmd->error_string = \"missing necessary objects\";\n@@ -2090,7 +2092,9 @@ static void execute_commands(struct command *commands,\n \t\topt.env = env.v;\n \t\topt.exclude_hidden_refs_section = \"receive\";\n \n-\t\tif (check_connected(iterate_receive_command_list, &data, &opt))\n+\t\tif (check_connected(the_repository,\n+\t\t\t\t    iterate_receive_command_list,\n+\t\t\t\t    &data, &opt))\n \t\t\tset_connectivity_errors(commands, si, transaction);\n \n \t\tif (use_sideband)\ndiff --git a/bundle.c b/bundle.c\nindex f55a521b2a..0a68e856f5 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -259,7 +259,7 @@ int verify_bundle(struct repository *r,\n \tif (ret)\n \t\tgoto cleanup;\n \n-\tif ((ret = check_connected(iterate_ref_map, &iter, &opts)))\n+\tif ((ret = check_connected(r, iterate_ref_map, &iter, &opts)))\n \t\terror(_(\"some prerequisite commits exist in the object store, \"\n \t\t\t\"but are not connected to the repository's history\"));\n \ndiff --git a/connected.c b/connected.c\nindex 929b9bd28d..34045f1032 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,5 +1,3 @@\n-#define USE_THE_REPOSITORY_VARIABLE\n-\n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n@@ -35,23 +33,24 @@ static int promised_object_cb(const struct object_id *oid UNUSED,\n  * objects in non-promisor packs, in which case we'll have to fall back to the\n  * rev-list-based connectivity checks. Returns a negative error code on error.\n  */\n-static int check_connected_promisor(oid_iterate_fn fn,\n+static int check_connected_promisor(struct repository *repo,\n+\t\t\t\t    oid_iterate_fn fn,\n \t\t\t\t    void *cb_data,\n \t\t\t\t    const struct object_id **oid)\n {\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = ODB_FOR_EACH_OBJECT_PROMISOR_ONLY,\n-\t\t.prefix_hex_len = the_repository->hash_algo->hexsz,\n+\t\t.prefix_hex_len = repo->hash_algo->hexsz,\n \t};\n \tint err;\n \n-\todb_reprepare(the_repository->objects);\n+\todb_reprepare(repo->objects);\n \tdo {\n \t\tbool found = false;\n \n \t\topts.prefix = *oid;\n \n-\t\terr = odb_for_each_object_ext(the_repository->objects, NULL,\n+\t\terr = odb_for_each_object_ext(repo->objects, NULL,\n \t\t\t\t\t      promised_object_cb, &found, &opts);\n \t\tif (err < 0)\n \t\t\treturn err;\n@@ -78,7 +77,8 @@ static int check_connected_promisor(oid_iterate_fn fn,\n  *\n  * Returns 0 if everything is connected, non-zero otherwise.\n  */\n-int check_connected(oid_iterate_fn fn, void *cb_data,\n+int check_connected(struct repository *repo,\n+\t\t    oid_iterate_fn fn, void *cb_data,\n \t\t    struct check_connected_options *opt)\n {\n \tstruct child_process rev_list = CHILD_PROCESS_INIT;\n@@ -101,8 +101,8 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \t\treturn err;\n \t}\n \n-\tif (repo_has_promisor_remote(the_repository)) {\n-\t\terr = check_connected_promisor(fn, cb_data, &oid);\n+\tif (repo_has_promisor_remote(repo)) {\n+\t\terr = check_connected_promisor(repo, fn, cb_data, &oid);\n \t\tif (err) {\n \t\t\tif (opt->err_fd)\n \t\t\t\tclose(opt->err_fd);\n@@ -119,7 +119,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \tstrvec_push(&rev_list.args,\"rev-list\");\n \tstrvec_push(&rev_list.args, \"--objects\");\n \tstrvec_push(&rev_list.args, \"--stdin\");\n-\tif (repo_has_promisor_remote(the_repository))\n+\tif (repo_has_promisor_remote(repo))\n \t\tstrvec_push(&rev_list.args, \"--exclude-promisor-objects\");\n \tif (!opt->is_deepening_fetch) {\n \t\tstrvec_push(&rev_list.args, \"--not\");\n@@ -160,7 +160,7 @@ int check_connected(oid_iterate_fn fn, void *cb_data,\n \t\tstrbuf_add(&idx_file, transport->pack_lockfiles.items[0].string,\n \t\t\t   base_len);\n \t\tstrbuf_addstr(&idx_file, \".idx\");\n-\t\tnew_pack = add_packed_git(the_repository, idx_file.buf,\n+\t\tnew_pack = add_packed_git(repo, idx_file.buf,\n \t\t\t\t\t  idx_file.len, 1);\n \t\tstrbuf_release(&idx_file);\n \t}\ndiff --git a/connected.h b/connected.h\nindex 16b2c84f2e..0624965baa 100644\n--- a/connected.h\n+++ b/connected.h\n@@ -2,6 +2,7 @@\n #define CONNECTED_H\n \n struct object_id;\n+struct repository;\n struct transport;\n \n /*\n@@ -66,7 +67,8 @@ struct check_connected_options {\n  *\n  * If \"opt\" is NULL, behaves as if CHECK_CONNECTED_INIT was passed.\n  */\n-int check_connected(oid_iterate_fn fn, void *cb_data,\n+int check_connected(struct repository *repo,\n+\t\t    oid_iterate_fn fn, void *cb_data,\n \t\t    struct check_connected_options *opt);\n \n #endif /* CONNECTED_H */\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 30a4dd6119..2cf8f8d653 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -2186,7 +2186,8 @@ struct ref *fetch_pack(struct fetch_pack_args *args,\n \t\topt.shallow_file = alternate_shallow_file;\n \t\tif (args->deepen)\n \t\t\topt.is_deepening_fetch = 1;\n-\t\tif (check_connected(iterate_ref_map, &iterator, &opt)) {\n+\t\tif (check_connected(the_repository, iterate_ref_map,\n+\t\t\t\t    &iterator, &opt)) {\n \t\t\terror(_(\"remote did not send all necessary objects\"));\n \t\t\tfree_refs(ref_cpy);\n \t\t\tref_cpy = NULL;\n-- \ngitgitgadget\n\n\n"},{"id":"556021","messageId":"7ef572bc05d96fea05289191c6738751b704fbfb.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 4/6] connected: add transfer.connectivityCheck config","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:57Z","receivedAt":"2026-10-23T13:11:57Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nAdd the transfer.connectivityCheck configuration option to choose\nthe algorithm used for the connectivity check.  For now the only\naccepted value is \"full\", which is the existing behavior and the\ndefault.  Unknown values fall back to \"full\" with a warning, so\nthat configuration written for a newer version of Git does not\nbreak fetches and pushes; \"full\" is always a correct check.\n\ncheck_connected() now always tells its rev-list subprocess which\nalgorithm to use with an internal --connectivity-check=<mode>\noption, which so far only accepts \"full\".  Passing the mode\nexplicitly means rev-list knows it is running a connectivity\ncheck, so later optimizations can be limited to that case.\n\nThis prepares for an incremental check added in the next commit.\n\nSuggested-by: Patrick Steinhardt <ps@pks.im>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc  | 16 ++++++++++++++++\n Documentation/rev-list-options.adoc |  5 +++++\n builtin/rev-list.c                  | 22 ++++++++++++++++++++++\n connected.c                         | 26 ++++++++++++++++++++++++++\n 4 files changed, 69 insertions(+)\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex f1ce50f4a6..811456f1bf 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -1,3 +1,19 @@\n+transfer.connectivityCheck::\n+\tChoose which algorithm to use for the connectivity check\n+\tperformed during object transfer operations such as\n+\tlinkgit:git-fetch[1] and linkgit:git-receive-pack[1].\n+\tThe connectivity check verifies that all objects reachable\n+\tfrom the incoming tips are available locally or, in a partial\n+\tclone, promised by a promisor remote.\n+\tThe variants are as follows:\n++\n+--\n+`full` (default);;\n+\tWalk the full object closure of the boundary commits.\n+--\n++\n+Unknown values are treated as `full`, with a warning.\n+\n transfer.credentialsInUrl::\n \tA configured URL can contain plaintext credentials in the form\n \t`<protocol>://<user>:<password>@<domain>/<path>`. You may want\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex bd9f345690..3979a142a5 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1102,6 +1102,11 @@ It is an error to use `--missing-only` without `--missing=print` or\n \tstronger than `--missing=allow-promisor` because it limits the\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n+\n+`--connectivity-check=full`::\n+\t(For internal use only.)  Used by `check_connected()` to\n+\tselect the connectivity check algorithm; see\n+\t`transfer.connectivityCheck` in linkgit:git-config[1].\n endif::git-rev-list[]\n \n `--no-walk[=(sorted|unsorted)]`::\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 0faa833fac..619855f4e4 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -29,6 +29,23 @@\n #include \"quote.h\"\n #include \"strbuf.h\"\n \n+enum connectivity_check_mode {\n+\tCONNECTIVITY_CHECK_FULL,\n+};\n+\n+struct connectivity_check {\n+\tenum connectivity_check_mode mode;\n+};\n+\n+static void parse_connectivity_check(const char *arg,\n+\t\t\t\t     struct connectivity_check *check)\n+{\n+\tif (!strcmp(arg, \"full\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n+\telse\n+\t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n+}\n+\n struct rev_list_info {\n \tstruct rev_info *revs;\n \tint flags;\n@@ -727,6 +744,7 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n+\tstruct connectivity_check connectivity_check = { 0 };\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -769,6 +787,8 @@ int cmd_rev_list(int argc,\n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\")) {\n \t\t\trepo->fetch_if_missing = 0;\n \t\t\trevs.exclude_promisor_objects = 1;\n+\t\t} else if (skip_prefix(arg, \"--connectivity-check=\", &arg)) {\n+\t\t\tparse_connectivity_check(arg, &connectivity_check);\n \t\t} else if (skip_prefix(arg, \"--missing=\", &arg)) {\n \t\t\tparse_missing_action_value(repo, arg);\n \t\t} else if (!strcmp(arg, \"--missing-only\")) {\n@@ -848,6 +868,8 @@ int cmd_rev_list(int argc,\n \n \t\tif (!strcmp(arg, \"--exclude-promisor-objects\"))\n \t\t\tcontinue; /* already handled above */\n+\t\tif (starts_with(arg, \"--connectivity-check=\"))\n+\t\t\tcontinue; /* already handled above */\n \t\tif (skip_prefix(arg, \"--missing=\", &arg))\n \t\t\tcontinue; /* already handled above */\n \ndiff --git a/connected.c b/connected.c\nindex 34045f1032..0147b7d3b3 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -1,4 +1,5 @@\n #include \"git-compat-util.h\"\n+#include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n@@ -66,6 +67,27 @@ static int check_connected_promisor(struct repository *repo,\n \treturn 1;\n }\n \n+/*\n+ * Fills in the --connectivity-check mode to pass to rev-list.\n+ */\n+static void connectivity_check_mode(struct repository *repo,\n+\t\t\t\t    struct strbuf *mode)\n+{\n+\tconst char *algorithm = NULL;\n+\n+\tif (repo_config_get_string_tmp(repo,\n+\t\t\t\t       \"transfer.connectivitycheck\",\n+\t\t\t\t       &algorithm))\n+\t\talgorithm = \"full\";\n+\tif (!strcasecmp(algorithm, \"full\"))\n+\t\tstrbuf_addstr(mode, \"full\");\n+\telse {\n+\t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n+\t\t\t  \"using 'full'\"), algorithm);\n+\t\tstrbuf_addstr(mode, \"full\");\n+\t}\n+}\n+\n /*\n  * If we feed all the commits we want to verify to this command\n  *\n@@ -84,6 +106,7 @@ int check_connected(struct repository *repo,\n \tstruct child_process rev_list = CHILD_PROCESS_INIT;\n \tFILE *rev_list_in;\n \tstruct check_connected_options defaults = CHECK_CONNECTED_INIT;\n+\tstruct strbuf mode = STRBUF_INIT;\n \tconst struct object_id *oid;\n \tint err = 0;\n \tstruct packed_git *new_pack = NULL;\n@@ -133,6 +156,9 @@ int check_connected(struct repository *repo,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n+\tconnectivity_check_mode(repo, &mode);\n+\tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n+\tstrbuf_release(&mode);\n \n \trev_list.git_cmd = 1;\n \tif (opt->env)\n-- \ngitgitgadget\n\n\n"},{"id":"556022","messageId":"9e84dd5c323453e48a7d6b0b2218c152c4b58823.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 5/6] connected: add incremental connectivity check via rev-list","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:58Z","receivedAt":"2026-10-23T13:11:58Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe full connectivity check uses rev-list to find commits\nreachable from the incoming tips but not from the\nalready-connected side, then walks their object closure.  Commit\ntraversal stops at the connectivity boundary, but trees and blobs\nreachable from that boundary still need to be walked so they can\nbe marked uninteresting, allocating a struct object for each one.\nOn repositories where the boundary commits have large trees, the\nconnectivity check for small incoming changes visits and tracks\nmore objects than needed.\n\nAdd an alternative connectivity check that verifies incoming\ncommits incrementally against their parents.\n\nThe verifier processes incoming commits with ancestors first and\ncompares each new tree against its trusted parent trees.  Entries\nalready seen on the trusted side are skipped by OID, so unchanged\nsubtrees need not remain at the same path to be recognized.\nChanged subtrees are recursively compared against same-path parent\nsubtrees, and new subtrees without a comparison base are verified\nfrom scratch.\n\nSee Documentation/technical/connectivity-check.adoc for the trust\ninvariants and detailed algorithm.\n\nThe incremental check runs inside rev-list, selected by the\ninternal --connectivity-check option.  After the normal\nrevision walk identifies the incoming commits, the verifier\nconsumes them before the usual object traversal; any pending\nnon-commit tips are still handled by the existing traversal.\n\nPartial-clone semantics are preserved: objects promised by a\npromisor remote are accepted as connected, and on-demand fetching\nis prevented by excluding promisor objects from traversal.\n\nA new transfer.connectivityCheck value \"incremental\" selects the\nnew mode, with \"full\" remaining the default.  Deepening fetches\nfall back to the full check because they do not have the normal\nexisting-reference boundary, and traversal instead follows the\neffective shallow boundary.\n\nClones and pushes into an empty repository also use the full\ncheck.  Without negative revisions there is no trusted boundary,\nand the incremental check would have to verify all of history\nfrom scratch like the full check does, with extra per-commit\noverhead on top.  On git.git, checking 70K commits without a\nboundary takes 5.3s with the full check and 9.2s with the\nincremental one.\n\np5413 results (best of 3), scaling one dimension at a time.\nEach fixture is a repository with a flat tree of many\ndirectories containing 100 files each.  The incoming set is a\nchain of commits on top of the existing history, with each\ncommit changing files in different directories round-robin.\nThe three axes vary the total repository tree size, the number\nof incoming commits, and the number of files changed per\nincoming commit.\n\nScaling tree size (10 incoming commits, 10 files/commit):\n\n    files    full  incr.  full/incr\n      5K    0.01s  0.01s    1.0x\n     50K    0.04s  0.01s    4.0x\n    200K    0.13s  0.01s   13.0x\n    800K    0.54s  0.03s   18.0x\n\nThe full mode must walk the trees of boundary commits, which\ngrows with overall tree size.  Incremental still scans the\nroot trees, but avoids descending into unchanged subtrees,\nso it grows much more slowly with repository size.\n\nScaling incoming commit count (200K files, 10 files/commit):\n\n    commits    full  incr.  full/incr\n          1    0.13s  0.01s   13.0x\n         10    0.13s  0.01s   13.0x\n        100    0.16s  0.06s    2.7x\n        500    0.29s  0.25s    1.2x\n       3000    1.14s  1.44s    0.8x\n       5000    1.87s  2.39s    0.8x\n      10000    3.45s  5.13s    0.7x\n\nWith many commits the per-commit overhead of scanning both\nthe new and parent root trees accumulates and incremental\nbecomes slower.  Breakeven is between 500 and 3000 commits\nand the ratio stabilizes near 0.7x for this fixture.\n\nScaling files per incoming commit (200K files, 10 commits):\n\n    files/commit    full  incr.  full/incr\n               1    0.13s  0.01s   13.0x\n              10    0.13s  0.01s   13.0x\n             100    0.15s  0.03s    5.0x\n             500    0.23s  0.13s    1.8x\n            1000    0.34s  0.27s    1.3x\n            2000    0.65s  0.56s    1.2x\n\nThe advantage shrinks as more files change per commit, but\nincremental stays slightly faster even at 2000 files/commit,\nwhere every directory is touched.\n\nSelected peak RSS measurements from the same fixtures:\n\n    case                        full    incr.  ratio\n    200K files, 10 commits      31 MB    12 MB   0.4x\n    800K files, 10 commits     110 MB    26 MB   0.2x\n    200K files, 5000 commits   155 MB   151 MB   1.0x\n\nIncremental uses less memory when it can prune enough of the\nboundary tree walk.  In the long-history case the two modes\nvisit similar object sets and memory converges.\n\nThe regression cases in the CPU benchmarks are in wall-clock\ntime rather than memory, primarily from scanning some trees\nmore than once: a commit's tree is read as a candidate when the\ncommit is verified, and again as a trusted parent tree when its\nchildren are verified.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc            |   8 +\n Documentation/rev-list-options.adoc           |   2 +-\n .../technical/connectivity-check.adoc         | 147 ++++\n Makefile                                      |   1 +\n builtin/rev-list.c                            |  27 +\n connected.c                                   |  14 +-\n meson.build                                   |   1 +\n t/meson.build                                 |   1 +\n t/perf/p5413-connectivity-check.sh            |  17 +-\n t/t5413-connectivity-check.sh                 | 774 ++++++++++++++++++\n t/t6700-tree-depth.sh                         |  17 +\n tree-verify.c                                 | 433 ++++++++++\n tree-verify.h                                 |  15 +\n 13 files changed, 1450 insertions(+), 7 deletions(-)\n create mode 100755 t/t5413-connectivity-check.sh\n create mode 100644 tree-verify.c\n create mode 100644 tree-verify.h\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex 811456f1bf..cba7597a58 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -10,6 +10,14 @@ transfer.connectivityCheck::\n --\n `full` (default);;\n \tWalk the full object closure of the boundary commits.\n+`incremental`;;\n+\tVerify incoming commits by diffing their trees against parent\n+\ttrees, recursively descending only into entries that differ.\n+\tThe largest benefits occur when incoming commits change a\n+\tsmall fraction of a large tree closure, and it can be slower\n+\tthan `full` when there are many incoming commits.\n+\tFalls back to `full` for clones, pushes into an empty\n+\trepository and deepening fetches.\n --\n +\n Unknown values are treated as `full`, with a warning.\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 3979a142a5..2023ad5f75 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1103,7 +1103,7 @@ It is an error to use `--missing-only` without `--missing=print` or\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n \n-`--connectivity-check=full`::\n+`--connectivity-check=(full|incremental)`::\n \t(For internal use only.)  Used by `check_connected()` to\n \tselect the connectivity check algorithm; see\n \t`transfer.connectivityCheck` in linkgit:git-config[1].\ndiff --git a/Documentation/technical/connectivity-check.adoc b/Documentation/technical/connectivity-check.adoc\nindex a112d5be3e..8b7d619d4c 100644\n--- a/Documentation/technical/connectivity-check.adoc\n+++ b/Documentation/technical/connectivity-check.adoc\n@@ -110,3 +110,150 @@ tag, tree, or blob, that object is not part of the commit walk.\n These non-commit tips are verified during the tree and blob\n traversal that follows the commit walk within the same rev-list\n invocation.\n+\n+Incremental connectivity check\n+------------------------------\n+\n+The incremental mode, selected by\n+`transfer.connectivityCheck=incremental`, avoids traversing the\n+full tree walk of the boundary commits.  Instead, it verifies\n+each incoming commit's tree against the already-trusted trees of\n+its parent commits.\n+\n+The same `--not --all` boundary as the full check identifies the\n+incoming commits.  The incremental mode reuses this boundary but\n+replaces what happens afterwards: rather than walking boundary\n+trees to mark objects uninteresting, it compares each incoming\n+commit's tree against its parents' trees, processing commits in\n+topological order (ancestors first).  This all happens inside\n+the same `rev-list` invocation, and the promisor fast path above\n+still runs first.\n+\n+Trust model\n+~~~~~~~~~~~\n+\n+A tree is trusted when its transitive object closure is known to\n+be connected.\n+\n+Every parent of an incoming commit is either on the\n+already-connected side of the `--not --all` boundary (a boundary\n+parent whose tree is trusted by definition) or is an earlier\n+incoming commit that was already verified due to the topological\n+processing order.  In both cases the parent's tree is trusted, so\n+all parent trees can be unconditionally used as comparison bases.\n+\n+Once an incoming commit's tree has been verified, it is trusted\n+and can serve as a comparison base for later descendants.  This\n+gives an inductive correctness argument: the base case is the\n+trust boundary established by `--not --all`, and the inductive\n+step is that verifying a commit extends the trusted set.\n+\n+Tree states\n+~~~~~~~~~~~\n+\n+The verifier tracks tree OIDs in three states:\n+\n+untrusted::\n+\tThe tree has not yet been established as connected.  This\n+\tis the implicit state of an OID not present in the state\n+\tmap.\n+\n+trusted::\n+\tThe tree is known to have a connected transitive closure,\n+\tbut its direct entries have not yet been published into the\n+\tverifier's trusted object sets.\n+\n+expanded::\n+\tThe tree is trusted and its direct non-gitlink entries\n+\thave also been published into the trusted object sets.\n+\n+State transitions are monotonic: a tree may move from untrusted\n+to trusted to expanded, but never backwards.  An untrusted tree\n+that is successfully verified goes directly to expanded.\n+\n+Blobs require only trusted/untrusted state: a blob becomes\n+trusted when it is found in a trusted tree or when its existence\n+and type have been verified directly.\n+\n+The trusted/expanded distinction is an optimization.  An expanded\n+parent does not need to be reread merely to publish entries that\n+are already trusted, though it may still be read when same-path\n+subtree bases are needed for recursive comparison.\n+\n+Algorithm\n+~~~~~~~~~\n+\n+At a high level:\n+\n+    verify(commits):\n+        sort topologically (ancestors first)\n+        for each commit:\n+            mark parent root trees as trusted\n+            verify_tree(commit.tree, parent root trees)\n+\n+    verify_tree(tree, base_trees):\n+        if tree already trusted: return\n+        read tree, collect entries not already trusted\n+        for each available base tree:\n+            publish its entries as trusted\n+            record same-path subtrees as bases\n+        for each collected entry:\n+            if now trusted: skip\n+            if blob: verify blob connectivity and type\n+            if tree: verify_tree(entry, its recorded bases)\n+        mark tree as expanded\n+\n+The important ordering within `verify_tree` is that entries from\n+the new tree are collected before the base trees are scanned, but\n+are processed only afterwards.  Trust learned from any base can\n+therefore eliminate work before recursive verification begins.\n+\n+Same-path parent subtrees are passed down as comparison bases\n+during recursive descent.  If no comparison base is available,\n+the new subtree is verified from scratch.\n+\n+Example\n+~~~~~~~\n+\n+Consider a commit that changes one file under `lib/` and moves an\n+unchanged subtree from `src/` to `dev/`:\n+\n+    Parent tree              New tree\n+    +-- src/   (aaa)         +-- dev/   (aaa)\n+    +-- lib/   (bbb)         +-- lib/   (ccc)\n+         +-- foo.c (ddd)          +-- foo.c (ddd)\n+         +-- bar.c (eee)          +-- bar.c (fff)\n+\n+Scanning the parent makes `aaa` trusted even though it moved from\n+`src/` to `dev/`, so that subtree is skipped.  The changed `ccc`\n+subtree is compared against its same-path parent `bbb`; scanning\n+`bbb` makes `ddd` and `eee` trusted, leaving only the new `fff`\n+blob to be checked.\n+\n+This illustrates two properties:\n+\n+* Trust is OID-based rather than path-based.  An unchanged subtree\n+  is recognized after a move.\n+\n+* Same-path parent subtrees provide recursive comparison bases.\n+  These bases improve pruning efficiency but are not required for\n+  correctness; a new subtree can always be verified from scratch.\n+\n+Multiple parents\n+~~~~~~~~~~~~~~~~\n+\n+For a merge commit, root trees from all parents are comparison\n+bases.  When several parents contain a same-path subtree, each\n+matching subtree is collected as a recursive comparison base.\n+\n+Entries published from any trusted parent become globally trusted,\n+so a matching tree or blob entry present in any parent can be\n+skipped while verifying the merge tree.\n+\n+Missing comparison bases\n+~~~~~~~~~~~~~~~~~~~~~~~~\n+\n+A promised base tree may not be locally available for comparison.\n+In that case the verifier skips that base and verifies the new\n+subtree without it.  The missing comparison can reduce pruning but\n+does not affect correctness.\ndiff --git a/Makefile b/Makefile\nindex a96be506b5..fa42e295e2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1339,6 +1339,7 @@ LIB_OBJS += trailer.o\n LIB_OBJS += transport-helper.o\n LIB_OBJS += transport.o\n LIB_OBJS += tree-diff.o\n+LIB_OBJS += tree-verify.o\n LIB_OBJS += tree-walk.o\n LIB_OBJS += tree.o\n LIB_OBJS += unpack-trees.o\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 619855f4e4..c07ad6e30c 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -28,9 +28,11 @@\n #include \"commit-reach.h\"\n #include \"quote.h\"\n #include \"strbuf.h\"\n+#include \"tree-verify.h\"\n \n enum connectivity_check_mode {\n \tCONNECTIVITY_CHECK_FULL,\n+\tCONNECTIVITY_CHECK_INCREMENTAL,\n };\n \n struct connectivity_check {\n@@ -42,6 +44,8 @@ static void parse_connectivity_check(const char *arg,\n {\n \tif (!strcmp(arg, \"full\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n+\telse if (!strcmp(arg, \"incremental\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n \telse\n \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n }\n@@ -991,6 +995,29 @@ int cmd_rev_list(int argc,\n \n \tprepare_maximal_independent(&revs);\n \n+\t/*\n+\t * Without a boundary of negative revisions, as in a clone or a\n+\t * push into an empty repository, every incoming commit is new.\n+\t * The incremental check would still work, but it would verify\n+\t * the whole object graph like the full check, with extra\n+\t * per-commit overhead on top, so use the full check.\n+\t */\n+\tif (!revs.limited)\n+\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\n+\tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n+\t\tstruct commit *commit;\n+\t\tstruct commit_list *new_commits = NULL;\n+\n+\t\twhile ((commit = get_revision(&revs)) != NULL)\n+\t\t\tcommit_list_insert(commit, &new_commits);\n+\n+\t\tif (verify_commits_incremental(repo, &new_commits,\n+\t\t\t\t\t       revs.exclude_promisor_objects))\n+\t\t\tdie(_(\"connectivity check failed\"));\n+\t\tcommit_list_free(new_commits);\n+\t}\n+\n \tif (revs.tree_objects)\n \t\tmark_edges_uninteresting(&revs, show_edge, 0);\n \ndiff --git a/connected.c b/connected.c\nindex 0147b7d3b3..c032987410 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -71,16 +71,28 @@ static int check_connected_promisor(struct repository *repo,\n  * Fills in the --connectivity-check mode to pass to rev-list.\n  */\n static void connectivity_check_mode(struct repository *repo,\n+\t\t\t\t    struct check_connected_options *opt,\n \t\t\t\t    struct strbuf *mode)\n {\n \tconst char *algorithm = NULL;\n \n+\t/*\n+\t * Deepening fetches do not have the usual boundary of\n+\t * existing references, so use the full check for them.\n+\t */\n+\tif (opt->is_deepening_fetch) {\n+\t\tstrbuf_addstr(mode, \"full\");\n+\t\treturn;\n+\t}\n+\n \tif (repo_config_get_string_tmp(repo,\n \t\t\t\t       \"transfer.connectivitycheck\",\n \t\t\t\t       &algorithm))\n \t\talgorithm = \"full\";\n \tif (!strcasecmp(algorithm, \"full\"))\n \t\tstrbuf_addstr(mode, \"full\");\n+\telse if (!strcasecmp(algorithm, \"incremental\"))\n+\t\tstrbuf_addstr(mode, \"incremental\");\n \telse {\n \t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n \t\t\t  \"using 'full'\"), algorithm);\n@@ -156,7 +168,7 @@ int check_connected(struct repository *repo,\n \tif (opt->progress)\n \t\tstrvec_pushf(&rev_list.args, \"--progress=%s\",\n \t\t\t     _(\"Checking connectivity\"));\n-\tconnectivity_check_mode(repo, &mode);\n+\tconnectivity_check_mode(repo, opt, &mode);\n \tstrvec_pushf(&rev_list.args, \"--connectivity-check=%s\", mode.buf);\n \tstrbuf_release(&mode);\n \ndiff --git a/meson.build b/meson.build\nindex 0a95d90d21..d94dedc26c 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -562,6 +562,7 @@ libgit_sources = [\n   'transport-helper.c',\n   'transport.c',\n   'tree-diff.c',\n+  'tree-verify.c',\n   'tree-walk.c',\n   'tree.c',\n   'unpack-trees.c',\ndiff --git a/t/meson.build b/t/meson.build\nindex f65eb04684..b23b4c90b0 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -654,6 +654,7 @@ integration_tests = [\n   't5410-receive-pack.sh',\n   't5411-proc-receive-hook.sh',\n   't5412-receive-report-hook.sh',\n+  't5413-connectivity-check.sh',\n   't5500-fetch-pack.sh',\n   't5501-fetch-push-alternates.sh',\n   't5502-quickfetch.sh',\ndiff --git a/t/perf/p5413-connectivity-check.sh b/t/perf/p5413-connectivity-check.sh\nindex 9e968d0106..250140a79a 100755\n--- a/t/perf/p5413-connectivity-check.sh\n+++ b/t/perf/p5413-connectivity-check.sh\n@@ -1,9 +1,9 @@\n #!/bin/sh\n \n-test_description='performance of connectivity check\n+test_description='performance of connectivity check modes\n \n-Benchmark the rev-list connectivity check, avoiding pack transfer\n-noise.\n+Compare the default and incremental rev-list connectivity modes\n+directly, avoiding pack transfer noise.\n \n Each repository has a flat tree of many directories with 100 files\n in each.  Three axes are scaled independently: tree size, commit\n@@ -51,9 +51,16 @@ test_perf_conn () {\n \t\tfi\n \t'\n \n-\ttest_perf \"$label\" --prereq PERL '\n+\ttest_perf \"$label (full)\" --prereq PERL '\n \t\tcat '\"${name}\"'_new |\n-\t\tgit -C '\"$name\"' rev-list \\\n+\t\tgit -C '\"$name\"' rev-list --connectivity-check=full \\\n+\t\t\t--objects --stdin --not --all --quiet \\\n+\t\t\t--exclude-promisor-objects\n+\t'\n+\n+\ttest_perf \"$label (incremental)\" --prereq PERL '\n+\t\tcat '\"${name}\"'_new |\n+\t\tgit -C '\"$name\"' rev-list --connectivity-check=incremental \\\n \t\t\t--objects --stdin --not --all --quiet \\\n \t\t\t--exclude-promisor-objects\n \t'\ndiff --git a/t/t5413-connectivity-check.sh b/t/t5413-connectivity-check.sh\nnew file mode 100755\nindex 0000000000..109ad17cff\n--- /dev/null\n+++ b/t/t5413-connectivity-check.sh\n@@ -0,0 +1,774 @@\n+#!/bin/sh\n+\n+test_description='connectivity check (transfer.connectivityCheck)'\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+. ./test-lib.sh\n+\n+test_oid_cache <<-\\EOF\n+missing sha1:0000000000000000000000000000000000000001\n+missing sha256:0000000000000000000000000000000000000000000000000000000000000001\n+EOF\n+\n+set_connectivity_check () {\n+\tgit -C \"$1\" config transfer.connectivityCheck \"$2\"\n+}\n+\n+# Run a connectivity check via rev-list directly.  Uses $mode\n+# (set by the enclosing for-loop) to choose full or incremental.\n+check_connected () {\n+\tprintf '%s\\n' \"$@\" |\n+\tgit rev-list --connectivity-check=\"$mode\" \\\n+\t\t--objects --stdin --not --all --quiet \\\n+\t\t--exclude-promisor-objects\n+}\n+\n+# Run git with a temporary index, leaving the real index untouched.\n+tmpgit () {\n+\tGIT_INDEX_FILE=.git/tmp-idx git \"$@\"\n+}\n+\n+# Create a commit with one file changed, without modifying HEAD,\n+# index, or worktree.  Prints the new commit OID on stdout.\n+# Usage: commit_with_change <parent> <path> <content>\n+commit_with_change () {\n+\tnew_blob=$(echo \"$3\" | git hash-object -w --stdin) &&\n+\ttmpgit read-tree \"$1\" &&\n+\ttmpgit update-index --replace \\\n+\t\t--cacheinfo \"100644,$new_blob,$2\" &&\n+\tnew_tree=$(tmpgit write-tree) &&\n+\trm -f .git/tmp-idx &&\n+\tgit commit-tree \"$new_tree\" -p \"$1\" -m \"modify $2\"\n+}\n+\n+# Check OIDs and optionally verify trace2 counts.\n+# Usage: check_connected_trace <trace-file> <trees> <blobs> <oid>...\n+# An empty string for <trees> or <blobs> skips that assertion.\n+check_connected_trace () {\n+\ttrace_file=$1 trees=$2 blobs=$3 &&\n+\tshift 3 &&\n+\ttest_env GIT_TRACE2_EVENT=\"$(pwd)/$trace_file\" \\\n+\t\tcheck_connected \"$@\" &&\n+\tif test \"$mode\" != incremental\n+\tthen\n+\t\treturn\n+\tfi &&\n+\tif test -n \"$trees\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity trees_loaded \"$trees\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi &&\n+\tif test -n \"$blobs\"\n+\tthen\n+\t\ttest_trace2_data_singular connectivity blobs_checked \"$blobs\" \\\n+\t\t\t<\"$trace_file\"\n+\tfi\n+}\n+\n+# Shared setup: a repo with several root-level files and nested dirs.\n+# The unchanged/ subtree (10 dirs x 10 files = 100 blobs, 11 trees)\n+# acts as a canary: any test asserting small tree/blob counts would\n+# fail dramatically if incremental accidentally walked into it.\n+\n+test_expect_success 'setup main repo' '\n+\tgit init main-repo &&\n+\t(\n+\t\tcd main-repo &&\n+\t\tfor i in $(test_seq 1 5)\n+\t\tdo\n+\t\t\techo \"file $i\" >\"file-$i.txt\" || return 1\n+\t\tdone &&\n+\t\tgit add file-*.txt &&\n+\t\tgit commit -m \"initial\" &&\n+\n+\t\tmkdir -p a/b/c &&\n+\t\techo deep >a/b/c/deep.txt &&\n+\t\techo other >a/other.txt &&\n+\t\tgit add a/b/c/deep.txt a/other.txt &&\n+\t\tgit commit -m \"add nested dirs\" &&\n+\n+\t\tfor i in $(test_seq 1 10)\n+\t\tdo\n+\t\t\td=\"unchanged/dir-$i\" &&\n+\t\t\tmkdir -p \"$d\" &&\n+\t\t\tfor j in $(test_seq 1 10)\n+\t\t\tdo\n+\t\t\t\techo \"$i $j\" >\"$d/file-$j.txt\" || return 1\n+\t\t\tdone\n+\t\tdone &&\n+\t\tgit add unchanged/ &&\n+\t\tgit commit -m \"add unchanged canary subtree\"\n+\t)\n+'\n+\n+test_expect_success 'setup replacement object repo' '\n+\tgit init replace-test &&\n+\t(\n+\t\tcd replace-test &&\n+\n+\t\ttest_commit --no-tag original file.txt &&\n+\t\toriginal=$(git rev-parse HEAD) &&\n+\t\torig_blob=$(git rev-parse HEAD:file.txt) &&\n+\n+\t\t# Orphan replacement commit with a different tree\n+\t\treplacement_tree=$(echo replaced | git hash-object -w --stdin |\n+\t\t\txargs -I{} git mktree <<-EOF\n+\t\t\t100644 blob {}\tfile.txt\n+\t\t\tEOF\n+\t\t) &&\n+\t\treplacement=$(git commit-tree -m \"replacement\" \\\n+\t\t\t\"$replacement_tree\") &&\n+\n+\t\tgit replace \"$original\" \"$replacement\" &&\n+\n+\t\t# Remove the original blob so only the replacement\n+\t\t# tree is complete.\n+\t\trm .git/objects/$(test_oid_to_path \"$orig_blob\") &&\n+\n+\t\t# Drop branch and HEAD so --not --all does not\n+\t\t# exclude the original commit.\n+\t\tgit update-ref -d refs/heads/main &&\n+\t\tgit update-ref -d HEAD &&\n+\n+\t\techo \"$original\" >.git/test-oid &&\n+\t\techo \"$orig_blob\" >.git/test-blob\n+\t)\n+'\n+\n+missing_oid=$(test_oid missing)\n+original_oid=$(cat replace-test/.git/test-oid)\n+orig_blob_oid=$(cat replace-test/.git/test-blob)\n+\n+for mode in full incremental\n+do\n+\n+# The full traversal reports a missing tree as \"bad tree object\".\n+case \"$mode\" in\n+full)\n+\tmissing_tree=\"bad tree object\"\n+\t;;\n+incremental)\n+\tmissing_tree=\"missing tree object\"\n+\t;;\n+esac\n+\n+# Corruption detection: craft broken object graphs and verify detection.\n+# All tests use main-repo without modifying its refs or worktree.\n+\n+test_expect_success \"$mode: rejects commit with missing blob\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects commit with missing subtree\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"40000 tree ${missing_oid}\\tdir\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tbad_commit=$(git commit-tree \"$bad_tree\" -p HEAD -m \"bad\") &&\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"$missing_tree $missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct tree tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tbad_tree=$(printf \"100644 blob ${missing_oid}\\tfile.txt\\n\" |\n+\t\t\tgit mktree --missing) &&\n+\t\ttest_expect_code 128 check_connected \"$bad_tree\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$missing_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"hello\" | git hash-object -w --stdin) &&\n+\t\tcheck_connected \"$blob_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects missing direct blob tip\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$missing_oid\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects blob OID reused as tree entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\tblob_oid=$(git rev-parse HEAD:file-1.txt) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"40000 subdir\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a tree\" err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: rejects tree OID reused as blob entry\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\ttree_oid=$(git rev-parse HEAD:a) &&\n+\t\tbin_oid=$(echo \"$tree_oid\" | hex2oct) &&\n+\n+\t\tbad_tree=$(printf \"100644 fakefile\\0$bin_oid\" |\n+\t\t\tgit hash-object -t tree -w --stdin) &&\n+\t\tbad_commit=$(git commit-tree -p HEAD -m \"child\" \"$bad_tree\") &&\n+\n+\t\ttest_expect_code 128 check_connected \"$bad_commit\" 2>err &&\n+\t\ttest_grep \"not a blob\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: checks multiple tips\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"tip-a\") &&\n+\t\tc2=$(commit_with_change HEAD file-2.txt \"tip-b\") &&\n+\t\tgit tag -a -m \"tagged\" multi-tag \"$c1\" &&\n+\t\ttag_oid=$(git rev-parse multi-tag) &&\n+\t\tgit tag -d multi-tag &&\n+\t\tcheck_connected \"$c2\" \"$tag_oid\"\n+\t)\n+'\n+\n+# Tree-diff optimization: verify trace2 counts.\n+\n+test_expect_success \"$mode: handles root commit (no parents)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tblob_oid=$(echo \"root-content\" | git hash-object -w --stdin) &&\n+\t\ttree_oid=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$blob_oid\" | git mktree) &&\n+\t\troot_oid=$(git commit-tree \"$tree_oid\" -m \"root\") &&\n+\n+\t\t# No parent trees, so the full tree is verified.\n+\t\t# 1 tree loaded (root), 1 blob checked.\n+\t\tcheck_connected_trace trace-root.txt 1 1 \"$root_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles single file change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD file-1.txt \"changed\") &&\n+\n+\t\t# 2 trees loaded (new root + parent root), 1 blob checked.\n+\t\tcheck_connected_trace trace-flat.txt 2 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles nested change\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\toid=$(commit_with_change HEAD a/b/c/deep.txt \"deep-changed\") &&\n+\t\t# 4 new trees + 4 parent trees = 8 loaded, 1 blob checked.\n+\t\tcheck_connected_trace trace-nested.txt 8 1 \"$oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles change-then-revert\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tc1=$(commit_with_change HEAD file-1.txt \"revert-tmp\") &&\n+\t\tc2=$(commit_with_change \"$c1\" file-1.txt \"file 1\") &&\n+\t\tc3=$(commit_with_change \"$c2\" file-1.txt \"revert-final\") &&\n+\n+\t\t# c1: new root + parent root = 2 loads.  c2: root matches\n+\t\t# HEAD (already trusted), skipped.  c3: new root + parent\n+\t\t# already expanded = 1 load.  Total: 3 trees, 2 blobs.\n+\t\tcheck_connected_trace trace-revert.txt 3 2 \"$c3\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles subtree moved to another path\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tmoved_tree=$(git ls-tree HEAD |\n+\t\t\tsed \"s/\ta$/\tmoved/\" |\n+\t\t\tgit mktree) &&\n+\t\tmoved=$(git commit-tree \"$moved_tree\" -p HEAD -m move) &&\n+\t\t# New root + parent root scanned, but the moved subtree\n+\t\t# (same OID) is trusted and not descended into.\n+\t\tcheck_connected_trace trace-move.txt 2 0 \"$moved\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles multi-parent merge\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\tleft=$(commit_with_change HEAD file-1.txt left) &&\n+\t\tright=$(commit_with_change HEAD file-2.txt right) &&\n+\t\tgit update-ref refs/heads/left \"$left\" &&\n+\t\tgit update-ref refs/heads/right \"$right\" &&\n+\t\tleft_blob=$(git rev-parse \"$left:file-1.txt\") &&\n+\t\tright_blob=$(git rev-parse \"$right:file-2.txt\") &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$left_blob,file-1.txt\" &&\n+\t\ttmpgit update-index --replace \\\n+\t\t\t--cacheinfo \"100644,$right_blob,file-2.txt\" &&\n+\t\tmerge_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tmerge=$(git commit-tree \"$merge_tree\" \\\n+\t\t\t-p \"$left\" -p \"$right\" -m merge) &&\n+\t\t# Both parent root trees are scanned as bases, so\n+\t\t# blobs from each parent are trusted without ODB checks.\n+\t\tcheck_connected_trace trace-merge.txt 3 0 \"$merge\" &&\n+\t\tgit update-ref -d refs/heads/left &&\n+\t\tgit update-ref -d refs/heads/right\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles merge with incoming and boundary parents\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# parent1 is incoming (not a ref), HEAD is boundary.\n+\t\tparent1=$(commit_with_change HEAD file-1.txt merge-inc) &&\n+\t\tmerge=$(git commit-tree \\\n+\t\t\t\"$(git rev-parse \"$parent1^{tree}\")\" \\\n+\t\t\t-p \"$parent1\" -p HEAD -m merge-mixed) &&\n+\t\t# parent1 verified first (topo order): 2 trees, 1 blob.\n+\t\t# merge tree = parent1 tree (already trusted): 0 extra.\n+\t\tcheck_connected_trace trace-merge-inc.txt 2 1 \"$merge\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles gitlink entries (submodules)\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"160000,$missing_oid,my-submodule\" &&\n+\t\tgitlink_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tgitlink_commit=$(git commit-tree \"$gitlink_tree\" -p HEAD \\\n+\t\t\t-m \"add gitlink\") &&\n+\n+\t\t# Gitlink entries are skipped -- the missing submodule\n+\t\t# commit OID does not cause a failure.\n+\t\tcheck_connected_trace trace-gitlink.txt 2 0 \"$gitlink_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles file-to-directory transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"foo\" is a blob at root.\n+\t\tblob_a=$(echo \"file-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,foo\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add foo as file\") &&\n+\n+\t\t# Child: \"foo\" becomes a directory (foo/bar.txt).\n+\t\tblob_b=$(echo \"dir-content\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove foo &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,foo/bar.txt\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"foo: file to directory\") &&\n+\t\tcheck_connected_trace trace-f2d.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: handles directory-to-file transition\" '\n+\t(\n+\t\tcd main-repo &&\n+\n+\t\t# Parent: \"bar/baz.txt\" exists (bar is a directory).\n+\t\tblob_a=$(echo \"nested\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree HEAD &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_a,bar/baz.txt\" &&\n+\t\tparent_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tparent=$(git commit-tree \"$parent_tree\" -p HEAD \\\n+\t\t\t-m \"add bar as directory\") &&\n+\n+\t\t# Child: \"bar\" becomes a plain file.\n+\t\tblob_b=$(echo \"flat\" | git hash-object -w --stdin) &&\n+\t\ttmpgit read-tree \"$parent\" &&\n+\t\ttmpgit update-index --remove bar/baz.txt &&\n+\t\ttmpgit update-index --add \\\n+\t\t\t--cacheinfo \"100644,$blob_b,bar\" &&\n+\t\tchild_tree=$(tmpgit write-tree) &&\n+\t\trm -f .git/tmp-idx &&\n+\t\tchild=$(git commit-tree \"$child_tree\" -p \"$parent\" \\\n+\t\t\t-m \"bar: directory to file\") &&\n+\t\tcheck_connected_trace trace-d2f.txt \"\" \"\" \"$child\"\n+\t)\n+'\n+\n+# Replacement objects.\n+\n+test_expect_success \"$mode: accepts with replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tcheck_connected \"$original_oid\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: rejects without replacement objects\" '\n+\t(\n+\t\tcd replace-test &&\n+\t\tGIT_NO_REPLACE_OBJECTS=1 &&\n+\t\texport GIT_NO_REPLACE_OBJECTS &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$original_oid\" 2>err &&\n+\t\ttest_grep \"missing blob object.*$orig_blob_oid\" err\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised blob\" '\n+\ttest_when_finished \"rm -rf prom-src prom-server.git prom-client\" &&\n+\tgit init prom-src &&\n+\ttest_commit -C prom-src --no-tag base file.txt original &&\n+\ttest_commit -C prom-src --no-tag \"add file2\" file2.txt extra &&\n+\tgit clone --bare prom-src prom-server.git &&\n+\tgit -C prom-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/prom-server.git\" prom-client &&\n+\t(\n+\t\tcd prom-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file2.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tnewname.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: does not lazy-fetch promised blob\" '\n+\ttest_when_finished \"rm -rf nofetch-src nofetch-server.git nofetch-client\" &&\n+\tgit init nofetch-src &&\n+\ttest_commit -C nofetch-src --no-tag base file.txt content &&\n+\tgit clone --bare nofetch-src nofetch-server.git &&\n+\tgit -C nofetch-server.git config uploadpack.allowfilter true &&\n+\tgit -C nofetch-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=blob:none \\\n+\t\t\"file://$(pwd)/nofetch-server.git\" nofetch-client &&\n+\t(\n+\t\tcd nofetch-client &&\n+\t\tpromised_blob=$(git rev-parse HEAD:file.txt) &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\" &&\n+\t\tnew_tree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$promised_blob\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-m \"root commit with promised blob\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_blob\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: accepts missing promised tree\" '\n+\ttest_when_finished \"rm -rf prom-tree-src prom-tree-server.git prom-tree-client\" &&\n+\tgit init prom-tree-src &&\n+\tmkdir -p prom-tree-src/a/b &&\n+\ttest_commit -C prom-tree-src --no-tag \"nested dirs\" a/b/file.txt deep &&\n+\tgit clone --bare prom-tree-src prom-tree-server.git &&\n+\tgit -C prom-tree-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-tree-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-tree-server.git\" prom-tree-client &&\n+\t(\n+\t\tcd prom-tree-client &&\n+\t\tpromised_tree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\" &&\n+\t\tnew_tree=$(printf \"40000 tree %s\\trenamed\\n\" \\\n+\t\t\t\"$promised_tree\" |\n+\t\t\tgit mktree --missing) &&\n+\t\tnew_commit=$(git commit-tree \"$new_tree\" \\\n+\t\t\t-p HEAD -m \"reuse promised tree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$promised_tree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies new subtree when parent subtree is promised\" '\n+\ttest_when_finished \"rm -rf prom-base-src prom-base-server.git prom-base-client\" &&\n+\tgit init prom-base-src &&\n+\tmkdir -p prom-base-src/a &&\n+\ttest_commit -C prom-base-src --no-tag \"base\" a/file.txt deep &&\n+\tgit clone --bare prom-base-src prom-base-server.git &&\n+\tgit -C prom-base-server.git config uploadpack.allowfilter true &&\n+\tgit -C prom-base-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --no-checkout --filter=tree:1 \\\n+\t\t\"file://$(pwd)/prom-base-server.git\" prom-base-client &&\n+\t(\n+\t\tcd prom-base-client &&\n+\t\tparent_subtree=$(git ls-tree HEAD -- a |\n+\t\t\tawk \"{print \\$3}\") &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\" &&\n+\t\tnew_blob=$(echo \"local-content\" | git hash-object -w --stdin) &&\n+\t\tnew_subtree=$(printf \"100644 blob %s\\tfile.txt\\n\" \\\n+\t\t\t\"$new_blob\" | git mktree) &&\n+\t\tnew_root=$(printf \"40000 tree %s\\ta\\n\" \\\n+\t\t\t\"$new_subtree\" | git mktree) &&\n+\t\tnew_commit=$(git commit-tree \"$new_root\" \\\n+\t\t\t-p HEAD -m \"replace promised subtree\") &&\n+\t\tcheck_connected \"$new_commit\" &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$parent_subtree\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: verifies local commit in partial clone\" '\n+\ttest_when_finished \"rm -rf pc-src pc-server.git pc-client\" &&\n+\tgit init pc-src &&\n+\ttest_commit -C pc-src --no-tag base file.txt &&\n+\tgit clone --bare pc-src pc-server.git &&\n+\tgit -C pc-server.git config uploadpack.allowfilter true &&\n+\tgit -C pc-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/pc-server.git\" pc-client &&\n+\t(\n+\t\tcd pc-client &&\n+\t\tlocal_commit=$(commit_with_change HEAD file.txt local-content) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: tolerates missing promisor parent commit\" '\n+\ttest_when_finished \"rm -rf mp-src mp-server.git mp-client\" &&\n+\tgit init mp-src &&\n+\ttest_commit -C mp-src --no-tag base file.txt &&\n+\tgit clone --bare mp-src mp-server.git &&\n+\tgit -C mp-server.git config uploadpack.allowfilter true &&\n+\tgit -C mp-server.git config uploadpack.allowanysha1inwant true &&\n+\tgit clone --filter=blob:none \\\n+\t\t\"file://$(pwd)/mp-server.git\" mp-client &&\n+\ttest_commit -C mp-src --no-tag c1 file.txt &&\n+\ttest_commit -C mp-src --no-tag c2 file.txt &&\n+\tc1=$(git -C mp-src rev-parse HEAD~1) &&\n+\tc2=$(git -C mp-src rev-parse HEAD) &&\n+\t(\n+\t\tcd mp-client &&\n+\t\t# Add a promisor pack holding only c2, so its parent c1\n+\t\t# is missing but promised (as after a lazy fetch that\n+\t\t# omits dependents).\n+\t\techo \"$c2\" |\n+\t\tgit -C ../mp-src pack-objects --stdout >c2.pack &&\n+\t\tgit index-pack --promisor --stdin <c2.pack &&\n+\t\ttest_must_fail env GIT_NO_LAZY_FETCH=1 \\\n+\t\t\tgit cat-file -e \"$c1\" &&\n+\t\ttree=$(git rev-parse HEAD^{tree}) &&\n+\t\tlocal_commit=$(git hash-object -t commit -w --stdin <<-EOF\n+\t\ttree $tree\n+\t\tparent $c1\n+\t\tauthor A U Thor <author@example.com> 1112912053 -0700\n+\t\tcommitter C O Mitter <committer@example.com> 1112912053 -0700\n+\n+\t\ton top of a missing promisor commit\n+\t\tEOF\n+\t\t) &&\n+\t\tcheck_connected \"$local_commit\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: respects shallow boundary\" '\n+\ttest_when_finished \"rm -rf shallow-src shallow\" &&\n+\tgit init shallow-src &&\n+\ttest_commit -C shallow-src --no-tag base file content-1 &&\n+\tmkdir shallow-src/sub &&\n+\ttest_commit -C shallow-src --no-tag change sub/other content-2 &&\n+\tgit clone --depth=1 \"file://$(pwd)/shallow-src\" shallow &&\n+\t(\n+\t\tcd shallow &&\n+\t\ttip=$(git rev-parse HEAD) &&\n+\t\tgit for-each-ref --format=\"delete %(refname)\" |\n+\t\t\tgit update-ref --no-deref --stdin &&\n+\t\tcheck_connected \"$tip\"\n+\t)\n+'\n+\n+test_expect_success \"$mode: deepening fetch succeeds\" '\n+\ttest_when_finished \"rm -rf deepen-src deepen-server.git deepen-client\" &&\n+\tgit init deepen-src &&\n+\ttest_commit -C deepen-src --no-tag c1 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c2 file.txt &&\n+\ttest_commit -C deepen-src --no-tag c3 file.txt &&\n+\tgit clone --bare deepen-src deepen-server.git &&\n+\tgit clone --depth=1 \"file://$(pwd)/deepen-server.git\" deepen-client &&\n+\tset_connectivity_check deepen-client $mode &&\n+\ttest -f deepen-client/.git/shallow &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/deepen-trace.txt\" \\\n+\t\tgit -C deepen-client fetch --deepen=2 origin main &&\n+\t# Incremental falls back to full for deepening fetches,\n+\t# so the trees_loaded event should not appear.\n+\ttest_grep ! trees_loaded deepen-trace.txt\n+'\n+\n+test_expect_success \"$mode: malformed tree detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\techo abc >malformed-tree &&\n+\t\tmalformed_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tmalformed-tree) &&\n+\t\tmalformed_commit=$(git commit-tree \"$malformed_tree\" \\\n+\t\t\t-p HEAD -m malformed) &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$malformed_commit\" 2>err\n+\t)\n+'\n+\n+test_expect_success PERL_TEST_HELPERS \\\n+\t\"$mode: mid-tree corruption detected\" '\n+\t(\n+\t\tcd main-repo &&\n+\t\t# Build a tree with one valid entry followed by garbage.\n+\t\tblob_oid=$(echo \"valid\" | git hash-object -w --stdin) &&\n+\t\tbin_oid=$(echo \"$blob_oid\" | hex2oct) &&\n+\t\tprintf \"100644 good\\0${bin_oid}GARBAGE\" >corrupt-mid-tree &&\n+\t\tcorrupt_tree=$(git hash-object --literally -t tree -w \\\n+\t\t\tcorrupt-mid-tree) &&\n+\t\tcorrupt_commit=$(git commit-tree \"$corrupt_tree\" \\\n+\t\t\t-p HEAD -m \"mid-tree corruption\") &&\n+\t\ttest_expect_code 128 check_connected \\\n+\t\t\t\"$corrupt_commit\" 2>err &&\n+\t\ttest_grep \"too-short tree object\" err\n+\t)\n+'\n+\n+done\n+\n+# The full check only checks that blobs exist, but the incremental\n+# check reads their headers and reports a corrupt blob.\n+test_expect_success 'incremental: rejects corrupt blob' '\n+\ttest_when_finished \"rm -rf corrupt-repo\" &&\n+\tgit init corrupt-repo &&\n+\t(\n+\t\tcd corrupt-repo &&\n+\t\ttest_commit base &&\n+\t\tblob=$(echo corrupt | git hash-object -w --stdin) &&\n+\t\ttree=$(printf \"100644 blob $blob\\tfile\\n\" | git mktree) &&\n+\t\tcommit=$(git commit-tree \"$tree\" -p HEAD -m corrupt) &&\n+\t\tpath=.git/objects/$(test_oid_to_path \"$blob\") &&\n+\t\tchmod +w \"$path\" &&\n+\t\techo garbage >\"$path\" &&\n+\t\tmode=incremental &&\n+\t\ttest_expect_code 128 check_connected \"$commit\" 2>err &&\n+\t\ttest_grep \"unable to read blob object $blob\" err\n+\t)\n+'\n+\n+# Algorithm selection.\n+\n+# Run a command with its trace2 events in \"trace\" and its stderr\n+# in \"err\".\n+run_traced () {\n+\trm -f trace &&\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace\" \"$@\" 2>err\n+}\n+\n+# Check which connectivity check the last run_traced used.\n+expect_mode () {\n+\tcase \"$1\" in\n+\tfull)\n+\t\ttest_grep ! trees_loaded trace\n+\t\t;;\n+\tincremental)\n+\t\ttest_grep trees_loaded trace\n+\t\t;;\n+\t*)\n+\t\tBUG \"unknown mode: $1\"\n+\t\t;;\n+\tesac\n+}\n+\n+test_expect_success 'setup source repository' '\n+\tgit init src &&\n+\ttest_commit -C src base file.txt &&\n+\ttest_commit -C src --no-tag update file.txt updated\n+'\n+\n+test_expect_success 'unknown transfer.connectivityCheck falls back to full' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C dst fetch ../src base:refs/heads/main &&\n+\tfor value in bogus\n+\tdo\n+\t\tset_connectivity_check dst \"$value\" &&\n+\t\trun_traced git -C dst fetch ../src main &&\n+\t\texpect_mode full &&\n+\t\ttest_grep \"unknown transfer.connectivityCheck algorithm .$value., using .full.\" err ||\n+\t\treturn 1\n+\tdone\n+'\n+\n+test_expect_success 'push uses incremental when configured' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C src push ../dst base:refs/heads/main &&\n+\tset_connectivity_check dst incremental &&\n+\trun_traced git -C src push ../dst main &&\n+\texpect_mode incremental\n+'\n+\n+test_expect_success 'fetch uses incremental when configured' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C dst fetch ../src base:refs/heads/main &&\n+\tset_connectivity_check dst incremental &&\n+\trun_traced git -C dst fetch ../src main &&\n+\texpect_mode incremental\n+'\n+\n+for mode in incremental\n+do\n+\ttest_expect_success \"push with $mode into an empty repository uses full\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\tgit init --bare dst &&\n+\t\tset_connectivity_check dst $mode &&\n+\t\trun_traced git -C src push ../dst main &&\n+\t\texpect_mode full\n+\t'\n+\n+\ttest_expect_success \"clone with $mode uses full\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\trun_traced git -c transfer.connectivityCheck=$mode \\\n+\t\t\tclone --no-local src dst &&\n+\t\texpect_mode full\n+\t'\n+done\n+\n+test_done\ndiff --git a/t/t6700-tree-depth.sh b/t/t6700-tree-depth.sh\nindex 0f6a2ad9b5..edb31e3914 100755\n--- a/t/t6700-tree-depth.sh\n+++ b/t/t6700-tree-depth.sh\n@@ -82,6 +82,23 @@ test_expect_success 'default limit for rev-list fails gracefully' '\n \ttest_must_fail git rev-list --objects big >/dev/null\n '\n \n+# The incremental check needs a boundary; without negative revisions\n+# rev-list falls back to the full check.\n+test_expect_success 'limit recursion of incremental check' '\n+\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n+\tgit $small_ok rev-list --connectivity-check=incremental \\\n+\t\t--objects small --not $boundary >/dev/null &&\n+\ttest_must_fail git $small_no rev-list \\\n+\t\t--connectivity-check=incremental --objects small \\\n+\t\t--not $boundary >/dev/null\n+'\n+\n+test_expect_success 'default limit for incremental check fails gracefully' '\n+\tboundary=$(git commit-tree $EMPTY_TREE -m boundary) &&\n+\ttest_must_fail git rev-list --connectivity-check=incremental \\\n+\t\t--objects big --not $boundary >/dev/null\n+'\n+\n test_expect_success 'limit recursion of diff-tree -r' '\n \tgit $small_ok diff-tree -r $EMPTY_TREE small &&\n \ttest_must_fail git $small_no diff-tree -r $EMPTY_TREE small\ndiff --git a/tree-verify.c b/tree-verify.c\nnew file mode 100644\nindex 0000000000..25db76d30b\n--- /dev/null\n+++ b/tree-verify.c\n@@ -0,0 +1,433 @@\n+#include \"git-compat-util.h\"\n+#include \"commit.h\"\n+#include \"gettext.h\"\n+#include \"hex.h\"\n+#include \"khash.h\"\n+#include \"object.h\"\n+#include \"odb.h\"\n+#include \"oid-array.h\"\n+#include \"oidset.h\"\n+#include \"tree.h\"\n+#include \"tree-walk.h\"\n+#include \"tree-verify.h\"\n+#include \"packfile.h\"\n+#include \"trace2.h\"\n+\n+enum tree_state {\n+\tTREE_UNTRUSTED = 0,\n+\tTREE_TRUSTED   = 1,\n+\tTREE_EXPANDED  = 2,\n+};\n+\n+/* Values are enum tree_state, stored as a byte to keep the map small. */\n+KHASH_INIT(oid_tree, struct object_id, unsigned char, 1,\n+\t   oidhash_by_value, oideq_by_value)\n+\n+static enum tree_state tree_map_get(kh_oid_tree_t *m,\n+\t\t\t\t    const struct object_id *oid)\n+{\n+\tkhint_t pos = kh_get_oid_tree(m, *oid);\n+\tif (pos == kh_end(m))\n+\t\treturn TREE_UNTRUSTED;\n+\treturn kh_val(m, pos);\n+}\n+\n+static void tree_map_add(kh_oid_tree_t *m, const struct object_id *oid,\n+\t\t\t enum tree_state state)\n+{\n+\tint added;\n+\tkhint_t pos = kh_put_oid_tree(m, *oid, &added);\n+\tif (added)\n+\t\tkh_val(m, pos) = state;\n+\telse if (state > kh_val(m, pos))\n+\t\tkh_val(m, pos) = state;\n+}\n+\n+struct work_item {\n+\tstruct name_entry entry;\n+\tstruct oid_array parent_trees;\n+};\n+\n+/* The entries of a tree that still need to be verified. */\n+struct work_list {\n+\tstruct work_item *items;\n+\tsize_t nr, alloc;\n+\tbool has_subtrees;\n+};\n+\n+static void work_list_clear(struct work_list *list)\n+{\n+\tfor (size_t i = 0; i < list->nr; i++)\n+\t\toid_array_clear(&list->items[i].parent_trees);\n+\tfree(list->items);\n+}\n+\n+struct verify_state {\n+\tkh_oid_tree_t *trees;\n+\tstruct oidset trusted_blobs;\n+\tsize_t trees_loaded;\n+\tsize_t blobs_checked;\n+\tint exclude_promisor_objects;\n+\n+\t/*\n+\t * Scratch space for reading one tree at a time.  It lives here\n+\t * rather than on the stack to keep the frame of the recursive\n+\t * verify_tree() small.\n+\t */\n+\tstruct tree_desc desc;\n+\tstruct name_entry entry;\n+};\n+\n+/*\n+ * Merge-walk the work list against one base tree entry, recording\n+ * same-path parent subtrees as recursive comparison bases.\n+ * Returns the updated work-list cursor.\n+ */\n+static size_t collect_subtree_bases(struct work_list *list, size_t wi,\n+\t\t\t\t    const struct name_entry *entry)\n+{\n+\twhile (wi < list->nr) {\n+\t\tstruct work_item *item = &list->items[wi];\n+\t\tint cmp = base_name_compare(item->entry.path,\n+\t\t\t\t\t    item->entry.pathlen,\n+\t\t\t\t\t    item->entry.mode,\n+\t\t\t\t\t    entry->path, entry->pathlen,\n+\t\t\t\t\t    entry->mode);\n+\t\tif (cmp > 0)\n+\t\t\tbreak;\n+\t\tif (cmp < 0) {\n+\t\t\twi++;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (S_ISDIR(item->entry.mode) && S_ISDIR(entry->mode))\n+\t\t\toid_array_append(&item->parent_trees, &entry->oid);\n+\t\treturn wi + 1;\n+\t}\n+\treturn wi;\n+}\n+\n+/*\n+ * Report an object that could not be read, telling a missing object\n+ * apart from one that exists but cannot be read, e.g. because it is\n+ * corrupt.\n+ */\n+static int unreadable_object_error(struct repository *repo,\n+\t\t\t\t   const struct object_id *oid,\n+\t\t\t\t   const char *type)\n+{\n+\tif (!odb_has_object(repo->objects, oid, 0))\n+\t\treturn error(_(\"missing %s object %s\"), type, oid_to_hex(oid));\n+\treturn error(_(\"unable to read %s object %s\"), type, oid_to_hex(oid));\n+}\n+\n+static int verify_blob(struct repository *repo,\n+\t\t       const struct object_id *oid,\n+\t\t       struct verify_state *vs)\n+{\n+\tstruct object_info oi = OBJECT_INFO_INIT;\n+\tenum object_type type;\n+\n+\tif (oidset_contains(&vs->trusted_blobs, oid))\n+\t\treturn 0;\n+\n+\tvs->blobs_checked++;\n+\toi.typep = &type;\n+\tif (odb_read_object_info_extended(repo->objects, oid, &oi,\n+\t\t\t\t\t  OBJECT_INFO_LOOKUP_REPLACE |\n+\t\t\t\t\t  OBJECT_INFO_SKIP_FETCH_OBJECT)) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, oid))\n+\t\t\treturn 0;\n+\t\treturn unreadable_object_error(repo, oid, \"blob\");\n+\t}\n+\tif (type != OBJ_BLOB)\n+\t\treturn error(_(\"object %s is a %s, not a %s\"),\n+\t\t\t     oid_to_hex(oid), type_name(type), \"blob\");\n+\toidset_insert(&vs->trusted_blobs, oid);\n+\treturn 0;\n+}\n+\n+/*\n+ * Read a tree object into *bufp.  Returns -1 if the object exists\n+ * but is not a tree.  Otherwise returns 0, leaving *bufp NULL if the\n+ * object could not be read.\n+ */\n+static int read_tree_object(struct object_database *odb,\n+\t\t\t    const struct object_id *oid,\n+\t\t\t    void **bufp, size_t *sizep)\n+{\n+\tstruct object_info oi = OBJECT_INFO_INIT;\n+\tenum object_type type;\n+\n+\toi.typep = &type;\n+\toi.sizep = sizep;\n+\toi.contentp = bufp;\n+\tif (odb_read_object_info_extended(odb, oid, &oi,\n+\t\t\t\t\t  OBJECT_INFO_LOOKUP_REPLACE |\n+\t\t\t\t\t  OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t*bufp = NULL;\n+\tif (*bufp && type != OBJ_TREE) {\n+\t\tFREE_AND_NULL(*bufp);\n+\t\treturn error(_(\"object %s is a %s, not a %s\"),\n+\t\t\t     oid_to_hex(oid), type_name(type), \"tree\");\n+\t}\n+\treturn 0;\n+}\n+\n+/* List the entries of a tree that are not yet known to be trusted. */\n+static void list_untrusted_entries(struct verify_state *vs,\n+\t\t\t\t   const struct object_id *tree_oid,\n+\t\t\t\t   void *tree_buf, size_t tree_size,\n+\t\t\t\t   struct work_list *list)\n+{\n+\tstruct name_entry *entry = &vs->entry;\n+\n+\tinit_tree_desc(&vs->desc, tree_oid, tree_buf, tree_size);\n+\twhile (tree_entry(&vs->desc, entry)) {\n+\t\tif (S_ISGITLINK(entry->mode))\n+\t\t\tcontinue;\n+\t\tif (S_ISDIR(entry->mode)) {\n+\t\t\tif (tree_map_get(vs->trees, &entry->oid) >=\n+\t\t\t    TREE_TRUSTED)\n+\t\t\t\tcontinue;\n+\t\t\tlist->has_subtrees = true;\n+\t\t} else {\n+\t\t\tif (oidset_contains(&vs->trusted_blobs, &entry->oid))\n+\t\t\t\tcontinue;\n+\t\t}\n+\t\tALLOC_GROW(list->items, list->nr + 1, list->alloc);\n+\t\tmemset(&list->items[list->nr], 0, sizeof(list->items[0]));\n+\t\tlist->items[list->nr++].entry = *entry;\n+\t}\n+}\n+\n+/*\n+ * Scan one trusted base tree: expand it if that has not happened yet,\n+ * and collect its same-path subtrees as bases for untrusted subtrees.\n+ */\n+static int scan_base_tree(struct repository *repo, struct verify_state *vs,\n+\t\t\t  const struct object_id *base_oid,\n+\t\t\t  struct work_list *list)\n+{\n+\tint expanded = tree_map_get(vs->trees, base_oid) >= TREE_EXPANDED;\n+\tstruct name_entry *entry = &vs->entry;\n+\tsize_t wi = 0, base_size;\n+\tvoid *base_buf;\n+\n+\tif (expanded && !list->has_subtrees)\n+\t\treturn 0;\n+\n+\tif (read_tree_object(repo->objects, base_oid, &base_buf, &base_size))\n+\t\treturn -1;\n+\tif (!base_buf) {\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, base_oid))\n+\t\t\treturn 0;\n+\t\treturn unreadable_object_error(repo, base_oid, \"tree\");\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tinit_tree_desc(&vs->desc, base_oid, base_buf, base_size);\n+\n+\twhile (tree_entry(&vs->desc, entry)) {\n+\t\tif (S_ISGITLINK(entry->mode))\n+\t\t\tcontinue;\n+\n+\t\t/* Same-path subtree as a base for an untrusted one. */\n+\t\tif (list->has_subtrees)\n+\t\t\twi = collect_subtree_bases(list, wi, entry);\n+\n+\t\t/* Expand: the base is trusted, so are its entries. */\n+\t\tif (!expanded) {\n+\t\t\tif (S_ISDIR(entry->mode))\n+\t\t\t\ttree_map_add(vs->trees, &entry->oid,\n+\t\t\t\t\t     TREE_TRUSTED);\n+\t\t\telse\n+\t\t\t\toidset_insert(&vs->trusted_blobs,\n+\t\t\t\t\t      &entry->oid);\n+\t\t}\n+\t}\n+\n+\tif (!expanded)\n+\t\ttree_map_add(vs->trees, base_oid, TREE_EXPANDED);\n+\n+\tfree(base_buf);\n+\treturn 0;\n+}\n+\n+/*\n+ * Verify a tree against the trusted base trees, loading as little as\n+ * possible:\n+ *\n+ * First list the entries that are not yet known to be trusted.  When\n+ * the parent trees have been expanded before, only the changed entries\n+ * remain.\n+ *\n+ * Then scan the trusted base trees (the same-path trees of the parent\n+ * commits).  A base that has not been expanded yet is read so that its\n+ * entries can be marked trusted; this often shows that listed entries\n+ * are trusted after all.  An already expanded base is still read when\n+ * there are untrusted subtrees, because its subtree at the same path is\n+ * what lets the untrusted subtree skip its own unchanged entries in\n+ * turn.\n+ *\n+ * Finally check that the remaining blobs exist, and recurse into the\n+ * remaining subtrees with the same-path subtrees as their bases.\n+ */\n+static int verify_tree(struct repository *repo,\n+\t\t       const struct object_id *tree_oid,\n+\t\t       const struct oid_array *trusted_bases,\n+\t\t       struct verify_state *vs, int depth)\n+{\n+\tstruct work_list list = { 0 };\n+\tsize_t i, tree_size;\n+\tvoid *tree_buf;\n+\tint ret = 0;\n+\n+\t/*\n+\t * Enforce core.maxTreeDepth like process_tree() in list-objects.c,\n+\t * so that the full and incremental checks reject the same trees.\n+\t */\n+\tif (depth > repo->settings.max_allowed_tree_depth)\n+\t\treturn error(_(\"exceeded maximum allowed tree depth\"));\n+\n+\tif (tree_map_get(vs->trees, tree_oid) >= TREE_TRUSTED)\n+\t\treturn 0;\n+\n+\tif (read_tree_object(repo->objects, tree_oid, &tree_buf, &tree_size))\n+\t\treturn -1;\n+\tif (!tree_buf) {\n+\t\t/*\n+\t\t * A missing tree that is promised can be fetched on demand,\n+\t\t * so it is not an error, as in the full check.\n+\t\t */\n+\t\tif (vs->exclude_promisor_objects &&\n+\t\t    is_promisor_object(repo, tree_oid))\n+\t\t\treturn 0;\n+\t\treturn unreadable_object_error(repo, tree_oid, \"tree\");\n+\t}\n+\n+\tvs->trees_loaded++;\n+\tlist_untrusted_entries(vs, tree_oid, tree_buf, tree_size, &list);\n+\n+\t/* Scan the base trees, unless nothing is left to verify. */\n+\tfor (i = 0; list.nr && i < trusted_bases->nr; i++) {\n+\t\tret = scan_base_tree(repo, vs, &trusted_bases->oid[i], &list);\n+\t\tif (ret)\n+\t\t\tgoto done;\n+\t}\n+\n+\t/*\n+\t * Check the remaining entries in tree order.  Verifying a subtree\n+\t * can show that later entries are trusted, which then makes them\n+\t * cheap to skip.\n+\t */\n+\tfor (i = 0; i < list.nr && !ret; i++) {\n+\t\tstruct work_item *item = &list.items[i];\n+\n+\t\tif (S_ISDIR(item->entry.mode))\n+\t\t\tret = verify_tree(repo, &item->entry.oid,\n+\t\t\t\t\t  &item->parent_trees, vs, depth + 1);\n+\t\telse\n+\t\t\tret = verify_blob(repo, &item->entry.oid, vs);\n+\t}\n+\tif (!ret)\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_EXPANDED);\n+\n+done:\n+\tfree(tree_buf);\n+\twork_list_clear(&list);\n+\treturn ret;\n+}\n+\n+static int verify_commit_tree(struct repository *repo,\n+\t\t\t      struct commit *commit,\n+\t\t\t      struct verify_state *vs)\n+{\n+\tstruct oid_array parent_trees = OID_ARRAY_INIT;\n+\tconst struct object_id *tree_oid;\n+\tstruct commit_list *p;\n+\tint ret;\n+\n+\t/*\n+\t * Parent trees are trusted: boundary parents are already\n+\t * connected, and earlier incoming parents were verified\n+\t * first due to the topological processing order.\n+\t *\n+\t * The revision walk skips missing parents that are promised\n+\t * (see process_parents()), so do the same here.  Such a parent\n+\t * contributes no base, which only means less of the new tree\n+\t * can be skipped.\n+\t */\n+\tfor (p = commit->parents; p; p = p->next) {\n+\t\tif (repo_parse_commit_gently(repo, p->item, 1) < 0) {\n+\t\t\tif (vs->exclude_promisor_objects &&\n+\t\t\t    is_promisor_object(repo, &p->item->object.oid))\n+\t\t\t\tcontinue;\n+\t\t\tret = error(_(\"unable to parse commit %s\"),\n+\t\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\t\tgoto out;\n+\t\t}\n+\t\ttree_oid = get_commit_tree_oid(p->item);\n+\t\tif (!tree_oid) {\n+\t\t\tret = error(_(\"unable to load root tree for commit %s\"),\n+\t\t\t\t    oid_to_hex(&p->item->object.oid));\n+\t\t\tgoto out;\n+\t\t}\n+\t\ttree_map_add(vs->trees, tree_oid, TREE_TRUSTED);\n+\t\toid_array_append(&parent_trees, tree_oid);\n+\t}\n+\n+\ttree_oid = get_commit_tree_oid(commit);\n+\tif (!tree_oid) {\n+\t\tret = error(_(\"unable to load root tree for commit %s\"),\n+\t\t\t    oid_to_hex(&commit->object.oid));\n+\t\tgoto out;\n+\t}\n+\tret = verify_tree(repo, tree_oid, &parent_trees, vs, 0);\n+out:\n+\toid_array_clear(&parent_trees);\n+\treturn ret;\n+}\n+\n+int verify_commits_incremental(struct repository *repo,\n+\t\t\t       struct commit_list **commits,\n+\t\t\t       int exclude_promisor_objects)\n+{\n+\tstruct verify_state vs = { 0 };\n+\tstruct commit_list *iter;\n+\tunsigned nr_before;\n+\tint ret = 0;\n+\n+\tvs.trees = kh_init_oid_tree();\n+\tvs.exclude_promisor_objects = exclude_promisor_objects;\n+\n+\t/*\n+\t * Ancestors must be verified before descendants so that parent\n+\t * trees can be trusted without re-verification.  Sort explicitly\n+\t * rather than relying on the caller's ordering.\n+\t *\n+\t * sort_in_topological_order() silently drops cycle members,\n+\t * so explicitly check if the size has changed.\n+\t */\n+\tnr_before = commit_list_count(*commits);\n+\tsort_in_topological_order(commits, REV_SORT_IN_GRAPH_ORDER);\n+\tif (commit_list_count(*commits) < nr_before) {\n+\t\tret = error(_(\"cycle detected in incoming commit graph\"));\n+\t\tgoto out;\n+\t}\n+\n+\t*commits = commit_list_reverse(*commits);\n+\n+\tfor (iter = *commits; iter && !ret; iter = iter->next)\n+\t\tret = verify_commit_tree(repo, iter->item, &vs);\n+\n+out:\n+\tkh_destroy_oid_tree(vs.trees);\n+\toidset_clear(&vs.trusted_blobs);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"trees_loaded\", (intmax_t)vs.trees_loaded);\n+\ttrace2_data_intmax(\"connectivity\", repo,\n+\t\t\t   \"blobs_checked\", (intmax_t)vs.blobs_checked);\n+\treturn ret;\n+}\ndiff --git a/tree-verify.h b/tree-verify.h\nnew file mode 100644\nindex 0000000000..ac7092da1c\n--- /dev/null\n+++ b/tree-verify.h\n@@ -0,0 +1,15 @@\n+#ifndef TREE_VERIFY_H\n+#define TREE_VERIFY_H\n+\n+struct commit_list;\n+struct repository;\n+\n+/*\n+ * Verify trees of commits incrementally against their parents.\n+ * Returns 0 on success, or reports an error and returns -1.\n+ */\n+int verify_commits_incremental(struct repository *repo,\n+\t\t\t       struct commit_list **commits,\n+\t\t\t       int exclude_promisor_objects);\n+\n+#endif /* TREE_VERIFY_H */\n-- \ngitgitgadget\n\n\n"},{"id":"556023","messageId":"98f67caa04bfe5b389846e99b069aeb8de221ecd.1792761119.git.gitgitgadget@gmail.com","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"[PATCH v5 6/6] connected: add \"auto\" connectivity check mode","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-23T13:11:59Z","receivedAt":"2026-10-23T13:11:59Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe incremental connectivity check pays a per-commit cost.  For\nevery incoming commit it reads the new trees along the changed\npaths, and the trees at the same paths in its parents to compare\nagainst.  A parent tree is often read again even though an earlier\ncommit already read it: on git.git, about a third of the tree reads\nfor 2000 incoming commits are such re-reads.  The full check\ninstead pays a mostly fixed cost for walking the boundary trees,\nso with many incoming commits the incremental check becomes the\nslower one (see the previous commit).  Users who mostly\nreceive small pushes but occasionally a large one should not have\nto choose between the two.\n\nAdd an \"auto\" value for transfer.connectivityCheck that picks the\nmode based on the incoming history.  When the revision walk is\nlimited, which is the case whenever the receiving repository has\nreferences, the incoming commit list is already computed by\nlimit_list(), so counting it is cheap.  \"auto\" then uses the\nincremental check for up to 1000 incoming commits, and the full\ncheck above that.\n\nThe cutoff is a fixed constant based on the p5413 fixture in the\nprevious commit, where breakeven is between 500 and 3000 commits.\nIt is conservative for repositories with large trees, where the\nfixed cost of the full check is higher.  Measured on real\nrepositories (rev-list against a single boundary commit, mean\nof at least 10 runs):\n\n    repository   files  commits    full  incr.  full/incr\n    internal A    556K     1000   1.71s  0.61s    2.8x\n    internal A    556K     4000   2.78s  2.83s    1.0x\n    internal B    288K     1000   1.78s  0.88s    2.0x\n    linux          95K      598   0.33s  0.12s    2.7x\n    linux          95K     1728   0.56s  0.35s    1.6x\n    git           4.9K      984   0.22s  0.27s    0.8x\n\nWith small trees incremental can be slower within the cutoff,\nbut both modes are fast enough there (0.22s vs 0.27s for git\nabove) that the difference does not matter in practice.\n\nNo single cutoff is optimal for every repository, so it can be set\nwith \"auto=<cutoff>\"; plain \"auto\" uses 1000.  A malformed cutoff\nis treated like any other unknown value: it falls back to \"full\"\nwith a warning.\n\nSuggested-by: Patrick Steinhardt <ps@pks.im>\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Documentation/config/transfer.adoc  |  6 ++++++\n Documentation/rev-list-options.adoc |  2 +-\n builtin/rev-list.c                  | 28 ++++++++++++++++++++++++++--\n connected.c                         |  9 ++++++++-\n t/t5413-connectivity-check.sh       | 25 +++++++++++++++++++++++--\n 5 files changed, 64 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/config/transfer.adoc b/Documentation/config/transfer.adoc\nindex cba7597a58..f77b25ff8b 100644\n--- a/Documentation/config/transfer.adoc\n+++ b/Documentation/config/transfer.adoc\n@@ -18,6 +18,12 @@ transfer.connectivityCheck::\n \tthan `full` when there are many incoming commits.\n \tFalls back to `full` for clones, pushes into an empty\n \trepository and deepening fetches.\n+`auto`;;\n+`auto=<cutoff>`;;\n+\tChoose between `full` and `incremental` with a heuristic.\n+\tCurrently `incremental` is used when there are at most\n+\t_<cutoff>_ incoming commits (1000 by default), and `full`\n+\totherwise.  The heuristic may change in future versions.\n --\n +\n Unknown values are treated as `full`, with a warning.\ndiff --git a/Documentation/rev-list-options.adoc b/Documentation/rev-list-options.adoc\nindex 2023ad5f75..285fb3809f 100644\n--- a/Documentation/rev-list-options.adoc\n+++ b/Documentation/rev-list-options.adoc\n@@ -1103,7 +1103,7 @@ It is an error to use `--missing-only` without `--missing=print` or\n \ttraversal, rather than just silencing errors about missing\n \tobjects.\n \n-`--connectivity-check=(full|incremental)`::\n+`--connectivity-check=(full|incremental|auto[=<cutoff>])`::\n \t(For internal use only.)  Used by `check_connected()` to\n \tselect the connectivity check algorithm; see\n \t`transfer.connectivityCheck` in linkgit:git-config[1].\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex c07ad6e30c..071a0e3cb5 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -29,23 +29,35 @@\n #include \"quote.h\"\n #include \"strbuf.h\"\n #include \"tree-verify.h\"\n+#include \"parse.h\"\n+\n+#define INCREMENTAL_MAX_COMMITS 1000\n \n enum connectivity_check_mode {\n \tCONNECTIVITY_CHECK_FULL,\n \tCONNECTIVITY_CHECK_INCREMENTAL,\n+\tCONNECTIVITY_CHECK_AUTO,\n };\n \n struct connectivity_check {\n \tenum connectivity_check_mode mode;\n+\tunsigned long max_commits;\n };\n \n static void parse_connectivity_check(const char *arg,\n \t\t\t\t     struct connectivity_check *check)\n {\n+\tconst char *cutoff;\n+\n \tif (!strcmp(arg, \"full\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_FULL;\n \telse if (!strcmp(arg, \"incremental\"))\n \t\tcheck->mode = CONNECTIVITY_CHECK_INCREMENTAL;\n+\telse if (!strcmp(arg, \"auto\"))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_AUTO;\n+\telse if (skip_prefix(arg, \"auto=\", &cutoff) &&\n+\t\t git_parse_ulong(cutoff, &check->max_commits))\n+\t\tcheck->mode = CONNECTIVITY_CHECK_AUTO;\n \telse\n \t\tdie(_(\"unknown --connectivity-check mode '%s'\"), arg);\n }\n@@ -748,7 +760,9 @@ int cmd_rev_list(int argc,\n \tint bisect_find_all = 0;\n \tint use_bitmap_index = 0;\n \tint filter_provided_objects = 0;\n-\tstruct connectivity_check connectivity_check = { 0 };\n+\tstruct connectivity_check connectivity_check = {\n+\t\t.max_commits = INCREMENTAL_MAX_COMMITS,\n+\t};\n \tconst char *show_progress = NULL;\n \tint ret = 0;\n \n@@ -1001,9 +1015,19 @@ int cmd_rev_list(int argc,\n \t * The incremental check would still work, but it would verify\n \t * the whole object graph like the full check, with extra\n \t * per-commit overhead on top, so use the full check.\n+\t *\n+\t * Otherwise the walk is limited, and revs.commits already holds\n+\t * all incoming commits, so counting them for \"auto\" is cheap.\n \t */\n-\tif (!revs.limited)\n+\tif (!revs.limited) {\n \t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\t} else if (connectivity_check.mode == CONNECTIVITY_CHECK_AUTO) {\n+\t\tif (commit_list_count(revs.commits) <=\n+\t\t    connectivity_check.max_commits)\n+\t\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_INCREMENTAL;\n+\t\telse\n+\t\t\tconnectivity_check.mode = CONNECTIVITY_CHECK_FULL;\n+\t}\n \n \tif (connectivity_check.mode == CONNECTIVITY_CHECK_INCREMENTAL) {\n \t\tstruct commit *commit;\ndiff --git a/connected.c b/connected.c\nindex c032987410..8ceaa45644 100644\n--- a/connected.c\n+++ b/connected.c\n@@ -3,6 +3,7 @@\n #include \"gettext.h\"\n #include \"hex.h\"\n #include \"odb.h\"\n+#include \"parse.h\"\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"connected.h\"\n@@ -74,7 +75,8 @@ static void connectivity_check_mode(struct repository *repo,\n \t\t\t\t    struct check_connected_options *opt,\n \t\t\t\t    struct strbuf *mode)\n {\n-\tconst char *algorithm = NULL;\n+\tconst char *algorithm = NULL, *cutoff;\n+\tunsigned long max_commits;\n \n \t/*\n \t * Deepening fetches do not have the usual boundary of\n@@ -93,6 +95,11 @@ static void connectivity_check_mode(struct repository *repo,\n \t\tstrbuf_addstr(mode, \"full\");\n \telse if (!strcasecmp(algorithm, \"incremental\"))\n \t\tstrbuf_addstr(mode, \"incremental\");\n+\telse if (!strcasecmp(algorithm, \"auto\"))\n+\t\tstrbuf_addstr(mode, \"auto\");\n+\telse if (skip_iprefix(algorithm, \"auto=\", &cutoff) &&\n+\t\t git_parse_ulong(cutoff, &max_commits))\n+\t\tstrbuf_addf(mode, \"auto=%lu\", max_commits);\n \telse {\n \t\twarning(_(\"unknown transfer.connectivityCheck algorithm '%s', \"\n \t\t\t  \"using 'full'\"), algorithm);\ndiff --git a/t/t5413-connectivity-check.sh b/t/t5413-connectivity-check.sh\nindex 109ad17cff..60edf99b63 100755\n--- a/t/t5413-connectivity-check.sh\n+++ b/t/t5413-connectivity-check.sh\n@@ -725,7 +725,7 @@ test_expect_success 'unknown transfer.connectivityCheck falls back to full' '\n \ttest_when_finished \"rm -rf dst\" &&\n \tgit init --bare dst &&\n \tgit -C dst fetch ../src base:refs/heads/main &&\n-\tfor value in bogus\n+\tfor value in bogus auto= auto=x\n \tdo\n \t\tset_connectivity_check dst \"$value\" &&\n \t\trun_traced git -C dst fetch ../src main &&\n@@ -753,7 +753,7 @@ test_expect_success 'fetch uses incremental when configured' '\n \texpect_mode incremental\n '\n \n-for mode in incremental\n+for mode in incremental auto\n do\n \ttest_expect_success \"push with $mode into an empty repository uses full\" '\n \t\ttest_when_finished \"rm -rf dst\" &&\n@@ -771,4 +771,25 @@ do\n \t'\n done\n \n+for value in auto auto=1\n+do\n+\ttest_expect_success \"push with $value uses incremental up to the cutoff\" '\n+\t\ttest_when_finished \"rm -rf dst\" &&\n+\t\tgit init --bare dst &&\n+\t\tgit -C src push ../dst base:refs/heads/main &&\n+\t\tset_connectivity_check dst $value &&\n+\t\trun_traced git -C src push ../dst main &&\n+\t\texpect_mode incremental\n+\t'\n+done\n+\n+test_expect_success 'push with auto uses full above the cutoff' '\n+\ttest_when_finished \"rm -rf dst\" &&\n+\tgit init --bare dst &&\n+\tgit -C src push ../dst base:refs/heads/main &&\n+\tset_connectivity_check dst auto=0 &&\n+\trun_traced git -C src push ../dst main &&\n+\texpect_mode full\n+'\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"556088","messageId":"xmqq4iec70yp.fsf@gitster.g","threadId":"66326","inReplyTo":"CAL71e4Ngsxv=JU_qzVGVyHPZiB2Qzvjn22NeVQAtZOdCr2vyXQ@mail.gmail.com","subject":"Re: [PATCH v4 5/6] connected: add incremental connectivity check via rev-list","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-23T17:58:54Z","receivedAt":"2026-10-23T17:58:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> This ends up nicer than what I had -- I can also get rid of the\n> BUG() guard for checking that flag state.\n\nThat's great.\n\n> Do you still want me to include this reasoning in the commit,\n> or is that now unnecessary given that it would no longer touch\n> fetch_if_missing?\n\nI do not care too deeply, but the next person who wants to add a\ncode to depend on fetch_if_missing may appreciate it, no?\n\n\n"},{"id":"556101","messageId":"xmqq4iec5jxi.fsf@gitster.g","threadId":"66326","inReplyTo":"pull.2211.v5.git.1792761119.gitgitgadget@gmail.com","subject":"Re: [PATCH v5 0/6] connected: add incremental connectivity check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-23T18:52:09Z","receivedAt":"2026-10-23T18:52:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> Next steps\n> ==========\n>\n> This series only addresses tree verification; the other significant cost is\n> finding the commit boundary, especially for repos with many refs. I already\n> have some prototypes for optimizing that too, and if this ends up landing,\n> that would be something I would start polishing up.\n>\n>\n> Changes since v4\n> ================\n>\n>  * The series no longer touches repo->fetch_if_missing, which Junio pointed\n>    out was a trap when cleared only for some modes. The incremental verifier\n>    instead reads blobs and trees with odb_read_object_info_extended() and\n>    OBJECT_INFO_SKIP_FETCH_OBJECT, so it never lazily fetches on its own;\n>    commit parsing already never fetches. The BUG() guard on the flag is\n>    gone, and parse_connectivity_check() no longer takes a repository. The\n>    full check is unchanged: in a partial clone check_connected() passes\n>    --exclude-promisor-objects as before.\n>  * A tree that cannot be read because it is corrupt is now reported as an\n>    error, like a corrupt blob, instead of dying.\n\nThanks, these incremental changes look good.\n\nWill replace.\n\n"}]}