[PATCH v2 2/2] packfile: fix corruption due to stale delta base cache entries
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Oct 6, 2026, 10:20 UTC
- Message-ID
- <20261006-pks-packfile-stale-delta-base-cache-v2-2-69669a2fc6ce@pks.im>
- In-Reply-To
- <20261006-pks-packfile-stale-delta-base-cache-v2-0-69669a2fc6ce@pks.im>
The delta base cache is a process-global hashmap that is keyed by the address of the `struct packed_git` plus the offset of the base object within that pack. Entries part of the cache are never removed when a pack is closed, and neither when the pack is subsequently freed. As a consequence, the cache may contain stale entries.
For a long time, the worst consequence of this leaking cache was that we held on to memory that we could've released. But the reason for this was that we didn't even free the packfiles, either. That has changed in 6f1e9394e2 (object: fix leaking packfiles when closing object store, 2024-08-08), where we plugged that leak.
Now that we free them, a new packfile may be allocated using the exact same address as a previously allocated one. And if the new packfile has both the same address and a similar layout, it may happen that a preexisting entry from a previously-allocated in the delta base cache would have the exact same key.
All of this sounds very theoretical, but we can actually trigger this bug somewhat reliably! When doing a merge in a repository with lots of submodules that have similar-looking packfiles we end up opening and then closing the object databases of each of the submodules in sequence. Because of the above mentioned commit we would close and free each of the packfiles part of the respective databases, but we wouldn't evict their delta base entries from the cache.
When using glibc, one of the packfiles will eventually get the exact same address, and that will then cause Git to read the wrong entry from the cache. Git detects this and aborts with an error:
$ git merge branch-b
error: Could not read 584ef938be4a749bfa13f68d5ac5545bc029e529
error: could not parse commit 584ef938be4a749bfa13f68d5ac5545bc029e529
error: failed to merge submodule G (repository corrupt)Now in this case we're lucky that Git detects this error because we try to read a commit from a different submodule via an object database that doesn't have it. But potentially, in an even more contrived scenario, we might even silently yield wrong data from the cache.
Fix this bug by evicting cache entries that belong to a specific pack when closing it.
Note that the added test reliably reproduces the above bug on my machine that uses NixOS at c59305bab206 (cosmic-applets: add missing runtime dependency (#566040), 2026-10-01) with glibc 2.44-25. But as we rely on specific allocation behaviour of glibc it is very likely that the test will not work on other platforms.
Reported-by: Guillaume Chauvel <guillaume.chauvel@gmail.com> Helped-by: Philippe Blain <levraiphilippeblain@gmail.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> --- packfile.c | 13 +++++++++++++ t/t6437-submodule-merge.sh | 47 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+)
diff --git a/packfile.c b/packfile.c index af1b837974..365c54c7dc 100644 --- a/packfile.c +++ b/packfile.c @@ -1253,6 +1253,18 @@ void clear_delta_base_cache(void) } } +static void delta_base_cache_evict_entry(struct packed_git *p) +{ + struct list_head *lru, *tmp; + + list_for_each_safe(lru, tmp, &delta_base_cache_lru) { + struct delta_base_cache_entry *entry = + list_entry(lru, struct delta_base_cache_entry, lru); + if (entry->key.p == p) + release_delta_base_cache(entry); + } +} + void close_pack(struct packed_git *p) { close_pack_windows(p); @@ -1261,6 +1273,7 @@ void close_pack(struct packed_git *p) close_pack_revindex(p); close_pack_mtimes(p); oidset_clear(&p->bad_objects); + delta_base_cache_evict_entry(p); } static void add_delta_base_cache(struct packed_git *p, off_t base_offset, diff --git a/t/t6437-submodule-merge.sh b/t/t6437-submodule-merge.sh index 1546d5f773..0ee3684206 100755 --- a/t/t6437-submodule-merge.sh +++ b/t/t6437-submodule-merge.sh @@ -514,4 +514,51 @@ test_expect_success 'merging should fail with no merge base' ' ) ' +test_expect_success 'merge with many packed submodules reports conflicts' ' + test_config_global protocol.file.allow always && + + # Create 16 submodules with two divergent branches each. + submodules="A B C D E F G H I J K L M N O P" && + for name in $submodules + do + git init source-$name && + test_commit -C source-$name $name-main && + git -C source-$name switch --create branch-a main && + git -C source-$name commit --allow-empty --message $name-branch-a && + git -C source-$name switch --create branch-b main && + git -C source-$name commit --allow-empty --message $name-branch-b || return 1 + done && + + # Create the superproject and add all submodules. + git init many-packed && + for name in $submodules + do + git -C many-packed submodule add --branch main "file://$PWD/source-$name" $name || return 1 + done && + git -C many-packed commit --message main && + + # Create two divergent commits in the superproject that update all + # submodules to the divergent branches. + for branch in branch-a branch-b + do + git -C many-packed switch -c $branch main && + for name in $submodules + do + git -C many-packed/$name switch $branch || return 1 + done && + git -C many-packed add $submodules && + git -C many-packed commit --message $branch || return 1 + done && + + # Clone the superproject to ensure that everything is well-packed and + # then merge the two branches, creating conflicts for every submodule. + git clone many-packed many-packed-clone && + git -C many-packed-clone submodule update --init && + git -C many-packed-clone switch branch-a && + test_expect_code 1 git -C many-packed-clone -c advice.submoduleMergeConflict=false merge branch-b >out 2>err && + grep "^CONFLICT (submodule)" out >conflicts && + test_line_count = 16 conflicts && + test_must_be_empty err +' + test_done
-- 2.56.0.406.ga2d225a756.dirty