[PATCH v3 3/6] odb/source-files: reload alternates on retry
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Oct 28, 2026, 07:11 UTC
- Message-ID
- <20261028-pks-odb-files-reload-alternates-v3-3-011ddb64b626@pks.im>
- In-Reply-To
- <20261028-pks-odb-files-reload-alternates-v3-0-011ddb64b626@pks.im>
When an object ID cannot be found we retry reading it from our packfiles after we have flushed the respective caches of the packed backends. But this misses one scenario: it may happen that the object databases' alternates get rewritten concurrently and all of its objects are moved to such an alternate. If so, we won't be able to find any of the objects anymore. This used to be working just fine, but with 5946a564cd (odb/source: make `read_object_info()` function pluggable, 2026-03-05) we have accidentally dropped this logic.
This scenario sounds somewhat unlikely, but we've actually been hitting it in the wild: when running long-lived git-cat-file(1) processes that we interactively query over many minutes, and that repository gets forked concurrently, then we move all objects into an object pool, write a new alternates file, and the git-cat-file(1) process will remain broken for the rest of its lifetime.
Fix this bug by re-preparing alternates when the object in question could not be found and then retrying the read. A couple things to note:
- This requires us to re-read loose objects, which is an operation
that we previously skipped on the retry. But we only need to do this
for the loose object directories that are actually new. - The result is still incomplete, as we only reload the alternates of
the primary object directory. But alternates are added recursively,
so in theory we'd also have to check whether alternates of any of
the alternates have changed. But we never did that before the
regression, either, so for now we don't care about this edge case. - It may happen that we reload when a quarantine directory is active.
If no alternates existed before that reload we may not have lazily
populated the map of object directories yet. So the assertion in
`odb_files_dir_is_usable()` does not necessarily hold anymore, and
we now have to populate the map with all object directories.Furthermore, this change causes a performance regression when asking for lots of missing objects. But that regression is basically in line with the performance we had before this mechanism got broken. The following benchmark demonstrates this when asking for 20k missing commits in an empty repository with a single empty alternate:
Benchmark 1: ./git -C /tmp/repo cat-file --batch-check (rev = 5946a564cd~) (50 runs)
mean ± σ min … max
Wall Time 289.9 ms ± 3.4 ms 285.8 ms … 307.5 ms
Memory 5.6 MiB ± 0.1 MiB 5.5 MiB … 5.7 MiB Benchmark 2: ./git -C /tmp/repo cat-file --batch-check (rev = 5946a564cd) (50 runs)
mean ± σ min … max
Wall Time 184.8 ms ± 2.5 ms 181.5 ms … 191.9 ms -36.2% (1.6x faster)
Memory 5.6 MiB ± 0.1 MiB 5.5 MiB … 5.7 MiB -0.1% Benchmark 3: ./git -C /tmp/repo cat-file --batch-check (rev = HEAD~) (50 runs)
mean ± σ min … max
Wall Time 189.1 ms ± 2.7 ms 185.9 ms … 196.1 ms -34.8% (1.5x faster)
Memory 5.9 MiB ± 0.1 MiB 5.6 MiB … 6.1 MiB +4.9% Benchmark 4: ./git -C /tmp/repo cat-file --batch-check (rev = HEAD) (50 runs)
mean ± σ min … max
Wall Time 293.2 ms ± 2.6 ms 289.3 ms … 303.2 ms +1.1%
Memory 5.9 MiB ± 0.1 MiB 5.6 MiB … 6.1 MiB +4.4%There's no helping this though, with the regression fixed we will be slower compared to before as we have to do more work. But the next commits will improve the status quo a bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im> --- odb/source-files.c | 21 +++++++++++++-------- t/t1006-cat-file.sh | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 8 deletions(-)
diff --git a/odb/source-files.c b/odb/source-files.c index c0375e7cb70..d66949b7de5 100644 --- a/odb/source-files.c +++ b/odb/source-files.c @@ -243,10 +243,11 @@ static bool odb_files_dir_is_usable(struct odb_source_files *files, * thing twice, or object directory itself. */ if (!hashmap_get_size(&files->dirs_by_path)) { - assert(!files->dirs->next); - hashmap_entry_init(&files->dirs->by_path_entry, - strihash(files->dirs->abspath)); - hashmap_add(&files->dirs_by_path, &files->dirs->by_path_entry); + for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) { + hashmap_entry_init(&dir->by_path_entry, + strihash(dir->abspath)); + hashmap_add(&files->dirs_by_path, &dir->by_path_entry); + } } if (!odb_files_dir_paths_cmp(files, path, normalized_objdir.buf)) @@ -425,6 +426,7 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source struct strbuf *errmsg) { struct odb_source_files *files = odb_source_files_downcast(source); + struct odb_files_dir **tail = NULL; enum odb_read_status status = ODB_READ_NOT_FOUND; enum odb_read_status ret; bool retry = false; @@ -448,10 +450,11 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source status = ret; } - for (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) { - if (retry) - break; - + /* + * When we have reprepared alternates then we only re-scan directories + * that were newly added. + */ + for (struct odb_files_dir *dir = tail ? *tail : files->dirs; dir; dir = dir->next) { ret = odb_source_read_object_info(&dir->loose->base, oid, oi, flags, status == ODB_READ_NOT_FOUND ? errmsg : NULL); if (!ret) @@ -461,6 +464,8 @@ static enum odb_read_status odb_source_files_read_object_info(struct odb_source } if (!retry && !(flags & OBJECT_INFO_QUICK)) { + tail = files->dirs_tail; + prepare_alternates(files, NULL); retry = true; goto retry; } diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh index 72fc8acc611..f2b24dd2787 100755 --- a/t/t1006-cat-file.sh +++ b/t/t1006-cat-file.sh @@ -1481,4 +1481,36 @@ test_objects_filter "object:type=commit" test_objects_filter "object:type=tag" test_objects_filter "object:type=tree" +test_expect_success PIPE 'concurrently written alternates are loaded' ' + test_when_finished "rm -rf repo alternate stdin stdout" && + git init repo && + git init alternate && + oid=$(git -C alternate hash-object -w --stdin </dev/null) && + mkfifo stdin && + mkfifo stdout && + { git -C repo cat-file --batch-check <stdin >stdout & } && + ( + exec 8>stdin && + exec 9<stdout && + + # Read the object the first time, without having alternates + # wired up yet. The object should be reported as missing. + echo "$oid" >&8 && + read line <&9 && + echo "$line" >actual && + echo "$oid missing" >expect && + test_cmp expect actual && + + # Read the object the second time, this time with the alternate + # set up. The backend should reload alternates and thus find + # the object. + echo "$PWD/alternate/.git/objects" >repo/.git/objects/info/alternates && + echo "$oid" >&8 && + read line <&9 && + echo "$line" >actual && + echo "$oid blob 0" >expect && + test_cmp expect actual + ) +' + test_done
-- 2.56.0.341.g393f52fd96.dirty