threads / bug / 66857

[BUG] index-pack lazily fetches REF_DELTA bases that are in the pack being indexed

Subject: [BUG] index-pack lazily fetches REF_DELTA bases that are in the pack being indexed

## tl;dr

In a blobless partial clone, indexing a thin pack with REF_DELTA entries triggers promisor fetches for bases the pack itself contains. Read the story.

replies: 0people: 1as markdown or json

לירון עדן· Nov 16, 2026, 17:38 UTC · lore

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

In a blobless partial clone, I fetched new commits from a server that sends a thin pack in which the deltas with a base inside the pack are REF_DELTA (the client did not negotiate ofs-delta).

The script below reproduces it with stock git over file://. Setting repack.useDeltaBaseOffset=false on the client stands in for a server that does not support ofs-delta.

----- 8< ----- #!/bin/sh # index-pack lazily fetches REF_DELTA bases that are inside the pack it # is indexing (partial clone + thin pack + REF_DELTA for in-pack bases). set -e D=$(mktemp -d) cd "$D"

commit () { git add . git -c user.name=t -c user.email=t@example.com commit -q -m "$1" }

git init -q server ( cd server # One wide tree, so that later versions of it delta well. for i in $(seq 1 300) do echo "base $i" >file$i done commit base git config uploadpack.allowFilter true git config uploadpack.allowAnySHA1InWant true )

git clone -q --no-checkout --filter=blob:none "file://$D/server" client

# Do not ask for ofs-delta, so every delta arrives as REF_DELTA. This is # what a client gets from a server that does not support ofs-delta. git -C client config repack.useDeltaBaseOffset false

old=$(git -C server rev-parse HEAD) ( cd server # Each new commit changes one file, i.e. one entry of the tree. for n in 1 2 3 4 5 6 7 8 do echo "new $n" >>file$n commit "new $n" done ) new=$(git -C server rev-parse HEAD)

GIT_TRACE2_EVENT="$D/trace2" GIT_TRACE_PACKET="$D/packet" \ git -C client fetch -q origin

echo "git version: $(git version)" echo "processes spawned below 'git fetch':" sed -n 's/.*"event":"cmd_name".*"hierarchy":"\([^"]*\)".*/\1/p' "$D/trace2" | sort | uniq -c

git -C server rev-list --objects --filter=blob:none "$old..$new" | cut -c1-40 | sort >"$D/in-pack" sed -n 's/.*fetch> want \([0-9a-f]\{40\}\).*/\1/p' "$D/packet" | grep -v "$new" | sort -u >"$D/lazy-wants"

echo "objects requested by the nested lazy fetch: $(wc -l <"$D/lazy-wants")" echo "of those, already in the pack being indexed: $(comm -12 "$D/lazy-wants" "$D/in-pack" | wc -l)" rm -rf "$D" ----- >8 -----

What did you expect to happen? (Expected behavior)

A single fetch. Every delta base is either in the received pack or already in the local object store, so nothing has to be fetched lazily.

What happened instead? (Actual behavior)

index-pack starts a nested lazy fetch for delta bases that are in the pack it is indexing. Output of the script with 2.56.0:

git version: git version 2.56.0 processes spawned below 'git fetch': 1 fetch 1 fetch/index-pack 1 fetch/index-pack/fetch 1 fetch/index-pack/fetch/index-pack 1 fetch/index-pack/fetch/maintenance 1 fetch/index-pack/fetch/upload-pack 1 fetch/index-pack/fetch/upload-pack/pack-objects 1 fetch/maintenance 1 fetch/upload-pack 1 fetch/upload-pack/pack-objects objects requested by the nested lazy fetch: 6 of those, already in the pack being indexed: 6

With the repack.useDeltaBaseOffset line removed (ofs-delta negotiated) there is no nested fetch and both counts are 0.

What's different between what you expected and what actually happened?

A redundant fetch is started from inside index-pack. In the script it costs one extra round trip. Against a real server it turned into unbounded recursion, described below.

Anything else you want to add:

Where it comes from, as far as I can tell: fix_unresolved_deltas() in builtin/index-pack.c, which is reached only when the pack is thin. In a repository with a promisor remote it first prefetches delta bases:

for (i = 0; i < nr_ref_deltas; i++) { struct ref_delta_entry *d = sorted_by_pos[i]; if (!odb_read_object_info_extended(the_repository->objects, &d->oid, NULL, OBJECT_INFO_FOR_PREFETCH)) continue; oid_array_append(&to_fetch, &d->oid); } promisor_remote_get_direct(the_repository, to_fetch.oid, to_fetch.nr);

This loop goes over all REF_DELTA entries, including the ones that were already resolved against a base inside the pack. The loop right after it skips those (real_type != OBJ_REF_DELTA), but the prefetch loop does not. The objects of the incoming pack are not in the object store yet, so their bases look missing and are requested from the promisor remote.

Skipping the already resolved entries would not cover everything: a delta chained onto another in-pack delta whose own base is outside the pack stays unresolved at this point, and its base has no known object ID yet. Resolving with the locally available bases first, and only then prefetching the bases of what is still unresolved, would avoid the request in that case too. I have not written or tested a patch.

How I ran into it: clients 2.50.1.windows.1 and 2.53.0 (Linux) against Azure DevOps Services, which did not advertise ofs-delta in the packet trace. The flow was:

git clone --no-checkout --filter=blob:none --single-branch \ --depth 1 <url> . git checkout <commit that is not in the clone>

The checkout lazily fetched the commit. The server answered with a thin pack of about 44,000 objects (commits and trees). Its out-of-pack bases were trees of the shallow commit, which the client had. index-pack then lazily requested about 9,700 objects, all of them in that pack. The server answered that request with another thin pack holding the same objects, the nested index-pack did the same again, and so on. Every level kept its fetch, remote-https, fetch-pack and index-pack processes alive and downloaded about 14 MB again. I saw 12 levels in about a minute before I interrupted it, and several hundred levels in an unattended job that ended only when it hit its memory limit.

4360734e90 (promisor-remote: prevent infinite recursion when lazy fetching, 2026-10-02) on master caps the nesting at 5, which bounds this, but the redundant requests are still made. The prefetch block above is the same in v2.56.1, master, next and seen as of 2026-11-15.

Fetching the commit explicitly before the checkout avoided it for me:

GIT_NO_LAZY_FETCH=1 git fetch --depth 1 origin <commit> git checkout <commit>

Please review the rest of the bug report below. You can delete any lines you don't wish to share.

[System Info]
git version:
git version 2.56.0
cpu: arm64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
rust: disabled
feature: fsmonitor--daemon
gettext: enabled
libcurl: 8.23.0
zlib: 1.2.12
SHA-1: SHA1_DC
SHA-256: SHA256_BLK
default-ref-format: files
default-hash: sha1
uname: Darwin 25.6.0 Darwin Kernel Version 25.6.0: Mon Oct 12 00:41:41
PDT 2026; root:xnu-12377.161.15.701.48~2/RELEASE_ARM64_T6050 arm64
compiler info: clang: 21.0.0 (clang-2100.3.34.2)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/bash

← back to recent threads