{"thread":{"id":"66653","subject":"[BUG] 2.56.0: filtered --unshallow of a moved ref aborts in pack-objects (should_include_obj) via inherited GIT_SHALLOW_FILE","startedAt":"2026-10-24T17:59:55Z","lastAt":"2026-10-27T07:03:05Z","messageCount":3,"participants":["Daniel Zoghalchali","Patrick Steinhardt"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"556180","messageId":"CAJwQ=KQynymg4CJOLkyebACYVQa+LhXB2UifniTASkLyArWejA@mail.gmail.com","threadId":"66653","inReplyTo":null,"subject":"[BUG] 2.56.0: filtered --unshallow of a moved ref aborts in pack-objects (should_include_obj) via inherited GIT_SHALLOW_FILE","fromName":"Daniel Zoghalchali","fromEmail":"daniel@latchkey.dev","sentAt":"2026-10-24T17:59:55Z","receivedAt":"2026-10-24T17:59:55Z","isPatch":false,"sender":{"key":"daniel@latchkey.dev","avatar":null},"body":"Hi,\n\nSince v2.56.0, a fetch that combines a tree filter with --unshallow\n(or --deepen) aborts in a non-partial shallow clone when the fetched\nref has moved past the local tip:\n\n  BUG: builtin/pack-objects.c:5098: should_include_obj should only be\ncalled on existing objects\n  error: pack-objects died of signal 6\n  fatal: could not finish pack-objects to repack local links\n  fatal: index-pack failed\n\nv2.55.0 completes the same fetch. We hit it in CI, where a job checks\nout a single commit at depth 1 and then unshallows its branch, which\nby then has moved on.\n\nReproduction (offline, any Linux):\n\n  set -e\n  W=$(mktemp -d); cd \"$W\"\n  git init -q -b dev seed\n  for i in 1 2 3 4; do\n    mkdir -p seed/src; echo \"$i\" > seed/src/f.txt\n    git -C seed add -A\n    git -C seed -c user.name=t -c user.email=t@t \\\n      commit -q -m \"c$i\"\n  done\n  git init -q --bare o.git\n  git -C seed push -q \"$W/o.git\" dev\n  git -C o.git config uploadpack.allowFilter true\n  git -C o.git config uploadpack.allowAnySHA1InWant true\n  git init -q co\n  git -C co remote add origin \"file://$W/o.git\"\n  TIP1=$(git -C seed rev-parse dev~1)\n  git -C co fetch -q --no-tags --depth=1 origin \\\n    \"+$TIP1:refs/remotes/origin/dev\"\n  git -C co checkout -q -B dev refs/remotes/origin/dev\n  git -C co fetch --no-tags --filter=tree:0 --unshallow origin \\\n    +refs/heads/dev:refs/remotes/origin/dev\n\nThe last command dies as above with 2.56.0, both Ubuntu 24.04's\ngit-core PPA build (1:2.56.0-0ppa1~ubuntu24.04.2) and v2.56.0 built\nfrom source. It succeeds with v2.55.0 built from source. With 2.56.0\nit also succeeds when dev has not moved past the checkout, with\n--filter=blob:none, or without a filter. --deepen=N with the same\nfilter aborts the same way.\n\nWhat happens:\n- fetch-pack runs \"git --shallow-file '' index-pack --promisor ...\".\n- The pack names the local tip as the parent of the new dev commit, so\n  index-pack's repack_local_links() starts \"git pack-objects\n  --exclude-promisor-objects-best-effort\".\n- Since d43b9acb05 (setup: unify setup of shallow file), --shallow-file\n  is applied by exporting GIT_SHALLOW_FILE, so that grandchild inherits\n  the empty shallow file.\n- The grandchild no longer treats the local tip as a shallow boundary.\n  It walks to the tip's parent, a promisor commit fetched without trees,\n  and is_not_in_promisor_pack_obj() hits the BUG on that parent's\n  missing root tree.\n- Before d43b9acb05, the grandchild read $GIT_DIR/shallow and stopped at\n  the tip.\n\nTo confirm this, we wrapped only the repack child. Unsetting\nGIT_SHALLOW_FILE for it makes 2.56.0 pass, and setting\nGIT_SHALLOW_FILE= for it makes 2.55.0 fail the same way, at its\nbuiltin/pack-objects.c:5004.\n\nSeparately, the BUG() is reachable without the environment change. On\n2.55.0, a filtered --unshallow of the local tip, followed by a\nfiltered fetch of a moved ref, also dies there, because the walk from\nthe local tip reaches promisor commits whose trees were never fetched.\n\nThanks,\nDaniel Zoghalchali\n\n"},{"id":"556340","messageId":"at9P0ctULX1BBD8y@pks.im","threadId":"66653","inReplyTo":"CAJwQ=KQynymg4CJOLkyebACYVQa+LhXB2UifniTASkLyArWejA@mail.gmail.com","subject":"Re: [BUG] 2.56.0: filtered --unshallow of a moved ref aborts in pack-objects (should_include_obj) via inherited GIT_SHALLOW_FILE","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-26T13:04:30Z","receivedAt":"2026-10-26T13:04:30Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nOn Sat, Oct 24, 2026 at 10:59:55AM -0700, Daniel Zoghalchali wrote:\n> Hi,\n> \n> Since v2.56.0, a fetch that combines a tree filter with --unshallow\n> (or --deepen) aborts in a non-partial shallow clone when the fetched\n> ref has moved past the local tip:\n> \n>   BUG: builtin/pack-objects.c:5098: should_include_obj should only be\n> called on existing objects\n>   error: pack-objects died of signal 6\n>   fatal: could not finish pack-objects to repack local links\n>   fatal: index-pack failed\n> \n> v2.55.0 completes the same fetch. We hit it in CI, where a job checks\n> out a single commit at depth 1 and then unshallows its branch, which\n> by then has moved on.\n> \n> Reproduction (offline, any Linux):\n> \n>   set -e\n>   W=$(mktemp -d); cd \"$W\"\n>   git init -q -b dev seed\n>   for i in 1 2 3 4; do\n>     mkdir -p seed/src; echo \"$i\" > seed/src/f.txt\n>     git -C seed add -A\n>     git -C seed -c user.name=t -c user.email=t@t \\\n>       commit -q -m \"c$i\"\n>   done\n>   git init -q --bare o.git\n>   git -C seed push -q \"$W/o.git\" dev\n>   git -C o.git config uploadpack.allowFilter true\n>   git -C o.git config uploadpack.allowAnySHA1InWant true\n>   git init -q co\n>   git -C co remote add origin \"file://$W/o.git\"\n>   TIP1=$(git -C seed rev-parse dev~1)\n>   git -C co fetch -q --no-tags --depth=1 origin \\\n>     \"+$TIP1:refs/remotes/origin/dev\"\n>   git -C co checkout -q -B dev refs/remotes/origin/dev\n>   git -C co fetch --no-tags --filter=tree:0 --unshallow origin \\\n>     +refs/heads/dev:refs/remotes/origin/dev\n\nThanks, this reproducer works for me.\n\n> The last command dies as above with 2.56.0, both Ubuntu 24.04's\n> git-core PPA build (1:2.56.0-0ppa1~ubuntu24.04.2) and v2.56.0 built\n> from source. It succeeds with v2.55.0 built from source. With 2.56.0\n> it also succeeds when dev has not moved past the checkout, with\n> --filter=blob:none, or without a filter. --deepen=N with the same\n> filter aborts the same way.\n> \n> What happens:\n> - fetch-pack runs \"git --shallow-file '' index-pack --promisor ...\".\n> - The pack names the local tip as the parent of the new dev commit, so\n>   index-pack's repack_local_links() starts \"git pack-objects\n>   --exclude-promisor-objects-best-effort\".\n> - Since d43b9acb05 (setup: unify setup of shallow file), --shallow-file\n>   is applied by exporting GIT_SHALLOW_FILE, so that grandchild inherits\n>   the empty shallow file.\n> - The grandchild no longer treats the local tip as a shallow boundary.\n>   It walks to the tip's parent, a promisor commit fetched without trees,\n>   and is_not_in_promisor_pack_obj() hits the BUG on that parent's\n>   missing root tree.\n> - Before d43b9acb05, the grandchild read $GIT_DIR/shallow and stopped at\n>   the tip.\n> \n> To confirm this, we wrapped only the repack child. Unsetting\n> GIT_SHALLOW_FILE for it makes 2.56.0 pass, and setting\n> GIT_SHALLOW_FILE= for it makes 2.55.0 fail the same way, at its\n> builtin/pack-objects.c:5004.\n\nMakes sense. Even though I think that the bug isn't actually that we\nstarted propagating GIT_SHALLOW_FILE, as we now basically behave like we\ndo it with all other global variables, too. And if we were to execute a\nnon-local command like git-receive-pack(1) we already know to unset that\nvalue again via `local_repo_env`.\n\nWhat's fishy is \"builtin/pack-objects.c:is_not_in_promisor_pack_obj()\",\nwhich explicitly checks whether a given object is in a promisor or not.\nBut in the case where we assume that an object may be a promisor object\nwe know that we're running in a repostitory that may not even have all\nof its objects in the first place -- that's the whole ponit of a partial\nclone after all. But when `odb_read_object_info_ext()` returns an error\nwe simply `BUG()` right away, which does not make any sense to me.\n\n> Separately, the BUG() is reachable without the environment change. On\n> 2.55.0, a filtered --unshallow of the local tip, followed by a\n> filtered fetch of a moved ref, also dies there, because the walk from\n> the local tip reaches promisor commits whose trees were never fetched.\n\nYeah, this bug has the same root cause and I was able to reproduce it,\ntoo. And indeed it confirms my above take as it already exists before\nthe change to how we propagate GIT_SHALLOW_FILE. So overall, I think the\nfix is as simple as this:\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ca3a891dfbc..e634173fb1f 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -5085,7 +5085,7 @@ static int is_not_in_promisor_pack_obj(struct object *obj, void *data UNUSED)\n        };\n\n        if (odb_read_object_info_extended(the_repository->objects, &obj->oid, &info, 0))\n-               BUG(\"should_include_obj should only be called on existing objects\");\n+               return 0;\n        return source_info.source->type != ODB_SOURCE_PACKED || !source_info.u.packed.pack->pack_promisor;\n }\n\nI'll think a bit more about this and send a patch tomorrow. Thanks!\n\nPatrick\n\n"},{"id":"556437","messageId":"auBMqRatQLqerRte@pks.im","threadId":"66653","inReplyTo":"at9P0ctULX1BBD8y@pks.im","subject":"Re: [BUG] 2.56.0: filtered --unshallow of a moved ref aborts in pack-objects (should_include_obj) via inherited GIT_SHALLOW_FILE","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-27T07:03:05Z","receivedAt":"2026-10-27T07:03:05Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 26, 2026 at 02:04:30PM +0100, Patrick Steinhardt wrote:\n> On Sat, Oct 24, 2026 at 10:59:55AM -0700, Daniel Zoghalchali wrote:\n> > Hi,\n> > \n> > Since v2.56.0, a fetch that combines a tree filter with --unshallow\n> > (or --deepen) aborts in a non-partial shallow clone when the fetched\n> > ref has moved past the local tip:\n> > \n> >   BUG: builtin/pack-objects.c:5098: should_include_obj should only be\n> > called on existing objects\n> >   error: pack-objects died of signal 6\n> >   fatal: could not finish pack-objects to repack local links\n> >   fatal: index-pack failed\n> > \n> > v2.55.0 completes the same fetch. We hit it in CI, where a job checks\n> > out a single commit at depth 1 and then unshallows its branch, which\n> > by then has moved on.\n> > \n> > Reproduction (offline, any Linux):\n> > \n> >   set -e\n> >   W=$(mktemp -d); cd \"$W\"\n> >   git init -q -b dev seed\n> >   for i in 1 2 3 4; do\n> >     mkdir -p seed/src; echo \"$i\" > seed/src/f.txt\n> >     git -C seed add -A\n> >     git -C seed -c user.name=t -c user.email=t@t \\\n> >       commit -q -m \"c$i\"\n> >   done\n> >   git init -q --bare o.git\n> >   git -C seed push -q \"$W/o.git\" dev\n> >   git -C o.git config uploadpack.allowFilter true\n> >   git -C o.git config uploadpack.allowAnySHA1InWant true\n> >   git init -q co\n> >   git -C co remote add origin \"file://$W/o.git\"\n> >   TIP1=$(git -C seed rev-parse dev~1)\n> >   git -C co fetch -q --no-tags --depth=1 origin \\\n> >     \"+$TIP1:refs/remotes/origin/dev\"\n> >   git -C co checkout -q -B dev refs/remotes/origin/dev\n> >   git -C co fetch --no-tags --filter=tree:0 --unshallow origin \\\n> >     +refs/heads/dev:refs/remotes/origin/dev\n> \n> Thanks, this reproducer works for me.\n> \n> > The last command dies as above with 2.56.0, both Ubuntu 24.04's\n> > git-core PPA build (1:2.56.0-0ppa1~ubuntu24.04.2) and v2.56.0 built\n> > from source. It succeeds with v2.55.0 built from source. With 2.56.0\n> > it also succeeds when dev has not moved past the checkout, with\n> > --filter=blob:none, or without a filter. --deepen=N with the same\n> > filter aborts the same way.\n> > \n> > What happens:\n> > - fetch-pack runs \"git --shallow-file '' index-pack --promisor ...\".\n> > - The pack names the local tip as the parent of the new dev commit, so\n> >   index-pack's repack_local_links() starts \"git pack-objects\n> >   --exclude-promisor-objects-best-effort\".\n> > - Since d43b9acb05 (setup: unify setup of shallow file), --shallow-file\n> >   is applied by exporting GIT_SHALLOW_FILE, so that grandchild inherits\n> >   the empty shallow file.\n> > - The grandchild no longer treats the local tip as a shallow boundary.\n> >   It walks to the tip's parent, a promisor commit fetched without trees,\n> >   and is_not_in_promisor_pack_obj() hits the BUG on that parent's\n> >   missing root tree.\n> > - Before d43b9acb05, the grandchild read $GIT_DIR/shallow and stopped at\n> >   the tip.\n> > \n> > To confirm this, we wrapped only the repack child. Unsetting\n> > GIT_SHALLOW_FILE for it makes 2.56.0 pass, and setting\n> > GIT_SHALLOW_FILE= for it makes 2.55.0 fail the same way, at its\n> > builtin/pack-objects.c:5004.\n> \n> Makes sense. Even though I think that the bug isn't actually that we\n> started propagating GIT_SHALLOW_FILE, as we now basically behave like we\n> do it with all other global variables, too. And if we were to execute a\n> non-local command like git-receive-pack(1) we already know to unset that\n> value again via `local_repo_env`.\n> \n> What's fishy is \"builtin/pack-objects.c:is_not_in_promisor_pack_obj()\",\n> which explicitly checks whether a given object is in a promisor or not.\n> But in the case where we assume that an object may be a promisor object\n> we know that we're running in a repostitory that may not even have all\n> of its objects in the first place -- that's the whole ponit of a partial\n> clone after all. But when `odb_read_object_info_ext()` returns an error\n> we simply `BUG()` right away, which does not make any sense to me.\n> \n> > Separately, the BUG() is reachable without the environment change. On\n> > 2.55.0, a filtered --unshallow of the local tip, followed by a\n> > filtered fetch of a moved ref, also dies there, because the walk from\n> > the local tip reaches promisor commits whose trees were never fetched.\n> \n> Yeah, this bug has the same root cause and I was able to reproduce it,\n> too. And indeed it confirms my above take as it already exists before\n> the change to how we propagate GIT_SHALLOW_FILE. So overall, I think the\n> fix is as simple as this:\n> \n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index ca3a891dfbc..e634173fb1f 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -5085,7 +5085,7 @@ static int is_not_in_promisor_pack_obj(struct object *obj, void *data UNUSED)\n>         };\n> \n>         if (odb_read_object_info_extended(the_repository->objects, &obj->oid, &info, 0))\n> -               BUG(\"should_include_obj should only be called on existing objects\");\n> +               return 0;\n>         return source_info.source->type != ODB_SOURCE_PACKED || !source_info.u.packed.pack->pack_promisor;\n>  }\n> \n> I'll think a bit more about this and send a patch tomorrow. Thanks!\n\nAnd here it is: <20261027-b4-pks-pack-objects-repack-links-with-missing-objects-v1-1-f50f90651274@pks.im>\n\nPatrick\n\n"}]}