{"thread":{"id":"66291","subject":"[BUG] git bundle create with bitmaps omits tree required by advertised ref","startedAt":"2026-09-08T08:23:50Z","lastAt":"2026-09-08T22:33:27Z","messageCount":2,"participants":["Peter Elmers","Taylor Blau"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"552188","messageId":"CALY5j-0K-LfowAavH8X3UfZ24eAsoX=xew=KTt=4uCfZrdwXQw@mail.gmail.com","threadId":"66291","inReplyTo":null,"subject":"[BUG] git bundle create with bitmaps omits tree required by advertised ref","fromName":"Peter Elmers","fromEmail":"peter.elmers@databricks.com","sentAt":"2026-09-08T08:23:37Z","receivedAt":"2026-09-08T08:23:50Z","isPatch":false,"body":"What did you do before the bug happened? (Steps to reproduce your issue)\n\nI created two sibling commits with the same root tree, stored one at a local\nbranch and the other at a remote-tracking ref, wrote a pack bitmap, and created\na bundle that included the local branch while excluding the remote-tracking\nref.\n\nAdding `-c pack.useBitmaps=false` appears to fix the issue.\n\nThe following script reproduces the issue:\n\n#!/bin/sh\nset -eu\n\nrepro_dir=$(mktemp -d)\nsource_repo=\"$repro_dir/source.git\"\nrecipient_repo=\"$repro_dir/recipient.git\"\n\ngit init -q --bare -b main \"$source_repo\"\ngit -C \"$source_repo\" config user.name A\ngit -C \"$source_repo\" config user.email a@example.com\n\nempty_tree=$(git -C \"$source_repo\" mktree </dev/null)\nbase=$(printf 'base\\n' | git -C \"$source_repo\" commit-tree \"$empty_tree\")\ngit -C \"$source_repo\" update-ref refs/heads/main \"$base\"\ngit clone -q --bare \"$source_repo\" \"$recipient_repo\"\n\nblob=$(printf 'change\\n' | git -C \"$source_repo\" hash-object -w --stdin)\nshared_tree=$(printf '100644 blob %s\\tfile\\n' \"$blob\" |\n  git -C \"$source_repo\" mktree)\nremote_tip=$(printf 'original\\n' |\n  git -C \"$source_repo\" commit-tree \"$shared_tree\" -p \"$base\")\nlocal_tip=$(printf 'rewritten\\n' |\n  git -C \"$source_repo\" commit-tree \"$shared_tree\" -p \"$base\")\n\ngit -C \"$source_repo\" update-ref refs/heads/feature \"$local_tip\"\ngit -C \"$source_repo\" update-ref refs/remotes/origin/feature \"$remote_tip\"\ngit -C \"$source_repo\" repack -q -ad --write-bitmap-index\ngit -C \"$source_repo\" bundle create \"$repro_dir/broken.bundle\" \\\n  feature ^refs/remotes/origin/feature\n\ngit -C \"$recipient_repo\" bundle verify \"$repro_dir/broken.bundle\"\ngit -C \"$recipient_repo\" bundle unbundle \"$repro_dir/broken.bundle\"\ngit -C \"$recipient_repo\" cat-file -e \"$local_tip^{tree}\"\n\nThe resulting commit and object topology is:\n\n                         refs/remotes/origin/feature (excluded)\n                                         |\n                                     remote_tip\n                                    /          \\\n                            parent /            \\ tree\n                                  /              \\\n                              base                shared_tree\n                                  \\              /\n                            parent \\            / tree\n                                    \\          /\n                                     local_tip\n                                         |\n                              refs/heads/feature (advertised)\n\nBoth tip commits have `base` as their parent and `shared_tree` as their root\ntree. The bundle advertises `local_tip` and excludes `remote_tip`.\n\nWhat did you expect to happen? (Expected behavior)\n\nAfter `git bundle verify` confirms that the recipient has every declared\nprerequisite and `git bundle unbundle` succeeds, every object required by the\nadvertised `feature` commit should be available. The final `git cat-file`\ncommand should exit successfully.\n\nWhat happened instead? (Actual behavior)\n\n`git bundle verify` reports that the bundle is okay and `git bundle unbundle`\nsucceeds, but the final command (cat-file) exits 128:\n\nfatal: Not a valid object name <local-tip>^{tree}\n\nThe advertised local commit object is present, but its root tree is absent.\n\nWhat's different between what you expected and what actually happened?\n\nThe bitmap-backed bundle omits an object required by an advertised ref without\ndeclaring the excluded sibling commit as a prerequisite. A recipient containing\nall declared prerequisites can therefore accept the bundle but cannot traverse\nor check out the advertised commit.\n\nAnything else you want to add:\n\nAdding `-c pack.useBitmaps=false` to `git bundle create` makes the final\n`git cat-file` command succeed.\n\n`pack.useBitmapBoundaryTraversal=true` did not prevent the omission in a\nseparate run of the same commit topology.\n\nSo it looks like a bitmap format bug.\n\nRelevant documentation:\n\nhttps://git-scm.com/docs/git-config#Documentation/git-config.txt-packuseBitmaps\nhttps://git-scm.com/docs/git-config#Documentation/git-config.txt-packuseBitmapBoundaryTraversal\nhttps://git-scm.com/docs/git-bundle#_object_prerequisites\n\n[System Info]\ngit version:\ngit version 2.54.0\ncpu: x86_64\nno commit associated with this build\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nrust: disabled\nSHA-1: SHA1_DC\nSHA-256: SHA256_BLK\ndefault-ref-format: files\ndefault-hash: sha1\nuname: Linux 7.0.0-1012-aws #12~24.04.1-Ubuntu SMP PREEMPT Wed Aug 12\n14:00:57 UTC 2026 x86_64\ncompiler info: gnuc: 9.4\nlibc info: glibc: 2.39\n$SHELL (typically, interactive shell): /usr/bin/zsh\n"},{"id":"552270","messageId":"aqCNL8y9wLLP4ovm@com-79390","threadId":"66291","inReplyTo":"CALY5j-0K-LfowAavH8X3UfZ24eAsoX=xew=KTt=4uCfZrdwXQw@mail.gmail.com","subject":"Re: [BUG] git bundle create with bitmaps omits tree required by advertised ref","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-09-08T22:33:19Z","receivedAt":"2026-09-08T22:33:27Z","isPatch":false,"body":"On Tue, Sep 08, 2026 at 10:23:37AM +0200, Peter Elmers wrote:\n> What did you do before the bug happened? (Steps to reproduce your issue)\n>\n> I created two sibling commits with the same root tree, stored one at a local\n> branch and the other at a remote-tracking ref, wrote a pack bitmap, and created\n> a bundle that included the local branch while excluding the remote-tracking\n> ref.\n>\n> Adding `-c pack.useBitmaps=false` appears to fix the issue.\n>\n> The following script reproduces the issue:\n\nInteresting. I reproduced what you wrote here using your script. I\nhighly suspect what's going on here is that the non-bitmap case\novercounts some objects beyond the boundary whereas the bitmap case\nbuilds an exact answer.\n\nThe non-bitmap traversal only marked boundary trees UNINTERESTING in\nthis case, so it happened to keep the shared tree.\n\nWe can fix this with something like the following (only lightly tested)\npatch, but it has some test fallout for cases where we generate bundles\nwith only tags (+CC Peff who may have some opinions).\n\n--- 8< ---\n\nSubject: [PATCH] bundle: restrict pack haves to recorded prerequisites\n\n`write_pack_data()` uses every UNINTERESTING tip as a pack have, but the\nbundle header records only boundary commits. If an included commit\nshares a tree with an excluded sibling, a bitmap walk can omit that tree\nwhile the header requires only their parent. A recipient with the parent\n  can verify and unbundle the result while lacking the advertised\n  commit's tree.\n\nRestrict pack haves to UNINTERESTING commits marked BOUNDARY, which\nare recorded as prerequisites. Other excluded tips must not suppress\nobjects that the advertised refs need.\n\nAs a consequence, there is a bit of test fallout in t6020.13. That test\nincludes the tags while excluding all branch tips. Its header records\nno prerequisites, but its pack previously contained only the three\ntag objects: the excluded tips caused their target histories to be\nomitted.\n\nBut that bundle was already incomplete! With no prerequisites, it must\nprovide the history reachable from its advertised tags. Restricting\npack haves to recorded prerequisites includes that history, increasing\nthe expected object count as below.\n\nReported-by: Peter Elmers <peter.elmers@databricks.com>\nSigned-off-by: Taylor Blau <ttaylorr@openai.com>\n---\n bundle.c               |  7 +++++--\n t/t6020-bundle-misc.sh | 14 ++++++++++++--\n 2 files changed, 17 insertions(+), 4 deletions(-)\n\ndiff --git a/bundle.c b/bundle.c\nindex f55a521b2a1..06a52b8e705 100644\n--- a/bundle.c\n+++ b/bundle.c\n@@ -359,10 +359,13 @@ static int write_pack_data(int bundle_fd, struct rev_info *revs, int progress)\n\n \tfor (i = 0; i < revs->pending.nr; i++) {\n \t\tstruct object *object = revs->pending.objects[i].item;\n-\t\tif (object->flags & UNINTERESTING)\n+\t\tif (object->flags & UNINTERESTING) {\n+\t\t\tif (!(object->flags & BOUNDARY))\n+\t\t\t\tcontinue; /* Not a bundle prerequisite. */\n \t\t\toid_array_append(&opts.haves, &object->oid);\n-\t\telse\n+\t\t} else {\n \t\t\toid_array_append(&opts.wants, &object->oid);\n+\t\t}\n \t}\n\n \tif (odb_generate_pack(revs->repo->objects, &generator, &opts)) {\ndiff --git a/t/t6020-bundle-misc.sh b/t/t6020-bundle-misc.sh\nindex 939d4214f4d..6fce6252a11 100755\n--- a/t/t6020-bundle-misc.sh\n+++ b/t/t6020-bundle-misc.sh\n@@ -446,8 +446,9 @@ test_expect_success 'create bundle 4 - with tags' '\n \t\tmake_user_friendly_and_stable_output >actual &&\n \ttest_cmp expect actual &&\n\n-\ttest_bundle_object_count       4.bdl 3 &&\n-\ttest_bundle_object_count stdin-4.bdl 3\n+\t# With no prerequisites, include the tag targets and their history.\n+\ttest_bundle_object_count       4.bdl 40 &&\n+\ttest_bundle_object_count stdin-4.bdl 40\n '\n\n test_expect_success 'clone from bundle' '\n@@ -784,4 +785,13 @@ do\n \t'\n done\n\n+test_expect_success 'bundle with bitmaps includes trees shared with an excluded sibling' '\n+\tcommit=$(git commit-tree main^{tree} -p main^ -m rewritten) &&\n+\tgit branch rewritten \"$commit\" &&\n+\ttest_when_finished \"git branch -D rewritten\" &&\n+\tgit repack -adb &&\n+\tgit -c pack.useBitmaps=true bundle create bitmap.bdl main..rewritten &&\n+\ttest_bundle_object_count bitmap.bdl 3\n+'\n+\n test_done\n\nbase-commit: b8242b093d9e941a34460d715e3ce616a34ac3fe\n--\n2.55.0.openai.744.g47c847ce2641\n\n"}]}