{"thread":{"id":"66667","subject":"[PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing","startedAt":"2026-10-26T15:46:02Z","lastAt":"2026-11-04T17:13:55Z","messageCount":7,"participants":["Marc Strapetz via GitGitGadget","Junio C Hamano","Marc Strapetz"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"556360","messageId":"pull.2281.git.1793029565.gitgitgadget@gmail.com","threadId":"66667","inReplyTo":null,"subject":"[PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing","fromName":"Marc Strapetz via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-26T15:46:02Z","receivedAt":"2026-10-26T15:46:02Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"A common feature-branch workflow is to push a branch, rebase it onto an\nupdated upstream, and force-push the rewritten commits. Large blobs added or\nchanged on the feature branch may survive the rebase unchanged. Although the\nreceiver already has those blobs through the old branch tip, Git can pack\nand send them again. This can turn a small history rewrite into a large\nupload.\n\nThis series teaches send-pack to request inspection of the old tip's tree\nbefore packing, so more receiver-known objects can be excluded.\n\nThe additional inspection is enabled only for forced non-fast-forward\nupdates of existing branches with locally available old tips. Ordinary\nfast-forward pushes do not enable it, including ones made with --force. No\nserver or wire-protocol change is needed.\n\nThe old tip is already a negative revision, but it may lie outside the\nboundary visited by the ordinary revision walk. Its tree is then not\nnecessarily inspected. Passing it to pack-objects as --inspect-have lets us\ncompare its root tree with the outgoing trees using the existing sparse tree\nmarking. This excludes additional receiver-known objects without walking the\nold tip's commit history, and prunes identical and receiver-only subtrees.\n\nShallow pushes keep their existing aggressive edge handling. Partial clones\nskip the extra tree inspection to avoid fetching missing objects, and\nsuccessful bitmap enumeration keeps its existing path. Old tips are resolved\nin pack-objects, where replacement refs are already disabled.\n\nI compared upstream 7e34d8174dcda3435d1404c9a30bef46afe6c13e with the\npatched source using trees from Chromium commit\n5b05cc16e960a021389ef9b543ab38a037c38d60. A depth-two clone supplies the\nobjects; the measured fixture has a complete, non-shallow history with a\ncommon base and sibling receiver/outgoing commits.\n\nBoth builds used GCC 15.2.0 and -g -O2 -Wall on an Intel i7-13700H under\nWSL2. Measurements used a warm cache, sparse enumeration, no bitmaps, one\npacking thread, two warm-up pairs and 15 measured pairs with alternating\nbuild order. Values below are upstream/patched; total wall times are medians\nin milliseconds, including process startup.\n\nCase Objects Pack bytes Total wall ms Reword 3 / 1 421 / 215 2.776 / 1.338\nSingle file 6 / 5 667 / 553 3.542 / 2.744 ~99% of files 552,724 / 552,723\n58,082,988 / 58,082,873 4,525.094 / 4,883.623\n\nIn the single-file case, enumeration increases from 0.758 to 0.982 ms, but\npack preparation decreases from 1.637 to 0.458 ms. In the last case, 506,468\nregular files receive distinct small replacement blobs; total packing time\nincreases by about 8%, saving only 115 pack bytes.\n\nThese are isolated local pack-objects measurements, not end-to-end pushes:\nnetwork transfer and receiving-server work are excluded. Where redundant\nlarge blobs materially inflate the pack, avoiding their transfer should\nfurther reduce end-to-end push time, especially on slower connections; that\nbenefit was not measured here.\n\nMost objects in this fixture are already excluded through the common base,\nso the pack-size savings are small. The widespread-change case tests\ninspection overhead on Chromium's directory structure, rather than realistic\ncompression of a whole-project rewrite. Feedback on this cost and the scope\nof the inspection would be particularly useful.\n\nLocal verification passed t5322-pack-objects-sparse.sh (19 tests) and\nt5516-fetch-push.sh (132 tests). Coverage includes identical trees, shared\nblobs under different trees, retained changed objects, subtree pruning,\npartial clones and local replacement refs.\n\nMarc Strapetz (3):\n  t5516: record redundant blobs after rebasing\n  pack-objects: inspect selected haves for shared objects\n  send-pack: avoid resending unchanged blobs after rebasing\n\n Documentation/git-pack-objects.adoc |   7 ++\n builtin/pack-objects.c              |  81 ++++++++++++++-\n odb.c                               |   1 +\n odb.h                               |   8 ++\n odb/source-files.c                  |   3 +\n send-pack.c                         |  30 ++++--\n t/t5322-pack-objects-sparse.sh      | 148 ++++++++++++++++++++++++++++\n t/t5516-fetch-push.sh               |  43 ++++++++\n 8 files changed, 314 insertions(+), 7 deletions(-)\n\n\nbase-commit: 7e34d8174dcda3435d1404c9a30bef46afe6c13e\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2281%2Fmstrap%2Fsend-pack-inspect-haves-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2281/mstrap/send-pack-inspect-haves-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2281\n-- \ngitgitgadget\n\n"},{"id":"556361","messageId":"84157f543eebc2ed347dd2165ed6046305e9226b.1793029565.git.gitgitgadget@gmail.com","threadId":"66667","inReplyTo":"pull.2281.git.1793029565.gitgitgadget@gmail.com","subject":"[PATCH 1/3] t5516: record redundant blobs after rebasing","fromName":"Marc Strapetz via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-26T15:46:03Z","receivedAt":"2026-10-26T15:46:03Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"From: Marc Strapetz <marc.strapetz@syntevo.com>\n\nA forced push of a rebased branch can resend an unchanged blob even\nthough the receiver has it through the advertised old tip.\n\nAdd object-count tests for --force, --force-with-lease and forced\nrefspecs. Confirm that the blob is unchanged by the rebase and record\nthe current three-object pack: the rebased commit, its tree and the\nunchanged blob. Disable push bitmaps to exercise normal enumeration.\n\nAlso cover local replacement refs, which must not cause objects\nmissing from the receiver to be omitted.\n\nAssisted-by: GPT-6 Astra, GPT-6.1 Sol\nSigned-off-by: Marc Strapetz <marc.strapetz@syntevo.com>\n---\n t/t5516-fetch-push.sh | 43 +++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 43 insertions(+)\n\ndiff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh\nindex b982b209bf..a46ba91ba0 100755\n--- a/t/t5516-fetch-push.sh\n+++ b/t/t5516-fetch-push.sh\n@@ -193,6 +193,49 @@ grep_wrote () {\n \tgrep 'write_pack_file/wrote.*\"value\":\"'$1'\"' $2\n }\n \n+for push_args in \\\n+\t\"--force origin topic\" \\\n+\t\"--force-with-lease origin topic\" \\\n+\t\"origin +topic\"\n+do\n+\ttest_expect_success \"push of rebased branch with $push_args\" '\n+\t\tmk_test testrepo heads/main &&\n+\t\ttest_when_finished \"rm -rf rebased rebased-push-event\" &&\n+\t\tgit clone testrepo rebased &&\n+\t\t(\n+\t\t\tcd rebased &&\n+\t\t\tgit checkout -b topic &&\n+\t\t\ttest_commit feature &&\n+\t\t\tgit push origin topic &&\n+\t\t\tgit checkout main &&\n+\t\t\ttest_commit upstream path1 &&\n+\t\t\tgit push origin main &&\n+\t\t\tgit checkout topic &&\n+\t\t\tgit rebase main &&\n+\t\t\ttest_cmp_rev feature:feature.t HEAD:feature.t &&\n+\t\t\tGIT_TRACE2_EVENT=\"$D/rebased-push-event\" \\\n+\t\t\tgit -c push.useBitmaps=false push $push_args\n+\t\t) &&\n+\t\tgrep_wrote 3 rebased-push-event # 1 commit, 1 tree, 1 blob\n+\t'\n+done\n+\n+test_expect_success 'forced update ignores replacement for old tip' '\n+\tmk_test testrepo heads/main &&\n+\ttest_when_finished \"rm -rf replaced\" &&\n+\tgit clone testrepo replaced &&\n+\t(\n+\t\tcd replaced &&\n+\t\ttest_commit old &&\n+\t\tgit push origin main &&\n+\t\told_tip=$(git rev-parse HEAD) &&\n+\t\tgit reset --hard HEAD^ &&\n+\t\ttest_commit new &&\n+\t\tgit replace \"$old_tip\" HEAD &&\n+\t\tgit push --force origin main\n+\t)\n+'\n+\n test_expect_success 'push without negotiation' '\n \tmk_empty testrepo &&\n \tgit push testrepo $the_first_commit:refs/remotes/origin/first_commit &&\n-- \ngitgitgadget\n\n\n"},{"id":"556362","messageId":"02087fc65ca9e2cceb1b738d6dc150c44b10f48b.1793029565.git.gitgitgadget@gmail.com","threadId":"66667","inReplyTo":"pull.2281.git.1793029565.gitgitgadget@gmail.com","subject":"[PATCH 2/3] pack-objects: inspect selected haves for shared objects","fromName":"Marc Strapetz via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-26T15:46:04Z","receivedAt":"2026-10-26T15:46:04Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"From: Marc Strapetz <marc.strapetz@syntevo.com>\n\nA receiver-known commit can be excluded from a revision walk without\nbecoming a boundary commit. Its tree then need not be inspected, so\nobjects shared with the outgoing history can still be packed.\n\nAccept --inspect-have <commit> lines in --revs input and carry the\nselected tips through the object database pack-generation options.\nCompare their root trees with the outgoing tips using the existing\nsparse tree marking. This identifies additional receiver-known\nobjects without walking the selected commits' histories and avoids\ndescending into identical or receiver-only subtrees.\n\nResolve the selected commits in pack-objects, where replacement refs\nare already disabled, so local replacements cannot substitute trees\nthe receiver does not have. Keep successful bitmap enumeration on its\nexisting path and skip the optional inspection in partial clones.\n\nCover identical root trees, shared blobs under different roots,\nretained changed objects, subtree pruning, non-commit input, partial\nclone fallback and shared objects in earlier outgoing commits.\n\nAssisted-by: GPT-6 Astra, GPT-6.1 Sol\nSigned-off-by: Marc Strapetz <marc.strapetz@syntevo.com>\n---\n Documentation/git-pack-objects.adoc |   7 ++\n builtin/pack-objects.c              |  81 ++++++++++++++-\n odb.c                               |   1 +\n odb.h                               |   8 ++\n odb/source-files.c                  |   3 +\n t/t5322-pack-objects-sparse.sh      | 148 ++++++++++++++++++++++++++++\n 6 files changed, 247 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc\nindex ccad938c5b..a98d4286fa 100644\n--- a/Documentation/git-pack-objects.adoc\n+++ b/Documentation/git-pack-objects.adoc\n@@ -70,6 +70,13 @@ base-name::\n \toutputs.  The objects on the resulting list are packed.\n \tBesides revisions, `--not` or `--shallow <SHA-1>` lines are\n \talso accepted.\n++\n+A `--inspect-have <commit>` line supplies the object ID of a commit known\n+to the receiver. Before non-bitmap enumeration, its root tree is compared\n+with the outgoing tips' root trees to identify additional objects that\n+need not be packed, without walking its commit history. Identical\n+subtrees and paths found only in the receiver-known trees are skipped.\n+This optional inspection is skipped in partial clones.\n \n --unpacked::\n \tThis implies `--revs`.  When processing the list of\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ca3a891dfb..9abb4b9fe3 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -4892,12 +4892,76 @@ static int get_object_list_path_walk(struct rev_info *revs)\n \treturn result;\n }\n \n+static void add_have_to_inspect(struct oidset *trees, const char *arg)\n+{\n+\tstruct object_id oid;\n+\tstruct commit *commit;\n+\tstruct tree *tree;\n+\tconst char *end;\n+\n+\tif (parse_oid_hex(arg, &oid, &end) || *end)\n+\t\tdie(_(\"not an object name '%s'\"), arg);\n+\tif (!odb_has_object(the_repository->objects, &oid, 0))\n+\t\treturn;\n+\tcommit = lookup_commit_reference_gently(the_repository, &oid, 1);\n+\tif (!commit)\n+\t\tdie(_(\"object %s is not a commit\"), arg);\n+\ttree = repo_get_commit_tree(the_repository, commit);\n+\tif (tree)\n+\t\toidset_insert(trees, &tree->object.oid);\n+}\n+\n+static void inspect_haves(struct rev_info *revs, struct oidset *trees)\n+{\n+\tstruct oidset_iter iter;\n+\tstruct object_id *oid;\n+\n+\t/* Avoid fetching missing trees for this optional optimization. */\n+\tif (!oidset_size(trees) || repo_has_promisor_remote(revs->repo))\n+\t\treturn;\n+\n+\t/* Mark receiver-known root trees as uninteresting. */\n+\toidset_iter_init(trees, &iter);\n+\twhile ((oid = oidset_iter_next(&iter))) {\n+\t\tstruct tree *tree = lookup_tree(revs->repo, oid);\n+\n+\t\tif (tree)\n+\t\t\ttree->object.flags |= UNINTERESTING;\n+\t}\n+\n+\t/*\n+\t * Add outgoing root trees for the same sparse tree marking used by\n+\t * mark_edges_uninteresting(). A receiver-known tip on the other side of\n+\t * a divergence can be a negative revision without its tree being visited\n+\t * by the ordinary walk. Combining its tree with the outgoing trees lets\n+\t * us exclude shared objects without walking that tip's commit history.\n+\t */\n+\tfor (size_t i = 0; i < revs->pending.nr; i++) {\n+\t\tstruct object *obj = revs->pending.objects[i].item;\n+\n+\t\tif (obj->flags & UNINTERESTING)\n+\t\t\tcontinue;\n+\t\tif (obj->type == OBJ_COMMIT) {\n+\t\t\tstruct tree *tree = repo_get_commit_tree(revs->repo,\n+\t\t\t\t\t\t\t(struct commit *)obj);\n+\n+\t\t\tif (tree)\n+\t\t\t\toidset_insert(trees, &tree->object.oid);\n+\t\t} else if (obj->type == OBJ_TREE) {\n+\t\t\toidset_insert(trees, &obj->oid);\n+\t\t}\n+\t}\n+\n+\tmark_trees_uninteresting_sparse(revs->repo, trees);\n+}\n+\n static void get_object_list(struct rev_info *revs, struct strvec *argv)\n {\n \tstruct setup_revision_opt s_r_opt = {\n \t\t.allow_exclude_promisor_objects = 1,\n \t};\n \tstruct repo_config_values *cfg = repo_config_values(the_repository);\n+\tstruct oidset have_trees = OIDSET_INIT;\n \tchar line[1000];\n \tint flags = 0;\n \tint save_warning;\n@@ -4931,6 +4995,11 @@ static void get_object_list(struct rev_info *revs, struct strvec *argv)\n \t\t\t\tuse_bitmap_index = 0;\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (starts_with(line, \"--inspect-have \")) {\n+\t\t\t\tadd_have_to_inspect(&have_trees, line + 15);\n+\t\t\t\twrite_bitmap_index = 0;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tdie(_(\"not a rev '%s'\"), line);\n \t\t}\n \t\tif (handle_revision_arg(line, revs, flags, REVARG_CANNOT_BE_FILENAME))\n@@ -4939,8 +5008,18 @@ static void get_object_list(struct rev_info *revs, struct strvec *argv)\n \n \tcfg->warn_on_object_refname_ambiguity = save_warning;\n \n-\tif (use_bitmap_index && !get_object_list_from_bitmap(revs))\n+\tif (use_bitmap_index && !get_object_list_from_bitmap(revs)) {\n+\t\toidset_clear(&have_trees);\n \t\treturn;\n+\t}\n+\n+\t/*\n+\t * Limit this inspection to non-bitmap enumeration to avoid extra tree\n+\t * reads on the bitmap fast path. Boundary bitmap traversal may still\n+\t * include receiver-known objects; that tradeoff is intentional.\n+\t */\n+\tinspect_haves(revs, &have_trees);\n+\toidset_clear(&have_trees);\n \n \tif (use_delta_islands)\n \t\tload_delta_islands(the_repository, progress);\ndiff --git a/odb.c b/odb.c\nindex 98c5c862f5..2bb27848ac 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -731,6 +731,7 @@ void odb_generate_pack_options_release(struct odb_generate_pack_options *opts)\n {\n \toid_array_clear(&opts->wants);\n \toid_array_clear(&opts->haves);\n+\toid_array_clear(&opts->haves_to_inspect);\n \toid_array_clear(&opts->shallows);\n }\n \ndiff --git a/odb.h b/odb.h\nindex 0ccc47ee73..10629b2044 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -667,6 +667,13 @@ struct odb_generate_pack_options {\n \t */\n \tstruct oid_array haves;\n \n+\t/*\n+\t * Receiver-known commit tips whose root trees should be examined\n+\t * to identify additional objects that need not be packed. This\n+\t * optional inspection does not walk their commit histories.\n+\t */\n+\tstruct oid_array haves_to_inspect;\n+\n \t/*\n \t * The shallow boundary that shall be used when computing object\n \t * reachability. When set, any shallow information of the repository\n@@ -751,6 +758,7 @@ struct odb_generate_pack_options {\n #define ODB_GENERATE_PACK_OPTIONS_INIT { \\\n \t.wants = OID_ARRAY_INIT, \\\n \t.haves = OID_ARRAY_INIT, \\\n+\t.haves_to_inspect = OID_ARRAY_INIT, \\\n \t.shallows = OID_ARRAY_INIT, \\\n \t.pack_fd = -1, \\\n }\ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex 455f409db0..bad9ecde4c 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -1176,6 +1176,9 @@ static int odb_source_files_generate_pack(struct odb_source *source UNUSED,\n \tfprintf(in, \"--not\\n\");\n \tfor (size_t i = 0; i < opts->haves.nr; i++)\n \t\tfprintf(in, \"%s\\n\", oid_to_hex(&opts->haves.oid[i]));\n+\tfor (size_t i = 0; i < opts->haves_to_inspect.nr; i++)\n+\t\tfprintf(in, \"--inspect-have %s\\n\",\n+\t\t\toid_to_hex(&opts->haves_to_inspect.oid[i]));\n \tfprintf(in, \"\\n\");\n \tfflush(in);\n \tif (ferror(in)) {\ndiff --git a/t/t5322-pack-objects-sparse.sh b/t/t5322-pack-objects-sparse.sh\nindex d39958c066..0cf48513eb 100755\n--- a/t/t5322-pack-objects-sparse.sh\n+++ b/t/t5322-pack-objects-sparse.sh\n@@ -138,4 +138,152 @@ test_expect_success 'pack.useSparse overridden' '\n \ttest_cmp required_objects.txt sparse_objects.txt\n '\n \n+test_expect_success 'setup inspect-have receiver and outgoing commits' '\n+\tgit checkout -b receiver main &&\n+\techo large >large.bin &&\n+\techo receiver >f2/f2/data.txt &&\n+\tgit add large.bin f2/f2/data.txt &&\n+\ttest_commit --no-tag receiver file.txt &&\n+\tgit checkout -b outgoing main &&\n+\tgit checkout receiver -- large.bin f2 &&\n+\tgit rm -r f3 &&\n+\ttest_commit --no-tag outgoing file.txt &&\n+\treceiver_tip=$(git rev-parse receiver) &&\n+\tcat >packinput.txt <<-EOF\n+\toutgoing\n+\t^main\n+\t^receiver\n+\t--inspect-have $receiver_tip\n+\tEOF\n+'\n+\n+test_expect_success 'inspect-have excludes an identical root tree' '\n+\ttree=$(git rev-parse receiver^{tree}) &&\n+\tsame_tip=$(git commit-tree \"$tree\" -p main -m \"Same tree\") &&\n+\tcat >same-tree-input.txt <<-EOF &&\n+\t$same_tip\n+\t^main\n+\t^receiver\n+\t--inspect-have $receiver_tip\n+\tEOF\n+\techo \"$same_tip\" >expect_objects.txt &&\n+\tgit pack-objects --stdout --revs <same-tree-input.txt >inspected.pack &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp_sorted expect_objects.txt inspected_objects.txt\n+'\n+\n+test_expect_success 'inspect-have excludes shared large blob from different root trees' '\n+\ttest_cmp_rev ! receiver^{tree} outgoing^{tree} &&\n+\ttest_cmp_rev receiver:large.bin outgoing:large.bin &&\n+\tblob=$(git rev-parse receiver:large.bin) &&\n+\tsed \"/^--inspect-have /d\" packinput.txt >ordinary-input.txt &&\n+\tgit pack-objects --stdout --revs <ordinary-input.txt >ordinary.pack &&\n+\tgit index-pack -o ordinary.idx ordinary.pack &&\n+\tgit show-index <ordinary.idx >index &&\n+\tawk \"{print \\$2}\" index >ordinary_objects.txt &&\n+\ttest_grep \"$blob\" ordinary_objects.txt &&\n+\tgit rev-parse outgoing outgoing^{tree} outgoing:file.txt >expect_objects.txt &&\n+\tgit pack-objects --stdout --revs <packinput.txt >inspected.pack &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp_sorted expect_objects.txt inspected_objects.txt\n+'\n+\n+test_expect_success 'inspect-have keeps changed blobs and trees' '\n+\tgit checkout -b outgoing-changed outgoing &&\n+\ttest_commit changed f2/f2/data.txt &&\n+\tsed \"s/^outgoing$/outgoing-changed/\" packinput.txt >changed-input.txt &&\n+\tgit rev-parse \\\n+\t\toutgoing \\\n+\t\toutgoing^{tree} \\\n+\t\toutgoing:file.txt \\\n+\t\toutgoing-changed \\\n+\t\toutgoing-changed^{tree} \\\n+\t\toutgoing-changed:f2 \\\n+\t\toutgoing-changed:f2/f2 \\\n+\t\toutgoing-changed:f2/f2/data.txt >changed_objects.txt &&\n+\tgit pack-objects --stdout --revs <changed-input.txt >inspected.pack &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp_sorted changed_objects.txt inspected_objects.txt\n+'\n+\n+test_expect_success 'inspect-have skips unchanged and receiver-only subtrees' '\n+\tgit rev-parse receiver:f2 receiver:f3 >trees &&\n+\tgit pack-objects --stdout --window=0 <trees >trees.pack &&\n+\tgit index-pack --stdin <trees.pack &&\n+\twhile read oid\n+\tdo\n+\t\tpath=$(test_oid_to_path \"$oid\") &&\n+\t\trm \".git/objects/$path\" || return 1\n+\tdone <trees &&\n+\t>pack-access &&\n+\tGIT_TRACE_PACK_ACCESS=\"$(pwd)/pack-access\" \\\n+\tgit pack-objects --stdout --revs <packinput.txt >inspected.pack &&\n+\ttest_must_be_empty pack-access &&\n+\tgit rev-parse outgoing outgoing^{tree} outgoing:file.txt >expect_objects.txt &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp_sorted expect_objects.txt inspected_objects.txt\n+'\n+\n+test_expect_success 'inspect-have rejects non-commit objects' '\n+\tblob=$(git rev-parse receiver:f2/f2/data.txt) &&\n+\techo \"--inspect-have $blob\" >invalid-input.txt &&\n+\ttest_must_fail git pack-objects --stdout --revs \\\n+\t\t<invalid-input.txt >invalid.pack 2>err &&\n+\ttest_grep \"is not a commit\" err\n+'\n+\n+test_expect_success 'inspect-have is ignored in partial clones' '\n+\ttest_config remote.missing.url \"$(pwd)/missing\" &&\n+\ttest_config remote.missing.promisor true &&\n+\tsed \"/^--inspect-have /d\" packinput.txt >partial-input.txt &&\n+\tgit pack-objects --stdout --revs --thin --window=0 \\\n+\t\t<partial-input.txt >ordinary.pack 2>err &&\n+\ttest_must_be_empty err &&\n+\tgit pack-objects --stdout --revs --thin --window=0 \\\n+\t\t<packinput.txt >inspected.pack 2>err &&\n+\ttest_must_be_empty err &&\n+\tgit index-pack -o ordinary.idx ordinary.pack &&\n+\tgit show-index <ordinary.idx >index &&\n+\tawk \"{print \\$2}\" index >ordinary_objects.txt &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp ordinary_objects.txt inspected_objects.txt\n+'\n+\n+test_expect_success 'inspect-have excludes shared blobs from earlier outgoing commits' '\n+\tgit checkout -b receiver-extra receiver &&\n+\ttest_commit --no-tag extra f2/f2/small &&\n+\tgit checkout -b outgoing-extra outgoing &&\n+\ttest_commit --no-tag extra f2/f2/small &&\n+\treceiver_extra_tip=$(git rev-parse receiver-extra) &&\n+\tcat >tip-input.txt <<-EOF &&\n+\toutgoing-extra\n+\t^main\n+\t^receiver-extra\n+\t--inspect-have $receiver_extra_tip\n+\tEOF\n+\tgit rev-parse \\\n+\t\toutgoing \\\n+\t\toutgoing^{tree} \\\n+\t\toutgoing:f2 \\\n+\t\toutgoing:f2/f2 \\\n+\t\toutgoing:file.txt \\\n+\t\toutgoing-extra \\\n+\t\toutgoing-extra^{tree} >expect_objects.txt &&\n+\tgit pack-objects --stdout --revs <tip-input.txt >inspected.pack &&\n+\tgit index-pack -o inspected.idx inspected.pack &&\n+\tgit show-index <inspected.idx >index &&\n+\tawk \"{print \\$2}\" index >inspected_objects.txt &&\n+\ttest_cmp_sorted expect_objects.txt inspected_objects.txt\n+'\n+\n test_done\n-- \ngitgitgadget\n\n\n"},{"id":"556363","messageId":"e13f12640853643e14288107ee03f99fc00bb0f1.1793029565.git.gitgitgadget@gmail.com","threadId":"66667","inReplyTo":"pull.2281.git.1793029565.gitgitgadget@gmail.com","subject":"[PATCH 3/3] send-pack: avoid resending unchanged blobs after rebasing","fromName":"Marc Strapetz via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-10-26T15:46:05Z","receivedAt":"2026-10-26T15:46:05Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"From: Marc Strapetz <marc.strapetz@syntevo.com>\n\nRebasing a pushed branch can put its old tip outside the boundary\ntraversed by pack-objects. Even with that tip among the negative\nrevisions, unchanged blobs may then be sent again.\n\nFor forced non-fast-forward branch updates, pass locally available\nadvertised old tips as haves_to_inspect. Limit this to refs selected\nfor sending; leave fast-forward and rejected updates out. Restrict\nthe hints to branches, which name commits, and skip shallow\nrepositories, whose thin-pack pushes already inspect negative tips\nwith --objects-edge-aggressive.\n\nUpdate the rebase tests to expect two packed objects instead of\nthree: the rebased commit and its tree. The unchanged blob is\nalready present on the receiver.\n\nAssisted-by: GPT-6 Astra, GPT-6.1 Sol\nSigned-off-by: Marc Strapetz <marc.strapetz@syntevo.com>\n---\n send-pack.c           | 30 ++++++++++++++++++++++++------\n t/t5516-fetch-push.sh |  2 +-\n 2 files changed, 25 insertions(+), 7 deletions(-)\n\ndiff --git a/send-pack.c b/send-pack.c\nindex f20460fbf4..de6d33cf75 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -42,17 +42,18 @@ int option_parse_push_signed(const struct option *opt,\n \tdie(\"bad %s argument: %s\", opt->long_name, arg);\n }\n \n-static void append_negative_object(struct repository *r,\n-\t\t\t\t   struct oid_array *haves,\n-\t\t\t\t   const struct object_id *oid)\n+static int append_negative_object(struct repository *r,\n+\t\t\t\t  struct oid_array *haves,\n+\t\t\t\t  const struct object_id *oid)\n {\n \t/*\n \t * The remote end may have advertised objects that we do not have in\n \t * our object database. Skip those, as we cannot use them as boundary.\n \t */\n \tif (!odb_has_object(r->objects, oid, 0))\n-\t\treturn;\n+\t\treturn 0;\n \toid_array_append(haves, oid);\n+\treturn 1;\n }\n \n /*\n@@ -89,8 +90,25 @@ static int pack_objects(struct repository *r,\n \t\tappend_negative_object(r, &opts.haves, &negotiated->oid[i]);\n \n \twhile (refs) {\n-\t\tif (!is_null_oid(&refs->old_oid))\n-\t\t\tappend_negative_object(r, &opts.haves, &refs->old_oid);\n+\t\tif (!is_null_oid(&refs->old_oid)) {\n+\t\t\tint added = append_negative_object(r, &opts.haves,\n+\t\t\t\t\t\t\t   &refs->old_oid);\n+\n+\t\t\t/*\n+\t\t\t * For forced pushes, inspect the old tip's tree to\n+\t\t\t * avoid sending objects the receiver already has.\n+\t\t\t * Shallow thin-pack pushes already inspect negative\n+\t\t\t * tips via --objects-edge-aggressive. Limit this\n+\t\t\t * optimization to branches because --inspect-have\n+\t\t\t * requires commits, while other refs may legitimately\n+\t\t\t * point to trees or blobs.\n+\t\t\t */\n+\t\t\tif (added && refs->forced_update && !opts.shallow &&\n+\t\t\t    starts_with(refs->name, \"refs/heads/\") &&\n+\t\t\t    (refs->status == REF_STATUS_OK ||\n+\t\t\t     refs->status == REF_STATUS_EXPECTING_REPORT))\n+\t\t\t\toid_array_append(&opts.haves_to_inspect, &refs->old_oid);\n+\t\t}\n \t\tif (!is_null_oid(&refs->new_oid))\n \t\t\toid_array_append(&opts.wants, &refs->new_oid);\n \t\trefs = refs->next;\ndiff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh\nindex a46ba91ba0..067b4ddc16 100755\n--- a/t/t5516-fetch-push.sh\n+++ b/t/t5516-fetch-push.sh\n@@ -216,7 +216,7 @@ do\n \t\t\tGIT_TRACE2_EVENT=\"$D/rebased-push-event\" \\\n \t\t\tgit -c push.useBitmaps=false push $push_args\n \t\t) &&\n-\t\tgrep_wrote 3 rebased-push-event # 1 commit, 1 tree, 1 blob\n+\t\tgrep_wrote 2 rebased-push-event # 1 commit, 1 tree\n \t'\n done\n \n-- \ngitgitgadget\n\n"},{"id":"556378","messageId":"xmqq1p9cs79b.fsf@gitster.g","threadId":"66667","inReplyTo":"pull.2281.git.1793029565.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-26T17:26:24Z","receivedAt":"2026-10-26T17:26:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Marc Strapetz via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> A common feature-branch workflow is to push a branch, rebase it onto an\n> updated upstream, and force-push the rewritten commits. Large blobs added or\n> changed on the feature branch may survive the rebase unchanged. Although the\n> receiver already has those blobs through the old branch tip, Git can pack\n> and send them again. This can turn a small history rewrite into a large\n> upload.\n\nThis sounds like the exact problem push negotiation was designed to\naddress.\n\n"},{"id":"556409","messageId":"729930b3-1be6-4612-823c-55cb20d555b8@syntevo.com","threadId":"66667","inReplyTo":"xmqq1p9cs79b.fsf@gitster.g","subject":"Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing","fromName":"Marc Strapetz","fromEmail":"marc.strapetz@syntevo.com","sentAt":"2026-10-26T21:23:46Z","receivedAt":"2026-10-26T21:23:46Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"On 26/10/2026 18:26, Junio C Hamano wrote:\n> This sounds like the exact problem push negotiation was designed to\n> address.\n\nThanks for pointing this out. This series aims to optimize Git's default\npacking for forced non-fast-forward updates, without adding a\nnegotiation phase. I should have explained that scope more clearly in\nthe cover letter:\n\n    `pack-objects` can resend unchanged objects from an advertised old\n    branch tip even though `send-pack` already supplies that tip as an\n    exclusion. This series improves default packing for forced updates\n    by applying the existing sparse tree marking to selected old tips.\n    It targets this inefficiency without the additional negotiation\n    exchanges of `push.negotiate` or the broader recursive tree marking\n    of `--objects-edge-aggressive`. Neither is enabled by default for\n    ordinary pushes from non-shallow repositories. This series targets\n    a narrower opportunity within that default behavior.\n\n    A divergent old tip can lie outside the boundary reached by the\n    ordinary revision walk. Passing it to `pack-objects` as\n    `--inspect-have` lets us compare its root tree with the outgoing\n    trees, pruning identical and receiver-only subtrees. The additional\n    inspection is limited to forced non-fast-forward updates of\n    existing branches with locally available old tips. Large blobs\n    retained across a rebase are a particularly costly example of the\n    redundant transfer this avoids.\n\nI would also add the following section at the end to explain how this\nrelates to the existing optimizations:\n\n    `push.negotiate` discovers additional common commits through a\n    separate fetch negotiation, potentially requiring multiple protocol\n    round trips. This can reduce the pack when ref advertisement alone\n    is insufficient, but adds latency even when the advertised tips\n    already suffice. For small pushes that latency can outweigh the\n    transfer savings. Even if negotiation achieves the same pack-size\n    reduction, improving the default packing path avoids the need to\n    pay that additional protocol cost.\n\n    `--objects-edge-aggressive` already considers excluded tips beyond\n    the ordinary walk's boundary, but recursively marks their trees.\n    The decision to enable it automatically only for shallow pushes was\n    made for performance reasons: the 2014 series [1] records a\n    regression from 4.785 to 23.322 seconds with 22,400 refs [2], and\n    keeps non-shallow repositories on `--objects-edge` for better\n    performance in the general case [3]. This series instead limits\n    inspection to the old tips of branches being forcibly updated and\n    uses sparse comparison to prune identical and receiver-only\n    subtrees.\n\n    The additional inspection requires no protocol exchange or walk of\n    the old tip's commit history. Ordinary fast-forward pushes keep\n    their existing path, including those made with `--force`. No server\n    or wire-protocol change is needed.\n\n    [1]\nhttps://lore.kernel.org/git/1419462340-769147-3-git-send-email-sandals@crustytoothpaste.net/T/\n    [2]\nhttps://github.com/git/git/commit/1684c1b219e02c91655ad929f752f4f864c72faf\n    [3]\nhttps://github.com/git/git/commit/2dacf26d0985521c0f30e535963a45257b63ea21\n\n\n"},{"id":"557466","messageId":"54bbe02c-3319-477e-a9d9-024ef7027142@syntevo.com","threadId":"66667","inReplyTo":"pull.2281.git.1793029565.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing","fromName":"Marc Strapetz","fromEmail":"marc.strapetz@syntevo.com","sentAt":"2026-11-04T17:13:55Z","receivedAt":"2026-11-04T17:13:55Z","isPatch":true,"sender":{"key":"marc.strapetz@syntevo.com","avatar":"https://avatars.githubusercontent.com/u/3380730?v=4"},"body":"On 26/10/2026 16:46, Marc Strapetz via GitGitGadget wrote:\n> This series teaches send-pack to request inspection of the old tip's tree\n> before packing, so more receiver-known objects can be excluded.\nI'd like to give this series one more try, because I think the default\npush path still misses a worthwhile optimization opportunity.\n\nRebase-and-force-update workflows are common enough that avoiding\nredundant transfer on the default path appears worth evaluating. Junio\npointed out that push negotiation addresses the same general problem.\nThis series instead targets the non-negotiated default path: it uses the\nalready advertised old destination tip, with no additional round trip,\nserver change, or protocol change.\n\nFor qualifying non-fast-forward branch updates, this series can avoid\nresending objects already present at the destination. This can reduce\npack size and network transfer, as well as client-side packing and\nserver-side receive and indexing work. The scope is narrow: ordinary\nfast-forward pushes are unaffected, and the additional inspection is\nlimited to forced updates with a locally available old tip.\n\nThe current measurements show both the potential and the tradeoff: the\nsmall cases improve, while the low-benefit worst-case fixture adds about\n8% packing time for a negligible size reduction. These measurements\ncover only local pack generation; they do not include the primary\nexpected benefits of a smaller pack: reduced network transfer and less\nreceive and indexing work on the server. I would appreciate another look\nat whether this targeted optimization is worthwhile on the default path,\nand whether its current scope is narrow enough.\n\nI've added Taylor and Derrick because of their work on pack generation\nand revision traversal. I'd be interested to hear whether this looks\nlike a reasonable tradeoff for rewritten branch updates, or whether\nthere is a simpler approach worth exploring.\n\n-Marc\n\n\n\n\n\n"}]}