threads / patch / 66667

patch, 3 partssend-pack: avoid resending unchanged objects after rebasing

Subject: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing

## tl;dr

Junio C Hamano said the problem is what push negotiation was designed for; the author says his approach is narrower and has no extra round trips. Read the story.

replies: 6people: 2as markdown or json

Marc Strapetz via GitGitGadget· Oct 26, 2026, 15:46 UTC · lore

A common feature-branch workflow is to push a branch, rebase it onto an updated upstream, and force-push the rewritten commits. Large blobs added or changed on the feature branch may survive the rebase unchanged. Although the receiver already has those blobs through the old branch tip, Git can pack and send them again. This can turn a small history rewrite into a large upload.

This series teaches send-pack to request inspection of the old tip's tree before packing, so more receiver-known objects can be excluded.

The additional inspection is enabled only for forced non-fast-forward updates of existing branches with locally available old tips. Ordinary fast-forward pushes do not enable it, including ones made with --force. No server or wire-protocol change is needed.

The old tip is already a negative revision, but it may lie outside the boundary visited by the ordinary revision walk. Its tree is then not necessarily inspected. Passing it to pack-objects as --inspect-have lets us compare its root tree with the outgoing trees using the existing sparse tree marking. This excludes additional receiver-known objects without walking the old tip's commit history, and prunes identical and receiver-only subtrees.

Shallow pushes keep their existing aggressive edge handling. Partial clones skip the extra tree inspection to avoid fetching missing objects, and successful bitmap enumeration keeps its existing path. Old tips are resolved in pack-objects, where replacement refs are already disabled.

I compared upstream 7e34d8174dcda3435d1404c9a30bef46afe6c13e with the patched source using trees from Chromium commit 5b05cc16e960a021389ef9b543ab38a037c38d60. A depth-two clone supplies the objects; the measured fixture has a complete, non-shallow history with a common base and sibling receiver/outgoing commits.

Both builds used GCC 15.2.0 and -g -O2 -Wall on an Intel i7-13700H under WSL2. Measurements used a warm cache, sparse enumeration, no bitmaps, one packing thread, two warm-up pairs and 15 measured pairs with alternating build order. Values below are upstream/patched; total wall times are medians in milliseconds, including process startup.

Case Objects Pack bytes Total wall ms Reword 3 / 1 421 / 215 2.776 / 1.338 Single file 6 / 5 667 / 553 3.542 / 2.744 ~99% of files 552,724 / 552,723 58,082,988 / 58,082,873 4,525.094 / 4,883.623

In the single-file case, enumeration increases from 0.758 to 0.982 ms, but pack preparation decreases from 1.637 to 0.458 ms. In the last case, 506,468 regular files receive distinct small replacement blobs; total packing time increases by about 8%, saving only 115 pack bytes.

These are isolated local pack-objects measurements, not end-to-end pushes: network transfer and receiving-server work are excluded. Where redundant large blobs materially inflate the pack, avoiding their transfer should further reduce end-to-end push time, especially on slower connections; that benefit was not measured here.

Most objects in this fixture are already excluded through the common base, so the pack-size savings are small. The widespread-change case tests inspection overhead on Chromium's directory structure, rather than realistic compression of a whole-project rewrite. Feedback on this cost and the scope of the inspection would be particularly useful.

Local verification passed t5322-pack-objects-sparse.sh (19 tests) and t5516-fetch-push.sh (132 tests). Coverage includes identical trees, shared blobs under different trees, retained changed objects, subtree pruning, partial clones and local replacement refs.

Marc Strapetz (3):
  t5516: record redundant blobs after rebasing
  pack-objects: inspect selected haves for shared objects
  send-pack: avoid resending unchanged blobs after rebasing
 Documentation/git-pack-objects.adoc |   7 ++
 builtin/pack-objects.c              |  81 ++++++++++++++-
 odb.c                               |   1 +
 odb.h                               |   8 ++
 odb/source-files.c                  |   3 +
 send-pack.c                         |  30 ++++--
 t/t5322-pack-objects-sparse.sh      | 148 ++++++++++++++++++++++++++++
 t/t5516-fetch-push.sh               |  43 ++++++++
 8 files changed, 314 insertions(+), 7 deletions(-)
base-commit: 7e34d8174dcda3435d1404c9a30bef46afe6c13e
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2281%2Fmstrap%2Fsend-pack-inspect-haves-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2281/mstrap/send-pack-inspect-haves-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2281
-- 
gitgitgadget
Marc Strapetz via GitGitGadget· Oct 26, 2026, 15:46 UTC · re: Marc Strapetz via GitGitGadget · lore

[PATCH 1/3] t5516: record redundant blobs after rebasing

From: Marc Strapetz <marc.strapetz@syntevo.com>

A forced push of a rebased branch can resend an unchanged blob even though the receiver has it through the advertised old tip.

Add object-count tests for --force, --force-with-lease and forced refspecs. Confirm that the blob is unchanged by the rebase and record the current three-object pack: the rebased commit, its tree and the unchanged blob. Disable push bitmaps to exercise normal enumeration.

Also cover local replacement refs, which must not cause objects missing from the receiver to be omitted.

Assisted-by: GPT-6 Astra, GPT-6.1 Sol
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
---
 t/t5516-fetch-push.sh | 43 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
Show changes to t/t5516-fetch-push.sh +43 −0
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index b982b209bf..a46ba91ba0 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -193,6 +193,49 @@ grep_wrote () {
 	grep 'write_pack_file/wrote.*"value":"'$1'"' $2
 }
 
+for push_args in \
+	"--force origin topic" \
+	"--force-with-lease origin topic" \
+	"origin +topic"
+do
+	test_expect_success "push of rebased branch with $push_args" '
+		mk_test testrepo heads/main &&
+		test_when_finished "rm -rf rebased rebased-push-event" &&
+		git clone testrepo rebased &&
+		(
+			cd rebased &&
+			git checkout -b topic &&
+			test_commit feature &&
+			git push origin topic &&
+			git checkout main &&
+			test_commit upstream path1 &&
+			git push origin main &&
+			git checkout topic &&
+			git rebase main &&
+			test_cmp_rev feature:feature.t HEAD:feature.t &&
+			GIT_TRACE2_EVENT="$D/rebased-push-event" \
+			git -c push.useBitmaps=false push $push_args
+		) &&
+		grep_wrote 3 rebased-push-event # 1 commit, 1 tree, 1 blob
+	'
+done
+
+test_expect_success 'forced update ignores replacement for old tip' '
+	mk_test testrepo heads/main &&
+	test_when_finished "rm -rf replaced" &&
+	git clone testrepo replaced &&
+	(
+		cd replaced &&
+		test_commit old &&
+		git push origin main &&
+		old_tip=$(git rev-parse HEAD) &&
+		git reset --hard HEAD^ &&
+		test_commit new &&
+		git replace "$old_tip" HEAD &&
+		git push --force origin main
+	)
+'
+
 test_expect_success 'push without negotiation' '
 	mk_empty testrepo &&
 	git push testrepo $the_first_commit:refs/remotes/origin/first_commit &&
-- 
gitgitgadget
Marc Strapetz via GitGitGadget· Oct 26, 2026, 15:46 UTC · re: Marc Strapetz via GitGitGadget · lore

[PATCH 2/3] pack-objects: inspect selected haves for shared objects

From: Marc Strapetz <marc.strapetz@syntevo.com>

A receiver-known commit can be excluded from a revision walk without becoming a boundary commit. Its tree then need not be inspected, so objects shared with the outgoing history can still be packed.

Accept --inspect-have <commit> lines in --revs input and carry the selected tips through the object database pack-generation options. Compare their root trees with the outgoing tips using the existing sparse tree marking. This identifies additional receiver-known objects without walking the selected commits' histories and avoids descending into identical or receiver-only subtrees.

Resolve the selected commits in pack-objects, where replacement refs are already disabled, so local replacements cannot substitute trees the receiver does not have. Keep successful bitmap enumeration on its existing path and skip the optional inspection in partial clones.

Cover identical root trees, shared blobs under different roots, retained changed objects, subtree pruning, non-commit input, partial clone fallback and shared objects in earlier outgoing commits.

Assisted-by: GPT-6 Astra, GPT-6.1 Sol
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
---
 Documentation/git-pack-objects.adoc |   7 ++
 builtin/pack-objects.c              |  81 ++++++++++++++-
 odb.c                               |   1 +
 odb.h                               |   8 ++
 odb/source-files.c                  |   3 +
 t/t5322-pack-objects-sparse.sh      | 148 ++++++++++++++++++++++++++++
 6 files changed, 247 insertions(+), 1 deletion(-)
Show changes to 6 files +247 −1

Documentation/git-pack-objects.adoc, builtin/pack-objects.c, odb.c, odb.h, odb/source-files.c, t/t5322-pack-objects-sparse.sh

diff --git a/Documentation/git-pack-objects.adoc b/Documentation/git-pack-objects.adoc
index ccad938c5b..a98d4286fa 100644
--- a/Documentation/git-pack-objects.adoc
+++ b/Documentation/git-pack-objects.adoc
@@ -70,6 +70,13 @@ base-name::
 	outputs.  The objects on the resulting list are packed.
 	Besides revisions, `--not` or `--shallow <SHA-1>` lines are
 	also accepted.
++
+A `--inspect-have <commit>` line supplies the object ID of a commit known
+to the receiver. Before non-bitmap enumeration, its root tree is compared
+with the outgoing tips' root trees to identify additional objects that
+need not be packed, without walking its commit history. Identical
+subtrees and paths found only in the receiver-known trees are skipped.
+This optional inspection is skipped in partial clones.
 
 --unpacked::
 	This implies `--revs`.  When processing the list of
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index ca3a891dfb..9abb4b9fe3 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4892,12 +4892,76 @@ static int get_object_list_path_walk(struct rev_info *revs)
 	return result;
 }
 
+static void add_have_to_inspect(struct oidset *trees, const char *arg)
+{
+	struct object_id oid;
+	struct commit *commit;
+	struct tree *tree;
+	const char *end;
+
+	if (parse_oid_hex(arg, &oid, &end) || *end)
+		die(_("not an object name '%s'"), arg);
+	if (!odb_has_object(the_repository->objects, &oid, 0))
+		return;
+	commit = lookup_commit_reference_gently(the_repository, &oid, 1);
+	if (!commit)
+		die(_("object %s is not a commit"), arg);
+	tree = repo_get_commit_tree(the_repository, commit);
+	if (tree)
+		oidset_insert(trees, &tree->object.oid);
+}
+
+static void inspect_haves(struct rev_info *revs, struct oidset *trees)
+{
+	struct oidset_iter iter;
+	struct object_id *oid;
+
+	/* Avoid fetching missing trees for this optional optimization. */
+	if (!oidset_size(trees) || repo_has_promisor_remote(revs->repo))
+		return;
+
+	/* Mark receiver-known root trees as uninteresting. */
+	oidset_iter_init(trees, &iter);
+	while ((oid = oidset_iter_next(&iter))) {
+		struct tree *tree = lookup_tree(revs->repo, oid);
+
+		if (tree)
+			tree->object.flags |= UNINTERESTING;
+	}
+
+	/*
+	 * Add outgoing root trees for the same sparse tree marking used by
+	 * mark_edges_uninteresting(). A receiver-known tip on the other side of
+	 * a divergence can be a negative revision without its tree being visited
+	 * by the ordinary walk. Combining its tree with the outgoing trees lets
+	 * us exclude shared objects without walking that tip's commit history.
+	 */
+	for (size_t i = 0; i < revs->pending.nr; i++) {
+		struct object *obj = revs->pending.objects[i].item;
+
+		if (obj->flags & UNINTERESTING)
+			continue;
+		if (obj->type == OBJ_COMMIT) {
+			struct tree *tree = repo_get_commit_tree(revs->repo,
+							(struct commit *)obj);
+
+			if (tree)
+				oidset_insert(trees, &tree->object.oid);
+		} else if (obj->type == OBJ_TREE) {
+			oidset_insert(trees, &obj->oid);
+		}
+	}
+
+	mark_trees_uninteresting_sparse(revs->repo, trees);
+}
+
 static void get_object_list(struct rev_info *revs, struct strvec *argv)
 {
 	struct setup_revision_opt s_r_opt = {
 		.allow_exclude_promisor_objects = 1,
 	};
 	struct repo_config_values *cfg = repo_config_values(the_repository);
+	struct oidset have_trees = OIDSET_INIT;
 	char line[1000];
 	int flags = 0;
 	int save_warning;
@@ -4931,6 +4995,11 @@ static void get_object_list(struct rev_info *revs, struct strvec *argv)
 				use_bitmap_index = 0;
 				continue;
 			}
+			if (starts_with(line, "--inspect-have ")) {
+				add_have_to_inspect(&have_trees, line + 15);
+				write_bitmap_index = 0;
+				continue;
+			}
 			die(_("not a rev '%s'"), line);
 		}
 		if (handle_revision_arg(line, revs, flags, REVARG_CANNOT_BE_FILENAME))
@@ -4939,8 +5008,18 @@ static void get_object_list(struct rev_info *revs, struct strvec *argv)
 
 	cfg->warn_on_object_refname_ambiguity = save_warning;
 
-	if (use_bitmap_index && !get_object_list_from_bitmap(revs))
+	if (use_bitmap_index && !get_object_list_from_bitmap(revs)) {
+		oidset_clear(&have_trees);
 		return;
+	}
+
+	/*
+	 * Limit this inspection to non-bitmap enumeration to avoid extra tree
+	 * reads on the bitmap fast path. Boundary bitmap traversal may still
+	 * include receiver-known objects; that tradeoff is intentional.
+	 */
+	inspect_haves(revs, &have_trees);
+	oidset_clear(&have_trees);
 
 	if (use_delta_islands)
 		load_delta_islands(the_repository, progress);
diff --git a/odb.c b/odb.c
index 98c5c862f5..2bb27848ac 100644
--- a/odb.c
+++ b/odb.c
@@ -731,6 +731,7 @@ void odb_generate_pack_options_release(struct odb_generate_pack_options *opts)
 {
 	oid_array_clear(&opts->wants);
 	oid_array_clear(&opts->haves);
+	oid_array_clear(&opts->haves_to_inspect);
 	oid_array_clear(&opts->shallows);
 }
 
diff --git a/odb.h b/odb.h
index 0ccc47ee73..10629b2044 100644
--- a/odb.h
+++ b/odb.h
@@ -667,6 +667,13 @@ struct odb_generate_pack_options {
 	 */
 	struct oid_array haves;
 
+	/*
+	 * Receiver-known commit tips whose root trees should be examined
+	 * to identify additional objects that need not be packed. This
+	 * optional inspection does not walk their commit histories.
+	 */
+	struct oid_array haves_to_inspect;
+
 	/*
 	 * The shallow boundary that shall be used when computing object
 	 * reachability. When set, any shallow information of the repository
@@ -751,6 +758,7 @@ struct odb_generate_pack_options {
 #define ODB_GENERATE_PACK_OPTIONS_INIT { \
 	.wants = OID_ARRAY_INIT, \
 	.haves = OID_ARRAY_INIT, \
+	.haves_to_inspect = OID_ARRAY_INIT, \
 	.shallows = OID_ARRAY_INIT, \
 	.pack_fd = -1, \
 }
diff --git a/odb/source-files.c b/odb/source-files.c
index 455f409db0..bad9ecde4c 100644
--- a/odb/source-files.c
+++ b/odb/source-files.c
@@ -1176,6 +1176,9 @@ static int odb_source_files_generate_pack(struct odb_source *source UNUSED,
 	fprintf(in, "--not\n");
 	for (size_t i = 0; i < opts->haves.nr; i++)
 		fprintf(in, "%s\n", oid_to_hex(&opts->haves.oid[i]));
+	for (size_t i = 0; i < opts->haves_to_inspect.nr; i++)
+		fprintf(in, "--inspect-have %s\n",
+			oid_to_hex(&opts->haves_to_inspect.oid[i]));
 	fprintf(in, "\n");
 	fflush(in);
 	if (ferror(in)) {
diff --git a/t/t5322-pack-objects-sparse.sh b/t/t5322-pack-objects-sparse.sh
index d39958c066..0cf48513eb 100755
--- a/t/t5322-pack-objects-sparse.sh
+++ b/t/t5322-pack-objects-sparse.sh
@@ -138,4 +138,152 @@ test_expect_success 'pack.useSparse overridden' '
 	test_cmp required_objects.txt sparse_objects.txt
 '
 
+test_expect_success 'setup inspect-have receiver and outgoing commits' '
+	git checkout -b receiver main &&
+	echo large >large.bin &&
+	echo receiver >f2/f2/data.txt &&
+	git add large.bin f2/f2/data.txt &&
+	test_commit --no-tag receiver file.txt &&
+	git checkout -b outgoing main &&
+	git checkout receiver -- large.bin f2 &&
+	git rm -r f3 &&
+	test_commit --no-tag outgoing file.txt &&
+	receiver_tip=$(git rev-parse receiver) &&
+	cat >packinput.txt <<-EOF
+	outgoing
+	^main
+	^receiver
+	--inspect-have $receiver_tip
+	EOF
+'
+
+test_expect_success 'inspect-have excludes an identical root tree' '
+	tree=$(git rev-parse receiver^{tree}) &&
+	same_tip=$(git commit-tree "$tree" -p main -m "Same tree") &&
+	cat >same-tree-input.txt <<-EOF &&
+	$same_tip
+	^main
+	^receiver
+	--inspect-have $receiver_tip
+	EOF
+	echo "$same_tip" >expect_objects.txt &&
+	git pack-objects --stdout --revs <same-tree-input.txt >inspected.pack &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp_sorted expect_objects.txt inspected_objects.txt
+'
+
+test_expect_success 'inspect-have excludes shared large blob from different root trees' '
+	test_cmp_rev ! receiver^{tree} outgoing^{tree} &&
+	test_cmp_rev receiver:large.bin outgoing:large.bin &&
+	blob=$(git rev-parse receiver:large.bin) &&
+	sed "/^--inspect-have /d" packinput.txt >ordinary-input.txt &&
+	git pack-objects --stdout --revs <ordinary-input.txt >ordinary.pack &&
+	git index-pack -o ordinary.idx ordinary.pack &&
+	git show-index <ordinary.idx >index &&
+	awk "{print \$2}" index >ordinary_objects.txt &&
+	test_grep "$blob" ordinary_objects.txt &&
+	git rev-parse outgoing outgoing^{tree} outgoing:file.txt >expect_objects.txt &&
+	git pack-objects --stdout --revs <packinput.txt >inspected.pack &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp_sorted expect_objects.txt inspected_objects.txt
+'
+
+test_expect_success 'inspect-have keeps changed blobs and trees' '
+	git checkout -b outgoing-changed outgoing &&
+	test_commit changed f2/f2/data.txt &&
+	sed "s/^outgoing$/outgoing-changed/" packinput.txt >changed-input.txt &&
+	git rev-parse \
+		outgoing \
+		outgoing^{tree} \
+		outgoing:file.txt \
+		outgoing-changed \
+		outgoing-changed^{tree} \
+		outgoing-changed:f2 \
+		outgoing-changed:f2/f2 \
+		outgoing-changed:f2/f2/data.txt >changed_objects.txt &&
+	git pack-objects --stdout --revs <changed-input.txt >inspected.pack &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp_sorted changed_objects.txt inspected_objects.txt
+'
+
+test_expect_success 'inspect-have skips unchanged and receiver-only subtrees' '
+	git rev-parse receiver:f2 receiver:f3 >trees &&
+	git pack-objects --stdout --window=0 <trees >trees.pack &&
+	git index-pack --stdin <trees.pack &&
+	while read oid
+	do
+		path=$(test_oid_to_path "$oid") &&
+		rm ".git/objects/$path" || return 1
+	done <trees &&
+	>pack-access &&
+	GIT_TRACE_PACK_ACCESS="$(pwd)/pack-access" \
+	git pack-objects --stdout --revs <packinput.txt >inspected.pack &&
+	test_must_be_empty pack-access &&
+	git rev-parse outgoing outgoing^{tree} outgoing:file.txt >expect_objects.txt &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp_sorted expect_objects.txt inspected_objects.txt
+'
+
+test_expect_success 'inspect-have rejects non-commit objects' '
+	blob=$(git rev-parse receiver:f2/f2/data.txt) &&
+	echo "--inspect-have $blob" >invalid-input.txt &&
+	test_must_fail git pack-objects --stdout --revs \
+		<invalid-input.txt >invalid.pack 2>err &&
+	test_grep "is not a commit" err
+'
+
+test_expect_success 'inspect-have is ignored in partial clones' '
+	test_config remote.missing.url "$(pwd)/missing" &&
+	test_config remote.missing.promisor true &&
+	sed "/^--inspect-have /d" packinput.txt >partial-input.txt &&
+	git pack-objects --stdout --revs --thin --window=0 \
+		<partial-input.txt >ordinary.pack 2>err &&
+	test_must_be_empty err &&
+	git pack-objects --stdout --revs --thin --window=0 \
+		<packinput.txt >inspected.pack 2>err &&
+	test_must_be_empty err &&
+	git index-pack -o ordinary.idx ordinary.pack &&
+	git show-index <ordinary.idx >index &&
+	awk "{print \$2}" index >ordinary_objects.txt &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp ordinary_objects.txt inspected_objects.txt
+'
+
+test_expect_success 'inspect-have excludes shared blobs from earlier outgoing commits' '
+	git checkout -b receiver-extra receiver &&
+	test_commit --no-tag extra f2/f2/small &&
+	git checkout -b outgoing-extra outgoing &&
+	test_commit --no-tag extra f2/f2/small &&
+	receiver_extra_tip=$(git rev-parse receiver-extra) &&
+	cat >tip-input.txt <<-EOF &&
+	outgoing-extra
+	^main
+	^receiver-extra
+	--inspect-have $receiver_extra_tip
+	EOF
+	git rev-parse \
+		outgoing \
+		outgoing^{tree} \
+		outgoing:f2 \
+		outgoing:f2/f2 \
+		outgoing:file.txt \
+		outgoing-extra \
+		outgoing-extra^{tree} >expect_objects.txt &&
+	git pack-objects --stdout --revs <tip-input.txt >inspected.pack &&
+	git index-pack -o inspected.idx inspected.pack &&
+	git show-index <inspected.idx >index &&
+	awk "{print \$2}" index >inspected_objects.txt &&
+	test_cmp_sorted expect_objects.txt inspected_objects.txt
+'
+
 test_done
-- 
gitgitgadget
Marc Strapetz via GitGitGadget· Oct 26, 2026, 15:46 UTC · re: Marc Strapetz via GitGitGadget · lore

[PATCH 3/3] send-pack: avoid resending unchanged blobs after rebasing

From: Marc Strapetz <marc.strapetz@syntevo.com>

Rebasing a pushed branch can put its old tip outside the boundary traversed by pack-objects. Even with that tip among the negative revisions, unchanged blobs may then be sent again.

For forced non-fast-forward branch updates, pass locally available advertised old tips as haves_to_inspect. Limit this to refs selected for sending; leave fast-forward and rejected updates out. Restrict the hints to branches, which name commits, and skip shallow repositories, whose thin-pack pushes already inspect negative tips with --objects-edge-aggressive.

Update the rebase tests to expect two packed objects instead of three: the rebased commit and its tree. The unchanged blob is already present on the receiver.

Assisted-by: GPT-6 Astra, GPT-6.1 Sol
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
---
 send-pack.c           | 30 ++++++++++++++++++++++++------
 t/t5516-fetch-push.sh |  2 +-
 2 files changed, 25 insertions(+), 7 deletions(-)
Show changes to 2 files +25 −7

send-pack.c, t/t5516-fetch-push.sh

diff --git a/send-pack.c b/send-pack.c
index f20460fbf4..de6d33cf75 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -42,17 +42,18 @@ int option_parse_push_signed(const struct option *opt,
 	die("bad %s argument: %s", opt->long_name, arg);
 }
 
-static void append_negative_object(struct repository *r,
-				   struct oid_array *haves,
-				   const struct object_id *oid)
+static int append_negative_object(struct repository *r,
+				  struct oid_array *haves,
+				  const struct object_id *oid)
 {
 	/*
 	 * The remote end may have advertised objects that we do not have in
 	 * our object database. Skip those, as we cannot use them as boundary.
 	 */
 	if (!odb_has_object(r->objects, oid, 0))
-		return;
+		return 0;
 	oid_array_append(haves, oid);
+	return 1;
 }
 
 /*
@@ -89,8 +90,25 @@ static int pack_objects(struct repository *r,
 		append_negative_object(r, &opts.haves, &negotiated->oid[i]);
 
 	while (refs) {
-		if (!is_null_oid(&refs->old_oid))
-			append_negative_object(r, &opts.haves, &refs->old_oid);
+		if (!is_null_oid(&refs->old_oid)) {
+			int added = append_negative_object(r, &opts.haves,
+							   &refs->old_oid);
+
+			/*
+			 * For forced pushes, inspect the old tip's tree to
+			 * avoid sending objects the receiver already has.
+			 * Shallow thin-pack pushes already inspect negative
+			 * tips via --objects-edge-aggressive. Limit this
+			 * optimization to branches because --inspect-have
+			 * requires commits, while other refs may legitimately
+			 * point to trees or blobs.
+			 */
+			if (added && refs->forced_update && !opts.shallow &&
+			    starts_with(refs->name, "refs/heads/") &&
+			    (refs->status == REF_STATUS_OK ||
+			     refs->status == REF_STATUS_EXPECTING_REPORT))
+				oid_array_append(&opts.haves_to_inspect, &refs->old_oid);
+		}
 		if (!is_null_oid(&refs->new_oid))
 			oid_array_append(&opts.wants, &refs->new_oid);
 		refs = refs->next;
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index a46ba91ba0..067b4ddc16 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -216,7 +216,7 @@ do
 			GIT_TRACE2_EVENT="$D/rebased-push-event" \
 			git -c push.useBitmaps=false push $push_args
 		) &&
-		grep_wrote 3 rebased-push-event # 1 commit, 1 tree, 1 blob
+		grep_wrote 2 rebased-push-event # 1 commit, 1 tree
 	'
 done
 
-- 
gitgitgadget
Junio C Hamano· Oct 26, 2026, 17:26 UTC · re: Marc Strapetz via GitGitGadget · lore

Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing

"Marc Strapetz via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 6 quoted lines
> A common feature-branch workflow is to push a branch, rebase it onto an
> updated upstream, and force-push the rewritten commits. Large blobs added or
> changed on the feature branch may survive the rebase unchanged. Although the
> receiver already has those blobs through the old branch tip, Git can pack
> and send them again. This can turn a small history rewrite into a large
> upload.

This sounds like the exact problem push negotiation was designed to address.

Marc Strapetz· Oct 26, 2026, 21:23 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing

On 26/10/2026 18:26, Junio C Hamano wrote:
> This sounds like the exact problem push negotiation was designed to
> address.

Thanks for pointing this out. This series aims to optimize Git's default packing for forced non-fast-forward updates, without adding a negotiation phase. I should have explained that scope more clearly in the cover letter:

    `pack-objects` can resend unchanged objects from an advertised old
    branch tip even though `send-pack` already supplies that tip as an
    exclusion. This series improves default packing for forced updates
    by applying the existing sparse tree marking to selected old tips.
    It targets this inefficiency without the additional negotiation
    exchanges of `push.negotiate` or the broader recursive tree marking
    of `--objects-edge-aggressive`. Neither is enabled by default for
    ordinary pushes from non-shallow repositories. This series targets
    a narrower opportunity within that default behavior.
    A divergent old tip can lie outside the boundary reached by the
    ordinary revision walk. Passing it to `pack-objects` as
    `--inspect-have` lets us compare its root tree with the outgoing
    trees, pruning identical and receiver-only subtrees. The additional
    inspection is limited to forced non-fast-forward updates of
    existing branches with locally available old tips. Large blobs
    retained across a rebase are a particularly costly example of the
    redundant transfer this avoids.

I would also add the following section at the end to explain how this relates to the existing optimizations:

    `push.negotiate` discovers additional common commits through a
    separate fetch negotiation, potentially requiring multiple protocol
    round trips. This can reduce the pack when ref advertisement alone
    is insufficient, but adds latency even when the advertised tips
    already suffice. For small pushes that latency can outweigh the
    transfer savings. Even if negotiation achieves the same pack-size
    reduction, improving the default packing path avoids the need to
    pay that additional protocol cost.
    `--objects-edge-aggressive` already considers excluded tips beyond
    the ordinary walk's boundary, but recursively marks their trees.
    The decision to enable it automatically only for shallow pushes was
    made for performance reasons: the 2014 series [1] records a
    regression from 4.785 to 23.322 seconds with 22,400 refs [2], and
    keeps non-shallow repositories on `--objects-edge` for better
    performance in the general case [3]. This series instead limits
    inspection to the old tips of branches being forcibly updated and
    uses sparse comparison to prune identical and receiver-only
    subtrees.
    The additional inspection requires no protocol exchange or walk of
    the old tip's commit history. Ordinary fast-forward pushes keep
    their existing path, including those made with `--force`. No server
    or wire-protocol change is needed.
    [1]
https://lore.kernel.org/git/1419462340-769147-3-git-send-email-sandals@crustytoothpaste.net/T/
    [2]
https://github.com/git/git/commit/1684c1b219e02c91655ad929f752f4f864c72faf
    [3]
https://github.com/git/git/commit/2dacf26d0985521c0f30e535963a45257b63ea21
Marc Strapetz· Nov 4, 2026, 17:13 UTC · re: Marc Strapetz via GitGitGadget · lore

Re: [PATCH 0/3] send-pack: avoid resending unchanged objects after rebasing

On 26/10/2026 16:46, Marc Strapetz via GitGitGadget wrote:
> This series teaches send-pack to request inspection of the old tip's tree
> before packing, so more receiver-known objects can be excluded.

I'd like to give this series one more try, because I think the default push path still misses a worthwhile optimization opportunity.

Rebase-and-force-update workflows are common enough that avoiding redundant transfer on the default path appears worth evaluating. Junio pointed out that push negotiation addresses the same general problem. This series instead targets the non-negotiated default path: it uses the already advertised old destination tip, with no additional round trip, server change, or protocol change.

For qualifying non-fast-forward branch updates, this series can avoid resending objects already present at the destination. This can reduce pack size and network transfer, as well as client-side packing and server-side receive and indexing work. The scope is narrow: ordinary fast-forward pushes are unaffected, and the additional inspection is limited to forced updates with a locally available old tip.

The current measurements show both the potential and the tradeoff: the small cases improve, while the low-benefit worst-case fixture adds about 8% packing time for a negligible size reduction. These measurements cover only local pack generation; they do not include the primary expected benefits of a smaller pack: reduced network transfer and less receive and indexing work on the server. I would appreciate another look at whether this targeted optimization is worthwhile on the default path, and whether its current scope is narrow enough.

I've added Taylor and Derrick because of their work on pack generation and revision traversal. I'd be interested to hear whether this looks like a reasonable tradeoff for rewritten branch updates, or whether there is a simpler approach worth exploring.

-Marc

← back to recent threads