git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/8] fetch-pack: use ref adv. to prune "have" sent

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jun 6, 2018, 20:47 UTC
Message-ID
<f12342fb2760eb0449c86c66bf44d39f5871be57.1528317619.git.jonathantanmy@google.com>
In-Reply-To
<cover.1528317619.git.jonathantanmy@google.com>

In negotiation using protocol v2, fetch-pack sometimes does not make full use of the information obtained in the ref advertisement: specifically, that if the server advertises a commit that the client also has, the client never needs to inform the server that it has the commit's parents, since it can just tell the server that it has the advertised commit and it knows that the server can and will infer the rest.

This is because, in do_fetch_pack_v2(), rev_list_insert_ref_oid() is invoked before everything_local(). This means that if we have a commit that is both our ref and their ref, it would be enqueued by rev_list_insert_ref_oid() as SEEN, and since it is thus already SEEN, everything_local() would not enqueue it.

If everything_local() were invoked first, as it is in do_fetch_pack() for protocol v0, then everything_local() would enqueue it with COMMON_REF | SEEN. The addition of COMMON_REF ensures that its parents are not sent as "have" lines.

Change the order in do_fetch_pack_v2() to be consistent with do_fetch_pack(), and to avoid sending unnecessary "have" lines.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 fetch-pack.c          |  6 +++---
 t/t5500-fetch-pack.sh | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 42 insertions(+), 3 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 09f5c83c4..114207b8e 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1391,9 +1391,6 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 				for_each_ref(clear_marks, NULL);
 			marked = 1;
 
-			for_each_ref(rev_list_insert_ref_oid, NULL);
-			for_each_cached_alternate(insert_one_alternate_object);
-
 			/* Filter 'ref' by 'sought' and those that aren't local */
 			mark_complete_and_common_ref(args, &ref);
 			filter_refs(args, &ref, sought, nr_sought);
@@ -1401,6 +1398,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 				state = FETCH_DONE;
 			else
 				state = FETCH_SEND_REQUEST;
+
+			for_each_ref(rev_list_insert_ref_oid, NULL);
+			for_each_cached_alternate(insert_one_alternate_object);
 			break;
 		case FETCH_SEND_REQUEST:
 			if (send_fetch_request(fd[1], args, ref, &common,
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index d4f435155..026ba9c9e 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -755,6 +755,45 @@ test_expect_success 'fetching deepen' '
 	)
 '
 
+test_expect_success 'use ref advertisement to prune "have" lines sent' '
+	rm -rf server client &&
+	git init server &&
+	test_commit -C server both_have_1 &&
+	git -C server tag -d both_have_1 &&
+	test_commit -C server both_have_2 &&
+
+	# In this test, the ref name that only the server has is a prefix of all
+	# other refs. This is because in protocol v2, the client sends
+	# "ref-prefix" to limit the ref advertisement. Naming the ref "bo" means
+	# that "ref-prefix refs/tags/bo*" is sent, resulting in the client also
+	# knowing about refs/tags/both_have_2, just as it would when it uses
+	# protocol v0.
+	git clone server client &&
+	test_commit -C server bo &&
+	test_commit -C client client_has &&
+
+	# In both protocol v0 and v2, ensure that the parent of both_have_2 is
+	# not sent as a "have" line. The client should know that the server has
+	# both_have_2, so it only needs to inform the server that it has
+	# both_have_2, and the server can infer the rest.
+
+	rm -f trace &&
+	cp -r client clientv0 &&
+	GIT_TRACE_PACKET="$(pwd)/trace" git -C clientv0 \
+		fetch origin bo &&
+	grep "have $(git -C client rev-parse client_has)" trace &&
+	grep "have $(git -C client rev-parse both_have_2)" trace &&
+	! grep "have $(git -C client rev-parse both_have_2^)" trace &&
+
+	rm -f trace &&
+	cp -r client clientv2 &&
+	GIT_TRACE_PACKET="$(pwd)/trace" git -C clientv2 -c protocol.version=2 \
+		fetch origin bo &&
+	grep "have $(git -C client rev-parse client_has)" trace &&
+	grep "have $(git -C client rev-parse both_have_2)" trace &&
+	! grep "have $(git -C client rev-parse both_have_2^)" trace
+'
+
 test_expect_success 'filtering by size' '
 	rm -rf server client &&
 	test_create_repo server &&
-- 
2.17.0.768.g1526ddbba1.dirty
Previous: Jonathan TanNext: Brandon Williams
Message 25 of 47 in “Refactor fetch negotiation into its own API”
  1. 0/6 Refactor fetch negotiation into its own APIJonathan Tan, Jun 4, 2018
  2. 1/6 fetch-pack: clear marks before everything_local()Jonathan Tan, Jun 4, 2018
  3. Jonathan NiederJun 5, 2018
  4. Jonathan TanJun 6, 2018
  5. 2/6 fetch-pack: truly stop negotiation upon ACK readyJonathan Tan, Jun 4, 2018
  6. Jonathan NiederJun 5, 2018
  7. Jonathan NiederJun 5, 2018
  8. Jonathan TanJun 6, 2018
  9. 3/6 fetch-pack: in protocol v2, enqueue commons firstJonathan Tan, Jun 4, 2018
  10. Jonathan NiederJun 5, 2018
  11. Jonathan TanJun 6, 2018
  12. 4/6 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 4, 2018
  13. Jonathan NiederJun 5, 2018
  14. Jonathan TanJun 6, 2018
  15. 5/6 fetch-pack: move common check and marking togetherJonathan Tan, Jun 4, 2018
  16. Jonathan NiederJun 6, 2018
  17. Jonathan TanJun 6, 2018
  18. 6/6 fetch-pack: introduce negotiator APIJonathan Tan, Jun 4, 2018
  19. Jonathan NiederJun 6, 2018
  20. Jonathan TanJun 6, 2018
  21. 0/8 Refactor fetch negotiation into its own APIJonathan Tan, Jun 6, 2018
  22. 1/8 fetch-pack: split up everything_local()Jonathan Tan, Jun 6, 2018
  23. Brandon WilliamsJun 14, 2018
  24. 2/8 fetch-pack: clear marks before re-markingJonathan Tan, Jun 6, 2018
  25. 4/8 fetch-pack: use ref adv. to prune "have" sentJonathan Tan, Jun 6, 2018
  26. Brandon WilliamsJun 14, 2018
  27. Junio C HamanoJun 14, 2018
  28. 5/8 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 6, 2018
  29. Brandon WilliamsJun 14, 2018
  30. Junio C HamanoJun 14, 2018
  31. 6/8 fetch-pack: move common check and marking togetherJonathan Tan, Jun 6, 2018
  32. 8/8 negotiator/default: use better style in commentsJonathan Tan, Jun 6, 2018
  33. Brandon WilliamsJun 14, 2018
  34. 7/8 fetch-pack: introduce negotiator APIJonathan Tan, Jun 6, 2018
  35. 3/8 fetch-pack: directly end negotiation if ACK readyJonathan Tan, Jun 6, 2018
  36. Brandon WilliamsJun 14, 2018
  37. Brandon WilliamsJun 14, 2018
  38. 0/7 Refactor fetch negotiation into its own APIJonathan Tan, Jun 14, 2018
  39. 1/7 fetch-pack: split up everything_local()Jonathan Tan, Jun 14, 2018
  40. 2/7 fetch-pack: clear marks before re-markingJonathan Tan, Jun 14, 2018
  41. 3/7 fetch-pack: directly end negotiation if ACK readyJonathan Tan, Jun 14, 2018
  42. Junio C HamanoJun 15, 2018
  43. 4/7 fetch-pack: use ref adv. to prune "have" sentJonathan Tan, Jun 14, 2018
  44. 5/7 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 14, 2018
  45. 6/7 fetch-pack: move common check and marking togetherJonathan Tan, Jun 14, 2018
  46. 7/7 fetch-pack: introduce negotiator APIJonathan Tan, Jun 14, 2018
  47. Brandon WilliamsJun 25, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.