git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 4/7] fetch-pack: use ref adv. to prune "have" sent

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Jun 14, 2018, 22:54 UTC
Message-ID
<be3774cd799b7ff538fefc2171bc46c164f2eae6.1529016350.git.jonathantanmy@google.com>
In-Reply-To
<cover.1529016350.git.jonathantanmy@google.com>

In negotiation using protocol v2, fetch-pack sometimes does not make full use of the information obtained in the ref advertisement: specifically, that if the server advertises a commit that the client also has, the client never needs to inform the server that it has the commit's parents, since it can just tell the server that it has the advertised commit and it knows that the server can and will infer the rest.

This is because, in do_fetch_pack_v2(), rev_list_insert_ref_oid() is invoked before mark_complete_and_common_ref(). This means that if we have a commit that is both our ref and their ref, it would be enqueued by rev_list_insert_ref_oid() as SEEN, and since it is thus already SEEN, mark_complete_and_common_ref() would not enqueue it.

If mark_complete_and_common_ref() were invoked first, as it is in do_fetch_pack() for protocol v0, then mark_complete_and_common_ref() would enqueue it with COMMON_REF | SEEN. The addition of COMMON_REF ensures that its parents are not sent as "have" lines.

Change the order in do_fetch_pack_v2() to be consistent with do_fetch_pack(), and to avoid sending unnecessary "have" lines.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 fetch-pack.c          |  6 +++---
 t/t5500-fetch-pack.sh | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 3 deletions(-)
diff --git a/fetch-pack.c b/fetch-pack.c
index 60adfc073..806c40021 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1392,9 +1392,6 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 				for_each_ref(clear_marks, NULL);
 			marked = 1;
 
-			for_each_ref(rev_list_insert_ref_oid, NULL);
-			for_each_cached_alternate(insert_one_alternate_object);
-
 			/* Filter 'ref' by 'sought' and those that aren't local */
 			mark_complete_and_common_ref(args, &ref);
 			filter_refs(args, &ref, sought, nr_sought);
@@ -1402,6 +1399,9 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
 				state = FETCH_DONE;
 			else
 				state = FETCH_SEND_REQUEST;
+
+			for_each_ref(rev_list_insert_ref_oid, NULL);
+			for_each_cached_alternate(insert_one_alternate_object);
 			break;
 		case FETCH_SEND_REQUEST:
 			if (send_fetch_request(fd[1], args, ref, &common,
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index d4f435155..e0cdc295d 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -755,6 +755,39 @@ test_expect_success 'fetching deepen' '
 	)
 '
 
+test_expect_success 'use ref advertisement to prune "have" lines sent' '
+	rm -rf server client &&
+	git init server &&
+	test_commit -C server both_have_1 &&
+	git -C server tag -d both_have_1 &&
+	test_commit -C server both_have_2 &&
+
+	git clone server client &&
+	test_commit -C server server_has &&
+	test_commit -C client client_has &&
+
+	# In both protocol v0 and v2, ensure that the parent of both_have_2 is
+	# not sent as a "have" line. The client should know that the server has
+	# both_have_2, so it only needs to inform the server that it has
+	# both_have_2, and the server can infer the rest.
+
+	rm -f trace &&
+	cp -r client clientv0 &&
+	GIT_TRACE_PACKET="$(pwd)/trace" git -C clientv0 \
+		fetch origin server_has both_have_2 &&
+	grep "have $(git -C client rev-parse client_has)" trace &&
+	grep "have $(git -C client rev-parse both_have_2)" trace &&
+	! grep "have $(git -C client rev-parse both_have_2^)" trace &&
+
+	rm -f trace &&
+	cp -r client clientv2 &&
+	GIT_TRACE_PACKET="$(pwd)/trace" git -C clientv2 -c protocol.version=2 \
+		fetch origin server_has both_have_2 &&
+	grep "have $(git -C client rev-parse client_has)" trace &&
+	grep "have $(git -C client rev-parse both_have_2)" trace &&
+	! grep "have $(git -C client rev-parse both_have_2^)" trace
+'
+
 test_expect_success 'filtering by size' '
 	rm -rf server client &&
 	test_create_repo server &&
-- 
2.17.0.582.gccdcbd54c4
Previous: Junio C HamanoNext: Jonathan Tan
Message 43 of 47 in “Refactor fetch negotiation into its own API”
  1. 0/6 Refactor fetch negotiation into its own APIJonathan Tan, Jun 4, 2018
  2. 1/6 fetch-pack: clear marks before everything_local()Jonathan Tan, Jun 4, 2018
  3. Jonathan NiederJun 5, 2018
  4. Jonathan TanJun 6, 2018
  5. 2/6 fetch-pack: truly stop negotiation upon ACK readyJonathan Tan, Jun 4, 2018
  6. Jonathan NiederJun 5, 2018
  7. Jonathan NiederJun 5, 2018
  8. Jonathan TanJun 6, 2018
  9. 3/6 fetch-pack: in protocol v2, enqueue commons firstJonathan Tan, Jun 4, 2018
  10. Jonathan NiederJun 5, 2018
  11. Jonathan TanJun 6, 2018
  12. 4/6 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 4, 2018
  13. Jonathan NiederJun 5, 2018
  14. Jonathan TanJun 6, 2018
  15. 5/6 fetch-pack: move common check and marking togetherJonathan Tan, Jun 4, 2018
  16. Jonathan NiederJun 6, 2018
  17. Jonathan TanJun 6, 2018
  18. 6/6 fetch-pack: introduce negotiator APIJonathan Tan, Jun 4, 2018
  19. Jonathan NiederJun 6, 2018
  20. Jonathan TanJun 6, 2018
  21. 0/8 Refactor fetch negotiation into its own APIJonathan Tan, Jun 6, 2018
  22. 1/8 fetch-pack: split up everything_local()Jonathan Tan, Jun 6, 2018
  23. Brandon WilliamsJun 14, 2018
  24. 2/8 fetch-pack: clear marks before re-markingJonathan Tan, Jun 6, 2018
  25. 4/8 fetch-pack: use ref adv. to prune "have" sentJonathan Tan, Jun 6, 2018
  26. Brandon WilliamsJun 14, 2018
  27. Junio C HamanoJun 14, 2018
  28. 5/8 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 6, 2018
  29. Brandon WilliamsJun 14, 2018
  30. Junio C HamanoJun 14, 2018
  31. 6/8 fetch-pack: move common check and marking togetherJonathan Tan, Jun 6, 2018
  32. 8/8 negotiator/default: use better style in commentsJonathan Tan, Jun 6, 2018
  33. Brandon WilliamsJun 14, 2018
  34. 7/8 fetch-pack: introduce negotiator APIJonathan Tan, Jun 6, 2018
  35. 3/8 fetch-pack: directly end negotiation if ACK readyJonathan Tan, Jun 6, 2018
  36. Brandon WilliamsJun 14, 2018
  37. Brandon WilliamsJun 14, 2018
  38. 0/7 Refactor fetch negotiation into its own APIJonathan Tan, Jun 14, 2018
  39. 1/7 fetch-pack: split up everything_local()Jonathan Tan, Jun 14, 2018
  40. 2/7 fetch-pack: clear marks before re-markingJonathan Tan, Jun 14, 2018
  41. 3/7 fetch-pack: directly end negotiation if ACK readyJonathan Tan, Jun 14, 2018
  42. Junio C HamanoJun 15, 2018
  43. 4/7 fetch-pack: use ref adv. to prune "have" sentJonathan Tan, Jun 14, 2018
  44. 5/7 fetch-pack: make negotiation-related vars localJonathan Tan, Jun 14, 2018
  45. 6/7 fetch-pack: move common check and marking togetherJonathan Tan, Jun 14, 2018
  46. 7/7 fetch-pack: introduce negotiator APIJonathan Tan, Jun 14, 2018
  47. Brandon WilliamsJun 25, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.