git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/2] fetch-pack: in protocol v2, in_vain only after ACK

From
Jonathan Nieder <jrnieder@gmail.com>
Date
Apr 26, 2020, 00:28 UTC
Message-ID
<20200426002802.GD877@gmail.com>
In-Reply-To
<51174e53527f9b29bb0a5ebb8726f07333113dfb.1587775989.git.jonathantanmy@google.com>
Hi,
Jonathan Tan wrote:
Show 20 quoted lines
> When fetching, Git stops negotiation when it has sent at least
> MAX_IN_VAIN (which is 256) "have" lines without having any of them
> ACK-ed. But this is supposed to trigger only after the first ACK, as
> pack-protocol.txt says:
> 
>   However, the 256 limit *only* turns on in the canonical client
>   implementation if we have received at least one "ACK %s continue"
>   during a prior round.  This helps to ensure that at least one common
>   ancestor is found before we give up entirely.
> 
> The code path for protocol v0 observes this, but not protocol v2,
> resulting in shorter negotiation rounds but significantly larger
> packfiles. Teach the code path for protocol v2 to check this criterion
> only after at least one ACK was received.
> 
> Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
> ---
>  fetch-pack.c          | 13 +++++++++----
>  t/t5500-fetch-pack.sh | 19 +++++++++++++++++++
>  2 files changed, 28 insertions(+), 4 deletions(-)
Makes a lot of sense.  Good find.
[...]
> --- a/fetch-pack.c
> +++ b/fetch-pack.c
[...]
Show 5 quoted lines
> @@ -1513,6 +1517,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
>  				break;
>  			case 1:
>  				in_vain = 0;
> +				seen_ack = 1;

not about this patch: can these return values from process_acks be made into an enum with named enumerators? That would make what's happening in the call site more obvious.

Show 27 quoted lines
>  				/* fallthrough */
>  			default:
>  				state = FETCH_SEND_REQUEST;
> diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
> index baa1a99f45..fcc5cc8ab0 100755
> --- a/t/t5500-fetch-pack.sh
> +++ b/t/t5500-fetch-pack.sh
> @@ -385,6 +385,25 @@ test_expect_success 'clone shallow with packed refs' '
>  	test_cmp count8.expected count8.actual
>  '
>  
> +test_expect_success 'in_vain not triggered before first ACK' '
> +	rm -rf myserver myclient trace &&
> +	git init myserver &&
> +	test_commit -C myserver foo &&
> +	git clone "file://$(pwd)/myserver" myclient &&
> +
> +	# MAX_IN_VAIN is 256. Because of batching, the client will send 496
> +	# (16+32+64+128+256) commits, not 256, before giving up. So create 496
> +	# irrelevant commits.
> +	test_commit_bulk -C myclient 496 &&
> +
> +	# The new commit that the client wants to fetch.
> +	test_commit -C myserver bar &&
> +
> +	GIT_TRACE_PACKET="$(pwd)/trace" git -C myclient fetch --progress origin &&
> +	cp trace /tmp/x &&
Leftover debugging line?
> +	test_i18ngrep "Total 3 " trace
Clever.

In some sense this is a fragile test, since the server could change how it reports progress some day. Would it make sense (perhaps as a followup patch) for this to use a trace2 log instead? For example, if we turn on tracing in the server, then since 9ed8790282 (pack-objects: write objects packed to trace2, 2019-04-11) it will report how many objects were in the pack it wrote.

After removing the "cp trace /tmp/x" line,
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Thanks.
Previous: Junio C HamanoNext: Jonathan Tan
Message 4 of 25 in “Protocol v2 in_vain fixes”
  1. 0/2 Protocol v2 in_vain fixesJonathan Tan, Apr 25, 2020
  2. 1/2 fetch-pack: in protocol v2, in_vain only after ACKJonathan Tan, Apr 25, 2020
  3. Junio C HamanoApr 25, 2020
  4. Jonathan NiederApr 26, 2020
  5. Jonathan TanApr 27, 2020
  6. Junio C HamanoApr 27, 2020
  7. 2/2 fetch-pack: in protocol v2, reset in_vain upon ACKJonathan Tan, Apr 25, 2020
  8. Jonathan NiederApr 26, 2020
  9. Jonathan TanApr 27, 2020
  10. 0/3 Protocol v2 in_vain fixesJonathan Tan, Apr 28, 2020
  11. 1/3 fetch-pack: return enum from process_acks()Jonathan Tan, Apr 28, 2020
  12. Jonathan NiederApr 28, 2020
  13. Junio C HamanoApr 28, 2020
  14. Michal SuchánekApr 28, 2020
  15. Junio C HamanoApr 28, 2020
  16. 2/3 fetch-pack: in protocol v2, in_vain only after ACKJonathan Tan, Apr 28, 2020
  17. Jonathan NiederApr 28, 2020
  18. Johannes SchindelinMay 6, 2020
  19. t5500: count objects through stderr, not traceJonathan Tan, May 6, 2020
  20. Junio C HamanoMay 6, 2020
  21. Junio C HamanoMay 6, 2020
  22. Johannes SchindelinMay 7, 2020
  23. Johannes SchindelinOct 13, 2020
  24. 3/3 fetch-pack: in protocol v2, reset in_vain upon ACKJonathan Tan, Apr 28, 2020
  25. Jonathan NiederApr 28, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.