git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/3] clone: remove double bundle list clear code

From
Justin Tobler <jltobler@gmail.com>
Date
Jul 26, 2024, 21:52 UTC
Message-ID
<ypkmgjgobu2tn36o7qzultltknaz35qysrfe4y4kce67uysdxk@edmxls7p36mx>
In-Reply-To
<20240724144957.3033840-2-toon@iotcl.com>
On 24/07/24 04:49PM, Toon Claes wrote:
Show 7 quoted lines
> The bundle list transport->bundles is filled by
> transport_get_remote_bundle_uri(). Only when the list is not used, it is
> cleared right away by calling clear_bundle_list().
> 
> This looks like we leak memory allocated for the list when
> transport->bundles *is* used. But in fact, transport->bundles is cleaned
> up in transport_disconnect() near the end of cmd_clone().

The `transport->bundles` is setup and initialized by `transport_get()` and populated by `transport_get_remote_bundle_uri()`. In its current form, we clean up `transport->bundles` if it is empty immediately after checking the remote for bundles. This is redundant though because the cleanup already occurs as part of `transport_disconnect()`.

Since `transport_disconnect()` is already responsible for freeing other parts of `transport`, it makes sense to let it be the one to handle it.

Show 36 quoted lines
> 
> Remove the double clean up of transport->bundles, and depend solely on
> transport_disconnect() to take care of it.
> 
> Also add a test case that hits this code, but due to other leaks we
> cannot mark it as leak-free.
> 
> Signed-off-by: Toon Claes <toon@iotcl.com>
> ---
>  builtin/clone.c             |  3 ---
>  t/t5558-clone-bundle-uri.sh | 28 +++++++++++++++++++++++++++-
>  2 files changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/builtin/clone.c b/builtin/clone.c
> index af6017d41a..aa507395a0 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -1419,9 +1419,6 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
>  			else if (fetch_bundle_list(the_repository,
>  						   transport->bundles))
>  				warning(_("failed to fetch advertised bundles"));
> -		} else {
> -			clear_bundle_list(transport->bundles);
> -			FREE_AND_NULL(transport->bundles);
>  		}
>  	}
> 
> diff --git a/t/t5558-clone-bundle-uri.sh b/t/t5558-clone-bundle-uri.sh
> index cd05321e17..2d6e690fbe 100755
> --- a/t/t5558-clone-bundle-uri.sh
> +++ b/t/t5558-clone-bundle-uri.sh
> @@ -1,6 +1,6 @@
>  #!/bin/sh
> 
> -test_description='test fetching bundles with --bundle-uri'
> +test_description='test clone with use of bundle-uri'

I assume this description was changed because the test pertains to clones and not fetches. Might be worth mentioning in the commit message.

Show 33 quoted lines
> 
>  . ./test-lib.sh
>  . "$TEST_DIRECTORY"/lib-bundle.sh
> @@ -438,6 +438,32 @@ test_expect_success 'negotiation: bundle list with all wanted commits' '
>  	test_grep ! "clone> want " trace-packet.txt
>  '
> 
> +test_expect_success 'bundles advertised by the server' '
> +	test_when_finished rm -f trace*.txt &&
> +	git clone clone-from clone-advertiser &&
> +	git -C clone-advertiser config uploadpack.advertiseBundleURIs true &&
> +	git -C clone-advertiser config bundle.version 1 &&
> +	git -C clone-advertiser config bundle.mode all &&
> +	git -C clone-advertiser config bundle.bundle-1.uri "file://$(pwd)/clone-from/bundle-1.bundle" &&
> +	git -C clone-advertiser config bundle.bundle-2.uri "file://$(pwd)/clone-from/bundle-2.bundle" &&
> +	git -C clone-advertiser config bundle.bundle-3.uri "file://$(pwd)/clone-from/bundle-3.bundle" &&
> +	git -C clone-advertiser config bundle.bundle-4.uri "file://$(pwd)/clone-from/bundle-4.bundle" &&
> +
> +	GIT_TRACE_PACKET="$(pwd)/trace-packet.txt" \
> +	git -c transfer.bundleURI=true clone clone-advertiser clone-advertised &&
> +	git -C clone-advertised for-each-ref --format="%(refname)" >refs &&
> +	grep "refs/bundles/" refs >actual &&
> +	cat >expect <<-\EOF &&
> +	refs/bundles/base
> +	refs/bundles/left
> +	refs/bundles/merge
> +	refs/bundles/right
> +	EOF
> +	test_cmp expect actual &&
> +	# We already have all needed commits so no "want" needed.
> +	test_grep ! "clone> want " trace-packet.txt
> +'
> +

Looks like this test is validating that a clone is retrieving bundles from a remote repository advertising said bundles. We to do so by checking for the `refs/bundles/*` references which should only exist if the advertised bundles were fetched. This makes sense to me.

From the commit message though, I thought the added test might have something to do with the changes to when `transport->bundles` was freed. I wonder if it would be a good idea to expland on this further in the commit message. Or since the added test is somewhat unrelated to this patch, maybe it should be a separate patch?

-Justin
Show 6 quoted lines
>  #########################################################################
>  # HTTP tests begin here
> 
> --
> 2.45.2
> 
Previous: Karthik NayakNext: Toon claes
Message 6 of 17 in “fetch: use bundle URIs when having creationToken heuristic”
  1. fetch: use bundle URIs when having creationToken heuristicToon Claes, Jul 22, 2024
  2. Junio C HamanoJul 22, 2024
  3. 0/3 fetch: use bundle URIs when having creationToken heuristicToon Claes, Jul 24, 2024
  4. 1/3 clone: remove double bundle list clear codeToon Claes, Jul 24, 2024
  5. Karthik NayakJul 26, 2024
  6. Justin ToblerJul 26, 2024
  7. Toon claesAug 2, 2024
  8. 2/3 transport: introduce transport_has_remote_bundle_uri()Toon Claes, Jul 24, 2024
  9. Karthik NayakJul 26, 2024
  10. Junio C HamanoJul 26, 2024
  11. 3/3 fetch: use bundle URIs when having creationToken heuristicToon Claes, Jul 24, 2024
  12. Karthik NayakJul 26, 2024
  13. Patrick SteinhardtJul 26, 2024
  14. Toon claesAug 2, 2024
  15. Patrick SteinhardtAug 22, 2024
  16. Toon ClaesSep 27, 2024
  17. builtin/clone: teach git-clone(1) the --ref= argumentToon Claes, Sep 27, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.