Re: [PATCH] bundle-uri: copy all bundle references ino the refs/bundle space

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 25, 2025, 18:14 UTC
Message-ID: <xmqqv7sxki36.fsf@gitster.g>
In-Reply-To: <pull.1897.git.git.1740489585344.gitgitgadget@gmail.com>

"Scott Chacon via GitGitGadget" <gitgitgadget@gmail.com> writes:

>     Furthermore, when I bundled just a tag (thinking it would have most
>     reachable objects) it completely failed to work because there were no
>     refs/heads/ available for negotiation - so it downloaded a huge file and
>     then still started from scratch on the fetch.

Nice finding.

>     Now I'm only getting an extra 43k objects, so 1% of the original total,
>     and the entire operation is a bit faster as well.

That makes all sense.

>     I'm not sure if there is a downside here, it seems clearly how you would
>     want the negotiation to go. It ends up with way more refs under
>     refs/bundle (now there is refs/bundle/origin/master, etc) but that's
>     being polluted by the head refs anyhow, right?

I am not sure what you mean by "being polluted by the head refs anyhow", but we should be equipped to deal with a repository with tons of local branches, so having the comparable number of remote-tracking branches instead of local branches now exposed in refs/bundle/remotes/* hierarchy should work equally as well, or we would have something we need to fix. So in principle I do not see a problem with this approach.

The mapping used to be "refs/heads/foo" to "refs/bundles/foo", but now "refs/heads/foo" is mapped to "refs/bundles/heads/foo", so that you would presumably be mapping "refs/remotes/origin/master" to "refs/bundles/remotes/origin/master", right? I hope existing users are *not* looking at their resulting refs/bundles/ hierarchy and somehow assuming the original mapping.

This is not something this "fix" changes, but unbundle_all_bundles() apparently is prepared to handle more than one bundles. I wonder what happens when multiple bundles expose the same branch pointing at different objects? The way I read unbundle_from_file() is that a new one overwrites the previous value, so even though we may have all unbundled many bundles, the objects from earlier bundles may lose their anchors, subject to be garbage-collected.

Imagine creating a bundle with two refs, refs/heads and refs/remotes, and append that bundle as the last bundle of a bunch of bundles full of local and remote-tracking branches, that have populated refs/bundles/ hierarchy with tons of refs. Now the last bundle is unbundled, and these two phoney refs would nuke everything that used to be under refs/bundles/heads/* and refs/bundles/remotes/* left by unpacking previous bundles, right?

>     Is this a reasonable change?

This is mostly Stolee's design, IIRC, so I have CC'ed; the work is mostly from 53a50892 (bundle-uri: create basic file-copy logic, 2022-08-09) that is more than 2 years ago.

Show 18 quoted lines

> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1897%2Fschacon%2Fsc-more-bundle-refs-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1897/schacon/sc-more-bundle-refs-v1
> Pull-Request: https://github.com/git/git/pull/1897
>
>  bundle-uri.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/bundle-uri.c b/bundle-uri.c
> index 744257c49c1..3371d56f4ce 100644
> --- a/bundle-uri.c
> +++ b/bundle-uri.c
> @@ -403,7 +403,7 @@ static int unbundle_from_file(struct repository *r, const char *file)
>  		const char *branch_name;
>  		int has_old;
>  
> -		if (!skip_prefix(refname->string, "refs/heads/", &branch_name))
> +		if (!skip_prefix(refname->string, "refs/", &branch_name))
>  			continue;

By skipping only "refs/", "branch_name" is no longer a branch name, which may be something we may want to fix, but if we were to address the "names from later bundles seem to overwrite names used by previous boundles, and unanchor objects obtained from them" problem, I suspect we'd rather want to create new and unique names there, without assuming or relying on that the names used by these bundles are reasonably unique, so this part of the code may have to change anyway, so we may not care too deeply at this point.