Re: should git maintenance prefetch be taught to honor remote.fetch refspec?

From: Derrick Stolee <stolee@gmail.com>
Date: Apr 4, 2021, 20:25 UTC
Message-ID: <f113284b-a7fe-ba7f-ce1c-d214efd5d0c6@gmail.com>
In-Reply-To: <xmqq1rbs4c6t.fsf@gitster.g>

On 4/2/2021 5:19 PM, Junio C Hamano wrote:

Show 31 quoted lines

> Derrick Stolee <stolee@gmail.com> writes:
> 
>> The other issue is that exact matches (no "*") have an exact
>> string in the destination, too, so replacing the _entire_
>> destination with "refs/prefetch/<remote>/*" breaks the refspec.
>> I think the substring approach will still work here.
> 
> I wonder if that matters.  If the exact format says
> 
> 	[remote "origin"]
> 	        url = git://git.kernel.org/pub/scm/git/git.git/
> 		fetch = +refs/heads/todo:refs/x/y/z
> 
> you can just add refs/prefetch/<remote>/ to the entire RHS to ensure
> that (1) the prefetch would not affect anything outside the 'prefetch'
> to break @{upstream} and friends, and (2) the prefetch from this remote
> would not affect anything used for other remotes.
> 
> IOW, the RHS, as long as it is hidden from normal operations and
> does not conflict with interaction with other repositories, where
> exactly in the refs hierarchy these objects are "parked" does not
> matter, I would think.
> 
> I do not recommend unparsed refspec and textually munging, by the
> way.  Doesn't
> 
> 	git fetch master:remotes/origin/master
> 
> first parse to normalize the src/dst sides to turn it into
> 
> 	git fetch refs/heads/master:refs/remotes/origin/master

I do not see evidence of this being reflected in my testing. The good news is that I'm creating a test scenario that will give us a clear way to test what the refspec parsing (and new output format logic) is doing.

Now, it might be that we are having an equivalent behavior to what you are suggesting because of the way refs are interpreted when those refs are created based on the refspec. This complicates things slightly, and I can work to see if there is a good way to modify the refspec parsing to make this an explicit assumption.

Show 5 quoted lines

> which is what you want to further redirect to the prefetch hierarchy
> 
> 	git fetch +refs/heads/master:refs/prefetch/origin/refs/remotes/origin/master
> 
> or whatever munging scheme is used?

The replacement will use the existing refspec's patterns for partitioning the refspace, except we add 'prefetch' and no longer insert the remote name ourselves. This allows us to have shorter refnames while respecting the given layout as much as possible.

> Also, I wonder if there should be a mechanism to prune the
> prefetched refs, but that is totally outside the scope of the
> problem we have been discussing in this thread.

 
Yes, this is a good idea. Prefetch refs are going to add the
"force" option (starts with '+') because these refs should be
updated automatically when the remote refs are force-pushed.
Having a mechanism for removing dropped refs is a good idea.

Thanks, -Stolee