git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] remote: make prune work for mixed mirror/non-mirror repos

From
Dennis Kaarsemaker <dennis@kaarsemaker.net>
Date
Jun 20, 2013, 23:07 UTC
Message-ID
<1371769636.17896.44.camel@localhost>
In-Reply-To
<7vppvgpfib.fsf@alter.siamese.dyndns.org>
(Sorry, I sent v2 before seeing this mail)
On do, 2013-06-20 at 15:46 -0700, Junio C Hamano wrote:
Show 24 quoted lines
> Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:
> 
> > When cloning a repo with --mirror, and adding more remotes later,
> > get_stale_heads for origin would mark all refs from other repos as stale. In
> > this situation, with refs-src and refs->dst both equal to refs/*, we should
> > ignore refs/remotes/* when looking for stale refs to prevent this from
> > happening.
> 
> I do not think it is a right solution to single out refs/remotes/*.
> 
> Going back to your original example:
> 
>     [remote "origin"]
>             url = git://github.com/git/git.git
>             fetch = +refs/*:refs/*
>             mirror = true
>     [remote "peff"]
>             url = git://github.com/peff/git.git
>             fetch = +refs/heads/*:refs/remotes/peff/*
> 
> Wouldn't you obtain "refs/remotes/github/html" from your "origin"
> via "git pull origin"?  What happens to your local copy of that ref,
> when it goes away from the origin and then you try to "fetch --prune
> origin" the next time with this patch (and without this patch)?

git pull origin gives me refs/html in this case. I did not try fetch --prune, but prune origin DTRT: if the html branch goes away at the origin, it goes away locally. Both with and without this patch.

It's refs/remotes/peff/somebranch that in this case *also* goes away without this patch, but is untouched with this patch

> What should happen?
Exactly that.
Show 5 quoted lines
> What if you had this instead of the above version of remote.peff.*?
> 
>     [remote "peff"]
>             url = git://github.com/peff/git.git
>             fetch = +refs/heads/*:refs/remotes/github/*
That doesn't change anything.
Show 16 quoted lines
> I think this is an unsolvable problem, and I _think_ the root cause
> of the issue is the configuration above that allows the RHS of
> different fetch refspecs to overlap.  refs/* is more generic and
> covers refs/remotes/peff/* and refs/remotes/github/*.  You cannot
> even know, just by looking at "origin" and your local repository,
> if refs/remotes/github/html you have should go away or it might have
> come from somewhere else.
> 
> The best we _could_ do, without contacting all the defined remotes,
> is probably to check each ref that we did not see from "origin" (for
> example, you find "refs/remotes/peff/frotz" that your origin does
> not have) and see if it could match RHS of fetch refspec of somebody
> else (e.g. RHS of "refs/heads/*:refs/remotes/peff/*" matches that
> ref).  Then we can conclude that refs/remotes/peff/frotz _might_
> have come from Peff's repository and not from "origin", and then we
> can optionally issue a warning and refrain from removing it.

I like that idea, though I also like the simplicity of simply singling out "remotes" as that's where normal remotes usually sit. And don't forget about tags (see patch v2).

Show 5 quoted lines
> This inevitably will have false positives and leave something that
> did originally came from "origin", because peff may no longer have
> 'frotz' branch in his repository.  I do not think we can do better
> than that, because we are trying to see if we can improve things
> without having to contact all the remotes.

But then the ref would have to be called "refs/remotes/peff/frotz" upstream. Hmm, that is of course completely possible: cloning something that's already a clone.

Show 18 quoted lines
> But if you go that route, the logic needs to go the same way when
> you are pruning against 'peff', and anything that you do not see in
> his repository right now but you have in refs/remotes/peff/ cannot
> be pruned, because it might have come from your origin via more
> generic refs/*:refs/* mapping.  It follows that you could never
> prune anything under refs/remotes/peff/* hierarchy.
> 
> You could introduce a "assume that more specific mapping never
> overlaps with a more generic mapping" rule (i.e. refs/* from RHS of
> remote.origin.fetch is more generic than refs/remotes/peff/* from
> RHS of remote.peff.fetch, and assume everything that you see in your
> local refs/remotes/peff/* came from peff and not from origin, I
> think, but at that point, is it worth the possible complexity to
> code that rule in the prune codepath and brittleness of that
> assumption that your origin will never add a new ref under that
> hierarchy, e.g. refs/remotes/peff/xyzzy?
> 
> So, I dunno.
Yeah, I'm starting to think this is not such a good idea. How about plan
B: issuing a warning when adding a remote with a refspec that also
matches another remote's refspec?

Or plan C: add a per-remote pruneIgnore setting that in this case I could set to refs/tags/* refs/remotes/* as I know it's correct? Could even be combined with plan B.

-- 
Dennis Kaarsemaker
www.kaarsemaker.net
Previous: Junio C HamanoNext: Junio C Hamano
Message 4 of 23 in “[BUG?] remote prune origin interacts badly with clone --mirror and multiple remotes”
  1. Dennis KaarsemakerJun 20, 2013
  2. remote: make prune work for mixed mirror/non-mirror reposDennis Kaarsemaker, Jun 20, 2013
  3. Junio C HamanoJun 20, 2013
  4. Dennis KaarsemakerJun 20, 2013
  5. Junio C HamanoJun 20, 2013
  6. Dennis KaarsemakerJun 20, 2013
  7. Junio C HamanoJun 20, 2013
  8. Jeff KingJun 20, 2013
  9. Dennis KaarsemakerJun 20, 2013
  10. Junio C HamanoJun 20, 2013
  11. remote: make prune work for mixed mirror/non-mirror reposDennis Kaarsemaker, Jun 20, 2013
  12. 0/3 Handling overlapping refspecs slightly smarterDennis Kaarsemaker, Jun 21, 2013
  13. 1/3 remote: Add warnings about mixin --mirror and other remotesDennis Kaarsemaker, Jun 21, 2013
  14. Junio C HamanoJun 21, 2013
  15. Dennis KaarsemakerJun 23, 2013
  16. Junio C HamanoJun 23, 2013
  17. Dennis KaarsemakerJun 23, 2013
  18. Junio C HamanoJun 23, 2013
  19. Dennis KaarsemakerJun 26, 2013
  20. Junio C HamanoJun 26, 2013
  21. 2/3 remote: Add test for prune and mixed --mirror and normal remotesDennis Kaarsemaker, Jun 21, 2013
  22. 3/3 remote: don't prune when detecting overlapping refspecsDennis Kaarsemaker, Jun 21, 2013
  23. Junio C HamanoJun 21, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.