threads / discuss / 53983

avoiding fetching specific refs from a remote

Subject: avoiding fetching specific refs from a remote

## tl;dr

10 messages between Aug 4, 2020 and Aug 13, 2020.

replies: 9people: 5as markdown or json

Jacob Keller· Aug 4, 2020, 21:53 UTC · lore
Hi,

I've recently had a few repositories switch their default branch from master to main. Currently, for backwards compatibility these repositories have kept master around as a synonym for main. It's unclear when this will be removed.

This results in the annoying completion behavior where:

$git checkout ma<TAB> main master

Essentially, because both main and master have similar names, tab completion requires remembering to type one additional character than I am used to.

I'd like to be able to customize my pull so that I either (a) exclude the reference when performing a fetch, or (b) exclude the reference when tab completing.

I don't know if there is any mechanism already in place to do this, so I'm trying to figure out which one would be most preferable on the list?

something like adding a remote.<name>.excludeRefs which would allow specifying a set of references to exclude from matching the refspec..

I realize I could customize the refspec to list the set of things I want, but then I would miss any potential future branches until I updated my remote config again.

I would like to be able to remove this annoyance for myself and colleagues.

It seems to me the easiest solution would be the sort of exclude option, as this enables generally skipping out on specific refs, and could be useful beyond just this initial impetus.

Thanks, Jake

Jeff King· Aug 5, 2020, 06:37 UTC · re: Jacob Keller · lore

Re: avoiding fetching specific refs from a remote

On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
Show 9 quoted lines
> This results in the annoying completion behavior where:
> 
> $git checkout ma<TAB>
> main
> master
> 
> Essentially, because both main and master have similar names, tab
> completion requires remembering to type one additional character than
> I am used to.

Heh, I didn't think anybody considered that implication so far; the fact that they were similar names has generally been considered a positive. I agree it would be annoying.

> I'd like to be able to customize my pull so that I either (a) exclude
> the reference when performing a fetch, or (b) exclude the reference
> when tab completing.
Not quite (b), but if you:
  - delete any local "master" branch, if you haven't already
  - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"

then it shouldn't complete "master". The second is necessary because we'd try to complete the magic "git checkout master" that auto-creates the branch from the remote version.

Show 6 quoted lines
> something like adding a remote.<name>.excludeRefs which would allow
> specifying a set of references to exclude from matching the refspec..
> 
> I realize I could customize the refspec to list the set of things I
> want, but then I would miss any potential future branches until I
> updated my remote config again.

This is definitely a reasonable thing to want, and it has come up off and on over the years. One search term for the list archive is "negative refspecs", though it turns up a lot of useless hits when the two words are not directly adjacent.

This old thread might be worth reading:
  https://lore.kernel.org/git/20140124090104.GA396@x4/
and there is even a patch in there:
  https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/

but I didn't go over it carefully enough to know whether it is utter trash, or something that could be used as a starting point.

-Peff
Junio C Hamano· Aug 5, 2020, 16:22 UTC · re: Jeff King · lore

Re: avoiding fetching specific refs from a remote

Jeff King <peff@peff.net> writes:
Show 7 quoted lines
>> Essentially, because both main and master have similar names, tab
>> completion requires remembering to type one additional character than
>> I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.

FWIW, I've been saying that 'maint' will become harder to type all along ever since I heard 'main'.

https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/
Show 15 quoted lines
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.

I think the idea is interesting. I sometimes find negative pathspecs quite useful, and I suspect negative refspecs would also be.

Jacob Keller· Aug 5, 2020, 20:34 UTC · re: Junio C Hamano · lore

Re: avoiding fetching specific refs from a remote

On Wed, Aug 5, 2020 at 9:22 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 16 quoted lines
>
> Jeff King <peff@peff.net> writes:
>
> >> Essentially, because both main and master have similar names, tab
> >> completion requires remembering to type one additional character than
> >> I am used to.
> >
> > Heh, I didn't think anybody considered that implication so far; the fact
> > that they were similar names has generally been considered a positive. I
> > agree it would be annoying.
>
> FWIW, I've been saying that 'maint' will become harder to type all
> along ever since I heard 'main'.
>
> https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/
>

Right. I'm generally for the name change, and I like having the remote honor the old name as a compatibility at least for some duration.

Show 19 quoted lines
> > This is definitely a reasonable thing to want, and it has come up off
> > and on over the years. One search term for the list archive is "negative
> > refspecs", though it turns up a lot of useless hits when the two words
> > are not directly adjacent.
> >
> > This old thread might be worth reading:
> >
> >   https://lore.kernel.org/git/20140124090104.GA396@x4/
> >
> > and there is even a patch in there:
> >
> >   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
> >
> > but I didn't go over it carefully enough to know whether it is utter
> > trash, or something that could be used as a starting point.
>
> I think the idea is interesting.  I sometimes find negative
> pathspecs quite useful, and I suspect negative refspecs would also
> be.

I've found negative pathspecs incredibly useful in some contexts, (with my only issue being to remember that if you specify a negative pathspec, you must also specify a positive one i.e. "." or something)

I think I'll try to revive that one patch Peff pointed out and then see if it makes sense or needs to be redone.

Jeff King· Aug 5, 2020, 20:34 UTC · re: Junio C Hamano · lore

Re: avoiding fetching specific refs from a remote

On Wed, Aug 05, 2020 at 09:22:47AM -0700, Junio C Hamano wrote:
Show 14 quoted lines
> Jeff King <peff@peff.net> writes:
> 
> >> Essentially, because both main and master have similar names, tab
> >> completion requires remembering to type one additional character than
> >> I am used to.
> >
> > Heh, I didn't think anybody considered that implication so far; the fact
> > that they were similar names has generally been considered a positive. I
> > agree it would be annoying.
> 
> FWIW, I've been saying that 'maint' will become harder to type all
> along ever since I heard 'main'.
> 
> https://lore.kernel.org/git/xmqqtuz9tq30.fsf@gitster.c.googlers.com/

True, but I thought that was just a reason against "main" in git.git itself. Most repos don't have "maint" (and we already conflicted on "ma" with the existing name :) though I agree that having one as a prefix of the other makes completion doubly painful). The interesting bit here is that the old name probably _isn't_ going away immediately, so this would affect everybody.

-Peff
Jacob Keller· Aug 5, 2020, 20:31 UTC · re: Jeff King · lore

Re: avoiding fetching specific refs from a remote

On Tue, Aug 4, 2020 at 11:37 PM Jeff King <peff@peff.net> wrote:
Show 31 quoted lines
>
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
>
> > This results in the annoying completion behavior where:
> >
> > $git checkout ma<TAB>
> > main
> > master
> >
> > Essentially, because both main and master have similar names, tab
> > completion requires remembering to type one additional character than
> > I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.
>
> > I'd like to be able to customize my pull so that I either (a) exclude
> > the reference when performing a fetch, or (b) exclude the reference
> > when tab completing.
>
> Not quite (b), but if you:
>
>   - delete any local "master" branch, if you haven't already
>
>   - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"
>
> then it shouldn't complete "master". The second is necessary because
> we'd try to complete the magic "git checkout master" that auto-creates
> the branch from the remote version.
>

I don't like this approach because I like having the guessing options available in most repositories.

Show 23 quoted lines
> > something like adding a remote.<name>.excludeRefs which would allow
> > specifying a set of references to exclude from matching the refspec..
> >
> > I realize I could customize the refspec to list the set of things I
> > want, but then I would miss any potential future branches until I
> > updated my remote config again.
>
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.
>
Thanks, it looks like this is at least sensible.
> -Peff
Philippe Blain· Aug 5, 2020, 22:12 UTC · re: Jeff King · lore

Re: avoiding fetching specific refs from a remote

Hi Peff,
Show 15 quoted lines
> Le 5 août 2020 à 02:37, Jeff King <peff@peff.net> a écrit :
> 
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
> 
>> something like adding a remote.<name>.excludeRefs which would allow
>> specifying a set of references to exclude from matching the refspec..
>> 
>> I realize I could customize the refspec to list the set of things I
>> want, but then I would miss any potential future branches until I
>> updated my remote config again.
> 
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.

I tried searching "negative refspecs" (*with* the double quotes) and I get 20 results so it seems to work. Although interestingly it doesn't find this message you wrote I'm responding to because "negative" and "refspec" are on separate lines... I'm CC-ing Eric in case he knows more about this limitation.

Philippe.
Eric Wong· Aug 7, 2020, 08:04 UTC · re: Philippe Blain · lore

Re: avoiding fetching specific refs from a remote

Philippe Blain <levraiphilippeblain@gmail.com> wrote:
Show 10 quoted lines
> > Le 5 août 2020 à 02:37, Jeff King <peff@peff.net> a écrit :
> > This is definitely a reasonable thing to want, and it has come up off
> > and on over the years. One search term for the list archive is "negative
> > refspecs", though it turns up a lot of useless hits when the two words
> > are not directly adjacent.
> 
> I tried searching "negative refspecs" (*with* the double quotes) and I get 20 results so it 
> seems to work. Although interestingly it doesn't find this message you wrote I'm responding to 
> because "negative" and "refspec" are on separate lines... I'm CC-ing Eric in case he knows more
> about this limitation.

Initially I thought it was phrase searching being disabled on lore (Xapian 1.2, "chert" backend). However, I just tried on lore and it shows Jeff's message from 2020-08-05 6:37 UTC:

 https://lore.kernel.org/git/?q=%22negative+refspecs%22

So maybe it was a delay due to Xapian's MVCC mechanism; but that shouldn't be HOURS off (maybe minutes at most), assuming you tried to search shortly before you sent the message I'm replying to at 22:12 UTC...

My v1 instance at public-inbox.org (which uses Xapian 1.4 and the "glass" backend) shows it, too:

 https://public-inbox.org/git/?q=%22negative+refspecs%22

You could also try NEAR and ADJ (I don't have much experience with those): https://xapian.org/docs/queryparser.html

Jacob Keller· Aug 12, 2020, 22:04 UTC · re: Jeff King · lore

Re: avoiding fetching specific refs from a remote

On Tue, Aug 4, 2020 at 11:37 PM Jeff King <peff@peff.net> wrote:
Show 55 quoted lines
>
> On Tue, Aug 04, 2020 at 02:53:30PM -0700, Jacob Keller wrote:
>
> > This results in the annoying completion behavior where:
> >
> > $git checkout ma<TAB>
> > main
> > master
> >
> > Essentially, because both main and master have similar names, tab
> > completion requires remembering to type one additional character than
> > I am used to.
>
> Heh, I didn't think anybody considered that implication so far; the fact
> that they were similar names has generally been considered a positive. I
> agree it would be annoying.
>
> > I'd like to be able to customize my pull so that I either (a) exclude
> > the reference when performing a fetch, or (b) exclude the reference
> > when tab completing.
>
> Not quite (b), but if you:
>
>   - delete any local "master" branch, if you haven't already
>
>   - set $GIT_COMPLETION_CHECKOUT_NO_GUESS to "1"
>
> then it shouldn't complete "master". The second is necessary because
> we'd try to complete the magic "git checkout master" that auto-creates
> the branch from the remote version.
>
> > something like adding a remote.<name>.excludeRefs which would allow
> > specifying a set of references to exclude from matching the refspec..
> >
> > I realize I could customize the refspec to list the set of things I
> > want, but then I would miss any potential future branches until I
> > updated my remote config again.
>
> This is definitely a reasonable thing to want, and it has come up off
> and on over the years. One search term for the list archive is "negative
> refspecs", though it turns up a lot of useless hits when the two words
> are not directly adjacent.
>
> This old thread might be worth reading:
>
>   https://lore.kernel.org/git/20140124090104.GA396@x4/
>
> and there is even a patch in there:
>
>   https://lore.kernel.org/git/20140125013433.GA22336@sigill.intra.peff.net/
>
> but I didn't go over it carefully enough to know whether it is utter
> trash, or something that could be used as a starting point.
>
> -Peff

I dug up the patch and have applied it to master. One concern I have is how do we tell what side the negative refspec applies to? It seems like we'd need to be able to distinguish whether it applies to the refname on the remote, or the refname on the local branch. The patch as-is assumes that negative refspecs only ever have "src".. I guess for fetch that means the name of the ref on the remote, and for push that means the name of the ref on the local end?

I was trying to modify the patch to add support to the push code flows so that it was more complete and could be submitted. I'll be sending what I have as an RFC soon.

Thanks, Jake

Jeff King· Aug 13, 2020, 08:48 UTC · re: Jacob Keller · lore

Re: avoiding fetching specific refs from a remote

On Wed, Aug 12, 2020 at 03:04:09PM -0700, Jacob Keller wrote:
Show 7 quoted lines
> I dug up the patch and have applied it to master. One concern I have
> is how do we tell what side the negative refspec applies to? It seems
> like we'd need to be able to distinguish whether it applies to the
> refname on the remote, or the refname on the local branch. The patch
> as-is assumes that negative refspecs only ever have "src".. I guess
> for fetch that means the name of the ref on the remote, and for push
> that means the name of the ref on the local end?

Yeah, unlike a normal refspec that can have both a src and dst, there really is only one "side" to a negative refspec, because it is not going anywhere. So selecting by src makes the most sense to me. We could also allow selecting by dst, like:

  git fetch origin refs/heads/*:refs/remotes/origin/* \
    ^refs/heads/foo           \ (1) omit remote foo branch; implies src
    ^refs/heads/bar:          \ (2) omit remote bar branch by explicit src
    ^:refs/remotes/origin/baz   (3) omit remote baz branch by explicit dst

I probably wouldn't bother with (2) or (3) unless somebody really wants them. We can do (1) now and then extend later without loss of compatibility.

Another thing to think about is how to handle overlap. E.g., in:
  ^refs/heads/foo refs/heads/*:refs/remotes/origin/*
should:
  1. The first take precedence over the second because we apply
     positive, then negative?
  2. Or should it be first over second because the first is specific and
     the second is a wildcard?
  3. Or should it be second over first because later refspecs override
     earlier?

I don't have a real preference, and I think there are many self-consistent schemes you could come up with. But it probably makes sense to think it through so that we don't get stuck supporting a half-baked behavior later.

> I was trying to modify the patch to add support to the push code flows
> so that it was more complete and could be submitted. I'll be sending
> what I have as an RFC soon.
Sounds good. Thanks for picking this up!
-Peff

← back to recent threads