threads / discuss / 46611

git fetch with refspec does not include tags?

Subject: git fetch with refspec does not include tags?

## tl;dr

12 messages between Aug 17, 2017 and Aug 20, 2017.

replies: 11people: 4as markdown or json

Carlsson, Magnus· Aug 17, 2017, 09:02 UTC · lore

Hi

In the git fetch documentation it states that by default you will fetch all tags that point into the history to the branches fetched.

"By default, any tag that points into the histories being fetched is also fetched; the effect is to fetch tags that point  at branches that you are interested in. This default behavior can be changed by using the --tags or --no-tags options or by configuring remote.<name>.tagOpt. By using a refspec that fetches tags explicitly, you can fetch tags that do not point into branches  you are interested in as well."

But for me I get tags if I do "git fetch" or "git fetch origin" but if I do "git fetch origin master" I don't get tags related to the master branch.

I understand that this might be due to me specifying a refspec and then it will only get that exact refspec, but for me it's not that clear from the documentation what I should expect. I read it as when I fetch something all related tags will come along.

Using: git version 2.13.0.rc1.15.gd2bbb7c

-- Magnus

MAGNUS CARLSSON Staff Software Engineer ARRIS

o: +46 13 36 75 92 e: magnus.carlsson@arris.com w: www.arris.com

ARRIS:  Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed.  It is confidential and may be attorney-client privileged.  In any event the Sender reserves, to the fullest extent, any "legal  advice privilege".  Any further distribution or copying of this message is strictly prohibited.  If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).

Jeff King· Aug 17, 2017, 09:28 UTC · re: Carlsson, Magnus · lore

Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 09:02:52AM +0000, Carlsson, Magnus wrote:

Show 19 quoted lines

> In the git fetch documentation it states that by default you will
> fetch all tags that point into the history to the branches fetched.
> 
> "By default, any tag that points into the histories being fetched is
> also fetched; the effect is to fetch tags that point  at branches that
> you are interested in. This default behavior can be changed by using
> the --tags or --no-tags options or by configuring
> remote.<name>.tagOpt. By using a refspec that fetches tags explicitly,
> you can fetch tags that do not point into branches  you are interested
> in as well."
> 
> But for me I get tags if I do "git fetch" or "git fetch origin" but if
> I do "git fetch origin master" I don't get tags related to the master
> branch.
> 
> I understand that this might be due to me specifying a refspec and
> then it will only get that exact refspec, but for me it's not that
> clear from the documentation what I should expect. I read it as when I
> fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So I'll try to untangle what's happening at least if not the reasoning. :)

I think the problem is not that you have a refspec, but that your refspec has no destination. Looking at the fetch code, we seem to turn on autotags only when the destination is a "real" ref and not just the default FETCH_HEAD. Which sort-of makes sense. If you're doing a one-off into FETCH_HEAD, you probably don't want to create tags, even if you have the objects they point to.

But this is further complicated by the opportunistic tracking-ref updates. You can see some interesting behavior with a setup like this:

  git init parent
  git -C parent commit --allow-empty -m one &&
  git -C parent tag -m foo mytag

  git init child
  cd child
  git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

  1. Definitely these defaults are under-documented. I couldn't find
     them anywhere in git-fetch(1).

  2. If we continue to follow the "are we storing any refs" rule for the
     default, possibly it should expand to "did we store anything,
     including opportunistic tracking-ref updates".

-Peff

Carlsson, Magnus· Aug 17, 2017, 11:29 UTC · re: Jeff King · lore

Re: git fetch with refspec does not include tags?

Thanks for your quick answer!

>  1. Definitely these defaults are under-documented. I couldn't find
>    them anywhere in git-fetch(1).

Yes, some sort of explanation would be good, especially since one of the first sentences state that you always get relevant tags.

>  2. If we continue to follow the "are we storing any refs" rule for the
>     default, possibly it should expand to "did we store anything,
>     including opportunistic tracking-ref updates".

Personally I think that I would prefer to always fetch relevant tags. If I don't want tags I can simply use the "--no-tags"

-- Magnus

MAGNUS CARLSSON Staff Software Engineer ARRIS

o: +46 13 36 75 92 e: magnus.carlsson@arris.com w: www.arris.com

ARRIS:  Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed.  It is confidential and may be attorney-client privileged.  In any event the Sender reserves, to the fullest extent, any "legal advice privilege".  Any further distribution or copying of this message is strictly prohibited.  If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).

________________________________________
From: Jeff King <peff@peff.net>
Sent: Thursday, August 17, 2017 11:28
To: Carlsson, Magnus
Cc: git@vger.kernel.org
Subject: Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 09:02:52AM +0000, Carlsson, Magnus wrote:

Show 19 quoted lines

> In the git fetch documentation it states that by default you will
> fetch all tags that point into the history to the branches fetched.
>
> "By default, any tag that points into the histories being fetched is
> also fetched; the effect is to fetch tags that point  at branches that
> you are interested in. This default behavior can be changed by using
> the --tags or --no-tags options or by configuring
> remote.<name>.tagOpt. By using a refspec that fetches tags explicitly,
> you can fetch tags that do not point into branches  you are interested
> in as well."
>
> But for me I get tags if I do "git fetch" or "git fetch origin" but if
> I do "git fetch origin master" I don't get tags related to the master
> branch.
>
> I understand that this might be due to me specifying a refspec and
> then it will only get that exact refspec, but for me it's not that
> clear from the documentation what I should expect. I read it as when I
> fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So I'll try to untangle what's happening at least if not the reasoning. :)

But this is further complicated by the opportunistic tracking-ref updates. You can see some interesting behavior with a setup like this:

  git init parent
  git -C parent commit --allow-empty -m one &&
  git -C parent tag -m foo mytag

  git init child
  cd child
  git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

  1. Definitely these defaults are under-documented. I couldn't find
     them anywhere in git-fetch(1).

  2. If we continue to follow the "are we storing any refs" rule for the
     default, possibly it should expand to "did we store anything,
     including opportunistic tracking-ref updates".

-Peff

Jeff King· Aug 17, 2017, 14:22 UTC · re: Carlsson, Magnus · lore

Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 11:29:48AM +0000, Carlsson, Magnus wrote:

Show 6 quoted lines

> >  2. If we continue to follow the "are we storing any refs" rule for the
> >     default, possibly it should expand to "did we store anything,
> >     including opportunistic tracking-ref updates".
> 
> Personally I think that I would prefer to always fetch relevant tags.
> If I don't want tags I can simply use the "--no-tags"

I think it's a bit more complex because "git pull" uses "git fetch" under the hood. In fact, your "git fetch origin master" is exactly what gets run when you do:

  git pull origin master

That's maybe OK. But I think one-off pulls like:

  git pull https://example.com/repo.git master

probably wouldn't want it. I'd have to give it some thought.

-Peff

Junio C Hamano· Aug 17, 2017, 19:41 UTC · re: Jeff King · lore

Re: git fetch with refspec does not include tags?

Jeff King <peff@peff.net> writes:

Show 11 quoted lines

> I think it's a bit more complex because "git pull" uses "git fetch"
> under the hood. In fact, your "git fetch origin master" is exactly what
> gets run when you do:
>
>   git pull origin master
>
> That's maybe OK. But I think one-off pulls like:
>
>   git pull https://example.com/repo.git master
>
> probably wouldn't want it. I'd have to give it some thought.

I agree with both. If you have named remote, you presumably are keeping copies of their branches as remote-tracking branches, and it may be fine to follow tags. An explicit URL used for one-off should not grab anything but the named thing, I would think.

Junio C Hamano· Aug 17, 2017, 19:38 UTC · re: Jeff King · lore

Re: git fetch with refspec does not include tags?

Jeff King <peff@peff.net> writes:

Show 6 quoted lines

>   # no tags, we just populate FETCH_HEAD because of the bare URL
>   git fetch ../parent
>
>   # this does fetch tags, because we're storing the result according to
>   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
>   git fetch origin

The above two look good.

Show 6 quoted lines

>   # this doesn't fetch tags, as the main command is "just" populating
>   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
>   # refs/remotes/origin/master, so let's update it on the side" kicks
>   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
>   # not the tags. Weird.
>   git fetch origin master

Yes, it looks weird, but I suspect that it is probably more correct not to fetch tags in this case. I wonder if it would be a solution not to do the "on the side" thing---after all the user didn't tell us to update refs/remotes/origin/master with this command line.

Then not following tags will be in line with all of the above reasoning above.

>   # and this one does fetch tags, because we have a real destination.
>   git fetch origin master:foo

Yup, that is expected.

> So what I'd say is:
>
>   1. Definitely these defaults are under-documented. I couldn't find
>      them anywhere in git-fetch(1).

Yup.

>   2. If we continue to follow the "are we storing any refs" rule for the
>      default, possibly it should expand to "did we store anything,
>      including opportunistic tracking-ref updates".

That also is a workable way to make things consistent.

Kevin Daudt· Aug 17, 2017, 20:22 UTC · re: Junio C Hamano · lore

Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:

Show 22 quoted lines

> Jeff King <peff@peff.net> writes:
> 
> >   # no tags, we just populate FETCH_HEAD because of the bare URL
> >   git fetch ../parent
> >
> >   # this does fetch tags, because we're storing the result according to
> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
> >   git fetch origin
> 
> The above two look good.
> 
> >   # this doesn't fetch tags, as the main command is "just" populating
> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
> >   # refs/remotes/origin/master, so let's update it on the side" kicks
> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
> >   # not the tags. Weird.
> >   git fetch origin master
> 
> Yes, it looks weird, but I suspect that it is probably more correct
> not to fetch tags in this case.  I wonder if it would be a solution
> not to do the "on the side" thing---after all the user didn't tell
> us to update refs/remotes/origin/master with this command line.

Isn't that how git fetch used to behave, or am I misunderstanding what you mean? It used to be that git fetch <remote> <branch> would not update any remote tracking branches.

From the 1.8.4 release notes:

Show 7 quoted lines

> "git fetch origin master" unlike "git fetch origin" or "git fetch"
> did not update "refs/remotes/origin/master"; this was an early
> design decision to keep the update of remote tracking branches
> predictable, but in practice it turns out that people find it more
> convenient to opportunistically update them whenever we have a
> chance, and we have been updating them when we run "git push" which
> already breaks the original "predictability" anyway.

Junio C Hamano· Aug 17, 2017, 20:38 UTC · re: Kevin Daudt · lore

Re: git fetch with refspec does not include tags?

Kevin Daudt <me@ikke.info> writes:

Show 37 quoted lines

> On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:
>> Jeff King <peff@peff.net> writes:
>> 
>> >   # no tags, we just populate FETCH_HEAD because of the bare URL
>> >   git fetch ../parent
>> >
>> >   # this does fetch tags, because we're storing the result according to
>> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
>> >   git fetch origin
>> 
>> The above two look good.
>> 
>> >   # this doesn't fetch tags, as the main command is "just" populating
>> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
>> >   # refs/remotes/origin/master, so let's update it on the side" kicks
>> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
>> >   # not the tags. Weird.
>> >   git fetch origin master
>> 
>> Yes, it looks weird, but I suspect that it is probably more correct
>> not to fetch tags in this case.  I wonder if it would be a solution
>> not to do the "on the side" thing---after all the user didn't tell
>> us to update refs/remotes/origin/master with this command line.
>
> Isn't that how git fetch used to behave, or am I misunderstanding what
> you mean? It used to be that git fetch <remote> <branch> would not
> update any remote tracking branches.
>
> From the 1.8.4 release notes:
>
>> "git fetch origin master" unlike "git fetch origin" or "git fetch"
>> did not update "refs/remotes/origin/master"; this was an early
>> design decision to keep the update of remote tracking branches
>> predictable, but in practice it turns out that people find it more
>> convenient to opportunistically update them whenever we have a
>> chance, and we have been updating them when we run "git push" which
>> already breaks the original "predictability" anyway.

No, you are not misunderstanding anything. The "pretend that we immediately turned around and fetched" done by "git push" was already breaking the predictability, but the change in 1.8.4 made it even worse. I am saying that going back to the old behaviour may be one option to address the issue being discussed in this thread.

Kevin Daudt· Aug 17, 2017, 20:43 UTC · re: Junio C Hamano · lore

Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 01:38:36PM -0700, Junio C Hamano wrote:

Show 46 quoted lines

> Kevin Daudt <me@ikke.info> writes:
> 
> > On Thu, Aug 17, 2017 at 12:38:58PM -0700, Junio C Hamano wrote:
> >> Jeff King <peff@peff.net> writes:
> >> 
> >> >   # no tags, we just populate FETCH_HEAD because of the bare URL
> >> >   git fetch ../parent
> >> >
> >> >   # this does fetch tags, because we're storing the result according to
> >> >   # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
> >> >   git fetch origin
> >> 
> >> The above two look good.
> >> 
> >> >   # this doesn't fetch tags, as the main command is "just" populating
> >> >   # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
> >> >   # refs/remotes/origin/master, so let's update it on the side" kicks
> >> >   # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
> >> >   # not the tags. Weird.
> >> >   git fetch origin master
> >> 
> >> Yes, it looks weird, but I suspect that it is probably more correct
> >> not to fetch tags in this case.  I wonder if it would be a solution
> >> not to do the "on the side" thing---after all the user didn't tell
> >> us to update refs/remotes/origin/master with this command line.
> >
> > Isn't that how git fetch used to behave, or am I misunderstanding what
> > you mean? It used to be that git fetch <remote> <branch> would not
> > update any remote tracking branches.
> >
> > From the 1.8.4 release notes:
> >
> >> "git fetch origin master" unlike "git fetch origin" or "git fetch"
> >> did not update "refs/remotes/origin/master"; this was an early
> >> design decision to keep the update of remote tracking branches
> >> predictable, but in practice it turns out that people find it more
> >> convenient to opportunistically update them whenever we have a
> >> chance, and we have been updating them when we run "git push" which
> >> already breaks the original "predictability" anyway.
> 
> No, you are not misunderstanding anything.  The "pretend that we
> immediately turned around and fetched" done by "git push" was
> already breaking the predictability, but the change in 1.8.4 made it
> even worse.  I am saying that going back to the old behaviour may be
> one option to address the issue being discussed in this thread.
>

Ok. The reason I'm bring this up is because we often had to tell users in the irc channel that git fetch <remote> <branch> did not update the remote tracking branches, which confused them, so reverting back might reintroduce this confusion again.

Jeff King· Aug 20, 2017, 07:47 UTC · re: Kevin Daudt · lore

Re: git fetch with refspec does not include tags?

On Thu, Aug 17, 2017 at 10:43:12PM +0200, Kevin Daudt wrote:

Show 10 quoted lines

> > No, you are not misunderstanding anything.  The "pretend that we
> > immediately turned around and fetched" done by "git push" was
> > already breaking the predictability, but the change in 1.8.4 made it
> > even worse.  I am saying that going back to the old behaviour may be
> > one option to address the issue being discussed in this thread.
> 
> Ok. The reason I'm bring this up is because we often had to tell users
> in the irc channel that git fetch <remote> <branch> did not update the
> remote tracking branches, which confused them, so reverting back might
> reintroduce this confusion again.

Yeah, I don't think we want to go back to the original behavior. I agree that it is partially to blame for the inconsistency that started this thread, but I think on balance it has saved much more confusion than it has started. And we can address that inconsistency with better tag rules (like the "autofollow if we wrote any real refs" thing).

I don't have a patch for that yet, so if anybody feels like taking a look, it would be much appreciated.

-Peff

Jeff King· Aug 20, 2017, 07:50 UTC · re: Jeff King · lore

Re: git fetch with refspec does not include tags?

On Sun, Aug 20, 2017 at 03:47:28AM -0400, Jeff King wrote:

Show 8 quoted lines

> Yeah, I don't think we want to go back to the original behavior. I agree
> that it is partially to blame for the inconsistency that started this
> thread, but I think on balance it has saved much more confusion than it
> has started. And we can address that inconsistency with better tag rules
> (like the "autofollow if we wrote any real refs" thing).
> 
> I don't have a patch for that yet, so if anybody feels like taking a
> look, it would be much appreciated.

Also: I don't think we've seen a patch yet for documenting the current
auto-follow behavior.  Even if we don't make a behavior change, let's
not forget to improve that, which should be much less work. :)

-Peff

Junio C Hamano· Aug 20, 2017, 15:51 UTC · re: Jeff King · lore

Re: git fetch with refspec does not include tags?

Jeff King <peff@peff.net> writes:

Show 14 quoted lines

> On Sun, Aug 20, 2017 at 03:47:28AM -0400, Jeff King wrote:
>
>> Yeah, I don't think we want to go back to the original behavior. I agree
>> that it is partially to blame for the inconsistency that started this
>> thread, but I think on balance it has saved much more confusion than it
>> has started. And we can address that inconsistency with better tag rules
>> (like the "autofollow if we wrote any real refs" thing).
>> 
>> I don't have a patch for that yet, so if anybody feels like taking a
>> look, it would be much appreciated.
>
> Also: I don't think we've seen a patch yet for documenting the current
> auto-follow behavior.  Even if we don't make a behavior change, let's
> not forget to improve that, which should be much less work. :)

Thanks for two thoughtful follow-ups.