git-fetch: default globally to --no-tags

Hi,

I have one main concern, and I can envision a few ways that git could solve it. I also see that there has been some previous discussion on reltaed topics over the years, but as far as I can tell, nothing has actually been implemented to address my concern.

--- My concern ---

When I fetch from a remote repository, the only ways to prevent fetching tags are:

  1) git fetch --no-tags <name>

  2) git config remote.<name>.tagopt --no-tags; git fetch <name>

The former requires extra typing for a case that (arguably) should be the default. The latter requires configuration for each repository. This doesn't scale well, if a user desires this behavior on all repositories they use.

I'd prefer something like this, to change the default tag-fetching behavior globally:

  git config --global remote.tagopt --no-tags

The closest I see is that I can disable tag-fetching globally on a per-remote name basis:

  git config --global remote.<name>.tagopt --no-tags

This brings me to a potential bug report:

--- Bug ---

When trying to use the remote.<name>.tagopt configuration option globally, I get something like this:

  $ git config --global remote.test.tagopt --no-tags
  $ git remote update
  ...
  Fetching test
  fatal: 'test' does not appear to be a git repository
  fatal: Could not read from remote repository.

  Please make sure you have the correct access rights
  and the repository exists.
  error: Could not fetch test
  ...

Expected behavior: if the local repository does not have a remote named 'test', then no additional output should be printed. If the local repository has a remote named test, then it should be fetched with the --no-tags option.

Actual behavior: git prints warnings about the 'test' remote, just because there is no remote named 'test.'

--- Motivations ---

This is all motivated by the fact that tag namespacing is completely broken in git. Tags are globally namespaced, and in a true DVCS environment, any particular developer has no control over another developer's tag naming conventions. So this namespace can easily become polluted, reducing the usefulness of tags as a whole [1]. This problem seems to have been acknowledged, and proposals appeared a few years ago [2]. But I don't see any solution for tag namespacing.

So, my current approach is to try to limit the damage done by accidentally fetching tags. Unfortunately, I can't find any good supported mechanisms to help me in the previous two sections.

--- TL;DR ---

My email boils down to two questions:

  (A) Has there been progress on implementing a proposal like in [2]?

  (B) Can we allow disabling (auto)tag-fetching globally? Like:

        git config --global remote.tagopt --no-tags

Thanks, Brian

[1] I could elaborate with some horror stories, to describe how this
    becomes a day-to-day nuisance in using git for me, but I'll avoid
    the details for now. Please accept that this is a usability bug.

[2]
    http://article.gmane.org/gmane.comp.version-control.git/165799
    http://article.gmane.org/gmane.comp.version-control.git/165885

On Tue, Nov 18, 2014 at 07:05:23PM -0800, Brian Norris wrote:

Yes, there is no way to globally specify --no-tags. I don't see it as a problem to add something like that.

I don't understand this assumption, though. Why don't you want to fetch tags from the remote side? Wouldn't this break something as simple as:

  git clone git://git.kernel.org/pub/scm/git/git.git
  cd git
  git checkout v2.1.3

(Actually, no, it wouldn't, because clone side-steps the usual tag-following procedure, but wouldn't you expect the same thing to work after a fetch in an existing repository?)

(And now having read all the way to the bottom of your email, I at least see why you dislike tags. But I am not convinced that turning them off completely does not come with its own set of usability problems).

Yeah, that would make sense to me (I do not know _why_ you would want to do it, but it would be consistent with other parts of git). If you are adding a new config option, though, please do not follow the odd and confusing "tagopt" config value, and make a boolean remote.autofollowtags or something.

That is the expected behavior (sort of). Any time you define a single key in a remote, that remote "exists", whether there is a remote.*.url key for it or not. You can use that to set options for remotes you specify only as a URL, though I do not know how widely-used that feature is.

IOW, git is not complaining here that there is no remote named 'test'. It is complaining that you have defined a remote called 'test', whose URL defaults to 'test' (because you did not specify one in the config), and that there is no repo called test. If you defined it as:

  git config --global remote.git://github.com/git/git.tagopt --no-tags

that would "work" in the sense that "git remote update" would download git in every repository. ;)

I think one could argue that "git remote update" should probably skip remotes for which no url is defined (but we _do_ still want to treat them as remotes internally, so that when you "git fetch ...", we apply the config correctly).

I don't know if I would say "completely broken", as it mostly gets the job done. But yes, there are other ways of doing it that would have advantages.

Not really, no. If you'd like to work on it, I'm sure people would be happy to review patches.

That seems reasonable to me to use remote.* as defaults for all remotes.

-Peff

Brian Norris <computersforpeace@gmail.com> writes:

You usually have TL;DR at the beginning to help people save time; having it at the end forces the whole thing to be read and would not help anybody. ;-)

I do not think so, and also I do not agree that "mirror everybody else's ref hierarchy into separate namespaces" is necessarily a good idea, especially for tags, whose reason of existence is to give people a way to have anchoring points they agree on to have a shared world view necessary to move things forward.

In other words, talks in [2] are attempting to solve a wrong problem. The problem people want to solve is to have a mechanism to keep private anchoring points that are not necessarily shared project wide, which tags in refs/tags hierarchy is *not*.

Like it or not, tags are meant to be used for globally shared anchoring points and the whole machinery (e.g. "fetch" that auto-follows tags, "clone" that gives refs/tags*:refs/tags/* refspec) is geared towards supporting that use pattern, which will be broken by moving tags to per-remote namespace.

I can see "git tag --local foo" that creates refs/local-tags/foo and also adding a mechanism to propagate local-tags/ hierarchy just like heads/ hierarchy is propagated per-remote as a solution to that problem that does not break the "release tags" use case, though.

Using remote.<variable> as a fallback for remote.<remote>.<variable> may be a useful addition, not limited to <variable>==tagopt case.

This is a tangent, but it is an important one because we are talking about "tagopt" specifically. I think we should start deprecating "*.tagopt --[no-]tags". It started as a quick-and-dirty hack back when "git fetch" was a shell script Porcelain, where it made it easy to write things like this in its implementation:

	tagOpt=$(git config "remote.$name.tagopt")
        git fetch $tagOpt $name $args

which gives an impression that any command line option can go there (e.g. as if you could set "remote.*.tagopt = --frotz --no-tags") and "git fetch" implementation, even after it is redone in C, must forever parse it as if it is part of a shell command line (e.g. splitting at $IFS, unquoting the shell quotes and interpreting as if they came in argv[]).

This is ugly and simply unmaintainable, and we should transition away from that, by doing something like:

 (1) Add remote.*.tags configuration, which defaults to 'follow',
     but can be set to 'true' or 'false'.  Accept '--tags' as a
     synonym to 'true' and '--no-tags' as a synonym to 'false'.

     * when set to 'follow', allow auto-following tags (the default).
     * when set to 'true', act as if --tags is given.
     * when set to 'false', act as if --no-tags is given.

 (2) Deprecate remote.*.tagopt configuration.  When it is used, give
     a warning about deprecation and encourage users to move to
     remote.*.tags setting.

 (3) Wait for several release cycles.

 (4) Remove remote.*.tags configuration support at a major version
     boundary.

Needless to say, support for remote.<variable> as a fallback for remote.<remote>.<variable> for any <variable> can be done in parallel to this tangent topic.

On Wed, Nov 19, 2014 at 10:45:48AM -0800, Junio C Hamano wrote:

I am not sure I agree here that the discussions in [2] were not handling this case. Here you are arguing for the tag-writer to distinguish between two separate namespaces: global and local.

But I think the proposals in [2] were about pushing that logic into the lookup phase. That is, pulling in all of the remote's tags as "refs/remotes/origin/tags/*", and then at lookup time checking multiple locations for tags (and preferring your local "refs/tags" to what is pulled from a remote).

I think that system is better because it gives flexibility in resolution to the viewer of the tags, not the writer. E.g., consider a project that is merging two different sub-projects, project1 and project2. Each of the sub-projects has their own global namespace with v1.0, v2.0, etc. They would never use local-tags; these are meant to be in a per-project global namespace.

But the superproject is pulling them both together; if it uses refs/tags, the global namespaces will clash. Instead, it would be more convenitn to have refs/remotes/project1/tags and so on. And then a lookup of "v1.0" can either prefer project1 to project2, vice-versa, or even respect neither. But the point is that the desired outcome is in the eye of the beholder, not the writer.

Thanks for writing this out. I touched on it in the other email I sent, but did not explain it very well. The transition you mentioned here is exactly what I was thinking of.

-Peff

Jeff King <peff@peff.net> writes:

With a separate local-tags hierarchy, the look-up part still has to be enhanced. After doing "git tag v2.0" and "git tag -l snapshot00", you would want to be able to say "git log snapshot00..v2.0" and have these found.

If you don't allow a private local-tags hierarchy, then those who make releases are burdened to be very careful not to contaminate their public repository --- "git tag snapshot00" cannot be used by them lightly just to mark their private state, if their day typically is concluded with "git push --follow-tags", as that will push out the "tags" that are meant to be private.

Aside from the mantra that it is a good thing to reduce the workload for people closer to the center of the project (hence likely to be bottleneck to slow down others) and a separate local-tags hierarchy is a way to achieve that, a separate local-tags hierarchy also helps people who do not make releases by letting them easily differenciate the global tags they get from the project and the tags they and other project participants create as private markers (their own will be found in refs/local-tags, others' in refs/remotes/$pal/local-tags).

Yeah, but isn't it an orthogonal issue? refs/tags/project{1,2}/* would be what I would recommend to use for "global" stuff whose purpose is to give people a shared world view.

On Wed, Nov 19, 2014 at 12:22:36PM -0800, Junio C Hamano wrote:

It's not that I want to disallow a private local-tags hierarchy. It's just that I consider it a completely orthogonal feature. In other words, the problem of tags in a global namespace is not solved at all by local-tags. It just helps people keep things out of the global namespace that they did not want to be there in the first place[1]. It does nothing for viewers who need to coalesce multiple global tag namespaces (either from multiple projects, or "proposals" of global tags they have pulled in from other non-canonical repositories).

How do they get that split? They cannot use tag auto-following, which puts the tags in the global refs/tags.

Certainly it can be done with git _today_ by setting up the appropriate refspecs. I think this is more about making things useful out of the box.

-Peff

[1] I am actually not convinced that the mixing of local and global tags
    is a huge problem in practice. We do not push tags by default, so
    local tags tend to stay local. OTOH, I think the world would be a
    better place if "git push --follow-tags" were the default, which
    would entail somehow separating global from local tags.