[1.8.0] Provide proper remote ref namespaces

On Wednesday 02 February 2011, Sverre Rabbelier wrote:

Here's my go at phrasing this in a proposal format. Feel free to revise and resend:

Proposal:

Currently, git stores remote refs in the local repo by default as follows:

  Remote repo    ->   Local repo
  ---------------------------------------------------------
  HEAD                refs/remotes/$remote/HEAD  (implicit)
  refs/heads/*        refs/remotes/$remote/*
  refs/tags/*         refs/tags/*                (implicit, autofollow)
  refs/replace/*      (TBD)
  refs/notes/*        (TBD)

Several users report that they are confused by the difference in how heads and tags are mapped, and by the implicit mappings that are not mentioned in the configured refspecs. Also, as users want to share ever more different types of refs (replace refs and notes refs have been discussed recently), the existing ref mappings (aka. refspecs) do not suggest a natural/intuitive mapping for the new ref types.

Instead, we should change the default ref mappings into the following:

  Remote repo    ->   Local repo
  --------------------------------------------------
  HEAD                refs/remotes/$remote/HEAD
  refs/heads/*        refs/remotes/$remote/heads/*
  refs/tags/*         refs/remotes/$remote/tags/*
  refs/replace/*      refs/remotes/$remote/replace/*
  refs/notes/*        refs/remotes/$remote/notes/*

In short, we make refs/remotes/$remote/* an actual mirror of selected parts of the remote's refs/* hierarchy. This provides consistent namespaces for remote refs that naturally allows adding new ref types in the future.

This change obviously affects our ref-handling code:

- Remote tags are now stored separate from local tags. When looking up a 
shorthand tag name (e.g. v1.7.4), we should consult local tags 
(refs/tags/v1.7.4) before remote tags (refs/remotes/*/tags/v1.7.4 [1]). See 
[2] for more details.

- Remote heads have moved into refs/remotes/$remote/heads/*, hence 
invalidating shorthand remote head names, like "origin/master". We should 
change the lookup code, so that a shorthand ref of the form "$remote/$head" 
where "$remote" happens to match a configured remote is eventually expanded 
into lookup for "refs/remotes/$remote/heads/$head" [3].

- We might want to generalize the handling of "$remote/$head" into allowing 
shorthands like "$remote/$tag", "$remote/$replace" and "$remote/$note" as 
well (provided, of course, that they match unambiguously).

- All fetch refspecs should be given explicitly.

Sub-proposal: While we are changing the default refspecs, we should also 
consider whether we want to keep the auto-following behavior that Git 
currently does for tags (don't fetch tags that refer to objects not 
otherwise fetched by another refspec). If we simply make an explicit 
"+refs/tags/*:refs/remotes/$remote/tags/*" refspec, we will lose the auto-
following behavior. If we do want to keep the auto-following behavior, we 
could for example add a "~" prefix to the refspec to trigger auto-following 
behavior (i.e. this refspec only applies to refs that happen to point at 
objects fetched by way of a different refspec). See 
http://thread.gmane.org/gmane.comp.version-control.git/160503/focus=160795 
for more details.

Risks:

Existing scripts/programs may make assumptions about the layout of remote refs without consulting the configured refspecs. However, such scripts/programs may also break today when non-default refspecs are used.

When remotes have conflicting tags (same tag name points to different objects), and the tag name does not exist locally (in refs/tags/*), looking up the shorthand tag name will result in an "ambiguous ref" error (instead of silently adopting whichever tag was fetched first). Although many consider this an improvement on the current behavior, there may be scenarios where this causes problems in external scripts/programs.

Existing scripts/programs that assume and depend on the current implicit refspecs (or the tag auto-following behavior), might encounter problems when we drop these in favor of explicit refspecs.

Migration plan:

The main part of this proposal is simply changing the default refspecs. As such, the proposal can be simulated in current Git versions by setting up custom refspecs according to the above table of ref mappings.

In v1.8.0, we should default to the new default refspecs when creating new remotes. However, existing remotes (created pre-v1.8.0) must continue to work as before, so we cannot simply remove the implicit refspecs (or tag auto-following). Instead we need to make sure that the implicit refspecs is NOT applied to the new-style remotes. Identifying new-style vs. old-style remotes can be done by looking at the refspec itself (old-style: "refs/remotes/$remote/*", new-style: "refs/remotes/$remote/heads/*"), or (worst case) by introducing a config variable specifying the desired behavior (defaulting to old-style).

When adding the new rules for looking up shorthand refs (described above), we should carefully verify that these won't cause regressions when applied to old-style refspecs in existing repos.

In a later major version we can consider removing the (by then ancient) implicit refspecs, and any other outdated compatibility measures code in our ref lookup code.

Have fun! :)

...Johan

[1]: The "refs/remotes/*/tags/v1.7.4" is not hardcoded, but rather the (default) result of mapping "refs/tags/v1.7.4" through each remote's refspecs as defined in the config.

[2]: When looking up a shorthand tag name (e.g. v1.7.4): If a local tag (refs/tags/v1.7.4) is found, then we have an unambiguous match. If no local tag is found, we look up the tag name in all configured remotes (using the method described in [1]). If the tag name exists in one or more remotes, and those remotes all agree on its ultimate object name (after applying e.g. ^{commit} or whatever is appropriate in the context of the lookup), then we also have an unambiguous match. However, if the tag name exists in multiple remotes, and they do NOT all agree on its ultimate object name, then the shorthand tag name is ambiguous and the lookup fails. The user can always resolve this ambiguity by creating a local tag (refs/tags/v1.7.4) pointing to the desired object.

[3]: As in [1], the "refs/remotes/$remote/heads/$head" is not hardcoded, but rather the result of mapping "refs/heads/$head" through the refspecs for $remote as defined in the config.