[1.8.0] Summary of the discussions

It seems that not many things need breaking, but we will break some.

Here are the ones that needed discussion and were discussed that I am aware of, with my comments (which shouldn't be read as final decision).

The next step will be to design the migration path, and execution.

 * "git branch --set-upstream <name>" should not be about setting the
   upstream of <name> to the current branch.

   From: Jay Soffian <jaysoffian@gmail.com>
   Date: Tue, 1 Feb 2011 01:57:03 -0500
   Message-ID: <AANLkTinUn2SMijphe3EmPMVOOwBjPB5ffFwwqZVxQmW0@mail.gmail.com>

   -- summary --

   The current behaviour is misdesigned. It should be setting the upstream
   of the current branch to <name>.

   -- comments --

   I agree.

 * "git push" default semantics will be "upstream" (renamed from
   "tracking"), not "matching".
   
   From: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
   Date: Mon, 14 Feb 2011 22:57:24 +0100
   Message-ID: <vpqr5bath2z.fsf@bauges.imag.fr>

   -- comments --

   Ok.

 * "git merge" without "what to merge" default to @{upstream}

   From: Junio C Hamano <gitster@pobox.com>
   Date: Mon, 31 Jan 2011 12:50:30 -0800
   Message-ID: <7vzkqg4x2h.fsf_-_@alter.siamese.dyndns.org>

   -- summary --

   Instead of failing, "git merge" to behave more in line with "git pull".

   -- comments --

   I think this makes sense.

 * Unify pathspec semantics

   From: Junio C Hamano <gitster@pobox.com>
   Date: Mon, 31 Jan 2011 09:07:14 -0800
   Message-ID: <7voc6x57el.fsf_-_@alter.siamese.dyndns.org>
   
   -- summary --

   Some commands and operations understand leading-path-components but not
   globs, while many others do. This inconsistency leads to some surprises.
   Make everybody understand globs.

   -- comments --

   This is already underway, I think.

 * Making more operations full-tree by default.

   From: Jeff King <peff@peff.net>
   Date: Wed, 9 Feb 2011 18:46:21 -0500
   Subject: Re: "git add -u" broken in git 1.7.4?
   Message-ID: <20110209234621.GA12575@sigill.intra.peff.net>

   -- summary --

   These two commands currently affect only paths within the current
   working directory when run from a subdirectory without a pathspec:

   . "add -u"
   . "add -A"

   Make them to add everything in the working tree, from the root level,
   even when they are run from a subdirectory.

   The following are left as exceptions, keeping the current behaviour to
   work within the current directory given no pathspecs:

   . "grep"
   . "archive"
   . "clean"

   Plumbing like these are also left as exceptions.

   . "ls-files"
   . "ls-tree"

   -- comments --

   As long as transition plan is made carefully, I think this is fine.

 * Extend "separate remote" to tags and notes.

   From: Johan Herland <johan@herland.net>
   Date: Mon, 07 Feb 2011 04:35:31 +0100
   Message-ID: <201102070435.31674.johan@herland.net>

   -- summary --

   Instead of using refs/remotes/origin/$branch to store copies of
   branches, which has two downsides:

    - HEAD as the "primary branch pointer" and as a branch name could
      collide;
    - tags are forced into a single global namespace. 

   Use refs/remotes/origin/{HEAD,heads/$branch,tags/$tag} to track what
   we copy from a remote in separate namespaces.

   -- comments --

   Making this coexist with repositories created before 1.8.0 might be
   tricky, but otherwise this is fine.

 * "git checkout refs/heads/<name>" checks out <name> branch.

    From: Martin von Zweigbergk <martin.von.zweigbergk@gmail.com>
    Date: Mon, 7 Feb 2011 06:01:51 -0500 (EST)
    Message-ID: <alpine.DEB.1.10.1102062234010.3788@debian>

    -- summary --

    In addition to "git checkout <name>" that names an existing branch,
    special case "git checkout refs/heads/<name>" to check the branch out,
    instead of detaching HEAD at the tip of the branch.  If you want to
    detach at a specific commit and wanted to disambiguate <name> that is
    both a branch and another ref by checking out refs/heads/<name>, your
    workflow will be broken by this change, but you can still say
    refs/heads/<name>^0 to work it around.

    -- comments --

    I do not think this backward incompatible breakage would seriously
    hurt in practice, and if people see newbies are hurt by getting a
    detached HEAD when they check out refs/heads/<name>, this proposal
    might be an improvement, but I have a feeling that this may be trying
    to solve a wrong problem.

    Why are these people _tempted_ to type refs/heads/ explicitly?  It
    could be our tutorial or documentation.  That might be the thing to
    fix.

 * "git fetch $from $branch..." to update tracking branches

   From: Thomas Rast <trast@student.ethz.ch>
   Date: Mon, 31 Jan 2011 22:44:09 +0100
   Message-ID: <201101312244.10047.trast@student.ethz.ch>

   -- summary --

   Traditionally, giving explicit refspecs from the command line was a way
   to disable the tracking copy specified with "remote.<name>.fetch"
   configuration. A good workflow should however not rely on such a
   "pretending that a bad update on the other side did not happen".
   Always update tracking branches we observed with "git fetch".

   -- comments --

   I think this makes sense.

 * Moving files around in the source tree, thinning the top-level.

   From: Nicolas Pitre <nico@fluxnic.net>
   Date: Mon, 31 Jan 2011 15:28:37 -0500 (EST)
   Message-ID: <alpine.LFD.2.00.1101311459000.8580@xanadu.home>

   -- summary --

   well not concrete enough concensus to summarize other than "too many
   files at the top is considered ugly".

   -- comments --

   I am not tempted to do the heavy-lifting myself; as long as the
   resulting tree looks sane, I won't object ;-)

On Thu, 17 Feb 2011, Junio C Hamano wrote:

At first I was going to say that I don't think special casing it should be necessary. It seemed like as long as the name resolves to something like refs/heads/<name>, the branch should be checked out. Then I think I realized why you said to special case it. Since checkout favors branches to tags, we would first have to check if refs/heads/<arg> is valid and if it is not, check if the argument resolves to refs/heads/<name> (maybe ignoring tags called "heads/<name>").

It wasn't until recently that I learned that "git checkout [refs/]heads/<name>" (and rebase <upstream> [refs/]heads/name) detaches the head, so it wasn't really to try to help newbies that I proposed changing it. It was just that the current behavior is not what I think most people would expect.

/Martin

Am 18.02.2011 02:02, schrieb Junio C Hamano:

Sorry for not having posted this earlier:

Proposal: Add a new "on-demand" mode to fetch and pull and make it the default.

When using the new "on-demand" mode every time new commits are fetched in the superproject they will be parsed for submodule commits. If these commits aren't present in a populated submodule, run "git fetch" inside that submodule. (Also see this thread for an in depth discussion: http://article.gmane.org/gmane.comp.version-control.git/158979/ )

Additionally change "git submodule update" to only then run fetch in a submodule when the commit it wants to check out is not already present there.

This behavior can be configured per submodule, per repo and globally.

Advantages:
* Disconnected operation. Right now it is really easy to forget to fetch
  all submodules before you get on a plane, possibly leaving you unable
  to check out certain revisions.
* "git diff --submodule" (which is used by git gui and gitk) needs those
   submodule commits to provide meaningful output
* "git merge" needs them to be able to merge submodule commits
* This is the least surprising behavior for new submodule users
(The upcoming recursive checkout will rely on the commits being present
too)

Risks: People might be annoyed by the extra time it takes to fetch the submodules when they only expect the superproject to be fetched. Others might be surprised that "git submodule update" doesn't do a fetch anymore.

Migration plan: Announce the change in behavior and document how to easily configure the old behavior when wanted and enable this functionality in 1.7.5.

On 11-02-19 11:40 AM, Jens Lehmann wrote:

I support the proposal, but just to be clear I'd rephrase that advantage as:

Disconnected operation. Right now it is really easy to forget to update populated submodules before you get on a plane, possibly leaving you unable to check out certain revisions.

(Obviously, I think the only-works-on-already-populated-submodules part is very important.)

		M.

Am 21.02.2011 16:12, schrieb Marc Branchaud:

Thanks and yes, that is exactly what this proposal is about.

Proposal: When switching branches, merging or resetting the work tree of populated submodules should be checked out too according to the commit recorded in the superproject. Make this the default for porcelain and optional for plumbing.

The same safety precautions that are used for files in the superproject apply to the changes present in the submodules, no local modifications may be overwritten unless -f is used. When the "update" config option is set to "merge" or "rebase" the submodule will be left unchanged.

The "update" config option will learn a new value "none" to let the user turn off this behavior for single submodules. A global config option and the command line parameter "--recurse-submodules" will also be added. This change will remove the need to call "git submodule update" for all populated submodules (except those who use the "update=merge" or "update=rebase" configuration settings).

Advantages:
* This makes submodules behave like the files in the superproject.
  Every time the work tree of the superproject changes, the work
  trees of the populated submodules are updated accordingly.
* This is the least surprising behavior for new submodule users.

Risks: The commands might run longer, for those users who use submodules to gain performance by putting large and/or many files into submodules this may be unacceptable. Also people might be surprised by submodule work trees changing without explicitly invoking "git submodule update". These commands will now fail in the presence of changed submodules where they would have succeeded before.

Migration plan: Announce the change in behavior and document how to easily configure the old behavior when needed and enable this functionality in 1.8.0.

Jens Lehmann <Jens.Lehmann@web.de> writes:

Nobody would object to make this facility available via config, I think, but I suspect that it is way too premature to decide which way should eventually be the default at this point.

My recommendation is to ease this in as an optional feature you can easily turn on with a config, let people try it for a few release cycles and then if we reach the concensus that this should be on by the default, start the usual "warn and tell how to keep the old behavior" one cycle before 1.8.0 (or 1.9.0 if we end up not having enough cycles before 1.8.0).

On 11-02-19 11:59 AM, Jens Lehmann wrote:

A big +1 from me for all the submodule work -- thanks Jens!

Could you clarify the proposal a bit? A few questions occurred to me as I read it:

* How is the initial set of populated submodules set up during a clone
operation?  Specifically, how would the origin repo specify which submodules
to recursively clone?  (My understanding is that the origin's .gitmodules
file, as it exists in whatever branch is being cloned, would specify
submodule.<name>.update values, and those would drive the recursion.)

* Which values of submodule.<name>.update would enable/disable recursion
during a clone?  Would just "checkout" enable it, or should "merge" and
"rebase" also trigger recursion when cloning?

* What happens when a clone's user manually populates one of the other
submodules that wasn't part of the initial set?  Automatic recursion into
this newly-populated submodule is controlled by the setting of the global
recurse-submodules option, right?

* What are the possible values of the global recurse-submodules option?
Here's what I came up with:

  all       -- Always recurse
  populated -- Only recurse into *all* currently-populated submodules
  respect   -- Respect each submodule's "update" option (better name?)
  none      -- Never recurse

* What will happen when I start checked out at commit A, with a populated
submodule, then check out commit B where that submodule doesn't exist, then
return to commit A?  How will whatever recursion settings I had at the start
be preserved?

		M.

Am 21.02.2011 17:13, schrieb Marc Branchaud:

That is what I have in mind. I tend towards updating all populated submodules on local operations (like switching branches, merging and so on) unless configured otherwise, while for cloning me thinks an explicit "submodule.<name>.update=checkout" or such should be necessary.

(But please note that cloning was not part of my proposal, I was only talking about the local operations for now ;-)

"merge" and "rebase" could do that too, but wait: That would make it impossible to say "I want his submodule merged/rebased but *not* cloned" ... Hmm, good point, I'll have to think about that some more ...

There will be a global and a per-submodule configuration. But yes, if the .git/config or .gitmodules don't specify anything for this submodule the global config will kick in. And if that isn't set I imagine that porcelain by default will recurse into populated submodules while plumbing won't.

For porcelain I tend to unify "all", "populated" and "respect": recurse into all populated submodules unless configured otherwise ("all" seems kind of superfluous, as I would respect the users choice not to populate a submodule after the initial clone). But for plumbing a "respect" option makes sense, using it could tell it to use the same defaults and config that porcelain uses to make writing scripts easier.

I think the same option that controls the cloning of submodules should control whether a new submodule will be populated or not. For submodules that already existed despite that it might be nice to remember and respect the users choice and restore it if it existed before.

Maybe I'll have to mention the next steps not covered by my proposal:

2) Checkout submodules on clone

3) Move .git-Directories into the .git of the superproject (this will
   enable us to delete and recreate submodules without cloning again or
   loosing local branches and/or tags).

After thinking about it again I now tend to agree with Junio that we should bring in the functionality without enabling it by default first and then later decide if and to what we change the defaults. So I draw back this proposal and will post it again when all three steps are implemented and can be properly evaluated.

Thanks both for your feedback!

On 11-02-21 01:30 PM, Jens Lehmann wrote:

Ooops, sorry for jumping the gun! I figured that since clone normally does a checkout that this would have to be worked out as part of the proposal.

A lot of what I'm confused about is how git would distinguish between 2 things:

 - How the user specifies submodule recursion within a local repo.

 - How a repo specifies initial submodule recursion for clones.

I'm happy to wait for your follow-up work to discuss this. Cloning aside, what you're proposing looks good to me.

I haven't thought enough about plumbing to tell if it makes sense to configure plumbing behavior explicitly. But at first glance it seems odd...

So, .gitmodules initially controls recursion. When a submodule gets populated, it gets an entry in .git/config which then determines the recursion behavior from then on. Changing branches might change .gitmodules, but anything in .git/config will persist so any customizations the user makes will also persist.

Sounds good to me!

		M.

Am 21.02.2011 20:56, schrieb Marc Branchaud:

Cool. When I'm done with the local repo stuff I'll continue working on the cloning part, so stay tuned for further discussions on that topic :-)

Plumbing shouldn't care about user config by default because the aim is to have predictable behavior. But e.g. git gui, which is using plumbing under the hood to do the actual checkout, should honor user configuration and porcelain defaults (because it is percieved by the user as being porcelain itself). So git gui would benefit from being able to simply tell plumbing "behave like you were porcelain" through an option instead of coding recursion, user configuration and the safety checks needed before checkout itself. And this would avoid code duplication too.

Yes. Upstream can give sane defaults but the user has the last word.

Great to hear!

Jens Lehmann <Jens.Lehmann@web.de> writes:

If .git/config entries are created _automatically_ based on the initial .gitmodules, doesn't that means it will cease to respect the upstream defaults in the case where .gitmodules changes, even if the user didn't actually change anything himself...?

-Miles

On 02/21/2011 07:51 PM, Miles Bader wrote:

Also consider that the .gitmodules' "upstream defaults" may have many changes within the existing repository. So, $(git checkout foo) will load some "defaults" and $(git checkout bar) may load different "defaults". I guess the defaults can be migrated into .git/config using $(git submodule init)...?

Phil

Am 22.02.2011 03:32, schrieb Phil Hord:

Yes, unfortunately that is the current behavior of the git-submodule command, it copies the "update" entry into .git/config on init (but not on sync, so it is not updated then if it changed). The other submodule flags ("ignore" & "fetchRecurseSubmodules" don't show that behavior, the user has to copy them herself to override upstream settings). I'm in favor of removing that copy and using the setting from .gitmodules when "update" is not found in .git/config, but this changes behavior. But maybe that is a worthwhile change for 1.8.0 preparing for the things to come?

Yes, and that is a feature. E.g. when you have a "working on foo" branch you can set the "fetchRecurseSubmodules" to "yes" there because you want to get all commits in the submodule to be able to commit some new ones in the superproject. But in every other branch setting that to "on-demand" will suffice.

Not with "git submodule init", as they then would be set in stone from the beginning. But I can imagine a "git submodule config <setting>" helper program, which copies the setting into .git/config or updates it there when wanted.