v4, 4 partsRe: [PATCH v4 0/4] git-submodule add: Add --local-branch option (summary)

On Tue, Nov 27, 2012 at 09:42:05PM -0500, W. Trevor King wrote:

Heiko pointed out that I likely hadn't looked at this branch at all, which is true. I'd confused it with his earlier:

On Fri, Nov 09, 2012 at 05:29:27PM +0100, Heiko Voigt wrote:

but the new floating_submodules_draft branch adds Heiko's function reorganization and floating submodule pull (with 'update --branch') on top of my v4 commits (instead of my branch-checkout with 'update --branch').

I do feel better about them today. To get a better feel for how people see this going forward, here is a summary of proposals to date:

v1. Record submodule.<name>.branch with 'submodule add --branch …',
    leave interpretation up to the user.

    Feedback (paraphrased):
      Phil Hord: that clobbers a configure option used by Gerritt and
        possibly others, rename to --record-branch.  Maybe we should
        export submodule.<name>.* as $submodule_* in 'foreach'?
      Nahor: what about corner cases (e.g. upstream renames branches)?
      Jens Lehmann: if you record it, people will expect Git to use
        it.  What about automatic pulls & commits?

v2. Add --record instead of using --branch to address Phil's v1
    comment.

    Feedback (paraphrased):
      Jens Lehmann: this is still a tiny optimization over using 'git
        config'.  People still use this option in different ways.
      Shawn Pearce: the Gerrit use is basically the same as Ævar's,
        but Gerrit has special handling for '.'.
      Jeff King: if we set up submodule.<name>.branch, we should
        either use it in Git, or make it very clear that we do not use
        it.  If we use Phil's $submodule_* export, we should clear the
        variables for recursive submodules.

v3. Renamed Added $submodule_* export to v2.  Give an example of
    Ævar's pull workflow when explaining that Git does not use the
    value internally.

    Feedback (paraphrased):
      Heiko Voigt: what about automatic pulls & commits?  You should
        allow .git/config overrides of the .gitmodules settings.  if
        we set up submodule.<name>.branch, we should use it in Git.
        How does the submodule know which remote branch to track?
        Junio's proposed 'diff' changes may hide $sha1 information.
      Junio C Hamano: if we set up submodule.<name>.branch, we should
        use it in Git.  Use something more specific than --record.
        Update 'git diff [$path]' and friends in the superproject
        compares the HEAD of the checkout of the submodule at $path
        with the tip of the branch named by submodule.$name.branch in
        .gitmodules of the superproject, instead of the commit that is
        recorded in the index of the superproject.
      Sascha Cunz: you can use 'git shortlog $OLD_SHA1..$NEW_SHA1' for
        the automatic commit message.
      Trevor King: actually, Ævar's update command only specifies the
        local branch name.  The remote is recorded for that branch in
        .git/modules/<name>/config.

v4. Rename --record to --local-branch.  Remove $submodule_* export.
    Add .git/config overrides for submodule.<name>.branch.  Add
    'submodule update --branch' to check out $sha1 as
    submodule.<name>.branch.

    Feedback (paraphrased):
      Heiko Voigt: who cares about the local branch name?  You should
        be pulling origin/$branch, but still using
        .git/modules/<name>/config to record the tracked branch.  You
        should also use 'submodule add --branch[=branch]' instead of
        '--local-branch'.  You should pull the 'update --branch'
        functionality out into a sub-function like
        'handle_tracked_branch_update'.
      Jens Lehmann: .git/config overrides are nice, but they should be
        optional.
      Trevor King: floating branches are following the submodule's
        remote, while 'update --rebase/--merge' are following the
        superproject's recorded $sha1.  Bundling remote-following and
        superproject-following in the same command may be confusing.

Here's what I think we need in v5:

* Jens' optional .git/config overrides.
* Return --local-branch handling to a side effect of --branch (as in
  v1).
* A new 'submodule pull' for tracking the submodule's remote, which is
  pulling --ff-only origin/$branch into a whatever state the submodule
  is currently in.  If any changes were made to submodule $shas,
  optionally commit them with a shortlog-generated commit message.
* Remove my current 'submodule update --branch' functionality.

Thoughs?  Do we need any more information for an automatic pull, or is

  $ git pull --ff-only origin/$branch

on a headless checkout sufficient?

Trevor

On Wed, Nov 28, 2012 at 08:09:03AM -0500, W. Trevor King wrote:

I thought of a better idea on the train. How about adding `--remote` to `submodule update` that overrides the gitlinked SHA-1 with the SHA-1 for origin/$branch? All of the other checkout/merge/rebase functionality is untouched. The only thing that changes is where we look for the update. I'm working up the patch now, and will submit v5 shortly.

From: "W. Trevor King" <wking@tremily.us>

On Wed, Nov 28, 2012 at 11:53:34AM -0500, W. Trevor King wrote:

Here it is.

Changes since v4:

* Remove `update --branch` in favor of the new `update --remote` logic
  as mentioned above.
* Optional config overrides for submodule.<name>.branch (as suggested
  by Jens)
* Save --branch as submodule.<name>.branch instead of requiring
  --local-branch.
* Restructure doc edits.

I'm a lot happier with this two-patch proposal. The first patch implements --remote and documents submodule.<name>.branch. The second patch just gives you a convenient way to set it. I haven't heard from anyone other than Heiko recently about the --branch/--remote-branch choice, so I returned to the simpler --branch side effect storage from v1. I'd be happy to submit v6 with explicit --remote-branch recording if desired.

W. Trevor King (2):
  submodule update: add --remote for submodule's upstream changes
  submodule add: If --branch is given, record it in .gitmodules

 Documentation/config.txt        |  9 +++++----
 Documentation/git-submodule.txt | 26 +++++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 30 +++++++++++++++++++++++++++++-
 t/t7400-submodule-basic.sh      |  1 +
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 6 files changed, 96 insertions(+), 6 deletions(-)

From: "W. Trevor King" <wking@tremily.us>

The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject.

However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly:

  $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull'

With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule.<name>.branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.).

Previous use of submodule.<name>.branch =======================================

Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in

  commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f
  Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
  Date:   Fri May 21 16:10:10 2010 +0000

    git-submodule foreach: Add $toplevel variable

Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name.

Although the --remote functionality is using `submodule.<name>.branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules/<name>/config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule.<name>.branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow.

Implementation details ======================

In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch:

  if test -z "$nofetch"
  then
    # Run fetch only if $sha1 isn't present or it
    # is not reachable from a ref.
    (clear_local_git_env; cd "$path" &&
      ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) &&
       test -z "$rev") || git-fetch)) ||
    die "$(eval_gettext "Unable to fetch in submodule path '\$path'")"
  fi

There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/config.txt        |  9 +++++----
 Documentation/git-submodule.txt | 24 +++++++++++++++++++++++-
 Documentation/gitmodules.txt    |  5 +++++
 git-submodule.sh                | 26 +++++++++++++++++++++++++-
 t/t7406-submodule-update.sh     | 31 +++++++++++++++++++++++++++++++
 5 files changed, 89 insertions(+), 6 deletions(-)

From: "W. Trevor King" <wking@tremily.us>

This allows you to easily record a submodule.<name>.branch option in .gitmodules when you add a new submodule. With this patch,

  $ git submodule add -b <branch> <repository> [<path>]
  $ git config -f .gitmodules submodule.<path>.branch <branch>

reduces to

  $ git submodule add -b <branch> <repository> [<path>]

This means that future calls to

  $ git submodule update --remote ...

will get updates from the same branch that you used to initialize the submodule, which is usually what you want.

Signed-off-by: W. Trevor King <wking@tremily.us>
---
 Documentation/git-submodule.txt | 2 ++
 git-submodule.sh                | 4 ++++
 t/t7400-submodule-basic.sh      | 1 +
 3 files changed, 7 insertions(+)