threads / patch / 35608

patch, 2 partsgit-submodule.sh: Support 'checkout' as a valid update command

Subject: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

## tl;dr

40 messages between Jan 5, 2014 and Jan 15, 2014. Diffs are folded; open one to read it.

replies: 39people: 6as markdown or json

Francesco Pretto· Jan 5, 2014, 02:50 UTC · lore
According to "Documentation/gitmodules.txt", 'checkout' is a valid
'submodule.<name>.update' command. Also "git-submodule.sh" refers to
it and processes it correctly. Reflect commit 'ac1fbb' to support this
syntax and also validates property values during 'update' command,
issuing a warning if the value found is unknwon.
---
 git-submodule.sh | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
Show changes to git-submodule.sh +13 −1
diff --git a/git-submodule.sh b/git-submodule.sh
index 2677f2e..1d041a7 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -622,7 +622,7 @@ cmd_init()
 		   test -z "$(git config submodule."$name".update)"
 		then
 			case "$upd" in
-			rebase | merge | none)
+			checkout | rebase | merge | none)
 				;; # known modes of updating
 			*)
 				echo >&2 "warning: unknown update mode '$upd' suggested for submodule '$name'"
@@ -805,6 +805,18 @@ cmd_update()
 			update_module=$update
 		else
 			update_module=$(git config submodule."$name".update)
+			case "$update_module" in
+			'')
+				;; # Unset update mode
+			checkout | rebase | merge | none)
+				;; # Known update modes
+			!*)
+				;; # Custom update command
+			*)
+				update_module=
+				echo >&2 "warning: invalid update mode for submodule '$name'"
+				;;
+			esac
 		fi
 
 		displaypath=$(relative_path "$prefix$sm_path")
-- 
1.8.5.2.230.g032cd47.dirty
Francesco Pretto· Jan 5, 2014, 02:50 UTC · re: Francesco Pretto · lore

[PATCH 2/2] Introduce git submodule attached update

At the current state, the following use-case is not supported very
well in git:
- a maintainer adds a submodule, checking out a specific branch of
the repository. He doesn't track the upstream submodule revision sha1;
- a developer checkout the repository branch decided by the maintainer.
Subsequent "merge" or "rebase" update operations don't detach the HEAD.
To ease the above use-case this patch:
- introduces a "submodule.<module>.attached" property that, when set
  to "true", ensures that the "update" operation will result in
  the HEAD attached to a branch;
- introduces "--attach|--dettach" switches to the submodule "update"
  command: they attach/detach the HEAD, overriding
  "submodule.<module>.attached" property value;
- introduces "--attached-update" switch to the "add" operation. It:
    * sets "submodule.<module>.attached" to true;
    * sets "submodule.<module>.ignore" to all.
Using the '--attach' switch or operating in a repository with
'submodule.<name>.attached' set to 'true' during "update" will:
- checkout a branch with an attached HEAD if the repository was just
cloned;
- perform a fast-forward only merge of changes if it's a 'checkout'
update operation;
- reattach the HEAD prior performing a 'merge', 'rebase' or '!command'
update operation if the HEAD was found detached. Orphaned commits
will also be merged back in the branch.
'--attach' or 'submodule.<name>.attached' set to true also implies '--remote'.
Using  the '--detach' switch or operating in a repository with
'submodule.<name>.attached' set to 'false' during "update" will:
- checkout a detached HEAD if the repository was just cloned;
- detach the HEAD prior performing a 'merge', 'rebase' or '!command'
update operation if the HEAD was found attached.

'submodule.<name>.attached' works similarly to 'submodule.<name>.update' property: git copies the values found in ".gitmodules" in ".git/config" when performing an "init" command. "update" looks for values in ".git/config" only.

'--attach' and '--detach' switches override an opposite behaviour of 'submodule.<name>.attached' properties.

The patch is strongly additive and doesn't break any submodule specific
test. It also adds some tests specific to the added feature.
---
 Documentation/git-submodule.txt    |  48 +++++--
 Documentation/gitmodules.txt       |  10 +-
 git-submodule.sh                   | 154 +++++++++++++++++++--
 t/t7410-submodule-attached-head.sh | 268 +++++++++++++++++++++++++++++++++++++
 4 files changed, 457 insertions(+), 23 deletions(-)
 create mode 100755 t/t7410-submodule-attached-head.sh
Show changes to 4 files +457 −23

Documentation/git-submodule.txt, Documentation/gitmodules.txt, git-submodule.sh, t/t7410-submodule-attached-head.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index bfef8a0..b97eefb 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -10,13 +10,14 @@ SYNOPSIS
 --------
 [verse]
 'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
-	      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
+	      [--reference <repository>] [--attached-update] [--depth <depth>]
+	      [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
-	      [-f|--force] [--rebase] [--reference <repository>] [--depth <depth>]
-	      [--merge] [--recursive] [--] [<path>...]
+	      [-f|--force] [--rebase] [--reference <repository>] [--attach | --detach]
+	      [--depth <depth>] [--merge] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
@@ -107,6 +108,9 @@ is the superproject and submodule repositories will be kept
 together in the same relative location, and only the
 superproject's URL needs to be provided: git-submodule will correctly
 locate the submodule using the relative URL in .gitmodules.
++
+If `--attached-update` is specified, the property `submodule.<name>.attached`
+will be set to `true` and `submodule.<name>.ignore` will be set to `all`.
 
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
@@ -156,12 +160,15 @@ it contains local modifications.
 update::
 	Update the registered submodules, i.e. clone missing submodules and
 	checkout the commit specified in the index of the containing repository.
-	This will make the submodules HEAD be detached unless `--rebase` or
-	`--merge` is specified or the key `submodule.$name.update` is set to
-	`rebase`, `merge` or `none`. `none` can be overridden by specifying
-	`--checkout`. Setting the key `submodule.$name.update` to `!command`
-	will cause `command` to be run. `command` can be any arbitrary shell
-	command that takes a single argument, namely the sha1 to update to.
+	This will make the submodules HEAD be detached unless `--attach` is
+	specified or `submodule.$name.attached` is set to `true`. The last setting
+	can always be overridden specifying `--detach`. Update mode can be
+	selected specifying `--checkout`, `--rebase` or `--merge` switches
+	or setting the key `submodule.$name.update` to `checkout`, `rebase`,
+	`merge` or `none`. `none` will cause the submodule to be skipped during
+	the update. Setting the key `submodule.$name.update` to `!command` will
+	cause `command` to be run. `command` can be any arbitrary shell command
+	that takes a single argument, namely the sha1 to update to.
 +
 If the submodule is not yet initialized, and you just want to use the
 setting as stored in .gitmodules, you can automatically initialize the
@@ -270,6 +277,23 @@ OPTIONS
 	be overridden by setting the `submodule.<name>.branch` option in
 	either `.gitmodules` or `.git/config` (with `.git/config` taking
 	precedence).
+
+--attached-update::
+	This option is only valid for the add command. Causes the add command
+	also to set the property `submodule.<name>.attached` to `true` and
+	the property `submodule.<name>.ignore` to `all`.
+
+--attach::
+	This option is only valid for the update commands. Causes the result
+	of an update operation to be an attached HEAD. In the update operation,
+	the branch named by 'submodule.<name>.branch' is checked out as the new
+	HEAD of the submodule repository. If 'submodule.<name>.branch' is not
+	set, the 'master' branch is checked out as the new HEAD of the
+	submodule. Note: `--attach` also implies `--remote`.
+
+--detach::
+	This option is only valid for the update command. Forces the result
+	of the update operation to be a detached HEAD in the submodule.
 +
 This works for any of the supported update procedures (`--checkout`,
 `--rebase`, etc.).  The only change is the source of the target SHA-1.
@@ -290,8 +314,7 @@ SHA-1.  If you don't want to fetch, you should use `submodule update
 --merge::
 	This option is only valid for the update command.
 	Merge the commit recorded in the superproject into the current branch
-	of the submodule. If this option is given, the submodule's HEAD will
-	not be detached. If a merge failure prevents this process, you will
+	of the submodule. If a merge failure prevents this process, you will
 	have to resolve the resulting conflicts within the submodule with the
 	usual conflict resolution tools.
 	If the key `submodule.$name.update` is set to `merge`, this option is
@@ -300,8 +323,7 @@ SHA-1.  If you don't want to fetch, you should use `submodule update
 --rebase::
 	This option is only valid for the update command.
 	Rebase the current branch onto the commit recorded in the
-	superproject. If this option is given, the submodule's HEAD will not
-	be detached. If a merge failure prevents this process, you will have
+	superproject. If a merge failure prevents this process, you will have
 	to resolve these failures with linkgit:git-rebase[1].
 	If the key `submodule.$name.update` is set to `rebase`, this option is
 	implicit.
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index f7be93f..9c436db 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -38,7 +38,8 @@ submodule.<name>.url::
 submodule.<name>.update::
 	Defines what to do when the submodule is updated by the superproject.
 	If 'checkout' (the default), the new commit specified in the
-	superproject will be checked out in the submodule on a detached HEAD.
+	superproject (or branch, with '--attach') will be checked out in
+	the submodule.
 	If 'rebase', the current branch of the submodule will be rebased onto
 	the commit specified in the superproject. If 'merge', the commit
 	specified in the superproject will be merged into the current branch
@@ -54,6 +55,13 @@ submodule.<name>.branch::
 	If the option is not specified, it defaults to 'master'.  See the
 	`--remote` documentation in linkgit:git-submodule[1] for details.
 
+submodule.<name>.attached::
+	Determine if the update operation will produce a detached HEAD or not.
+	Valid values are `true` or `false`. If the property is set to `true`
+	and `submodule.<name>.branch` is not set, the branch `master` will
+	be checked out. If `submodule.<name>.branch` is set the branch
+	specified will be checked out instead.
+
 submodule.<name>.fetchRecurseSubmodules::
 	This option can be used to control recursive fetching of this
 	submodule. If this option is also present in the submodules entry in
diff --git a/git-submodule.sh b/git-submodule.sh
index 1d041a7..bc6df2b 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,11 +5,11 @@
 # Copyright (c) 2007 Lars Hjemli
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
-USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
+USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--attached-update] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
-   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--attach | --detach] [--merge] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
@@ -36,6 +36,9 @@ update=
 prefix=
 custom_name=
 depth=
+attach=
+detach=
+attached_update=
 
 # The function takes at most 2 arguments. The first argument is the
 # URL that navigates to the submodule origin repo. When relative, this URL
@@ -352,6 +355,9 @@ cmd_add()
 			custom_name=$2
 			shift
 			;;
+		--attached-update)
+			attached_update=yes
+			;;
 		--depth)
 			case "$2" in '') usage ;; esac
 			depth="--depth=$2"
@@ -491,6 +497,12 @@ Use -f if you really want to add it." >&2
 	then
 		git config -f .gitmodules submodule."$sm_name".branch "$branch"
 	fi &&
+	if test -n "$attached_update"
+	then
+		# We'll stay stick to the HEAD, no need to track revision sha1
+		git config -f .gitmodules submodule."$sm_name".attached "true"
+		git config -f .gitmodules submodule."$sm_name".ignore "all"
+	fi &&
 	git add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
@@ -632,6 +644,22 @@ cmd_init()
 			git config submodule."$name".update "$upd" ||
 			die "$(eval_gettext "Failed to register update mode for submodule path '\$displaypath'")"
 		fi
+
+		# Copy "attached" setting when it is not set yet
+		if attached="$(git config -f .gitmodules submodule."$name".attached)" &&
+		   test -n "$attached" &&
+		   test -z "$(git config submodule."$name".attached)"
+		then
+			case "$attached" in
+			true | false)
+				;; # Valid attach flag values
+			*)
+				echo >&2 "warning: invalid attach flag value for submodule '$name'"
+				;;
+			esac
+			git config submodule."$name".attached "$attached" ||
+			die "$(eval_gettext "Failed to register attach option for submodule path '\$displaypath'")"
+		fi
 	done
 }
 
@@ -750,6 +778,14 @@ cmd_update()
 		--reference=*)
 			reference="$1"
 			;;
+		--attach)
+			if test -n "$detach" ; then usage ; fi
+			attach=1
+			;;
+		--detach)
+			if test -n "$attach" ; then usage ; fi
+			detach=1
+			;;
 		-m|--merge)
 			update="merge"
 			;;
@@ -800,6 +836,28 @@ cmd_update()
 		name=$(module_name "$sm_path") || exit
 		url=$(git config submodule."$name".url)
 		branch=$(get_submodule_config "$name" branch master)
+		attach_module=
+		detach_module=
+		if test -n "$attach" -o -n "$detach"
+		then
+			attach_module=$attach
+			detach_module=$detach
+		else
+			attached=$(git config submodule."$name".attached)
+			case "$attached" in
+			'')
+				;; # Unset attach flag
+			true)
+				attach_module=1
+				;;
+			false)
+				detach_module=1
+				;;
+			*)
+				echo >&2 "warning: invalid attach flag value for submodule '$name'"
+				;;
+			esac
+		fi
 		if ! test -z "$update"
 		then
 			update_module=$update
@@ -848,7 +906,16 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
 		fi
 
-		if test -n "$remote"
+		head_rev_ref=$(clear_local_git_env; cd "$sm_path" && git rev-parse --abbrev-ref HEAD) ||
+		die "$(eval_gettext "Unable to determine revision ref in submodule path '\$sm_path'")"
+		head_detached=
+		if test "$head_rev_ref" = "HEAD"
+		then
+			# Determine if the HEAD is detached
+			head_detached="true"
+		fi
+
+		if test -n "$remote" -o -n "$attach_module"
 		then
 			if test -z "$nofetch"
 			then
@@ -862,7 +929,8 @@ Maybe you want to use 'update --init'?")"
 			die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
 		fi
 
-		if test "$subsha1" != "$sha1" -o -n "$force"
+		if test "$subsha1" != "$sha1" || test -n "$attach_module" -a -n "$head_detached" ||
+			test -n "$detach_module" -a -z "$head_detached" || test -n "$force"
 		then
 			subforce=$force
 			# If we don't already have a -f flag and the submodule has never been checked out
@@ -882,40 +950,108 @@ Maybe you want to use 'update --init'?")"
 			fi
 
 			# Is this something we just cloned?
+			just_cloned=
 			case ";$cloned_modules;" in
 			*";$name;"*)
 				# then there is no local change to integrate
-				update_module= ;;
+				update_module="checkout"
+				just_cloned=yes
+				;;
 			esac
 
+			if test -z "$update_module"
+			then
+				# Fallback to checkout
+				update_module="checkout"
+			fi
+
+			command_attach=:
+			suffix_attach=
+			if test "$update_module" != "checkout"
+			then
+				if test -n "$attach_module" -a -n "$head_detached"
+				then
+					# We need to reattach to the branch
+					command_attach="git checkout $subforce -q"
+					suffix_attach=$branch
+				elif test -n "$detach_module" -a -z "$head_detached"
+				then
+					# We need to detach from the branch
+					command_attach="git checkout $subforce -q"
+					suffix_attach=$sha1
+				fi
+			fi
+
+			command_pre=:
+			suffix_pre=
+			command_post=:
+			suffix_pre=
+			suffix=
 			must_die_on_failure=
+			custom_update=
 			case "$update_module" in
 			rebase)
 				command="git rebase"
+				suffix=$sha1
 				die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$displaypath'")"
 				say_msg="$(eval_gettext "Submodule path '\$displaypath': rebased into '\$sha1'")"
 				must_die_on_failure=yes
+				if test -n "$attach_module" -a -n "$head_detached" && test "$subsha1" != "$sha1"
+				then
+					# After the rebase, we merge orphaned commits in the branch
+					command_post="git merge"
+					suffix_post=$subsha1
+				fi
 				;;
 			merge)
+				if test -n "$attach_module" -a -n "$head_detached" && test "$subsha1" != "$sha1"
+				then
+					# Prior the rebase, we merge orphaned commits in in the branch
+					command_pre="git merge"
+					suffix_pre=$subsha1
+				fi
 				command="git merge"
+				suffix=$sha1
 				die_msg="$(eval_gettext "Unable to merge '\$sha1' in submodule path '\$displaypath'")"
 				say_msg="$(eval_gettext "Submodule path '\$displaypath': merged in '\$sha1'")"
 				must_die_on_failure=yes
 				;;
+			checkout)
+				if test -n "$attach_module"
+				then
+					command="git checkout $subforce -q"
+					suffix=$branch
+					die_msg="$(eval_gettext "Unable to checkout banch '\$branch' in submodule path '\$displaypath'")"
+					say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out branch '\$branch'")"
+					if test -z "$just_cloned" -a && test "$subsha1" != "$sha1"
+					then
+						# Perform a fast-forward only merge of the origin
+						command_post="git merge $subforce --ff-only"
+						suffix_post="origin/$branch"
+					fi
+				else
+					command="git checkout $subforce -q"
+					suffix=$sha1
+					die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
+					say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
+				fi
+				;;
 			!*)
 				command="${update_module#!}"
+				suffix=$sha1
 				die_msg="$(eval_gettext "Execution of '\$command \$sha1' failed in submodule  path '\$prefix\$sm_path'")"
 				say_msg="$(eval_gettext "Submodule path '\$prefix\$sm_path': '\$command \$sha1'")"
 				must_die_on_failure=yes
+				custom_update=yes
 				;;
 			*)
-				command="git checkout $subforce -q"
-				die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
-				say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
+				# Valid user configurable update modes are already filtered above
+				die "$(eval_gettext "Unexpected update mode in the current flow")"
 				;;
 			esac
 
-			if (clear_local_git_env; cd "$sm_path" && $command "$sha1")
+			if (clear_local_git_env; cd "$sm_path" && $command_attach "$suffix_attach" &&
+				$command_pre "$suffix_pre" && $command "$suffix" && $command_post "$suffix_pro")
 			then
 				say "$say_msg"
 			elif test -n "$must_die_on_failure"
diff --git a/t/t7410-submodule-attached-head.sh b/t/t7410-submodule-attached-head.sh
new file mode 100755
index 0000000..04b3018
--- /dev/null
+++ b/t/t7410-submodule-attached-head.sh
@@ -0,0 +1,268 @@
+#!/bin/sh
+#
+# Copyright (c) 2014 Francesco Pretto
+#
+
+test_description='Support for submodules with attached head
+
+This test verifies the sanity of the add and update git submodule commands with
+or without the --attached-update, --attach, --detach switches or the
+submoudule.<module>.attach property set
+'
+
+TEST_NO_CREATE_REPO=true
+. ./test-lib.sh
+
+submodurl1=$(pwd -P)/repo1
+submodurl2=$(pwd -P)/repo2
+repourl=$(pwd -P)/repo
+
+test_expect_success 'setup - create repository "repo1" to be used as submodule' '
+	mkdir repo1 &&
+	(
+		cd repo1 &&
+		git init &&
+		git config receive.denyCurrentBranch ignore &&
+		echo a >a &&
+		git add a &&
+		git commit -m "repo1 commit 1"
+	)
+'
+
+test_expect_success 'setup - reate repository "repo2" to be used as submodule' '
+	mkdir repo2 &&
+	(
+		cd repo2 &&
+		git init &&
+		git config receive.denyCurrentBranch ignore &&
+		echo a >a &&
+		git add a &&
+		git commit -m "repo2 commit 1"
+	)
+'
+
+test_expect_success 'setup - create repository "repo" to be added with sumodules' '
+	mkdir repo &&
+	(
+		cd repo &&
+		git init &&
+		git config receive.denyCurrentBranch ignore &&
+		echo a >a &&
+		git add a &&
+		git commit -m "repo commit 1"
+	)
+'
+
+test_expect_success 'setup - clone repository "repo" in "repoclone"' '
+	git clone "$repourl" repoclone
+'
+
+test_expect_success 'setup - add "mod1" as regular submodule of "repo"' '
+	(
+		cd repo &&
+		git submodule add "$submodurl1" submod1
+	)
+'
+
+test_expect_success 'setup - add "mod2" as update attached HEAD submodule of "repo"' '
+	(
+		cd repo &&
+		git submodule add --attached-update "$submodurl2" submod2
+	)
+'
+
+test_expect_success 'setup - commit submodules in repo' '
+	(
+		cd repo &&
+		git add . &&
+		git commit -m "Added submodules"
+	)
+'
+
+test_expect_success 'init submodules in cloned repo' '
+	(
+		cd repoclone &&
+		git pull &&
+		git submodule init
+	)
+'
+
+test_expect_success 'update submodules in cloned repo' '
+	(
+		cd repoclone &&
+		git submodule update
+	)
+'
+
+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
+	(
+		cd repoclone/submod1 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
+	)
+'
+
+test_expect_success 'assert submod2 HEAD is attached in cloned repo' '
+	(
+		cd repoclone/submod2 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
+	)
+'
+
+test_expect_success 'update submodules with --attach in cloned repo' '
+	(
+		cd repoclone &&
+		git submodule update --attach
+	)
+'
+
+test_expect_success 'assert submod1 HEAD is attached in cloned repo' '
+	(
+		cd repoclone/submod1 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
+	)
+'
+
+test_expect_success 'update submodules with --detach in cloned repo' '
+	(
+		cd repoclone &&
+		git submodule update --detach
+	)
+'
+
+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
+	(
+		cd repoclone/submod1 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
+	)
+'
+
+test_expect_success 'assert submod2 HEAD is detached in cloned repo' '
+	(
+		cd repoclone/submod2 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
+	)
+'
+
+test_expect_success 'update submodules in cloned repo (will restore HEAD states)' '
+	(
+		cd repoclone &&
+		git submodule update
+	)
+'
+
+test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
+	(
+		cd repoclone/submod1 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
+	)
+'
+
+test_expect_success 'assert submod2 HEAD is attached in cloned repo' '
+	(
+		cd repoclone/submod2 &&
+		test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
+	)
+'
+
+test_expect_success 'setup - add update operation to submodules' '
+	(
+		cd repo &&
+		git config  -f .gitmodules submodule.submod1.update merge &&
+		git config  -f .gitmodules submodule.submod2.update rebase &&
+		git add . &&
+		git commit -m "updated submodules"
+	)
+'
+
+test_expect_success 'setup - update cloned repo and reinitialize submodules' '
+	(
+		cd repoclone &&
+		git pull &&
+		git submodule init
+	)
+'
+
+test_expect_success 'add some content to repo2' '
+	(
+		cd repo2 &&
+		echo b >b &&
+		git add b &&
+		git commit -m "repo2 commit 2"
+	)
+'
+
+test_expect_success 'update sumodules in cloned repo and verify that submod2 matches repo2' '
+	(
+		cd repoclone &&
+		git submodule update &&
+		test -e submod2/b
+	)
+'
+
+test_expect_success 'prepend some content to repo1/a' '
+	(
+		cd repo1 &&
+		echo -e "b\na" >a &&
+		git add a &&
+		git commit -m "repo1 commit 2"
+	)
+'
+
+test_expect_success 'append some content in repoclone/submod1 and commit' '
+	(
+		cd repoclone/submod1 &&
+		echo c >>a &&
+		git add a &&
+		git commit -m "submod1 commit 1"
+	)
+'
+
+test_expect_success 'update repoclone submodules with --attach' '
+	(
+		cd repoclone &&
+		git submodule update --attach
+	)
+'
+
+test_expect_success 'verify repoclone submod1 merge with reattached orphaned commits was correct' '
+	(
+		cd repoclone/submod1 &&
+		test "$(<a)" = "$'b\na\nc'"
+	)
+'
+
+test_expect_success 'setup - set operation checkout to submodule sumod1 in repo' '
+	(
+		cd repo &&
+		git config  -f .gitmodules submodule.submod1.update checkout &&
+		git add . &&
+		git commit -m "updated submodules"
+	)
+'
+
+test_expect_success 'setup - update cloned repo and reinitialize submodules' '
+	(
+		cd repoclone &&
+		git pull &&
+		git submodule init
+	)
+'
+
+test_expect_success 'add some content to repo1' '
+	(
+		cd repo1 &&
+		echo b >b &&
+		git add b &&
+		git commit -m "repo1 commit 3"
+	)
+'
+
+test_expect_success 'update submodule submod2 (merge ff-only) and verify it matches repo2' '
+	(
+		cd repoclone &&
+		git submodule update &&
+		test -e submod2/b
+	)
+'
+
+test_done
-- 
1.8.5.2.230.g032cd47.dirty
Francesco Pretto· Jan 5, 2014, 19:55 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

(Hmmpth, forgot signoff...)
To whom it may interest, added some CC.
2014/1/5 Francesco Pretto <ceztko@gmail.com>:
Show 717 quoted lines
> At the current state, the following use-case is not supported very
> well in git:
> - a maintainer adds a submodule, checking out a specific branch of
> the repository. He doesn't track the upstream submodule revision sha1;
> - a developer checkout the repository branch decided by the maintainer.
> Subsequent "merge" or "rebase" update operations don't detach the HEAD.
>
> To ease the above use-case this patch:
> - introduces a "submodule.<module>.attached" property that, when set
>   to "true", ensures that the "update" operation will result in
>   the HEAD attached to a branch;
> - introduces "--attach|--dettach" switches to the submodule "update"
>   command: they attach/detach the HEAD, overriding
>   "submodule.<module>.attached" property value;
> - introduces "--attached-update" switch to the "add" operation. It:
>     * sets "submodule.<module>.attached" to true;
>     * sets "submodule.<module>.ignore" to all.
>
> Using the '--attach' switch or operating in a repository with
> 'submodule.<name>.attached' set to 'true' during "update" will:
> - checkout a branch with an attached HEAD if the repository was just
> cloned;
> - perform a fast-forward only merge of changes if it's a 'checkout'
> update operation;
> - reattach the HEAD prior performing a 'merge', 'rebase' or '!command'
> update operation if the HEAD was found detached. Orphaned commits
> will also be merged back in the branch.
>
> '--attach' or 'submodule.<name>.attached' set to true also implies '--remote'.
>
> Using  the '--detach' switch or operating in a repository with
> 'submodule.<name>.attached' set to 'false' during "update" will:
> - checkout a detached HEAD if the repository was just cloned;
> - detach the HEAD prior performing a 'merge', 'rebase' or '!command'
> update operation if the HEAD was found attached.
>
> 'submodule.<name>.attached' works similarly to 'submodule.<name>.update'
> property: git copies the values found in ".gitmodules" in ".git/config" when
> performing an "init" command. "update" looks for values in ".git/config"
> only.
>
> '--attach' and '--detach' switches override an opposite behaviour
> of 'submodule.<name>.attached' properties.
>
> The patch is strongly additive and doesn't break any submodule specific
> test. It also adds some tests specific to the added feature.
> ---
>  Documentation/git-submodule.txt    |  48 +++++--
>  Documentation/gitmodules.txt       |  10 +-
>  git-submodule.sh                   | 154 +++++++++++++++++++--
>  t/t7410-submodule-attached-head.sh | 268 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 457 insertions(+), 23 deletions(-)
>  create mode 100755 t/t7410-submodule-attached-head.sh
>
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index bfef8a0..b97eefb 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -10,13 +10,14 @@ SYNOPSIS
>  --------
>  [verse]
>  'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
> -             [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
> +             [--reference <repository>] [--attached-update] [--depth <depth>]
> +             [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
>  'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
>  'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
> -             [-f|--force] [--rebase] [--reference <repository>] [--depth <depth>]
> -             [--merge] [--recursive] [--] [<path>...]
> +             [-f|--force] [--rebase] [--reference <repository>] [--attach | --detach]
> +             [--depth <depth>] [--merge] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
>               [commit] [--] [<path>...]
>  'git submodule' [--quiet] foreach [--recursive] <command>
> @@ -107,6 +108,9 @@ is the superproject and submodule repositories will be kept
>  together in the same relative location, and only the
>  superproject's URL needs to be provided: git-submodule will correctly
>  locate the submodule using the relative URL in .gitmodules.
> ++
> +If `--attached-update` is specified, the property `submodule.<name>.attached`
> +will be set to `true` and `submodule.<name>.ignore` will be set to `all`.
>
>  status::
>         Show the status of the submodules. This will print the SHA-1 of the
> @@ -156,12 +160,15 @@ it contains local modifications.
>  update::
>         Update the registered submodules, i.e. clone missing submodules and
>         checkout the commit specified in the index of the containing repository.
> -       This will make the submodules HEAD be detached unless `--rebase` or
> -       `--merge` is specified or the key `submodule.$name.update` is set to
> -       `rebase`, `merge` or `none`. `none` can be overridden by specifying
> -       `--checkout`. Setting the key `submodule.$name.update` to `!command`
> -       will cause `command` to be run. `command` can be any arbitrary shell
> -       command that takes a single argument, namely the sha1 to update to.
> +       This will make the submodules HEAD be detached unless `--attach` is
> +       specified or `submodule.$name.attached` is set to `true`. The last setting
> +       can always be overridden specifying `--detach`. Update mode can be
> +       selected specifying `--checkout`, `--rebase` or `--merge` switches
> +       or setting the key `submodule.$name.update` to `checkout`, `rebase`,
> +       `merge` or `none`. `none` will cause the submodule to be skipped during
> +       the update. Setting the key `submodule.$name.update` to `!command` will
> +       cause `command` to be run. `command` can be any arbitrary shell command
> +       that takes a single argument, namely the sha1 to update to.
>  +
>  If the submodule is not yet initialized, and you just want to use the
>  setting as stored in .gitmodules, you can automatically initialize the
> @@ -270,6 +277,23 @@ OPTIONS
>         be overridden by setting the `submodule.<name>.branch` option in
>         either `.gitmodules` or `.git/config` (with `.git/config` taking
>         precedence).
> +
> +--attached-update::
> +       This option is only valid for the add command. Causes the add command
> +       also to set the property `submodule.<name>.attached` to `true` and
> +       the property `submodule.<name>.ignore` to `all`.
> +
> +--attach::
> +       This option is only valid for the update commands. Causes the result
> +       of an update operation to be an attached HEAD. In the update operation,
> +       the branch named by 'submodule.<name>.branch' is checked out as the new
> +       HEAD of the submodule repository. If 'submodule.<name>.branch' is not
> +       set, the 'master' branch is checked out as the new HEAD of the
> +       submodule. Note: `--attach` also implies `--remote`.
> +
> +--detach::
> +       This option is only valid for the update command. Forces the result
> +       of the update operation to be a detached HEAD in the submodule.
>  +
>  This works for any of the supported update procedures (`--checkout`,
>  `--rebase`, etc.).  The only change is the source of the target SHA-1.
> @@ -290,8 +314,7 @@ SHA-1.  If you don't want to fetch, you should use `submodule update
>  --merge::
>         This option is only valid for the update command.
>         Merge the commit recorded in the superproject into the current branch
> -       of the submodule. If this option is given, the submodule's HEAD will
> -       not be detached. If a merge failure prevents this process, you will
> +       of the submodule. If a merge failure prevents this process, you will
>         have to resolve the resulting conflicts within the submodule with the
>         usual conflict resolution tools.
>         If the key `submodule.$name.update` is set to `merge`, this option is
> @@ -300,8 +323,7 @@ SHA-1.  If you don't want to fetch, you should use `submodule update
>  --rebase::
>         This option is only valid for the update command.
>         Rebase the current branch onto the commit recorded in the
> -       superproject. If this option is given, the submodule's HEAD will not
> -       be detached. If a merge failure prevents this process, you will have
> +       superproject. If a merge failure prevents this process, you will have
>         to resolve these failures with linkgit:git-rebase[1].
>         If the key `submodule.$name.update` is set to `rebase`, this option is
>         implicit.
> diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
> index f7be93f..9c436db 100644
> --- a/Documentation/gitmodules.txt
> +++ b/Documentation/gitmodules.txt
> @@ -38,7 +38,8 @@ submodule.<name>.url::
>  submodule.<name>.update::
>         Defines what to do when the submodule is updated by the superproject.
>         If 'checkout' (the default), the new commit specified in the
> -       superproject will be checked out in the submodule on a detached HEAD.
> +       superproject (or branch, with '--attach') will be checked out in
> +       the submodule.
>         If 'rebase', the current branch of the submodule will be rebased onto
>         the commit specified in the superproject. If 'merge', the commit
>         specified in the superproject will be merged into the current branch
> @@ -54,6 +55,13 @@ submodule.<name>.branch::
>         If the option is not specified, it defaults to 'master'.  See the
>         `--remote` documentation in linkgit:git-submodule[1] for details.
>
> +submodule.<name>.attached::
> +       Determine if the update operation will produce a detached HEAD or not.
> +       Valid values are `true` or `false`. If the property is set to `true`
> +       and `submodule.<name>.branch` is not set, the branch `master` will
> +       be checked out. If `submodule.<name>.branch` is set the branch
> +       specified will be checked out instead.
> +
>  submodule.<name>.fetchRecurseSubmodules::
>         This option can be used to control recursive fetching of this
>         submodule. If this option is also present in the submodules entry in
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 1d041a7..bc6df2b 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -5,11 +5,11 @@
>  # Copyright (c) 2007 Lars Hjemli
>
>  dashless=$(basename "$0" | sed -e 's/-/ /')
> -USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
> +USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--attached-update] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
>     or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
> -   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> +   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--attach | --detach] [--merge] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
>     or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
> @@ -36,6 +36,9 @@ update=
>  prefix=
>  custom_name=
>  depth=
> +attach=
> +detach=
> +attached_update=
>
>  # The function takes at most 2 arguments. The first argument is the
>  # URL that navigates to the submodule origin repo. When relative, this URL
> @@ -352,6 +355,9 @@ cmd_add()
>                         custom_name=$2
>                         shift
>                         ;;
> +               --attached-update)
> +                       attached_update=yes
> +                       ;;
>                 --depth)
>                         case "$2" in '') usage ;; esac
>                         depth="--depth=$2"
> @@ -491,6 +497,12 @@ Use -f if you really want to add it." >&2
>         then
>                 git config -f .gitmodules submodule."$sm_name".branch "$branch"
>         fi &&
> +       if test -n "$attached_update"
> +       then
> +               # We'll stay stick to the HEAD, no need to track revision sha1
> +               git config -f .gitmodules submodule."$sm_name".attached "true"
> +               git config -f .gitmodules submodule."$sm_name".ignore "all"
> +       fi &&
>         git add --force .gitmodules ||
>         die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
>  }
> @@ -632,6 +644,22 @@ cmd_init()
>                         git config submodule."$name".update "$upd" ||
>                         die "$(eval_gettext "Failed to register update mode for submodule path '\$displaypath'")"
>                 fi
> +
> +               # Copy "attached" setting when it is not set yet
> +               if attached="$(git config -f .gitmodules submodule."$name".attached)" &&
> +                  test -n "$attached" &&
> +                  test -z "$(git config submodule."$name".attached)"
> +               then
> +                       case "$attached" in
> +                       true | false)
> +                               ;; # Valid attach flag values
> +                       *)
> +                               echo >&2 "warning: invalid attach flag value for submodule '$name'"
> +                               ;;
> +                       esac
> +                       git config submodule."$name".attached "$attached" ||
> +                       die "$(eval_gettext "Failed to register attach option for submodule path '\$displaypath'")"
> +               fi
>         done
>  }
>
> @@ -750,6 +778,14 @@ cmd_update()
>                 --reference=*)
>                         reference="$1"
>                         ;;
> +               --attach)
> +                       if test -n "$detach" ; then usage ; fi
> +                       attach=1
> +                       ;;
> +               --detach)
> +                       if test -n "$attach" ; then usage ; fi
> +                       detach=1
> +                       ;;
>                 -m|--merge)
>                         update="merge"
>                         ;;
> @@ -800,6 +836,28 @@ cmd_update()
>                 name=$(module_name "$sm_path") || exit
>                 url=$(git config submodule."$name".url)
>                 branch=$(get_submodule_config "$name" branch master)
> +               attach_module=
> +               detach_module=
> +               if test -n "$attach" -o -n "$detach"
> +               then
> +                       attach_module=$attach
> +                       detach_module=$detach
> +               else
> +                       attached=$(git config submodule."$name".attached)
> +                       case "$attached" in
> +                       '')
> +                               ;; # Unset attach flag
> +                       true)
> +                               attach_module=1
> +                               ;;
> +                       false)
> +                               detach_module=1
> +                               ;;
> +                       *)
> +                               echo >&2 "warning: invalid attach flag value for submodule '$name'"
> +                               ;;
> +                       esac
> +               fi
>                 if ! test -z "$update"
>                 then
>                         update_module=$update
> @@ -848,7 +906,16 @@ Maybe you want to use 'update --init'?")"
>                         die "$(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
>                 fi
>
> -               if test -n "$remote"
> +               head_rev_ref=$(clear_local_git_env; cd "$sm_path" && git rev-parse --abbrev-ref HEAD) ||
> +               die "$(eval_gettext "Unable to determine revision ref in submodule path '\$sm_path'")"
> +               head_detached=
> +               if test "$head_rev_ref" = "HEAD"
> +               then
> +                       # Determine if the HEAD is detached
> +                       head_detached="true"
> +               fi
> +
> +               if test -n "$remote" -o -n "$attach_module"
>                 then
>                         if test -z "$nofetch"
>                         then
> @@ -862,7 +929,8 @@ Maybe you want to use 'update --init'?")"
>                         die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")"
>                 fi
>
> -               if test "$subsha1" != "$sha1" -o -n "$force"
> +               if test "$subsha1" != "$sha1" || test -n "$attach_module" -a -n "$head_detached" ||
> +                       test -n "$detach_module" -a -z "$head_detached" || test -n "$force"
>                 then
>                         subforce=$force
>                         # If we don't already have a -f flag and the submodule has never been checked out
> @@ -882,40 +950,108 @@ Maybe you want to use 'update --init'?")"
>                         fi
>
>                         # Is this something we just cloned?
> +                       just_cloned=
>                         case ";$cloned_modules;" in
>                         *";$name;"*)
>                                 # then there is no local change to integrate
> -                               update_module= ;;
> +                               update_module="checkout"
> +                               just_cloned=yes
> +                               ;;
>                         esac
>
> +                       if test -z "$update_module"
> +                       then
> +                               # Fallback to checkout
> +                               update_module="checkout"
> +                       fi
> +
> +                       command_attach=:
> +                       suffix_attach=
> +                       if test "$update_module" != "checkout"
> +                       then
> +                               if test -n "$attach_module" -a -n "$head_detached"
> +                               then
> +                                       # We need to reattach to the branch
> +                                       command_attach="git checkout $subforce -q"
> +                                       suffix_attach=$branch
> +                               elif test -n "$detach_module" -a -z "$head_detached"
> +                               then
> +                                       # We need to detach from the branch
> +                                       command_attach="git checkout $subforce -q"
> +                                       suffix_attach=$sha1
> +                               fi
> +                       fi
> +
> +                       command_pre=:
> +                       suffix_pre=
> +                       command_post=:
> +                       suffix_pre=
> +                       suffix=
>                         must_die_on_failure=
> +                       custom_update=
>                         case "$update_module" in
>                         rebase)
>                                 command="git rebase"
> +                               suffix=$sha1
>                                 die_msg="$(eval_gettext "Unable to rebase '\$sha1' in submodule path '\$displaypath'")"
>                                 say_msg="$(eval_gettext "Submodule path '\$displaypath': rebased into '\$sha1'")"
>                                 must_die_on_failure=yes
> +                               if test -n "$attach_module" -a -n "$head_detached" && test "$subsha1" != "$sha1"
> +                               then
> +                                       # After the rebase, we merge orphaned commits in the branch
> +                                       command_post="git merge"
> +                                       suffix_post=$subsha1
> +                               fi
>                                 ;;
>                         merge)
> +                               if test -n "$attach_module" -a -n "$head_detached" && test "$subsha1" != "$sha1"
> +                               then
> +                                       # Prior the rebase, we merge orphaned commits in in the branch
> +                                       command_pre="git merge"
> +                                       suffix_pre=$subsha1
> +                               fi
>                                 command="git merge"
> +                               suffix=$sha1
>                                 die_msg="$(eval_gettext "Unable to merge '\$sha1' in submodule path '\$displaypath'")"
>                                 say_msg="$(eval_gettext "Submodule path '\$displaypath': merged in '\$sha1'")"
>                                 must_die_on_failure=yes
>                                 ;;
> +                       checkout)
> +                               if test -n "$attach_module"
> +                               then
> +                                       command="git checkout $subforce -q"
> +                                       suffix=$branch
> +                                       die_msg="$(eval_gettext "Unable to checkout banch '\$branch' in submodule path '\$displaypath'")"
> +                                       say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out branch '\$branch'")"
> +                                       if test -z "$just_cloned" -a && test "$subsha1" != "$sha1"
> +                                       then
> +                                               # Perform a fast-forward only merge of the origin
> +                                               command_post="git merge $subforce --ff-only"
> +                                               suffix_post="origin/$branch"
> +                                       fi
> +                               else
> +                                       command="git checkout $subforce -q"
> +                                       suffix=$sha1
> +                                       die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
> +                                       say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
> +                               fi
> +                               ;;
>                         !*)
>                                 command="${update_module#!}"
> +                               suffix=$sha1
>                                 die_msg="$(eval_gettext "Execution of '\$command \$sha1' failed in submodule  path '\$prefix\$sm_path'")"
>                                 say_msg="$(eval_gettext "Submodule path '\$prefix\$sm_path': '\$command \$sha1'")"
>                                 must_die_on_failure=yes
> +                               custom_update=yes
>                                 ;;
>                         *)
> -                               command="git checkout $subforce -q"
> -                               die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
> -                               say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
> +                               # Valid user configurable update modes are already filtered above
> +                               die "$(eval_gettext "Unexpected update mode in the current flow")"
>                                 ;;
>                         esac
>
> -                       if (clear_local_git_env; cd "$sm_path" && $command "$sha1")
> +                       if (clear_local_git_env; cd "$sm_path" && $command_attach "$suffix_attach" &&
> +                               $command_pre "$suffix_pre" && $command "$suffix" && $command_post "$suffix_pro")
>                         then
>                                 say "$say_msg"
>                         elif test -n "$must_die_on_failure"
> diff --git a/t/t7410-submodule-attached-head.sh b/t/t7410-submodule-attached-head.sh
> new file mode 100755
> index 0000000..04b3018
> --- /dev/null
> +++ b/t/t7410-submodule-attached-head.sh
> @@ -0,0 +1,268 @@
> +#!/bin/sh
> +#
> +# Copyright (c) 2014 Francesco Pretto
> +#
> +
> +test_description='Support for submodules with attached head
> +
> +This test verifies the sanity of the add and update git submodule commands with
> +or without the --attached-update, --attach, --detach switches or the
> +submoudule.<module>.attach property set
> +'
> +
> +TEST_NO_CREATE_REPO=true
> +. ./test-lib.sh
> +
> +submodurl1=$(pwd -P)/repo1
> +submodurl2=$(pwd -P)/repo2
> +repourl=$(pwd -P)/repo
> +
> +test_expect_success 'setup - create repository "repo1" to be used as submodule' '
> +       mkdir repo1 &&
> +       (
> +               cd repo1 &&
> +               git init &&
> +               git config receive.denyCurrentBranch ignore &&
> +               echo a >a &&
> +               git add a &&
> +               git commit -m "repo1 commit 1"
> +       )
> +'
> +
> +test_expect_success 'setup - reate repository "repo2" to be used as submodule' '
> +       mkdir repo2 &&
> +       (
> +               cd repo2 &&
> +               git init &&
> +               git config receive.denyCurrentBranch ignore &&
> +               echo a >a &&
> +               git add a &&
> +               git commit -m "repo2 commit 1"
> +       )
> +'
> +
> +test_expect_success 'setup - create repository "repo" to be added with sumodules' '
> +       mkdir repo &&
> +       (
> +               cd repo &&
> +               git init &&
> +               git config receive.denyCurrentBranch ignore &&
> +               echo a >a &&
> +               git add a &&
> +               git commit -m "repo commit 1"
> +       )
> +'
> +
> +test_expect_success 'setup - clone repository "repo" in "repoclone"' '
> +       git clone "$repourl" repoclone
> +'
> +
> +test_expect_success 'setup - add "mod1" as regular submodule of "repo"' '
> +       (
> +               cd repo &&
> +               git submodule add "$submodurl1" submod1
> +       )
> +'
> +
> +test_expect_success 'setup - add "mod2" as update attached HEAD submodule of "repo"' '
> +       (
> +               cd repo &&
> +               git submodule add --attached-update "$submodurl2" submod2
> +       )
> +'
> +
> +test_expect_success 'setup - commit submodules in repo' '
> +       (
> +               cd repo &&
> +               git add . &&
> +               git commit -m "Added submodules"
> +       )
> +'
> +
> +test_expect_success 'init submodules in cloned repo' '
> +       (
> +               cd repoclone &&
> +               git pull &&
> +               git submodule init
> +       )
> +'
> +
> +test_expect_success 'update submodules in cloned repo' '
> +       (
> +               cd repoclone &&
> +               git submodule update
> +       )
> +'
> +
> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
> +       (
> +               cd repoclone/submod1 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
> +       )
> +'
> +
> +test_expect_success 'assert submod2 HEAD is attached in cloned repo' '
> +       (
> +               cd repoclone/submod2 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
> +       )
> +'
> +
> +test_expect_success 'update submodules with --attach in cloned repo' '
> +       (
> +               cd repoclone &&
> +               git submodule update --attach
> +       )
> +'
> +
> +test_expect_success 'assert submod1 HEAD is attached in cloned repo' '
> +       (
> +               cd repoclone/submod1 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
> +       )
> +'
> +
> +test_expect_success 'update submodules with --detach in cloned repo' '
> +       (
> +               cd repoclone &&
> +               git submodule update --detach
> +       )
> +'
> +
> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
> +       (
> +               cd repoclone/submod1 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
> +       )
> +'
> +
> +test_expect_success 'assert submod2 HEAD is detached in cloned repo' '
> +       (
> +               cd repoclone/submod2 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
> +       )
> +'
> +
> +test_expect_success 'update submodules in cloned repo (will restore HEAD states)' '
> +       (
> +               cd repoclone &&
> +               git submodule update
> +       )
> +'
> +
> +test_expect_success 'assert submod1 HEAD is detached in cloned repo' '
> +       (
> +               cd repoclone/submod1 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" = "HEAD"
> +       )
> +'
> +
> +test_expect_success 'assert submod2 HEAD is attached in cloned repo' '
> +       (
> +               cd repoclone/submod2 &&
> +               test "$(git rev-parse --abbrev-ref HEAD)" != "HEAD"
> +       )
> +'
> +
> +test_expect_success 'setup - add update operation to submodules' '
> +       (
> +               cd repo &&
> +               git config  -f .gitmodules submodule.submod1.update merge &&
> +               git config  -f .gitmodules submodule.submod2.update rebase &&
> +               git add . &&
> +               git commit -m "updated submodules"
> +       )
> +'
> +
> +test_expect_success 'setup - update cloned repo and reinitialize submodules' '
> +       (
> +               cd repoclone &&
> +               git pull &&
> +               git submodule init
> +       )
> +'
> +
> +test_expect_success 'add some content to repo2' '
> +       (
> +               cd repo2 &&
> +               echo b >b &&
> +               git add b &&
> +               git commit -m "repo2 commit 2"
> +       )
> +'
> +
> +test_expect_success 'update sumodules in cloned repo and verify that submod2 matches repo2' '
> +       (
> +               cd repoclone &&
> +               git submodule update &&
> +               test -e submod2/b
> +       )
> +'
> +
> +test_expect_success 'prepend some content to repo1/a' '
> +       (
> +               cd repo1 &&
> +               echo -e "b\na" >a &&
> +               git add a &&
> +               git commit -m "repo1 commit 2"
> +       )
> +'
> +
> +test_expect_success 'append some content in repoclone/submod1 and commit' '
> +       (
> +               cd repoclone/submod1 &&
> +               echo c >>a &&
> +               git add a &&
> +               git commit -m "submod1 commit 1"
> +       )
> +'
> +
> +test_expect_success 'update repoclone submodules with --attach' '
> +       (
> +               cd repoclone &&
> +               git submodule update --attach
> +       )
> +'
> +
> +test_expect_success 'verify repoclone submod1 merge with reattached orphaned commits was correct' '
> +       (
> +               cd repoclone/submod1 &&
> +               test "$(<a)" = "$'b\na\nc'"
> +       )
> +'
> +
> +test_expect_success 'setup - set operation checkout to submodule sumod1 in repo' '
> +       (
> +               cd repo &&
> +               git config  -f .gitmodules submodule.submod1.update checkout &&
> +               git add . &&
> +               git commit -m "updated submodules"
> +       )
> +'
> +
> +test_expect_success 'setup - update cloned repo and reinitialize submodules' '
> +       (
> +               cd repoclone &&
> +               git pull &&
> +               git submodule init
> +       )
> +'
> +
> +test_expect_success 'add some content to repo1' '
> +       (
> +               cd repo1 &&
> +               echo b >b &&
> +               git add b &&
> +               git commit -m "repo1 commit 3"
> +       )
> +'
> +
> +test_expect_success 'update submodule submod2 (merge ff-only) and verify it matches repo2' '
> +       (
> +               cd repoclone &&
> +               git submodule update &&
> +               test -e submod2/b
> +       )
> +'
> +
> +test_done
> --
> 1.8.5.2.230.g032cd47.dirty
>
Heiko Voigt· Jan 5, 2014, 20:33 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

On Sun, Jan 05, 2014 at 03:50:49AM +0100, Francesco Pretto wrote:
Show 6 quoted lines
> At the current state, the following use-case is not supported very
> well in git:
> - a maintainer adds a submodule, checking out a specific branch of
> the repository. He doesn't track the upstream submodule revision sha1;
> - a developer checkout the repository branch decided by the maintainer.
> Subsequent "merge" or "rebase" update operations don't detach the HEAD.

Could you please extend the description of your use-case so we can understand your goal better?

The following questions directly pop into my mind:
 - What means the maintainer does not track the submodules sha1? Does
   that mean the superproject always refers to submodule commits using
   branches?
 - What happens if you want to go back to an earlier revision? Lets say
   a tagged release? How is ensured that you get the correct revision in
   the submodules?
 - In which situations does the developer or maintainer switch between
   your attached/detached mode?
 - What is the "repository branch" which is given to the developer by
   the maintainer used for? Who creates this branch and who merges into
   it?
 - What are these subsequent "merge" or "rebase" update operations? Do
   you mean everyone has submodule.name.update configured to merge or
   rebase?
Still puzzled.
Cheers Heiko
Francesco Pretto· Jan 5, 2014, 21:46 UTC · re: Heiko Voigt · lore

Re: [PATCH 2/2] Introduce git submodule attached update

2014/1/5 Heiko Voigt <hvoigt@hvoigt.net>:
>
> Could you please extend the description of your use-case so we can
> understand your goal better?
>
Just in case you missed the first patch iteration[1].
Show 5 quoted lines
> The following questions directly pop into my mind:
>
>  - What means the maintainer does not track the submodules sha1? Does
>    that mean the superproject always refers to submodule commits using
>    branches?

It means he doesn't need to control other developers commit to be checked out so he sets "submodule.<name>.ignore" to "all". In this way he and the developers can work actively in their submodule copy.

>  - What happens if you want to go back to an earlier revision? Lets say
>    a tagged release? How is ensured that you get the correct revision in
>    the submodules?

"submodule.<name>.branch" is one setting that is not copied in ".git/config" by "git submodule init". "git submodule update" will use the setting in ".gitmodules" if not overridden voluntarily by the developer in ".git/config". The maintainer can change that setting in ".gitmodules" and commit the change. Modifies will be propagated by the next "git pull && git submodule update" of the developer in the superproject.

>  - In which situations does the developer or maintainer switch between
>    your attached/detached mode?

The developer/maintainer does so optionally and voluntarily and it effects only its private working tree.

>  - What is the "repository branch" which is given to the developer by
>    the maintainer used for? Who creates this branch and who merges into
>    it?

The branch of course must exist prior submodule adding. In this use-case it does not really matter who creates it and who merges into it. Everyone with the right to merge into it has to work in the submodule seamlessly, as it was working on separate clone of the same repository used as the submodule.

>  - What are these subsequent "merge" or "rebase" update operations? Do
>    you mean everyone has submodule.name.update configured to merge or
>    rebase?
>

subsequent "merge" or "rebase" update operations are just the ones after the initial clone/checkout, nothing particular.

Greetings, Francesco

[1] http://marc.info/?l=git&m=138836829531511&w=2
Heiko Voigt· Jan 6, 2014, 14:06 UTC · re: Francesco Pretto · lore

Re: Re: [PATCH 2/2] Introduce git submodule attached update

On Sun, Jan 05, 2014 at 10:46:11PM +0100, Francesco Pretto wrote:
Show 10 quoted lines
> 2014/1/5 Heiko Voigt <hvoigt@hvoigt.net>:
> > The following questions directly pop into my mind:
> >
> >  - What means the maintainer does not track the submodules sha1? Does
> >    that mean the superproject always refers to submodule commits using
> >    branches?
> 
> It means he doesn't need to control other developers commit to be
> checked out so he sets "submodule.<name>.ignore" to "all". In this way
> he and the developers can work actively in their submodule copy.

So practically speaking: You mean that the value of submodule.<name>.ignore is set to "all" in the master branch of the superproject? From your other email referring to svn:externals I figure that.

Show 11 quoted lines
> >  - What happens if you want to go back to an earlier revision? Lets say
> >    a tagged release? How is ensured that you get the correct revision in
> >    the submodules?
> 
> "submodule.<name>.branch" is one setting that is not copied in
> ".git/config" by "git submodule init". "git submodule update" will use
> the setting in ".gitmodules" if not overridden voluntarily by the
> developer in ".git/config". The maintainer can change that setting in
> ".gitmodules" and commit the change. Modifies will be propagated by
> the next "git pull && git submodule update" of the developer in the
> superproject.

I do not understand how does that ensure you get the correct submodule revision when checking out a tagged release? To get a precise revision the superproject needs to track a sha1 of a submodule commit. I do not see how that has anything to do with submodule.<name>.branch?

Show 5 quoted lines
> >  - In which situations does the developer or maintainer switch between
> >    your attached/detached mode?
> 
> The developer/maintainer does so optionally and voluntarily and it
> effects only its private working tree.

This does not answer my question. I would like to find out the reason why one would do the switch.

Show 9 quoted lines
> >  - What is the "repository branch" which is given to the developer by
> >    the maintainer used for? Who creates this branch and who merges into
> >    it?
> 
> The branch of course must exist prior submodule adding. In this
> use-case it does not really matter who creates it and who merges into
> it. Everyone with the right to merge into it has to work in the
> submodule seamlessly, as it was working on separate clone of the same
> repository used as the submodule.

o Here is the same. I am searching for a description like:

If the developer works on a feature that needs a submodule change he:
  - creates a submodule branch
  - configures that submodule branch in the superproject:
  	git config -f .gitmodules submodule.common.branch dev/some-feature
	git commit -am "TEMP: track submodule common on branch"
 - and pushes out his superproject branch
The submodule branch is then posted for review and continued to work on.

Once everyone involved is happy with the submodule change the branch in there gets merged to master.

Now the branch in the superproject is modified to drop the change in .gitmodules and the sha1 reference in the superproject is updated to the current master of the superproject.

The superproject branch is posted for review.
...

Could you describe something like this for your workflow? A complete change lifecycle when a developer works, as you call it, "actively" in a submodule?

Show 7 quoted lines
> >  - What are these subsequent "merge" or "rebase" update operations? Do
> >    you mean everyone has submodule.name.update configured to merge or
> >    rebase?
> >
> 
> subsequent "merge" or "rebase" update operations are just the ones
> after the initial clone/checkout, nothing particular.

To clarify you are talking about issuing "git merge" or "git rebase" commands in the superproject?

Cheers Heiko
Francesco Pretto· Jan 6, 2014, 17:47 UTC · re: Heiko Voigt · lore

Re: Re: [PATCH 2/2] Introduce git submodule attached update

Dear Heiko, my replies below. I also take a couple excerpts from other emails, as I prefer to not flame on different threads :) .

2014/1/6 Heiko Voigt <hvoigt@hvoigt.net>:
Show 10 quoted lines
> On Sun, Jan 05, 2014 at 10:46:11PM +0100, Francesco Pretto wrote:
>> It means he doesn't need to control other developers commit to be
>> checked out so he sets "submodule.<name>.ignore" to "all". In this way
>> he and the developers can work actively in their submodule copy.
>
> So practically speaking: You mean that the value of
> submodule.<name>.ignore is set to "all" in the master branch of the
> superproject? From your other email referring to svn:externals I figure
> that.
>

Correct, but this works also if the branch of the superproject is a different branch than "master". I think you are right in a point, see the next reply.

Show 5 quoted lines
> The workflow could always be changed to allow recording revisions. Which
> is why you use git in the first place right? If you discard revisions
> for submodules tracking down regression bugs can become a big problem or
> completely impossible. Try using git bisect on such a history.
>

Ok, you are right: setting "submodule.<name>.ignore" to "all" by default with a switch "--attached" in "git submodule add" is too much. My point is just make sure users will checkout an attached HEAD.

Show 13 quoted lines
>> "submodule.<name>.branch" is one setting that is not copied in
>> ".git/config" by "git submodule init". "git submodule update" will use
>> the setting in ".gitmodules" if not overridden voluntarily by the
>> developer in ".git/config". The maintainer can change that setting in
>> ".gitmodules" and commit the change. Modifies will be propagated by
>> the next "git pull && git submodule update" of the developer in the
>> superproject.
>
> I do not understand how does that ensure you get the correct submodule
> revision when checking out a tagged release? To get a precise revision
> the superproject needs to track a sha1 of a submodule commit. I do not
> see how that has anything to do with submodule.<name>.branch?
>

"submodule.<name>.attacched" set to true implies "--remote". sha1 of the latest commit is taken from "origin/$branch". In this way you get the latest commit of that branch, and you do a 'merge', 'rebase', 'checkout' or '!command' according to the configured ''. This mechanism is already in the patch at the current state.

Show 9 quoted lines
>> >  - In which situations does the developer or maintainer switch between
>> >    your attached/detached mode?
>>
>> The developer/maintainer does so optionally and voluntarily and it
>> effects only its private working tree.
>
> This does not answer my question. I would like to find out the reason
> why one would do the switch.
>

The developer does it voluntarily, at his responsibility, because he may decide to partecipate more actively to the development of the submodule and still want to use a simple "git submodule update" to updates his submodules, overriding its configuration as it can be done for other properties like, for example, "branch". This ability is of course already possible reattaching the HEAD manually but you loose the convenient ability to use "git submodule update".

Show 32 quoted lines
>> The branch of course must exist prior submodule adding. In this
>> use-case it does not really matter who creates it and who merges into
>> it. Everyone with the right to merge into it has to work in the
>> submodule seamlessly, as it was working on separate clone of the same
>> repository used as the submodule.
> o
> Here is the same. I am searching for a description like:
>
> If the developer works on a feature that needs a submodule change he:
>   - creates a submodule branch
>   - configures that submodule branch in the superproject:
>         git config -f .gitmodules submodule.common.branch dev/some-feature
>         git commit -am "TEMP: track submodule common on branch"
>  - and pushes out his superproject branch
>
> The submodule branch is then posted for review and continued to work on.
>
> Once everyone involved is happy with the submodule change the branch in
> there gets merged to master.
>
> Now the branch in the superproject is modified to drop the change in
> .gitmodules and the sha1 reference in the superproject is updated to the
> current master of the superproject.
>
> The superproject branch is posted for review.
>
> ...
>
> Could you describe something like this for your workflow? A complete
> change lifecycle when a developer works, as you call it, "actively" in a
> submodule?
>

I'm really sorry, I thought this was already clear from the first patch iteration. I will go more in depth:

Say we have our actual projects "project1" and "project2". Say we have a project "common". This "common" project is *not* independent: it exists only to serve "project1" and "project2" and it's tested only in the scope of "project1" and "project2". Also "common" is very actively developed and follows the same lifecyle of "project1" and "project2" on separate branches. I think that it's important that you get this point: developers of "common" don't clone it separately, as it would be impossible to test it, they clone "project1" and "project2" and expect to find it inside one of these.

Let say now how "common" can be shared between "project1" and
"project2" so developers of "project1" don't break "project2" working
on "common" and the other way around. "common" has the following
stable branches:
- master;
- master-project1;
- master-project2;

The maintainer of "project1" at master branch sets "common" as submodule at branch "master-project1". The maintainer of "project2" at master branch sets "common" as submodule at branch "master-project2". Periodically a maintainer of "common" will pull both master-project1 and master-project1 on a branch "next". Maintainers of both "project1" and "project1", in coordination, are responsible to make next to be merged in master, so they can both sync their master-project1 and master-project2 in the "common" submodule.

Let say now how for example {"project1" and "common"} can evolve. As told, these can't really evolve separately. Maintainer of "project1" prepares a branch called "staging-featureA". "featureA" is big and will require more people to work on the same feature for days/weeks. Maintainer of "project1" also prepares a branch "project1-staging-featureA" on "common" and set ".gitmodules" of "project1" to point to "project1-staging-featureA". Developers of featureA would like to do this:

$ git pull
$ git checkout staging-featureA
$ git submodule update      # clones an attached HEAD of common on the branch
                                        #
'submodule.common.project1.staging-featureA'
$ .... start coding in common seamlessly as they where in project1 ....

Also developers do frequently rebase: $ git pull --rebase $ git submodule update

Or maybe a shortcut of this: "git submodule update" should be given the possibility to go "--remote" by default. Of course if "common" of the developer is in a branch different that 'submodule.<name>.branch' "git submodule update" has not to switch the branch.

Show 7 quoted lines
>> Maybe who coded submodules at first was thinking that the best way to
>> contribute to a project is to checkout that repository, and not work
>> in the submodule. As said, this works well when the submodule
>> repository is a full project, and not a bunch of shared code.
>
>Why not work in the submodule? See explanation above.
>

Because, as said above, the submodule is not independent. It does not have proper code that test it and the best test case is using the submodule in the scope of the superproject.

Show 11 quoted lines
>> >  - What are these subsequent "merge" or "rebase" update operations? Do
>> >    you mean everyone has submodule.name.update configured to merge or
>> >    rebase?
>> >
>>
>> subsequent "merge" or "rebase" update operations are just the ones
>> after the initial clone/checkout, nothing particular.
>
> To clarify you are talking about issuing "git merge" or "git rebase"
> commands in the superproject?
>

No, I'm talking about "git submoudule update" with "--merge" or "--rebase switches or "submodule.name.update" configured.

2014/1/6 Heiko Voigt <hvoigt@hvoigt.net>:
Show 6 quoted lines
> I am not so sure. svn:externals was IMO a hack in SVN to bind projects
> together. It does not record the revision and so has nothing to do
> with version control. If you simply want to always checkout the
> development tip of some project you could do something like this:
>
>        git submodule foreach 'git fetch && git checkout origin/master'

This can be very unconvenient if the reccomended *starting* branch to where attach the HEAD is not "master":

git submodule foreach 'branch="$(git config -f $toplevel/.gitmodules submodule.$name.branch)"; git checkout origin/$branch

Of course the developer after they may want to move to a local branch (or may not: please don't forget shared repositories), but the above command may difficult to be taught. Also with the comit[1] that blocks copying of !command to ".git/config" and sets default "none", you made it harder to offer a mantainer decided default update behavior like the one I described.

Show 5 quoted lines
> The demand for this 'missing feature' which we call the 'floating
> submodules' model has been around for some time but until now we could
> convince people that its not a feature but you are actually loosing
> history information.
>

Putting your reasoning to the extreme consequences: why does not "git clone" clone a dettached HEAD? In your workflow I should probably never start coding from "master", but people frequently do instead. Also, as you observed, it's possible to track submodule revision sha1 also on an attached HEAD. If I don't want anyone to track the revision I can already set 'ignore' property.

My bottom line:
- For what I understand, detached HEAD it's a way to say "hey, you
have to stay on this commit. Also don't even think you can push to the
upstream branch". This sometimes can't be spurious, as in the use case
I wrote above: access control on the remote repositories should be
enough. I think maintainers should have the option to make developers
to clone a repository starting with an attached HEAD on the branch
suggested in submodule.$name.branch;
- "git submodule update" is missing a property to do automatically
"--remote". I think in the use case I wrote it's really handy to have
a "git submodule update" to act like this.
Thank you for reading and sorry for the long email :)

Greetings, Francesco

[1] http://marc.info/?l=git&m=138610752125816&w=2
David Engster· Jan 6, 2014, 19:21 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

Francesco Pretto writes:
Show 8 quoted lines
> 2014/1/6 Heiko Voigt <hvoigt@hvoigt.net>:
>> Could you describe something like this for your workflow? A complete
>> change lifecycle when a developer works, as you call it, "actively" in a
>> submodule?
>>
>
> I'm really sorry, I thought this was already clear from the first
> patch iteration. I will go more in depth:

While I have some trouble understanding all the details of Francesco's description, I find the idea of "attaching submodules to branches" very useful. I think I could well use that to simplify my merging grunt work for GNU Emacs (which, in case you're wondering, is probably switching to git as its VCS). I don't mean to hijack this thread, and I guess my use case is a bit different than what Francesco has in mind; still, I think it is similar enough that my use case could help in talking about the details of his patch; if not, please feel free to ignore it.

GNU Emacs ships with some pretty large packages, namely Gnus, Org and CEDET, which are also available as "stand-alone" versions for manual installation, and their development happens in separate upstream repositories. Since I'm a CEDET developer, I'll use it as an example in the following.

First off, it is important to note that merges are always bi-directional: not only is new CEDET code pulled into Emacs, but Emacs developers also change things which have to be merged back upstream. So far, merging between the two repositories was done manually by me, which is error-prone (and boring). I think that by pulling in CEDET directly as a submodule, this merging could be made easier. Most importantly, my hope is that more people than me could do it. :-)

Here's how I would like this to work; first the CEDET -> Emacs part, which is rather straight-forward:

- The CEDET repository has two branches: 'master' and 'stable'.
- The Emacs repository imports CEDET's 'stable' branch as a submodule.
- CEDET's main development happens in 'master', and the CEDET developers
  are responsible for merging stable code to 'stable'. They will then
  make a new commit for the submodule in Emacs accordingly.

The Emacs -> CEDET part is more hairy. Most of the time, the fixes happening in the Emacs repository for CEDET are very small and/or trivial and can usually be considered "always stable": fixes for spelling, compiler warnings, or small refactorings like renames, etc. This kind of "merging back to CEDET upstream" should hence be as easy as possible for Emacs developers:

- When an Emacs developer changes something in the CEDET submodule, the
  changes they commit should by default automatically land in CEDET's
  'stable' branch. That means that when they enter the submodule, they
  should be in the branch 'stable' instead of being detached, and a push
  should update the 'stable' branch in CEDET accordingly. The submodule
  must then be committed as well.
- It is then up to the CEDET developers to merge these changes into the
  'master' branch of the CEDET repo.

I know that the "correct" workflow would be to always use feature branches, but it'd be nice if that could be avoided if one so chooses.

A little picture in the hope that it makes things clearer:
             +-----------+
             |  master   | <--
+-------+    +-----------+    | Merges to/from master
| CEDET |                     | done only by CEDET developers
+-------+                     | 
             +-----------+    |
             |  stable   | <--  <--------
             +-----------+               |
                                         |
                                         |
                                         | Any Emacs developer
                                         | can push and commit
                                         | submodule
+--------+    +----------------------+   |
| Emacs  | -- | lisp/cedet submodule | <-
+--------+    +----------------------+

AFAICS the main problem with this approach is that one always has to think of committing the new SHA1 of the submodule. If I understand Francesco correctly, he wants to eliminate the need for that by simply always taking the head of the attached branch. I also think that would be a nice feature, since in the above drawing, the lisp/cedet submodule should always follow the 'stable' branch in CEDET upstream. However, as Heiko notes, the history must be preserved to be able to go back to earlier revisions, so there must be some kind of commit for the submodule when 'stable' changes; maybe that could be automated somehow?

-David
W. Trevor King· Jan 7, 2014, 19:27 UTC · re: David Engster · lore

Re: [PATCH 2/2] Introduce git submodule attached update

On Mon, Jan 06, 2014 at 08:21:24PM +0100, David Engster wrote:
Show 16 quoted lines
>              +-----------+
>              |  master   | <--
> +-------+    +-----------+    | Merges to/from master
> | CEDET |                     | done only by CEDET developers
> +-------+                     | 
>              +-----------+    |
>              |  stable   | <--  <--------
>              +-----------+               |
>                                          |
>                                          |
>                                          | Any Emacs developer
>                                          | can push and commit
>                                          | submodule
> +--------+    +----------------------+   |
> | Emacs  | -- | lisp/cedet submodule | <-
> +--------+    +----------------------+

This looks reasonable, and except for the detached-HEAD after the initial update-clone, I think Git already supports everything you need. If you set submodule.cedet.update to 'rebase' (or 'merge') you can easily integrate your local master changes with cedet/master (e.g. if a CEDET dev updates cedet/master before the Emacs dev has a chance to push their fix). With the non-checkout update mode, you'll also stay on your checked-out master branch during 'submodule update' calls.

Show 7 quoted lines
> AFAICS the main problem with this approach is that one always has to
> think of committing the new SHA1 of the submodule.
> …
> However, as Heiko notes, the history must be preserved to be able to
> go back to earlier revisions, so there must be some kind of commit
> for the submodule when 'stable' changes; maybe that could be
> automated somehow?

If an Emacs dev in the submodule makes the CEDET change, you could use a post-commit hook (in the CEDET submodule) to also commit the change to the Emacs superproject). However, commiting only the submodule bump may not be what you want. Maybe there are other superproject changes that should be committed alongside the submodule bump. Maybe there is stuff in the superprojects's staging area that should *not* be committed alongside the submodule bump. This ambiguity makes it tricky for Git to automatically do “the right thing”.

If cedet/master is updated independently by the CEDET devs, there's no way for the local Emacs repo to know about the change, so it's impossible to automatically update Emacs (without polling for CEDET updates or some other transgression ;).

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 7, 2014, 04:10 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

On Mon, Jan 06, 2014 at 06:47:58PM +0100, Francesco Pretto wrote:
> I'm really sorry, I thought this was already clear from the first
> patch iteration. I will go more in depth:
For me anyway, this extra detail is very helpful.  Thanks :).
Show 10 quoted lines
> Maintainer of "project1" also prepares a branch
> "project1-staging-featureA" on "common" and set ".gitmodules" of
> "project1" to point to "project1-staging-featureA". Developers of
> featureA would like to do this:
> 
> $ git pull
> $ git checkout staging-featureA
> $ git submodule update      # clones an attached HEAD of common on the branch
>                             # 'submodule.common.project1.staging-featureA'
> $ .... start coding in common seamlessly as they where in project1 ....

So the checked-out branch switches depending on the local superproject branch. That sounds nice, but I'm not sure where the superproject-branch-to-local-submodule-branch mapping would be stored. We currently do this for remote-tracking submodule branches with an in-tree .gitmodules (which can differ between submodule branches) with local overides in a single out-of-tree .git/config (which is independent of the checked out branch). Ideally we'd have a way to add local overrides on a per-superproject-branch basis, but I don't know what that would look like.

Show 6 quoted lines
> Also developers do frequently rebase:
> $ git pull --rebase
> $ git submodule update
> 
> Or maybe a shortcut of this: "git submodule update" should be given
> the possibility to go "--remote" by default.

Rebasing the superproject and then updating the submodules (to the superproject's gitlinked commits) is not the same as a --remote update (to the subproject's upstream branch tip).

> Of course if "common" of the developer is in a branch different that
> 'submodule.<name>.branch' "git submodule update" has not to switch
> the branch.
I don't understand what you're saying here.
Show 11 quoted lines
> >> Maybe who coded submodules at first was thinking that the best
> >> way to contribute to a project is to checkout that repository,
> >> and not work in the submodule. As said, this works well when the
> >> submodule repository is a full project, and not a bunch of shared
> >> code.
> >
> >Why not work in the submodule? See explanation above.
> 
> Because, as said above, the submodule is not independent. It does
> not have proper code that test it and the best test case is using
> the submodule in the scope of the superproject.

You can cd into the submodule, and develop it as an independent repository. When you want to test your changes, just cd back into the superproject and run your test suite.

Show 12 quoted lines
> 2014/1/6 Heiko Voigt <hvoigt@hvoigt.net>:
> > I am not so sure. svn:externals was IMO a hack in SVN to bind projects
> > together. It does not record the revision and so has nothing to do
> > with version control. If you simply want to always checkout the
> > development tip of some project you could do something like this:
> >
> >        git submodule foreach 'git fetch && git checkout origin/master'
> 
> This can be very unconvenient if the reccomended *starting* branch to
> where attach the HEAD is not "master":
> git submodule foreach 'branch="$(git config -f $toplevel/.gitmodules
> submodule.$name.branch)"; git checkout origin/$branch
Which is equivalent to:
  $ git submodule update --remote --checkout

except for branch-vs-detached-HEAD. If you are doing local development, I'd recommend setting up submodule.<name>.update to a non-checkout strategy and using:

  $ git submodule update --remote

which will integrate the upstream changes with any local changes (updating whichever local submodule branch you had checked out).

> Also with the comit[1] that blocks copying of !command to
> ".git/config" and sets default "none", you made it harder to offer a
> mantainer decided default update behavior like the one I described.

The maintainer can still suggest checkout/pull/rebase, and the developer can still clear remove the none from .git/config after initializing the submodule. You only need to do this once per submodule.

> I think maintainers should have the option to make developers to
> clone a repository starting with an attached HEAD on the branch
> suggested in submodule.$name.branch;

I agree, and want to use a non-checkout submodule.<name>.update mode to identify developers who would want this. My v2 patch switches on submodule.<name>.branch, but I'll update it in v3 to switch on submodule.<name>.update. There's no need to confuse this with additional attach/detach functionality.

> - "git submodule update" is missing a property to do automatically
> "--remote". I think in the use case I wrote it's really handy to have
> a "git submodule update" to act like this.

You can already add aliases, but a remote/local-gitlink config variable would be nice too.

Here's an attempted summary of our desires, and my ideal route forward:

* Preferred local submodule branches for each superproject branch.
  * Not currently supported by Git.
  * Requires some sort of per-superproject-branch .git/config.
  * Fall back to the remote-tracking submodule.<name>.branch?
* Auto checkout of the preferred branch
  * Can do this at clone-update time with my patch.
  * For later submodule branch switches, maybe we want:
      git submodule checkout [-b <branch>] [<paths>…]
    Then if a user blows off their detached HEAD, at least they'll
    feel a bit sheepish afterwards.
* Configurable (remote or local) default update source (so folks who
  primarily update --remote don't have to have long command lines).
  * New submodule.<name>.source = {remote|local} config
  * New 'update [--source={local|remote}]' option
  * Deprecate 'update --remote' with a long phase out.
  However:
  * Maybe they should just setup an alias instead?

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 7, 2014, 22:36 UTC · re: W. Trevor King · lore

Preferred local submodule branches (was: Introduce git submodule attached update)

On Tue, Jan 07, 2014 at 10:51:34PM +0100, Francesco Pretto wrote:
Show 10 quoted lines
> 2014/1/7 W. Trevor King <wking@tremily.us>:
> >
> > I'd be happy to hear ideas about superproject-branch-specific local
> > overrides to a hypothetical submodule.<name>.local-branch, in the
> > event that a developer doesn't like a default set in .gitmodules.  If
> > I could think of a way to do that, we could avoid this heuristic
> > approach, and make the local submodule.<name>.local-branch
> > vs. remote-tracking submodule.<name>.branch distinction more obvious.
> 
> Uh, I think you got it wrong in the other thread:

I'm grafting this discussion back on to the thread where I proposed submodule.<name>.local-branch.

> I didn't proposed such feature.
Right.  I proposed this feature after reading your proposed workflow.
> I just wanted the attached submodule use case to be supported and of
> course "--branch means attached" is even easier to get this.

As I understood it, the '--branch means attached' stuff was tied up with automatic --remote updates.

There are three branches that submodule folks usually care about:
1. The linked $sha1 in the superproject (set explicitly for every
   superproject commit, and thus for every superproject branch).
2. The remote-tracking submodule.<name>.branch that lives in the
   upstream submodule.<name>.url repository.
3. The submodule's locally checked out branch, which we currently let
   the developer setup by hand, which is used integrated with one of
   the other two branches during non-checkout updates.

Git is currently a bit weak on conveniently handling type-3 branches. “Just use what the developer has setup” works well for many basic workflows, but falls short for:

* Cloning-updates, where we currently always setup a detached HEAD.
* Workflows where the preferred type-3 branch depends on the
  superproject branch.

The former is easy to fix [1] if you accept submodule.<name>.branch as a guess, but this conflates the type-2 and type-3 branches.

For the latter, you'd want something like:
On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:
Show 8 quoted lines
> * Auto checkout of the preferred branch
>   * Can do this at clone-update time with my patch.
>   * For later submodule branch switches, maybe we want:
> 
>       git submodule checkout [-b <branch>] [<paths>…]
> 
>     Then if a user blows off their detached HEAD, at least they'll
>     feel a bit sheepish afterwards.
which would likely need some of Jens' new core checkout handling [2].

Cheers, Trevor

[1]: Using something along the lines of my
     http://article.gmane.org/gmane.comp.version-control.git/239967
[2]: http://article.gmane.org/gmane.comp.version-control.git/240117
-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 7, 2014, 23:52 UTC · re: W. Trevor King · lore

Re: Preferred local submodule branches (was: Introduce git submodule attached update)

On Tue, Jan 07, 2014 at 02:36:25PM -0800, W. Trevor King wrote:
Show 38 quoted lines
> There are three branches that submodule folks usually care about:
> 
> 1. The linked $sha1 in the superproject (set explicitly for every
>    superproject commit, and thus for every superproject branch).
> 2. The remote-tracking submodule.<name>.branch that lives in the
>    upstream submodule.<name>.url repository.
> 3. The submodule's locally checked out branch, which we currently let
>    the developer setup by hand, which is used integrated with one of
>    the other two branches during non-checkout updates.
> 
> Git is currently a bit weak on conveniently handling type-3 branches.
> “Just use what the developer has setup” works well for many basic
> workflows, but falls short for:
> 
> * Cloning-updates, where we currently always setup a detached HEAD.
> * Workflows where the preferred type-3 branch depends on the
>   superproject branch.
> 
> The former is easy to fix [1] if you accept submodule.<name>.branch as
> a guess, but this conflates the type-2 and type-3 branches.
> 
> For the latter, you'd want something like:
> 
> On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:
> > * Auto checkout of the preferred branch
> >   * Can do this at clone-update time with my patch.
> >   * For later submodule branch switches, maybe we want:
> > 
> >       git submodule checkout [-b <branch>] [<paths>…]
> > 
> >     Then if a user blows off their detached HEAD, at least they'll
> >     feel a bit sheepish afterwards.
> 
> which would likely need some of Jens' new core checkout handling [2].
> 
> [1]: Using something along the lines of my
>      http://article.gmane.org/gmane.comp.version-control.git/239967
> [2]: http://article.gmane.org/gmane.comp.version-control.git/240117

For example, in Jonathan's recent version of Jens' series, the initial-setup and update functionality are moving into C. See:

* populate_submodule() [1] for the initial-clone setup (calling
  'read-tree'), and
* update_submodule() [2] for subsequent updates (calling 'checkout -q'
  with an optional '-f')

this is where any submodule.<name>.local-branch would come into play, if we decide to go down that route. It doesn't look like the C updates have the auto-clone functionality that the Bash updates have. I'm not sure if that's in the pipe or not. I'm not as familiar with the C implementation though, so maybe I'm missing the mark here.

Cheers, Trevor

[1]: http://article.gmane.org/gmane.comp.version-control.git/239698 [2]: http://article.gmane.org/gmane.comp.version-control.git/239699

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 8, 2014, 03:47 UTC · re: W. Trevor King · lore

Re: Preferred local submodule branches

On Wed, Jan 08, 2014 at 03:12:44AM +0100, Francesco Pretto wrote:
Show 6 quoted lines
> 2014/1/8 W. Trevor King <wking@tremily.us>:
> > Note that I've moved away from “submodule.<name>.branch
> > set means attached” towards “we should set per-superproject-branch
> > submodule.<name>.local-branch explicitly” [1].
> 
> Honestly, I'm having an hard time to follow this thread.

I tried to refocus things (with a new subject) in this sub-thread. Hopefully that helps make the discussion more linear ;).

> Also, you didn't update the patch.

I'm waiting [1] to see how the C-level checkout by Jens and Jonathan progresses [2,3] before writing more code.

> If you were endorsed by someone (Junio, Heiko, ...) for the
> "submodule.<name>.local-branch" feature please show me where.

As far as I know, no-one else has endorsed this idea (yet :). Heiko has expressed concern [4], but not convincingly enough (yet :) to win me over ;).

Show 6 quoted lines
> I somehow understand the point of the
> "submodule.<name>.local-branch" property, but I can't "see" the the
> workflow. Please, show me some hypothetical scripting example with
> as much complete as possible workflow (creation, developer update,
> mantainers creates feature branch, developer update, developer
> attach to another branch).

I've put this at the bottom of the message to avoid bothering the tl;dr crowd, although they have probably long since tuned us out ;).

> Also, consider I proposed to support the attached HEAD path to
> reduce complexity and support a simpler use case for git
> submodules. I would be disappointed if the complexity is reduced in
> a way and augmented in another.

Agreed. I think we're all looking for the least-complex solution that covers all (or most) reasonable workflows.

Show 12 quoted lines
> > On Wed, Jan 08, 2014 at 01:17:49AM +0100, Francesco Pretto wrote:
> >> # Attach the submodule HEAD to <branch>.
> >> # Also set ".git/config" 'submodule.<module>.branch' to <branch>
> >> $ git submodule head -b <branch> --attach <module>
> > [...]
> > I also prefer 'checkout' to 'head', because 'checkout'
> > already exists in non-submodule Git for switching between local
> > branches.
> 
> I can agree with similarity to other git commands, but 'checkout'
> does not give me the idea of something that writes to ".git/config"
> or ".gitmodules".

Neither does 'head'. We have precedence in 'git submodule add' for embracing and extending a core git command with additional .gitmodules manipulation. I think it's easier to pick up the submodule jargon when we add submodule-specific side-effects to submodule-specific commands named after their core analogs than it would be if we pick unique names for the submodule-specific commands.

Show 15 quoted lines
> >> # Unset  ".git/config" 'submodule.<module>.branch'
> >> # Also attach or detach the HEAD according to what is in ".gitmodules":
> >> # with Trevor's patch 'submodule.<module>.branch' set means attached,
> >> # unset means detached
> >> $ git submodule head --reset <module>
> >
> > To me this reads “always detach HEAD” (because it unsets
> > submodule.<name>.branch, and submodule.<name>.branch unset means
> > detached).
> 
> I disagree: this would remove only the value in ".git/config". If the
> value is till present in ".gitmodules", as I wrote above, the behavior
> of what is in the index should be respected as for the other
> properties. Also it gives a nice meaning to a switch like --reset :
> return to how it was before.

Ah, that makes more sense. I had confused .git/config with “.gitmodules and .git/config”.

Show 17 quoted lines
> >> NOTE: feature branch part!
> >>
> >> # Set ".gitmodules" 'submodule.<module>.branch' to <branch>
> >> $ git submodule head -b <branch> --attach --index <module>
> >>
> >> # Unset ".gitmodules" 'submodule.<module>.branch'
> >> $ git submodule head --reset --index <module>
> >> ---------------------------------------------------------------------
> >
> > These are just manipulating .gitmodules.  I think we also need
> > per-superproject-branch configs under the superproject's .git/ for
> > developer overrides.
> 
> I disagree: in my idea the --index switch is a maintainer only command
> to modify the behavior of the developers and touch only indexed files
> (.gitmodules, or create a new submodule branch). It expressly don't
> touch .git/config.

Something that just touches the config files is syntactic sugar, so I avoided a more detailed review and moved on to address what I saw as a more fundamental issue (preferred submodule local branches on a per-superproject-branch level).

Here's a detailed workflow for the {my-feature, my-feature, master} example I roughed out before [5].

  # create the subproject
  mkdir subproject &&
  (
    cd subproject &&
    git init &&
    echo 'Hello, world' > README &&
    git add README &&
    git commit -m 'Subproject v1'
  ) &&
  # create the superproject
  mkdir superproject
  (
    cd superproject &&
    git init &&
    git submodule add ../subproject submod &&
    git config -f .gitmodules submodule.submod.update merge &&
    git commit -am 'Superproject v1' &&
    ( # 'submodule update' doesn't look in .gitmodules (yet [6]) for a
      # default update mode.  Copy submodule.submod.update over to
      # .git/config
      git submodule init
    )
  ) &&
  # start a feature branch on the superproject
  (
    cd superproject &&
    #git checkout -b my-feature --recurse-submodules &&
    ( # 'git submodule checkout --recurse-submodules' doesn't exist yet, so...
      git checkout -b my-feature &&
      git config -f .gitmodules submodule.submod.local-branch my-feature &&
      cd submod &&
      git checkout -b my-feature
    ) &&
    (
      cd submod &&
      echo 'Add the subproject side of this feature' > my-feature &&
      git add my-feature &&
      git commit -m 'Add my feature to the subproject'
    ) &&
    echo 'Add the superproject side of this feature' > my-feature &&
    git add my-feature &&
    git commit -m 'Add the feature to the superproject'
  ) &&
  # meanwhile, the subproject has been advancing
  (
    cd subproject &&
    echo 'Goodbye, world' >> README &&
    git commit -am 'Subproject v2'
  ) &&
  # we need to get that critical advance into the superproject quick!
  (
    cd superproject &&
    # update the master branch
    #git checkout --recurse-submodules master
    ( # 'git checkout --recurse-submodules' doesn't exist yet [2,3].
      # Even with that patch, 'git checkout' won't respect
      # submodule.<name>.local-branch without further work.
      git checkout master &&
      cd submod &&
      git checkout master  # don't pull in our my-feature work
    )
    git submodule update --remote &&
    git commit -am 'Catch submod up with Subproject v2' &&
    # update the my-feature branch
    git checkout my-feature
    ( # 'git checkout' doesn't mess with submodules
      cd submod &&
      git checkout my-feature
    )
    git submodule update --remote &&
    git commit -am 'Catch submod up with Subproject v2' &&
    # what does the history look like?
    (
      cd submod &&
      git --no-pager log --graph --date-order --oneline --decorate --all
      # *   3a22cef (HEAD, my-feature) Merge commit 'd53958b18277ce5bd6c734e9597a69bb878b31e1' into my-feature
      # |\  
      # * | 8322dcc Add my feature to the subproject
      # | * d53958b (origin/master, origin/HEAD, master) Subproject v2
      # |/  
      # * 9813010 Subproject v1
    ) &&
    git ls-tree master submod &&
    # 160000 commit d53958b18277ce5bd6c734e9597a69bb878b31e1  submod
    git ls-tree my-feature submod
    # 160000 commit 3a22cef30db57f1b89251f3e434fa0bd0f1b99a2  submod
  )
  git --version
  # git version 1.8.3.2
The currently-ugly bits could be fixed with:
* 'git submodule update' falling back on .gitmodules for
  submodule.<name>.update [6].
* 'git submodule checkout -b my-feature --recurse-submodules' should
  checkout the submodule.<name>.local-branch configured for the
  super-project's my-feature branch (but only if that wouldn't destroy
  some current submodule information).  This would build on work in
  Jens and Jonathans' branch [2,3].

Cheers, Trevor

[1]: http://article.gmane.org/gmane.comp.version-control.git/240127 [2]: http://article.gmane.org/gmane.comp.version-control.git/240117 [3]: http://thread.gmane.org/gmane.comp.version-control.git/239695 [4]: http://article.gmane.org/gmane.comp.version-control.git/240178 [5]: http://article.gmane.org/gmane.comp.version-control.git/240190 [6]: http://article.gmane.org/gmane.comp.version-control.git/239246

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 8, 2014, 04:06 UTC · re: W. Trevor King · lore

Re: Preferred local submodule branches

On Tue, Jan 07, 2014 at 07:47:08PM -0800, W. Trevor King wrote:
Show 16 quoted lines
>     #git checkout --recurse-submodules master
>     ( # 'git checkout --recurse-submodules' doesn't exist yet [2,3].
>       # Even with that patch, 'git checkout' won't respect
>       # submodule.<name>.local-branch without further work.
>       git checkout master &&
>       cd submod &&
>       git checkout master  # don't pull in our my-feature work
>     )
>     git submodule update --remote &&
>     git commit -am 'Catch submod up with Subproject v2' &&
>     # update the my-feature branch
>     git checkout my-feature
>     ( # 'git checkout' doesn't mess with submodules
>       cd submod &&
>       git checkout my-feature
>     )
Oops, the my-feature checkout block should have been:
    #git checkout --recurse-submodules my-feature
    ( # 'git checkout --recurse-submodules' doesn't exist yet...
      git checkout my-feature &&
      cd submod &&
      git checkout my-feature
    )

mirroring the earlier master checkout block. Sorry for the sloppy editing.

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 9, 2014, 06:17 UTC · re: W. Trevor King · lore

[RFC v3 0/4] Preferred local submodule branches

From: "W. Trevor King" <wking@tremily.us>

In another branch of the submodule thread Francesco kicked off, I mentioned that we could store the preferred local submodule branch on a per-superbranch level if we used the .git/modules/<submodule-name>/config for local overrides [1]. Here's a patch series that greatly extends my v2 "submodule: Respect requested branch on all clones" series [2] to also support automatic, recursive submodule checkouts, as I outlined here [3]. After this series, I can get through:

  # create the subproject
  mkdir subproject &&
  (
    cd subproject &&
    git init &&
    echo 'Hello, world' > README &&
    git add README &&
    git commit -m 'Subproject v1'
  ) &&
  # create the superproject
  mkdir superproject
  (
    cd superproject &&
    git init &&
    git submodule add ../subproject submod &&
    git config -f .gitmodules submodule.submod.update merge &&
    git commit -am 'Superproject v1' &&
    ( # 'submodule update' doesn't look in .gitmodules (yet [4]) for a
      # default update mode.  Copy submodule.submod.update over to
      # .git/config
      git submodule init
    )
  ) &&
  # start a feature branch on the superproject
  (
    cd superproject &&
    #git checkout -b my-feature --recurse-submodules &&
    ( # 'git submodule checkout --recurse-submodules' doesn't exist yet, so...
      git checkout -b my-feature &&
      git submodule checkout -b --gitmodules
    ) &&
    (
      cd submod &&
      echo 'Add the subproject side of this feature' > my-feature &&
      git add my-feature &&
      git commit -m 'Add my feature to the subproject'
    ) &&
    echo 'Add the superproject side of this feature' > my-feature &&
    git add my-feature &&
    git commit -am 'Add the feature to the superproject'
  ) &&
  # meanwhile, the subproject has been advancing
  (
    cd subproject &&
    echo 'Goodbye, world' >> README &&
    git commit -am 'Subproject v2'
  ) &&
  # we need to get that critical advance into the superproject quick!
  (
    cd superproject &&
    # update the master branch
    #git checkout --recurse-submodules master
    ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6].
      # Even with that patch, 'git checkout' won't respect
      # submodule.<name>.local-branch without further work.
      git checkout master &&
      git submodule checkout
    ) &&
    git submodule update --remote &&
    git commit -am 'Catch submod up with Subproject v2' &&
    # update the my-feature branch
    #git checkout --recurse-submodules my-feature &&
    ( # 'git checkout --recurse-submodules' doesn't exist yet [5,6].
      git checkout my-feature &&
      git submodule checkout
    ) &&
    git submodule update --remote &&
    git commit -am 'Catch submod up with Subproject v2' &&
    # what does the history look like?
    (
      cd submod &&
      git --no-pager log --graph --date-order --oneline --decorate --all
      # *   16d9e3e (HEAD, my-feature) Merge commit 'f5e134d5747ee4a206e96d8c017f92f5b29a07f3' into my-feature
      # |\  
      # | * f5e134d (origin/master, origin/HEAD, master) Subproject v2
      # * | 0a1cd07 Add my feature to the subproject
      # |/  
      # * c2d32ba Subproject v1
    ) &&
    printf 'master: ' &&
    git ls-tree master submod &&
    # master: 160000 commit f5e134d5747ee4a206e96d8c017f92f5b29a07f3  submod
    printf 'my-feature: ' &&
    git ls-tree my-feature submod
    # my-feature: 160000 commit 16d9e3ea2fb57e7a166587203abdb328f90895d1  submod
  )
  git --version
  # git version 1.8.5.2.237.g01c62c6

I think the first three patches are fairly solid. The last one gets through the above script, but I'd need a more thorough test suite before I trusted it. I tried to be detailed in the commit messages, but of course, we'd want some user-facing documentation if we actually merged something like this series. I'm sending it to the list mostly to explain my current views and re-focus debate [1].

[1]: http://article.gmane.org/gmane.comp.version-control.git/240240 [2]: http://article.gmane.org/gmane.comp.version-control.git/239967 [3]: http://article.gmane.org/gmane.comp.version-control.git/240192 [4]: http://article.gmane.org/gmane.comp.version-control.git/239246 [5]: http://thread.gmane.org/gmane.comp.version-control.git/239695 [6]: http://article.gmane.org/gmane.comp.version-control.git/240117

Cheers, Trevor

W. Trevor King (4):
  submodule: Add helpers for configurable local branches
  submodule: Teach 'update' to preserve local branches
  submodule: Teach 'add' about a configurable local-branch
  submodule: Add a new 'checkout' command
 git-submodule.sh | 152 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 138 insertions(+), 14 deletions(-)
-- 
1.8.5.2.237.g01c62c6
W. Trevor King· Jan 9, 2014, 06:17 UTC · re: W. Trevor King · lore

[RFC v3 1/4] submodule: Add helpers for configurable local branches

From: "W. Trevor King" <wking@tremily.us>
There are three branches that submodule folks usually care about:
1. The linked $sha1 in the superproject (set explicitly for every
   superproject commit, and thus for every superproject branch).
2. The remote-tracking submodule.<name>.branch that tracks a branch in
   upstream submodule.<name>.url repository.
3. The submodule's locally checked out branch, which we currently let
   the developer setup by hand, which is integrated with one of the
   other two branches by non-checkout update modes.

Git is currently a bit weak on conveniently handling branch #3. "Just use what the developer has setup" works well for many basic workflows, but falls short for:

* Cloning-updates, where we currently always setup a detached HEAD.
  This is easy to fix if you accept submodule.<name>.branch or the
  branch pointed to by the cloned repository's HEAD as a guess, but
  this conflates branch #2 and branch #3, which may confuse users.
* Workflows where the preferred #3 branch depends on the superproject
  branch.  For example, if the remote subproject has only a master
  branch, but the local superproject needs to develop several
  submodule feature branches simultaneously, you can have a situation
  like this:
    Superproject branch  Submodule branch  Subproject branch
    ===================  ================  =================
    master               master            master
    feature-1            feature-1         master
    feature-2            feature-2         master
    feature-3            feature-2         master

In order to checkout the appropriate submodule branch for a given superproject branch, we need a way to specify the preferred submodule branch for a given superproject branch. This commit adds two helper functions:

* get_current_branch, to determine which superproject branch you're
  on, and
* get_local_branch, to determine the preferred submodule branch for
  that superproject branch.
The lookup chain for the local-branch is:
1. superproject.<superproject-branch>.local-branch in the submodule's
   config (superproject/.git/modules/<submodule-name>/config).  This
   is where the developer can store local per-superproject-branch
   overrides (e.g. if they wanted to use submodule branch feature-1
   with superproject branch feature-3).
2. submodule.<submodule-name>.local-branch in the superproject's
   config.  This is where the developer can store local
   cross-superproject-branch overrides (e.g. if they wanted to use
   submodule branch master for any superproject branch that didn't
   have a per-superproject-branch override).
3. submodule.<submodule-name>.local-branch in the superproject's
   .gitmodules file.  Because the gitmodules file is stored in the
   superproject's versioned tree, it is automatically
   superproject-branch-specific.  For example:
     $ git cat-file -p feature-1:.gitmodules
     ...
     [submodule "submod"]
         ...
         local-branch = feature-1
     $ git cat-file -p feature-3:.gitmodules
     ...
     [submodule "submod"]
         ...
         local-branch = feature-2
   this is where the project-wide defaults are setup and shared
   between developers.
4. The default local-branch is 'master'.
The new get_local_branch function handles the first step in this
chain.  The next two steps are already covered by the existing
get_submodule_config.
---
 git-submodule.sh | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
Show changes to git-submodule.sh +33 −0
diff --git a/git-submodule.sh b/git-submodule.sh
index 2677f2e..56fc3f1 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -220,6 +220,39 @@ get_submodule_config () {
 	printf '%s' "${value:-$default}"
 }
 
+#
+# Print a submodule's configured local branch name
+#
+# $1 = superproject branch
+# $2 = default (from the superproject's .gitmodules)
+#
+# To be called from the submodule root directory.
+#
+get_local_branch ()
+{
+	superproject_branch="$1"
+	default="${2:-master}"
+	if test -z "${superproject_branch}"
+	then
+		value=""
+	else
+		value=$(git config superproject."$superproject_branch".local-branch)
+	fi
+	printf '%s' "${value:-$default}"
+}
+
+#
+# Print the currently checked out branch of the current repository
+#
+# $1 = default
+#
+get_current_branch ()
+{
+	default="$1"
+	branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null) ||
+	branch=""
+	printf '%s' "${branch:-$default}"
+}
 
 #
 # Map submodule path to submodule name
-- 
1.8.5.2.237.g01c62c6
W. Trevor King· Jan 9, 2014, 06:17 UTC · re: W. Trevor King · lore

[RFC v3 2/4] submodule: Teach 'update' to preserve local branches

From: "W. Trevor King" <wking@tremily.us>
There's no sense in setting up a local branch if we're just going to
go back to a detached HEAD with every checkout-mode update.  This
commit replaces the checkout with a reset, updating whatever the
locally checked out branch (or detached HEAD) happens to be.  While it
is tempting to checkout a new local-branch here (as we did after the
clone), it's more consistent to follow the lead of the other update
modes and just use the currently checked out branch.
---
 git-submodule.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
Show changes to git-submodule.sh +3 −3
diff --git a/git-submodule.sh b/git-submodule.sh
index 56fc3f1..c5ea7bd 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -930,9 +930,9 @@ Maybe you want to use 'update --init'?")"
 				must_die_on_failure=yes
 				;;
 			*)
-				command="git checkout $subforce -q"
-				die_msg="$(eval_gettext "Unable to checkout '\$sha1' in submodule path '\$displaypath'")"
-				say_msg="$(eval_gettext "Submodule path '\$displaypath': checked out '\$sha1'")"
+				command="git reset --hard -q"
+				die_msg="$(eval_gettext "Unable to reset branch to '\$sha1' in submodule path '\$displaypath'")"
+				say_msg="$(eval_gettext "Submodule path '\$displaypath': reset branch to '\$sha1'")"
 				;;
 			esac
 
-- 
1.8.5.2.237.g01c62c6
W. Trevor King· Jan 9, 2014, 06:17 UTC · re: W. Trevor King · lore

[RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch

From: "W. Trevor King" <wking@tremily.us>

This patch teaches 'git submodule add' to look for a preferred local-branch, and to checkout that branch after the initial clone. The local branch will always point at the commit checked out by the internal 'git clone' operation. For example:

  $ git submodule add git://example.com/subproject.git submod

will checkout the branch pointed to by the cloned repository's HEAD, and call the local branch 'master'.

  $ git submodule add -b my-feature git://example.com/subproject.git submod

will checkout the branch pointed to by the cloned repository's my-feature, and *still* call the local branch 'master'.

'git submodule add' does not always make an initial clone (e.g. if a git repository already exists at the target path). In cases where 'git submodule add' does not clone a repository, we just leave the local branch alone.

This commit also shifts the post-clone branch checkout logic from
cmd_add to module_clone, so it can be shared with cmd_update.  The
previous code only checked out the requested branch in cmd_add but not
in cmd_update; this left the user on a detached HEAD after an update
initially cloned, and subsequent updates kept the HEAD detached,
unless the user moved to the desired branch himself.  Now, unless the
user explicitly asks to work on a detached HEAD, subsequent updates
all happen on the specified branch, which matches the end-user
expectation much better.
---
 git-submodule.sh | 23 +++++++++++++----------
 1 file changed, 13 insertions(+), 10 deletions(-)
Show changes to git-submodule.sh +13 −10
diff --git a/git-submodule.sh b/git-submodule.sh
index c5ea7bd..7cee0bf 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -339,7 +339,19 @@ module_clone()
 	echo "gitdir: $rel/$a" >"$sm_path/.git"
 
 	rel=$(echo $a | sed -e 's|[^/][^/]*|..|g')
-	(clear_local_git_env; cd "$sm_path" && GIT_WORK_TREE=. git config core.worktree "$rel/$b")
+	superproject_branch=$(get_current_branch)
+	default_local_branch=$(get_submodule_config "$sm_name" local-branch)
+	(
+		clear_local_git_env
+		cd "$sm_path" &&
+		GIT_WORK_TREE=. git config core.worktree "$rel/$b" &&
+		local_branch=$(get_local_branch "${superproject_branch}" "${default_local_branch}") &&
+		# ash fails to wordsplit ${branch:+-b "$branch"...}
+		case "$branch" in
+		'') git checkout -f -q -B "$local_branch" ;;
+		?*) git checkout -f -q -B "$local_branch" "origin/$branch" ;;
+		esac
+	) || die "$(eval_gettext "Unable to checkout submodule '\$sm_path'")"
 }
 
 isnumber()
@@ -503,15 +515,6 @@ Use -f if you really want to add it." >&2
 			fi
 		fi
 		module_clone "$sm_path" "$sm_name" "$realrepo" "$reference" "$depth" || exit
-		(
-			clear_local_git_env
-			cd "$sm_path" &&
-			# ash fails to wordsplit ${branch:+-b "$branch"...}
-			case "$branch" in
-			'') git checkout -f -q ;;
-			?*) git checkout -f -q -B "$branch" "origin/$branch" ;;
-			esac
-		) || die "$(eval_gettext "Unable to checkout submodule '\$sm_path'")"
 	fi
 	git config submodule."$sm_name".url "$realrepo"
 
-- 
1.8.5.2.237.g01c62c6
Francesco Pretto· Jan 15, 2014, 00:18 UTC · re: W. Trevor King · lore

Re: [RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch

I've matured this opinion about "local-branch" some days ago, but I couldn't join the discussion because I was extremely busy. Hope it's is still current (and correct).

2014/1/9 W. Trevor King <wking@tremily.us>
Show 21 quoted lines
>
> @@ -339,7 +339,19 @@ module_clone()
>         echo "gitdir: $rel/$a" >"$sm_path/.git"
>
>         rel=$(echo $a | sed -e 's|[^/][^/]*|..|g')
> -       (clear_local_git_env; cd "$sm_path" && GIT_WORK_TREE=. git config core.worktree "$rel/$b")
> +       superproject_branch=$(get_current_branch)
> +       default_local_branch=$(get_submodule_config "$sm_name" local-branch)
> +       (
> +               clear_local_git_env
> +               cd "$sm_path" &&
> +               GIT_WORK_TREE=. git config core.worktree "$rel/$b" &&
> +               local_branch=$(get_local_branch "${superproject_branch}" "${default_local_branch}") &&
> +               # ash fails to wordsplit ${branch:+-b "$branch"...}
> +               case "$branch" in
> +               '') git checkout -f -q -B "$local_branch" ;;
> +               ?*) git checkout -f -q -B "$local_branch" "origin/$branch" ;;
> +               esac
> +       ) || die "$(eval_gettext "Unable to checkout submodule '\$sm_path'")"
>  }
>
also
2014/1/8 W. Trevor King <wking@tremily.us>:
Show 25 quoted lines
>  To elaborate the idea I sketched out here [2], say
> you want:
>
>   Superproject branch  Submodule branch  Upstream branch
>   ===================  ================  ===============
>   master               master            master
>   super-feature        master            master
>   my-feature           my-feature        master
>   other-feature        other-feature     other-feature
>
> That's only going to work with per-superproject-branch configs for
> both the local and remote branches.  Using the same name for both
> local and remote branches does not work.
>
> Let me motivate each of the combinations in the above table:
>
> * master, master, master: The stable trunk.
> * super-feature, master, master: A superproject feature that works
>   with the stock submodule.
> * my-feature, my-feature, master: A superproject feature that needs an
>   improved submodule, but wants to integrate upstream master changes
>   during development.
> * other-feature, other-feature, other-feature: A superproject feature
>   that needs an improved submodule, and wants to integrate
>   other-feature changes that are also being developed upstream

The "local-branch" feature means to my brain the following: I, maintainer, decide for you, developer, what name should be the branch you are checking out. While, in general, it makes sense for a developer to switch to a differently named "feature branch" that can pull the original remote branch if he's actively developing (on any repository, not only a submodule), this leads me to the following questions: would it be good to introduce such enforcement? Do we allow something similar on regular repositories? In short I believe this workflow may reflect a personal attitude. In that case I'm unsure if git should ease it so specifically.

W. Trevor King· Jan 15, 2014, 01:02 UTC · re: Francesco Pretto · lore

Re: [RFC v3 3/4] submodule: Teach 'add' about a configurable local-branch

On Wed, Jan 15, 2014 at 01:18:12AM +0100, Francesco Pretto wrote:
> I've matured this opinion about "local-branch" some days ago, but I
> couldn't join the discussion because I was extremely busy. Hope it's
> is still current (and correct).

I think the discussion is still open, but actions are postponed until 'checkout --recurse-submodules' lands [1].

> The "local-branch" feature means to my brain the following: I,
> maintainer, decide for you, developer, what name should be the
> branch you are checking out.

The goal is to have “I, your faithful Git command, check out for you, oh wise developer, the superproject branch you requested, along with the local submodule branch associated with that super project branch.” ;) Maybe that would be more clear if the localBranch settings are purely local (stored only in out-of-tree configs), and not contained in the superproject's .gitmodules file? As this series stands, that would just drop step 3 from the lookup chain [2]. That step is below all the local out-of-tree locations though, and I see no non-psychological reason to keep folks from sharing reasonable default names for local branches.

Cheers, Trevor

[1]: http://article.gmane.org/gmane.comp.version-control.git/240420 [2]: http://article.gmane.org/gmane.comp.version-control.git/240251

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
W. Trevor King· Jan 9, 2014, 06:17 UTC · re: W. Trevor King · lore

[RFC v3 4/4] submodule: Add a new 'checkout' command

From: "W. Trevor King" <wking@tremily.us>

This borrows a good deal of the cmd_foreach logic to iterate through submodules (potentially recursively), checking out the preferred local branch for each submodule (as appropriate for the current superproject branch). Ideally, this logic would be bundled into the forthcoming:

  $ git checkout --recurse-submodules

logic that Jens Lehmann and Jonathan Nieder are working up in C. Until that happens, you can simulate that checkout behaviour with:

  $ git checkout some-branch
  $ git submodule checkout --recursive

The relevant superproject branch used to determine the preferred submodule branch is the submodules immediate parent, not the top-level superproject. For example, with the following submodule inheritance:

  superproject (branch super-1)
  `-- midproject (branch mid-1)
      `-- subproject (branch sub-1)

The .gitmodules configs should look like (assuming there are no local overrides):

  $ git cat-file -p super-1:.gitmodules
  ...
  [submodule "midproject"]
       ...
       local-branch = mid-1
  $ cd midproject
  $ git cat-file -p mid-1:.gitmodules
  ...
  [submodule "subproject"]
       ...
       local-branch = sub-1
The super-1 branch need not even exist in the midproject repository.

This commit handles branch switches inside existing submodules. Handling (or even detecting) submodules that are created and destroyed or moving submodules that change path between the initial and final superproject branch is put off to future patches.

I also added minimal support for initial branch creation. Create your initial branch with:

  $ git checkout -b my-feature
  $ git submodule checkout --recursive -b --gitmodules

which will create new 'my-feature' branches in each submodule (or die trying). It will also save 'my-feature' to the superproject's .gitmodules' submodule.<name>.local-branch for future checkouts. After setting up a branch like this, future checkouts (from some other branch) will look like:

  $ git checkout my-feature
  $ git submodule checkout --recursive
---
 git-submodule.sh | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 89 insertions(+), 1 deletion(-)
Show changes to git-submodule.sh +89 −1
diff --git a/git-submodule.sh b/git-submodule.sh
index 7cee0bf..16cebb1 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,6 +6,7 @@
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
 USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
+   or: $dashless [--quiet] checkout [--recursive] [-b|-B] [--gitmodules] [--] [<path>...]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
@@ -36,6 +37,10 @@ update=
 prefix=
 custom_name=
 depth=
+create_new_branch=
+checkout_options=
+save_in_gitmodules=
+default_local_branch="master"
 
 # The function takes at most 2 arguments. The first argument is the
 # URL that navigates to the submodule origin repo. When relative, this URL
@@ -532,6 +537,89 @@ Use -f if you really want to add it." >&2
 }
 
 #
+# Checkout the appropriate local branch for each submodule
+#
+cmd_checkout()
+{
+	# parse $args after "submodule ... checkout".
+	while test $# -ne 0
+	do
+		case "$1" in
+		-q|--quiet)
+			GIT_QUIET=1
+			;;
+		--recursive)
+			recursive=1
+			;;
+		-b|-B)
+			checkout_options="${checkout_options} $1"
+			create_new_branch=1
+			;;
+		--gitmodules)
+			save_in_gitmodules=1
+			;;
+		--)
+			shift
+			break
+			;;
+		-*)
+			usage
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done
+
+	toplevel=$(pwd)
+
+	superproject_branch=$(get_current_branch)
+	if test -n "$create_new_branch"
+	then
+		default_local_branch="${superproject_branch}"
+	fi
+
+	# dup stdin so that it can be restored when running the external
+	# command in the subshell (and a recursive call to this function)
+	exec 3<&0
+
+	module_list "$@" |
+	while read mode sha1 stage sm_path
+	do
+		die_if_unmatched "$mode"
+		name=$(module_name "$sm_path") || exit
+		displaypath=$(relative_path "$prefix$sm_path")
+		if test -e "$sm_path"/.git
+		then
+			say "$(eval_gettext "Entering '\$displaypath'")"
+			name=$(module_name "$sm_path")
+			super_local_branch=$(get_submodule_config "$name" local-branch "${default_local_branch}")
+			(
+				prefix="$prefix$sm_path/"
+				clear_local_git_env
+				cd "$sm_path" &&
+				local_branch=$(get_local_branch "${superproject_branch}" "${super_local_branch}") &&
+				git checkout ${checkout_options} "${local_branch}" &&
+				if test -n "$recursive"
+				then
+					cmd_checkout
+				fi
+			) <&3 3<&- ||
+			die "$(eval_gettext "Stopping at '\$displaypath'; script returned non-zero status.")"
+			if test -n "${save_in_gitmodules}"
+			then
+				(
+					local_branch=$(clear_local_git_env && cd "$sm_path" && get_current_branch) &&
+					git config -f .gitmodules submodule."${name}".local-branch "${local_branch}"
+				) ||
+				die "$(eval_gettext "Could not save local-branch for '\$displaypath'")"
+			fi
+		fi
+	done
+}
+
+#
 # Execute an arbitrary command sequence in each checked out
 # submodule
 #
@@ -1378,7 +1466,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | deinit | update | status | summary | sync)
+	add | checkout | foreach | init | deinit | update | status | summary | sync)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.8.5.2.237.g01c62c6
W. Trevor King· Jan 12, 2014, 01:08 UTC · re: W. Trevor King · lore

Tight submodule bindings (was: Preferred local submodule branches)

On Wed, Jan 08, 2014 at 10:17:51PM -0800, W. Trevor King wrote:
Show 11 quoted lines
> In another branch of the submodule thread Francesco kicked off, I
> mentioned that we could store the preferred local submodule branch on
> a per-superbranch level if we used the
> .git/modules/<submodule-name>/config for local overrides [1].  Here's
> a patch series that greatly extends my v2 "submodule: Respect
> requested branch on all clones" series [2] to also support automatic,
> recursive submodule checkouts, as I outlined here [3].
> 
> [1]: http://article.gmane.org/gmane.comp.version-control.git/240240
> [2]: http://article.gmane.org/gmane.comp.version-control.git/239967
> [3]: http://article.gmane.org/gmane.comp.version-control.git/240192

While mulling over better ways to explain my local-branch idea, I've come up with a more tightly bound model that may help break the silence that has greeted the “Preferred local submodule branches” series ;). That series doesn't have strong options on update mechanics, which leads to wishy-washy exchanges where nobody has a clear mental picture:

On Thu, Jan 09, 2014 at 10:40:52PM +0100, Jens Lehmann wrote:
Show 31 quoted lines
> Am 09.01.2014 20:55, schrieb W. Trevor King:
> > On Thu, Jan 09, 2014 at 08:23:07PM +0100, Jens Lehmann wrote:
> >> Am 09.01.2014 18:32, schrieb W. Trevor King:
> >>>> when superproject branches are merged (with and without conflicts),
> >>>
> >>> I don't think this currently does anything to the submodule itself,
> >>> and that makes sense to me (use 'submodule update' or my 'submodule
> >>> checkout' if you want such effects).  We should keep the current logic
> >>> for updating the gitlinked $sha.  In the case that the
> >>> .gitmodule-configured local-branches disagree, we should give the
> >>> usual conflict warning (and <<<===>>> markup) and let the user resolve
> >>> the conflict in the usual way.
> >>
> >> For me it makes lots of sense that in recursive checkout mode the
> >> merged submodules are already checked out (if possible) right after
> >> a superproject merge, making another "submodule update" unnecessary
> >> (the whole point of recursive update is to make "submodule update"
> >> obsolete, except for "--remote").
> > 
> > If you force the user to have the configured local-branch checked out
> > before a non-checkout operations with checkout side-effects (as we
> > currently do for other kinds of dirty trees), I think you'll avoid
> > most (all?) of the branch-clobbering problems.
> 
> I'm thinking that a local branch works in two directions: It should
> make it easy to follow an upstream branch and also make changes to it
> (and publish those) if necessary. But neither local nor upstream
> changes take precedence, so the user should either use "merge" or
> "rebase" as update strategy or be asked to resolve the conflict
> manually when "checkout" is configured and the branches diverged.
> Does that make sense?

The current series is only weakly bound (you can explicitly call git submodule checkout' to change to the preferred local submodule branch), and the current Git is extremely weakly bound (you have to cd into the submodule and change branches by hand). The following extrapolates the “Preferred local submodule branches” series to a tightly-bound ideal.

Gitlinked commit hash ---------------------

The submodule model revolves around links to commits (“gitlinks”):
  $ git ls-tree HEAD
  100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f    .gitmodules
  160000 commit fbfa124c29362f180026bf0074630e8bd0ff4550  submod

These are effectively switchable trees. The tree referenced by commit fbfa124 is 492781c:

  $ (cd submod/ && git cat-file commit fbfa124)
  tree 492781c581d4dec380a61ef5ec69a104de448a74
  …

If you init the submodule, subsequent checkouts will check out that tree, just like 'git checkout' would do if you'd had a superproject tree like:

  $ git ls-tree HEAD
  100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f    .gitmodules
  040000 tree 492781c581d4dec380a61ef5ec69a104de448a74    submod

For folks who treat the submodule as a black box (and do no local development), switchable trees are all they care about. They can easily checkout (or not, with deinit), the submodule tree at a gitlinked hash, and everything is nice and reproducible. The fact that 'submod' is stored as a commit object and not a tree, is just a convenient marker for optional init/deinit/remote-update-integration functionality.

Additional metadata, the initial checkout, and syncing down -----------------------------------------------------------

However, folks who do local submodule development will care about which submodule commit is responsible for that tree, because that's going to be the base of their local development. They also care about additional out-of-tree information, including the branch that commit is on. For already-initialized submodules, there are existing places in the submodule config to store this configuration:

1. HEAD for the checked-out branch,
2. branch.<name>.remote → remote.<name>.url for the upstream
   subproject URL,
4. branch.<name>.rebase (or pull.rebase) to prefer rebase over merge
   for integration,
5. …

You need somewhere in-tree to store this destined-to-be-out-of-tree information, so that superproject developers that have not yet initialized the submodule will know what values are suggested by the superproject maintainers. That's where .gitmodules comes in, because storing all of this fairly static, locally overridable information in the gitlink itself would be nonsensical (said Linus in 2007 [1]). When you checkout a submodule for the first time, Git should take the default information from .gitmodules and file it away in the submodule's appropriate out-of-tree config locations. The out-of-tree data listed above should be stored in:

1. submodule.<name>.local-branch
2. submodule.<name>.url
4. submodule.<name>.update
5. …

Once you have an in-tree way to specify defaults for this out-of-tree information, you're going to have developers like me that just want to stick with the defaults, following them through changes. That means you'd like to have the “copy .gitmodules defaults into your submodule's config” functionality that usually happens on the initial submodule checkout happen on *every superproject-initiated checkout*. In fact, I think life is easier for everyone if this is the default, and we add a new option (submodule.<name>.sync = false) that says “don't overwrite optional settings in my submodule's out-of-tree config on checkout” for for folks who want to opt out. Don't worry, this is not going to clobber people, because we'll be syncing the other way too.

Syncing up ----------

In the previous section I explained how data should flow from .gitmodules into out-of-tree configs. What about the other direction? We currently let folks handle this by hand, but I'd prefer a tighter integration between the submodule config and the superproject tree to avoid losing work. That means changes to tracked submodule status (checked-out hash, checked-out branch, upstream URL, upstream branch, default integration strategy, …) should trigger dirty-tree status just like uncommitted changes to in-tree files. 'git add' (or stash) on the dirty submodule would store changed commit hashes in the index, pull changed out-of-tree configs back into the in-tree .gitmodules, and add the new .gitmodules to the index. If the working .gitmodules was already dirty (vs. the index), the add/stash should die without making any changes. If the user has disabled syncing between .gitmodules and the submodule's out-of-tree configs, then don't worry about optional settings. Always sync the required settings, which at this point would just be submodule.<name>.local-branch.

Purely local metadata ---------------------

Some metadata does not make sense in the superproject tree. For example, whether a submodule is interesting enough to checkout (init/deinit) or whether you want to auto-sync optional metadata .gitmodules defaults. This metadata should live in the superproject's out-of-tree config, and should not be stored in the in-tree .gitmodules. Since you *will* want to share the upstream URL, I proposed using an explicit submodule.<name>.active setting to store the “do I care” information [2], instead of overloading submodule.<name>.url (I'd auto-sync the .gitmodule's submodule.<name>.url with the subproject's remote.origin.url unless the user opted out of .gitmodules syncing).

Subsequent checkouts --------------------

Now that we have strict linking between the submodule state (both in-tree and out-of-tree configs) and the superproject tree (gitlink and .gitmodules), changing between superproject branches is really easy:

1. Make sure the working tree is not dirty.  If it is, ask the user to
   either add-and-commit or stash, and then die to let them do so.
2. Checkout the new superproject branch.
   2.1. For each old submodule that doesn't exist in the new branch,
        blow away the submodule directory (assuming a new-style
        .git/modules/… layout, and not an old-style submod/.git/…
        layout).
   2.2. For each gitlinked submodule that didn't exist in the old
        branch, setup the submodule as if you were doing the initial
        cloning checkout (forcing a new local-branch to point at the
        gitlinked commit).  If you find local out-of-tree
        *superproject* configs that conflict with the .gitmodules
        values, prefer the superproject configs.  Clobber submodule
        configs and local branches at will (modulo
        submodule.<name>.sync), because any submodule configs that the
        user wanted to keep should have been added to the superproject
        branch earlier (or stashed).

Integrating other branches --------------------------

Merges and rebases can alter the submodule's in-tree configs (and create and remove submodules). The existing logic for merging .gitmodules and gitlinks works well, so stick with that. In the event that there are unresolvable conflicts, bail out and let the user resolve the conflicts and use 'git commit' to finish checking out the resolved state.

Issues ------

I like the current submodule integration configuration:
* submodule.<name>.branch (specify the remote branch to integrate, but
  I'd prefer submodule.<name>.integration-ref for clarity).
* submodule.<name>.update (specify how to integrate it, but I'd prefer
  submodule.<name>.integration-mode for clarity).
more than the current core integration configuration:
* branch.<name>.merge (with branch.<name>.remote, the branch to remote
  branch to integrate via merging).
* branch.<name>.rebase (override branch.<name>.merge to integrate via
  rebasing).

These seem to mix the orthogonal concepts of integration target and integration mode, and the divergence from the .gitmodules representation makes syncing awkward.

Summary -------

New .gitmodules options:
* submodule.<name>.local-branch, store the submodule's HEAD, must stay
  in sync for checkouts.
New .git/config options:
* submodule.<name>.active, for init/deinit.
* submodule.<name>.sync, for whether you want to automatically sync
  the submodule's out-of-tree configs up to .gitmodules before
  checkout operations, and sync back from .gitmodules (possibly
  altered on the new branch) into the submodule's out-of-tree configs
  during checkout.

With this tighter binding, submodule information is either tracked in the superproject, or explicitly not touched by the superproject. That makes it much harder to break things or clobber a user's work, and also much easier to keep submodules up to date with superproject changes. Users shouldn't have to explicitly manage their submodules to carry out routine core tasks like checking out other branches.

I see no reason to add --recurse-submodule flags to 'git checkout' (and merge, …). Anything that happens post-clone should recurse through submodules automatically, and use the submodule.<name>.active setting to decide when recursion is desired.

I think the ideal submodule-specific interface would be just:
* git submodule [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
                [--reference <repository>] [--] <repository> [<path>]
* git submodule [--quiet] init [--] [<path>...]
* git submodule [--quiet] deinit [-f|--force] [--] <path>...
* git submodule [--quiet] foreach [--recursive] <command>
The current 'git submodule update --remote' would just be:
  $ git submodule foreach 'git pull'

because all of the local-branch checkouts would have already been handled. Similarly, a global push would be just:

  $ git submodule foreach 'git push'

You get all the per-submodule configuration (for triangular workflows, etc.) for free, with no submodule-specific confusion.

So, is this:
* Interesting enough to be worth pursuing?
* Simple enough to be easily understood?

I'd be happy to mock this up in shell, but only if anyone else would be interested enough to review the implementation ;). Then I'll look into integrating the preferred model (this tightly bound proposal, or v3's looser bindings, or <your idea here>) in C, building on Jens and Jonathan's work.

Cheers, Trevor

[1]: http://article.gmane.org/gmane.comp.version-control.git/44162 [2]: http://article.gmane.org/gmane.comp.version-control.git/211042

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Jens Lehmann· Jan 13, 2014, 19:37 UTC · re: W. Trevor King · lore

Re: Tight submodule bindings

Thanks for the writeup, comments below.
Am 12.01.2014 02:08, schrieb W. Trevor King:
Show 31 quoted lines
> Gitlinked commit hash
> ---------------------
> 
> The submodule model revolves around links to commits (“gitlinks”):
> 
>   $ git ls-tree HEAD
>   100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f    .gitmodules
>   160000 commit fbfa124c29362f180026bf0074630e8bd0ff4550  submod
> 
> These are effectively switchable trees.  The tree referenced by commit
> fbfa124 is 492781c:
> 
>   $ (cd submod/ && git cat-file commit fbfa124)
>   tree 492781c581d4dec380a61ef5ec69a104de448a74
>   …
> 
> If you init the submodule, subsequent checkouts will check out that
> tree, just like 'git checkout' would do if you'd had a superproject
> tree like:
> 
>   $ git ls-tree HEAD
>   100644 blob 189fc359d3dc1ed5019b9834b93f0dfb49c5851f    .gitmodules
>   040000 tree 492781c581d4dec380a61ef5ec69a104de448a74    submod
> 
> For folks who treat the submodule as a black box (and do no local
> development), switchable trees are all they care about.  They can
> easily checkout (or not, with deinit), the submodule tree at a
> gitlinked hash, and everything is nice and reproducible.  The fact
> that 'submod' is stored as a commit object and not a tree, is just a
> convenient marker for optional init/deinit/remote-update-integration
> functionality.

But there are users (like me) who do not treat submodules as black boxes and nonetheless do development in them with update set to checkout (after creating a feature branch of course ;-).

Show 26 quoted lines
> Additional metadata, the initial checkout, and syncing down
> -----------------------------------------------------------
> 
> However, folks who do local submodule development will care about
> which submodule commit is responsible for that tree, because that's
> going to be the base of their local development.  They also care about
> additional out-of-tree information, including the branch that commit
> is on.  For already-initialized submodules, there are existing places
> in the submodule config to store this configuration:
> 
> 1. HEAD for the checked-out branch,
> 2. branch.<name>.remote → remote.<name>.url for the upstream
>    subproject URL,
> 4. branch.<name>.rebase (or pull.rebase) to prefer rebase over merge
>    for integration,
> 5. …
> 
> You need somewhere in-tree to store this destined-to-be-out-of-tree
> information, so that superproject developers that have not yet
> initialized the submodule will know what values are suggested by the
> superproject maintainers.  That's where .gitmodules comes in, because
> storing all of this fairly static, locally overridable information in
> the gitlink itself would be nonsensical (said Linus in 2007 [1]).
> When you checkout a submodule for the first time, Git should take the
> default information from .gitmodules and file it away in the
> submodule's appropriate out-of-tree config locations.

I disagree, that only makes sense for the URL setting (and this currently only happens with the update setting, which I intend to change). Everything else should be taken from .gitmodules unless the user wants to override it. The only setting I'm not so sure about is the local branch setting, as that might have to propagate into the submodule.

Show 14 quoted lines
>  The out-of-tree
> data listed above should be stored in:
> 
> 1. submodule.<name>.local-branch
> 2. submodule.<name>.url
> 4. submodule.<name>.update
> 5. …
> 
> Once you have an in-tree way to specify defaults for this out-of-tree
> information, you're going to have developers like me that just want to
> stick with the defaults, following them through changes.  That means
> you'd like to have the “copy .gitmodules defaults into your
> submodule's config” functionality that usually happens on the initial
> submodule checkout happen on *every superproject-initiated checkout*.

You don't need to copy it every time when you simply use the .gitmodules file as fallback, no?

Show 6 quoted lines
> In fact, I think life is easier for everyone if this is the default,
> and we add a new option (submodule.<name>.sync = false) that says
> “don't overwrite optional settings in my submodule's out-of-tree
> config on checkout” for for folks who want to opt out.  Don't worry,
> this is not going to clobber people, because we'll be syncing the
> other way too.
Yet another flag to make peoples life easier? I don't think so ;-)
Show 19 quoted lines
> Syncing up
> ----------
> 
> In the previous section I explained how data should flow from
> .gitmodules into out-of-tree configs.  What about the other direction?
> We currently let folks handle this by hand, but I'd prefer a tighter
> integration between the submodule config and the superproject tree to
> avoid losing work.  That means changes to tracked submodule status
> (checked-out hash, checked-out branch, upstream URL, upstream branch,
> default integration strategy, …) should trigger dirty-tree status just
> like uncommitted changes to in-tree files.  'git add' (or stash) on
> the dirty submodule would store changed commit hashes in the index,
> pull changed out-of-tree configs back into the in-tree .gitmodules,
> and add the new .gitmodules to the index.  If the working .gitmodules
> was already dirty (vs. the index), the add/stash should die without
> making any changes.  If the user has disabled syncing between
> .gitmodules and the submodule's out-of-tree configs, then don't worry
> about optional settings.  Always sync the required settings, which at
> this point would just be submodule.<name>.local-branch.

Such a logic might make sense. And without copying stuff from .gitmodules someplace else it becomes even easier ;-)

Show 9 quoted lines
> Purely local metadata
> ---------------------
> 
> Some metadata does not make sense in the superproject tree.  For
> example, whether a submodule is interesting enough to checkout
> (init/deinit) or whether you want to auto-sync optional metadata
> .gitmodules defaults.  This metadata should live in the superproject's
> out-of-tree config, and should not be stored in the in-tree
> .gitmodules.

Not always. It makes a lot of sense to let upstream mark a submodule as "too big and you won't need it anyway" in the .gitmodules file.

Show 6 quoted lines
>  Since you *will* want to share the upstream URL, I
> proposed using an explicit submodule.<name>.active setting to store
> the “do I care” information [2], instead of overloading
> submodule.<name>.url (I'd auto-sync the .gitmodule's
> submodule.<name>.url with the subproject's remote.origin.url unless
> the user opted out of .gitmodules syncing).

That is wrong as it would break horribly when you check out an old commit with a now dead submodule URL and that gets automatically synced.

Show 10 quoted lines
> Subsequent checkouts
> --------------------
> 
> Now that we have strict linking between the submodule state (both
> in-tree and out-of-tree configs) and the superproject tree (gitlink
> and .gitmodules), changing between superproject branches is really
> easy:
> 
> 1. Make sure the working tree is not dirty.  If it is, ask the user to
>    either add-and-commit or stash, and then die to let them do so.

This condition is too hard, relax that to "a trivial merge can switch from current state to target state" and make it behave just like branch switching in the superproject. After all submodules should behave as much as possible like content of the superproject.

Show 6 quoted lines
> 2. Checkout the new superproject branch.
> 
>    2.1. For each old submodule that doesn't exist in the new branch,
>         blow away the submodule directory (assuming a new-style
>         .git/modules/… layout, and not an old-style submod/.git/…
>         layout).
Yep.
Show 6 quoted lines
>    2.2. For each gitlinked submodule that didn't exist in the old
>         branch, setup the submodule as if you were doing the initial
>         cloning checkout (forcing a new local-branch to point at the
>         gitlinked commit).  If you find local out-of-tree
>         *superproject* configs that conflict with the .gitmodules
>         values, prefer the superproject configs.
Yup, our working title for that is "autoinit".
Show 5 quoted lines
>  Clobber submodule
>         configs and local branches at will (modulo
>         submodule.<name>.sync), because any submodule configs that the
>         user wanted to keep should have been added to the superproject
>         branch earlier (or stashed).

I don't think I like this part, but I admit I do not fully understand what you mean here. Clobbering stuff the user did doesn't sound very nice.

Show 9 quoted lines
> Integrating other branches
> --------------------------
> 
> Merges and rebases can alter the submodule's in-tree configs (and
> create and remove submodules).  The existing logic for merging
> .gitmodules and gitlinks works well, so stick with that.  In the event
> that there are unresolvable conflicts, bail out and let the user
> resolve the conflicts and use 'git commit' to finish checking out the
> resolved state.
Agreed.
Show 9 quoted lines
> Issues
> ------
> 
> I like the current submodule integration configuration:
> 
> * submodule.<name>.branch (specify the remote branch to integrate, but
>   I'd prefer submodule.<name>.integration-ref for clarity).
> * submodule.<name>.update (specify how to integrate it, but I'd prefer
>   submodule.<name>.integration-mode for clarity).
But we won't rename those now.
Show 10 quoted lines
> more than the current core integration configuration:
> 
> * branch.<name>.merge (with branch.<name>.remote, the branch to remote
>   branch to integrate via merging).
> * branch.<name>.rebase (override branch.<name>.merge to integrate via
>   rebasing).
> 
> These seem to mix the orthogonal concepts of integration target and
> integration mode, and the divergence from the .gitmodules
> representation makes syncing awkward.

I'm still hoping we might come up with a solution that doesn't need the syncing.

Show 7 quoted lines
> Summary
> -------
> 
> New .gitmodules options:
> 
> * submodule.<name>.local-branch, store the submodule's HEAD, must stay
>   in sync for checkouts.

I'm still not convinced that the current branch setting couldn't be extended to carry that information, but no objections against configuring such a branch. But what do you mean with "must stay in sync for checkouts"?

> New .git/config options:
> 
> * submodule.<name>.active, for init/deinit.

I understand an option for automatic init (autoinit), but not for automatic deinit. Is the latter really useful?

Show 5 quoted lines
> * submodule.<name>.sync, for whether you want to automatically sync
>   the submodule's out-of-tree configs up to .gitmodules before
>   checkout operations, and sync back from .gitmodules (possibly
>   altered on the new branch) into the submodule's out-of-tree configs
>   during checkout.
Not needed if you use .gitmodules as fallback.
Show 6 quoted lines
> With this tighter binding, submodule information is either tracked in
> the superproject, or explicitly not touched by the superproject.  That
> makes it much harder to break things or clobber a user's work, and
> also much easier to keep submodules up to date with superproject
> changes.  Users shouldn't have to explicitly manage their submodules
> to carry out routine core tasks like checking out other branches.
Agreed.
> I see no reason to add --recurse-submodule flags to 'git checkout'
> (and merge, …).  Anything that happens post-clone should recurse
> through submodules automatically, and use the submodule.<name>.active
> setting to decide when recursion is desired.

Backwards compatibility and testing. Let's first implement that and provide a config option to enable it for real world testing, and then let's discuss changing the default later.

Show 7 quoted lines
> I think the ideal submodule-specific interface would be just:
> 
> * git submodule [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
>                 [--reference <repository>] [--] <repository> [<path>]
> * git submodule [--quiet] init [--] [<path>...]
> * git submodule [--quiet] deinit [-f|--force] [--] <path>...
> * git submodule [--quiet] foreach [--recursive] <command>
Ok.
Show 6 quoted lines
> The current 'git submodule update --remote' would just be:
> 
>   $ git submodule foreach 'git pull'
> 
> because all of the local-branch checkouts would have already been
> handled.

Nope, that does different things to submodules where "branch" isn't configured, right?

>  Similarly, a global push would be just:
> 
>   $ git submodule foreach 'git push'
What's wrong with:
$ git push --recurse-submodules=on-demand

And it'll push the superproject at the same time. Extra points for already being implemented ;-)

Show 7 quoted lines
> You get all the per-submodule configuration (for triangular workflows,
> etc.) for free, with no submodule-specific confusion.
> 
> So, is this:
> 
> * Interesting enough to be worth pursuing?
> * Simple enough to be easily understood?

The thoughts about the branch workflow are really interesting. Some other proposals overshoot a bit in my opinion ;-)

Show 5 quoted lines
> I'd be happy to mock this up in shell, but only if anyone else would
> be interested enough to review the implementation ;).  Then I'll look
> into integrating the preferred model (this tightly bound proposal, or
> v3's looser bindings, or <your idea here>) in C, building on Jens and
> Jonathan's work.

The update modes (cleaning removed submodules and creating new ones) are better handled in my recursive checkout series. But I believe we can at least prototype the branch handling in shell.

W. Trevor King· Jan 13, 2014, 20:07 UTC · re: Jens Lehmann · lore

Re: Tight submodule bindings

On Mon, Jan 13, 2014 at 08:37:37PM +0100, Jens Lehmann wrote:
Show 12 quoted lines
> Am 12.01.2014 02:08, schrieb W. Trevor King:
> > For folks who treat the submodule as a black box (and do no local
> > development), switchable trees are all they care about.  They can
> > easily checkout (or not, with deinit), the submodule tree at a
> > gitlinked hash, and everything is nice and reproducible.  The fact
> > that 'submod' is stored as a commit object and not a tree, is just
> > a convenient marker for optional
> > init/deinit/remote-update-integration functionality.
> 
> But there are users (like me) who do not treat submodules as
> black boxes and nonetheless do development in them with update
> set to checkout (after creating a feature branch of course ;-).

I'm still not clear on how this works for you ;). Can you sketch out an example shell history showing how you use checkout updates to do this?

Show 10 quoted lines
> > When you checkout a submodule for the first time, Git should take
> > the default information from .gitmodules and file it away in the
> > submodule's appropriate out-of-tree config locations.
> 
> I disagree, that only makes sense for the URL setting (and this
> currently only happens with the update setting, which I intend to
> change). Everything else should be taken from .gitmodules unless
> the user wants to override it. The only setting I'm not so sure
> about is the local branch setting, as that might have to propagate
> into the submodule.

I think copying into the submodule's out-of-tree config is the way to go, because users won't always be driving the submodule from the superproject. If the settings are in the submodule's out-of-tree config, everything will be consistent betwee stuff run from the superproject and stuff run from submodule itself. It also allows us to use familiar configuration commands inside the submodule, and have those automatically mapped back into the .gitmodules file for us.

Show 8 quoted lines
> > In fact, I think life is easier for everyone if this is the
> > default, and we add a new option (submodule.<name>.sync = false)
> > that says “don't overwrite optional settings in my submodule's
> > out-of-tree config on checkout” for for folks who want to opt out.
> > Don't worry, this is not going to clobber people, because we'll be
> > syncing the other way too.
> 
> Yet another flag to make peoples life easier? I don't think so ;-)

I'm fine if there is no opt-out, and the syncing is mandatory, but I imagine that folks who want a local (unshared, not in .gitmodules) URL would complain.

Show 13 quoted lines
> > Purely local metadata
> > ---------------------
> > 
> > Some metadata does not make sense in the superproject tree.  For
> > example, whether a submodule is interesting enough to checkout
> > (init/deinit) or whether you want to auto-sync optional metadata
> > .gitmodules defaults.  This metadata should live in the
> > superproject's out-of-tree config, and should not be stored in the
> > in-tree .gitmodules.
> 
> Not always. It makes a lot of sense to let upstream mark a
> submodule as "too big and you won't need it anyway" in the
> .gitmodules file.
Good.  Then there's no need for this special class of settings.
Show 10 quoted lines
> > Since you *will* want to share the upstream URL, I proposed using
> > an explicit submodule.<name>.active setting to store the “do I
> > care” information [2], instead of overloading submodule.<name>.url
> > (I'd auto-sync the .gitmodule's submodule.<name>.url with the
> > subproject's remote.origin.url unless the user opted out of
> > .gitmodules syncing).
> 
> That is wrong as it would break horribly when you check out an old
> commit with a now dead submodule URL and that gets automatically
> synced.

If you've already checked out the submodule with a current URL, you should already have the old commit locally, and Git will use it without trying to re-fetch from the broken old URL.

Show 15 quoted lines
> > Subsequent checkouts
> > --------------------
> > 
> > Now that we have strict linking between the submodule state (both
> > in-tree and out-of-tree configs) and the superproject tree (gitlink
> > and .gitmodules), changing between superproject branches is really
> > easy:
> > 
> > 1. Make sure the working tree is not dirty.  If it is, ask the user to
> >    either add-and-commit or stash, and then die to let them do so.
> 
> This condition is too hard, relax that to "a trivial merge can
> switch from current state to target state" and make it behave just
> like branch switching in the superproject. After all submodules
> should behave as much as possible like content of the superproject.
Sounds good to me.
Show 9 quoted lines
> >  Clobber submodule
> >         configs and local branches at will (modulo
> >         submodule.<name>.sync), because any submodule configs that
> >         the user wanted to keep should have been added to the
> >         superproject branch earlier (or stashed).
> 
> I don't think I like this part, but I admit I do not fully
> understand what you mean here. Clobbering stuff the user did doesn't
> sound very nice.

It's fine because we forced them to commit or stash any (not trivially mergable) changes before starting the checkout command.

Show 12 quoted lines
> > Summary
> > -------
> > 
> > New .gitmodules options:
> > 
> > * submodule.<name>.local-branch, store the submodule's HEAD, must
> >   stay in sync for checkouts.
> 
> I'm still not convinced that the current branch setting couldn't be
> extended to carry that information, but no objections against
> configuring such a branch. But what do you mean with "must stay in
> sync for checkouts"?

That checkout-inducing commands should die if the .gitmodule's local-branch and the submodule's HEAD don't match.

Show 6 quoted lines
> > New .git/config options:
> > 
> > * submodule.<name>.active, for init/deinit.
> 
> I understand an option for automatic init (autoinit), but not for
> automatic deinit. Is the latter really useful?

This isn't auto-anything. This is just “I think the submodule is interesting, please turn it on” (i.e. I ran “git submodule init <submod>”).

Only active submodules should get all the syncing, setup, and teardown logic that goes along with submodule checkout. Inactive submodules are ignored.

Show 9 quoted lines
> > I see no reason to add --recurse-submodule flags to 'git checkout'
> > (and merge, …).  Anything that happens post-clone should recurse
> > through submodules automatically, and use the
> > submodule.<name>.active setting to decide when recursion is
> > desired.
> 
> Backwards compatibility and testing. Let's first implement that and
> provide a config option to enable it for real world testing, and
> then let's discuss changing the default later.
Ok.
Show 9 quoted lines
> > The current 'git submodule update --remote' would just be:
> > 
> >   $ git submodule foreach 'git pull'
> > 
> > because all of the local-branch checkouts would have already been
> > handled.
> 
> Nope, that does different things to submodules where "branch" isn't
> configured, right?

It does the same thing. Without submodule.<name>.branch configured, you just integrate the subproject's master.

Show 10 quoted lines
> >  Similarly, a global push would be just:
> > 
> >   $ git submodule foreach 'git push'
> 
> What's wrong with:
> 
> $ git push --recurse-submodules=on-demand
> 
> And it'll push the superproject at the same time. Extra points for
> already being implemented ;-)

That's a strong argument ;). I still don't think the new-in-1.7.4 UI change will add value. The new-in-1.7.7 --recurse-submodules=check would still be useful.

Show 9 quoted lines
> > I'd be happy to mock this up in shell, but only if anyone else
> > would be interested enough to review the implementation ;).  Then
> > I'll look into integrating the preferred model (this tightly bound
> > proposal, or v3's looser bindings, or <your idea here>) in C,
> > building on Jens and Jonathan's work.
> 
> The update modes (cleaning removed submodules and creating new ones)
> are better handled in my recursive checkout series. But I believe we
> can at least prototype the branch handling in shell.

I'll prototype it, and keep trying to convince you about the syncing ;). I think the main arguments for syncing are:

* No divergent configs between superproject-initiated actions and
  submodule-initiated actions.
* No work clobbered, or accidentally uncommitted, due to syncing
  submodule -> superproject before checkout-inducing commands.

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Junio C Hamano· Jan 13, 2014, 22:13 UTC · re: W. Trevor King · lore

Re: Tight submodule bindings

"W. Trevor King" <wking@tremily.us> writes:
Show 8 quoted lines
> Additional metadata, the initial checkout, and syncing down
> -----------------------------------------------------------
>
> However, folks who do local submodule development will care about
> which submodule commit is responsible for that tree, because that's
> going to be the base of their local development.  They also care about
> additional out-of-tree information, including the branch that commit
> is on.
Well, please step back a bit.

They do not have to care about what local branch they use to build follow-up work based on that commit. In fact, they would want to be able to develop more than one histories on top, which means more than one branches they can name themselves.

The only thing they care about is where the result of their development _goes_, that is the URL and the branch of the remote they are pushing back to.

I have a feeling that this is not specific for submodules---if you did this:

	git init here
        cd here
        git fetch $there master
        git reset --hard FETCH_HEAD

and are given the resulting working tree to start hacking on, you would not know where the history came from, or where your result wants to go.

So "the branch that commit is on" is a wrong thing to focus on. "The branch the history built on top of the commit wants to go" may be closer and these two are different.

Show 9 quoted lines
>  For already-initialized submodules, there are existing places
> in the submodule config to store this configuration:
>
> 1. HEAD for the checked-out branch,
> 2. branch.<name>.remote → remote.<name>.url for the upstream
>    subproject URL,
> 4. branch.<name>.rebase (or pull.rebase) to prefer rebase over merge
>    for integration,
> 5. …
What happened to 3 ;-)?

And also branch.<name>.merge may say on which of _their_ branch the commit you learn in the superproject tree would be found. If you are using centralized workflow, that would be the branch at your central repository to update with your push, too.

In any case, "local-branch" is wrong from two aspects:
 1. (obvious) It does not follow our naming convention not to use
    dashed-names for configuration variables.
 2. You do not care about the names you use locally.  The only thing
    you care about is where people meet at the central repository,
    i.e. where your result is pushed to.
Show 5 quoted lines
> Syncing up
> ----------
>
> In the previous section I explained how data should flow from
> .gitmodules into out-of-tree configs.

s/should/you think should/, I think, but another way may be not to copy and read from there, which may be a lot simpler. Then upon switching branches of top-level superproject (which would update .gitmodules to the version on the new branch), you may get different settings automatically. But see below.

Show 6 quoted lines
> ...  Since you *will* want to share the upstream URL, I
> proposed using an explicit submodule.<name>.active setting to store
> the “do I care” information [2], instead of overloading
> submodule.<name>.url (I'd auto-sync the .gitmodule's
> submodule.<name>.url with the subproject's remote.origin.url unless
> the user opted out of .gitmodules syncing).

It may not be a good idea to blindly update to whatever happens to be in .gitmodules, especially once submodule.*.url is initialized.

I think we would need a bit more sophisticated mechanism than "use from .git/config if set, otherwise use from .gitmodules", at least for the URL. It may not be limited to the URL, and other pieces of metainformation about submodules may need similar handling, but I'd refrain from extending the scope of discussion needlessly at this point.

Imagine that your embedded appliance project used to use a submodule from git://k.org/linux-2.6 as its kernel component and now the upstream of it is instead called just git://k.org/linux; the URL specified by submodule.kernel.url in .gitmodules for the entry submodule.kernel.path=kernel would have changed from the former to the latter sometime in the superproject's history. Switching back to an old version in the superproject to fix an old bug in the maintenance track of the superproject would still want to push associated fixes to the kernel to k.org/linux, not linux-2.6, the latter of which may now be defunct [*1*]. One way to make it work semi-automatically is to keep track of what the user has seen in .gitmodules and offer chances to update entries in .git/config. If you cloned the superproject recently, you would only know about the new git://k.org/linux URL and that would be copied to .git/config (which the current code does). In addition, you would remember that we saw git://k.org/linux URL (which the current code does not). Upon switching back to an old version, we could notice that the URL in .gitmodules, which is git://k.org/linux-2.6, is not something the user has seen, and at that point we could ask the user to tell us what URL should be used, record the answer _and_ the fact that we saw that old URL as well. Then until the superproject updates the URL the next time to a value that we have never seen, the user can keep using the right URL without being asked [*2*].

Show 6 quoted lines
> 2. Checkout the new superproject branch.
>
>    2.1. For each old submodule that doesn't exist in the new branch,
>         blow away the submodule directory (assuming a new-style
>         .git/modules/… layout, and not an old-style submod/.git/…
>         layout).
Sure.
Show 10 quoted lines
>    2.2. For each gitlinked submodule that didn't exist in the old
>         branch, setup the submodule as if you were doing the initial
>         cloning checkout (forcing a new local-branch to point at the
>         gitlinked commit).  If you find local out-of-tree
>         *superproject* configs that conflict with the .gitmodules
>         values, prefer the superproject configs.  Clobber submodule
>         configs and local branches at will (modulo
>         submodule.<name>.sync), because any submodule configs that the
>         user wanted to keep should have been added to the superproject
>         branch earlier (or stashed).
See above.
[Footnote]

*1* On the other hand, the switch of the submodule URL in the superproject may have been between two separate projects (e.g. you used to build your embedded appliance using BSD kernel but recent versions use Linux kernel)---in such a project, you would want the submodule URL to follow what is in .gitmodules when you switch between old and new versions in the superproject. But our recommendation in such a case is to use different names for submodules that is bound at the same path in the superproject so that we can keep them as two separate repositories in .git/mdoules/ of the superproject. So at least for the URL, there is no reason to use the old version that appears in .gitmodules of the superproject even when you checkout an old version of it.

*2* This "remembering" may have to be more than "have we seen this" one-bit per different values. For URL, I think the one-bit is enough, but for other things, it might make sense to keep track of "In the version of superproject with X in .gitmodules, the user wants to use value Y" for each values X the user has seen.

W. Trevor King· Jan 14, 2014, 02:44 UTC · re: Junio C Hamano · lore

Re: Tight submodule bindings

On Mon, Jan 13, 2014 at 02:13:46PM -0800, Junio C Hamano wrote:
Show 15 quoted lines
> "W. Trevor King" <wking@tremily.us> writes:
> 
> > Additional metadata, the initial checkout, and syncing down
> > -----------------------------------------------------------
> >
> > However, folks who do local submodule development will care about
> > which submodule commit is responsible for that tree, because
> > that's going to be the base of their local development.  They also
> > care about additional out-of-tree information, including the
> > branch that commit is on.
> 
> Well, please step back a bit.
> 
> They do not have to care about what local branch they use to build
> follow-up work based on that commit.

They do if they want to checkout the banch out again later, before pushing it somewhere public.

> In fact, they would want to be able to develop more than one
> histories on top, which means more than one branches they can name
> themselves.

Agreed, bug for each superproject branch they will still have a single submodule branch that should be checked out by default when they checkout that superproject branch.

> The only thing they care about is where the result of their
> development _goes_, that is the URL and the branch of the remote
> they are pushing back to.

Maybe they're just doing local development? I think the remote branch(es) you pull from and push to are important, but not the only thing you might care about.

Show 15 quoted lines
> I have a feeling that this is not specific for submodules---if you
> did this:
> 
> 	git init here
>         cd here
>         git fetch $there master
>         git reset --hard FETCH_HEAD
> 
> and are given the resulting working tree to start hacking on, you
> would not know where the history came from, or where your result
> wants to go.  
> 
> So "the branch that commit is on" is a wrong thing to focus on.
> "The branch the history built on top of the commit wants to go" may
> be closer and these two are different.

That makes sense. I don't think the former (as distinct from the latter) is of any interest to anybody. I don't care what the branch name was when the past history was developed. I don't even really care about the new branch name. I do care that checking out a superproject branch gives me the same branch (with pull/push configs, etc.) that I had the last time I was on that superproject branch.

Show 11 quoted lines
> >  For already-initialized submodules, there are existing places
> > in the submodule config to store this configuration:
> >
> > 1. HEAD for the checked-out branch,
> > 2. branch.<name>.remote → remote.<name>.url for the upstream
> >    subproject URL,
> > 4. branch.<name>.rebase (or pull.rebase) to prefer rebase over merge
> >    for integration,
> > 5. …
> 
> What happened to 3 ;-)?
I can't count? :p
> In any case, "local-branch" is wrong from two aspects:
> 
>  1. (obvious) It does not follow our naming convention not to use
>     dashed-names for configuration variables.

I'll use localBranch in my mockup ;). Although skimming through config.txt shows a number of alllowercase settings as well as camelCase.

>  2. You do not care about the names you use locally.  The only thing
>     you care about is where people meet at the central repository,
>     i.e. where your result is pushed to.
I also care about local-checkout consistency, as described above.
Show 11 quoted lines
> > Syncing up
> > ----------
> >
> > In the previous section I explained how data should flow from
> > .gitmodules into out-of-tree configs.
> 
> s/should/you think should/, I think, but another way may be not to
> copy and read from there, which may be a lot simpler.  Then upon
> switching branches of top-level superproject (which would update
> .gitmodules to the version on the new branch), you may get different
> settings automatically.

That only works for superproject-level commands that know about the .gitmodules file. If you cd into the submodule and work there directly, your actions will be using the submodule's out-of-tree config. I think most of the time folks will want those out-of-tree configs to match the settings in the superproject's .gitmodules, hence the submodule.<name>.sync defaulting to true.

Show 9 quoted lines
> > ...  Since you *will* want to share the upstream URL, I proposed
> > using an explicit submodule.<name>.active setting to store the “do
> > I care” information [2], instead of overloading
> > submodule.<name>.url (I'd auto-sync the .gitmodule's
> > submodule.<name>.url with the subproject's remote.origin.url
> > unless the user opted out of .gitmodules syncing).
> 
> It may not be a good idea to blindly update to whatever happens to
> be in .gitmodules, especially once submodule.*.url is initialized.

Why not? We're blindly updating it to the value that was previously pulled out of the submodule's out-of-tree config. If the user doesn't like what's happening to .gitmodules upstream and doesn't want to keep a patched version locally, they can always turn off submodule.<name>.sync.

Show 10 quoted lines
> Imagine that your embedded appliance project used to use a submodule
> from git://k.org/linux-2.6 as its kernel component and now the
> upstream of it is instead called just git://k.org/linux; the URL
> specified by submodule.kernel.url in .gitmodules for the entry
> submodule.kernel.path=kernel would have changed from the former to
> the latter sometime in the superproject's history.  Switching back
> to an old version in the superproject to fix an old bug in the
> maintenance track of the superproject would still want to push
> associated fixes to the kernel to k.org/linux, not linux-2.6, the
> latter of which may now be defunct [*1*].

The checkout would work (because the old gitlinked commit is already in the local repository), but the push would not. I don't think it would be difficult to recover from that manually (and just specify the full URL when pushing). You could also:

1. Commit your fix.
2. Checkout a more modern superproject branch (which will load the
   current URL into the submodule's config).
3. Push the fix.
4. Continue to work on the modern branch.
That doesn't sound much more difficult than the ideal:
1. Commit your fix.
2. Push the fix.
3. Checkout a more modern superproject branch (which will load the
   current URL into the submodule's config).
4. Continue to work on the modern branch.

If you expect to be back making more superproject/subproject joint bugfixes in future, I think it makes sense to start a maintenance branch of the superproject that updates the .gitmodules URL to point at the modern location.

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Heiko Voigt· Jan 7, 2014, 22:51 UTC · re: W. Trevor King · lore

Re: Re: [PATCH 2/2] Introduce git submodule attached update

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:
Show 16 quoted lines
> Here's an attempted summary of our desires, and my ideal route
> forward:
> 
> * Preferred local submodule branches for each superproject branch.
>   * Not currently supported by Git.
>   * Requires some sort of per-superproject-branch .git/config.
>   * Fall back to the remote-tracking submodule.<name>.branch?
> 
> * Auto checkout of the preferred branch
>   * Can do this at clone-update time with my patch.
>   * For later submodule branch switches, maybe we want:
> 
>       git submodule checkout [-b <branch>] [<paths>…]
> 
>     Then if a user blows off their detached HEAD, at least they'll
>     feel a bit sheepish afterwards.

Well, for development on a detached HEAD in a submodule we are currently not very careful anyway. A simple

	git submodule update

will already blow away any detached HEAD work. But AFAIK it should trigger the "you are leaving commits from a detached HEAD behind" warning, so there is some safeguard and recovery.

Cheers Heiko
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)

iEYEARECAAYFAlLMhPAACgkQjLR3Aoip+rqP6wCeIhtpWLJC3XVO3nu2ViQTbHPg T5wAoLLEZ256GOOjBxoTKo2/FmfvQGLp =+bqm -----END PGP SIGNATURE-----

W. Trevor King· Jan 7, 2014, 23:14 UTC · re: Heiko Voigt · lore

Re: [PATCH 2/2] Introduce git submodule attached update

On Tue, Jan 07, 2014 at 11:51:28PM +0100, Heiko Voigt wrote:
Show 24 quoted lines
> On Mon, Jan 06, 2014 at 08:10:04PM -0800, W. Trevor King wrote:
> > Here's an attempted summary of our desires, and my ideal route
> > forward:
> > 
> > * Preferred local submodule branches for each superproject branch.
> >   * Not currently supported by Git.
> >   * Requires some sort of per-superproject-branch .git/config.
> >   * Fall back to the remote-tracking submodule.<name>.branch?
> > 
> > * Auto checkout of the preferred branch
> >   * Can do this at clone-update time with my patch.
> >   * For later submodule branch switches, maybe we want:
> > 
> >       git submodule checkout [-b <branch>] [<paths>…]
> > 
> >     Then if a user blows off their detached HEAD, at least they'll
> >     feel a bit sheepish afterwards.
> 
> Well, for development on a detached HEAD in a submodule we are currently
> not very careful anyway. A simple
> 
> 	git submodule update
> 
> will already blow away any detached HEAD work.

Only if you use the checkout strategy. With --merge or --rebase, you'll have the $sha1 (or upstream remote with --remote) integrated with your detached HEAD work. You end up with a new detached HEAD containing the result of the integration (just confirmed with tests using Git v1.8.3.2). That seems reasonable to me, so I'm happy with the integration logic.

> But AFAIK it should trigger the "you are leaving commits from a
> detached HEAD behind" warning, so there is some safeguard and
> recovery.

I did not see those in testing with Git v1.8.3.2, likely because of the '-f -q' we pass to 'git checkout' for checkout-mode updates.

Regardless of branch integration issues, I think a per-superproject-branch preferred submodule branch is important for 'git checkout' to work in the superproject. If you want:

* submodule branch master for superproject branch master, and
* submodule branch my-feature for superproject branch my-feature,
  $ git checkout my-feature

in the superproject is currently going to leave you with the submodule on master, which is not convenient ;). I think we should come up with a better solution to the superproject checkout problem before adding in additional complications due to branch integration ;).

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Junio C Hamano· Jan 7, 2014, 18:56 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

Francesco Pretto <ceztko@gmail.com> writes:
Show 14 quoted lines
>>> >  - In which situations does the developer or maintainer switch between
>>> >    your attached/detached mode?
>>>
>>> The developer/maintainer does so optionally and voluntarily and it
>>> effects only its private working tree.
>>
>> This does not answer my question. I would like to find out the reason
>> why one would do the switch.
>
> The developer does it voluntarily, at his responsibility, because he
> may decide to partecipate more actively to the development of the
> submodule and still want to use a simple "git submodule update" to
> updates his submodules, overriding its configuration as it can be done
> for other properties like, for example, "branch".

It is still unclear to me why we need attached/detached mode for that. The developer may want to do an exploratory development, whose result is unknown to deserve to be committed on the specified branch at the beginning, and choose to build on a detached HEAD, which is a perfectly normal thing to do. But the standard way to do so, whether the developer is working in the top-level superproject or in a submodule, would be to just do:

	cd $there && git checkout HEAD^0

or use whatever commit the state to be detached is at instead of "HEAD" in the above example, no?

Francesco Pretto· Jan 7, 2014, 19:44 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/2] Introduce git submodule attached update

2014/1/7 Junio C Hamano <gitster@pobox.com>:
Show 20 quoted lines
> Francesco Pretto <ceztko@gmail.com> writes:
>> The developer does it voluntarily, at his responsibility, because he
>> may decide to partecipate more actively to the development of the
>> submodule and still want to use a simple "git submodule update" to
>> updates his submodules, overriding its configuration as it can be done
>> for other properties like, for example, "branch".
>
> It is still unclear to me why we need attached/detached mode for
> that.  The developer may want to do an exploratory development,
> whose result is unknown to deserve to be committed on the specified
> branch at the beginning, and choose to build on a detached HEAD,
> which is a perfectly normal thing to do.  But the standard way to do
> so, whether the developer is working in the top-level superproject
> or in a submodule, would be to just do:
>
>         cd $there && git checkout HEAD^0
>
> or use whatever commit the state to be detached is at instead of
> "HEAD" in the above example, no?
>

Because of the overlapping change with the the other patch proposed by Trevor, and to not generate confusion, I will stop for now pursuing for an "attach|detach" command/switch specific for submodules, waiting for Trevors's patch possible acceptance. After that I will see it still makes sense or not.

Junio C Hamano· Jan 7, 2014, 19:07 UTC · re: Francesco Pretto · lore

Re: [PATCH 2/2] Introduce git submodule attached update

Francesco Pretto <ceztko@gmail.com> writes:
Show 11 quoted lines
> My bottom line:
> - For what I understand, detached HEAD it's a way to say "hey, you
> have to stay on this commit. Also don't even think you can push to the
> upstream branch". This sometimes can't be spurious, as in the use case
> I wrote above: access control on the remote repositories should be
> enough. I think maintainers should have the option to make developers
> to clone a repository starting with an attached HEAD on the branch
> suggested in submodule.$name.branch;
> - "git submodule update" is missing a property to do automatically
> "--remote". I think in the use case I wrote it's really handy to have
> a "git submodule update" to act like this.

The short version I read in the message is that your workflow, in which partipants want to work on a branch, gets frustrating with the current system only because the default update/initial cloning detaches HEAD and will stay in that state until the user gets out of the detached state manually. Once that initial detachment is fixed, there is no more major issue, as update will stay on that branch.

Am I reading you correctly?
Francesco Pretto· Jan 7, 2014, 19:25 UTC · re: Junio C Hamano · lore

Re: [PATCH 2/2] Introduce git submodule attached update

2014/1/7 Junio C Hamano <gitster@pobox.com>:
Show 23 quoted lines
> Francesco Pretto <ceztko@gmail.com> writes:
>
>> My bottom line:
>> - For what I understand, detached HEAD it's a way to say "hey, you
>> have to stay on this commit. Also don't even think you can push to the
>> upstream branch". This sometimes can't be spurious, as in the use case
>> I wrote above: access control on the remote repositories should be
>> enough. I think maintainers should have the option to make developers
>> to clone a repository starting with an attached HEAD on the branch
>> suggested in submodule.$name.branch;
>> - "git submodule update" is missing a property to do automatically
>> "--remote". I think in the use case I wrote it's really handy to have
>> a "git submodule update" to act like this.
>
> The short version I read in the message is that your workflow, in
> which partipants want to work on a branch, gets frustrating with the
> current system only because the default update/initial cloning
> detaches HEAD and will stay in that state until the user gets out of
> the detached state manually. Once that initial detachment is fixed,
> there is no more major issue, as update will stay on that branch.
>
> Am I reading you correctly?
>
Yep, you got it correctly.
Francesco Pretto· Jan 5, 2014, 23:22 UTC · re: Heiko Voigt · lore

Re: [PATCH 2/2] Introduce git submodule attached update

2014/1/5 Heiko Voigt <hvoigt@hvoigt.net>:
> Could you please extend the description of your use-case so we can
> understand your goal better?
>

Maybe I found better words to explain you my goal: the current git submodule use-case threats the submodule as a project independent dependency. My use case threats the submodule as part of the superproject repository. It could be easier to say that in this way submodules would behave very similarly to "svn:externals", something that is actually missing in git. My goal is obtain this without altering git behavior for the existing use case.

>  - In which situations does the developer or maintainer switch between
>    your attached/detached mode?

As I told you in the other answer this is voluntary done by the developer, as he prefers. I came to the conclusion that the "--attach|--detach" switches for the "update" command are not that useful and can be removed. It's still possible to obtain the switch between detached/attached very easily in this way:

# Attach submodule $ git config submodule.<name>.attached "true" $ git submodule update

# Detach submodule $ git config submodule.<name>.attached "false" $ git submodule update

# Unset property in both ".gitmodules" and ".git/config" means -> do nothing $ git config --unset submodule.<name>.attached $ git submodule update

Also my "submodule.<name>.attached" property at the moment behaves like "submodule.<name>.update": it is copied in ".git/config" by "git submodule init". This is probably a mistake: the overridden value should be stored in ".git/config" only at the developer will, so the maintainer has still a chance to modify it in ".gitmodules" and propagate the behavior.

I would send an updated patch but at this point I prefer to wait for a full review.

Thank you, Francesco

Heiko Voigt· Jan 6, 2014, 14:18 UTC · re: Francesco Pretto · lore

Re: Re: [PATCH 2/2] Introduce git submodule attached update

On Mon, Jan 06, 2014 at 12:22:23AM +0100, Francesco Pretto wrote:
Show 12 quoted lines
> 2014/1/5 Heiko Voigt <hvoigt@hvoigt.net>:
> > Could you please extend the description of your use-case so we can
> > understand your goal better?
> >
> 
> Maybe I found better words to explain you my goal: the current git
> submodule use-case threats the submodule as a project independent
> dependency. My use case threats the submodule as part of the
> superproject repository. It could be easier to say that in this way
> submodules would behave very similarly to "svn:externals", something
> that is actually missing in git. My goal is obtain this without
> altering git behavior for the existing use case.

I am not so sure. svn:externals was IMO a hack in SVN to bind projects together. It does not record the revision and so has nothing to do with version control. If you simply want to always checkout the development tip of some project you could do something like this:

	git submodule foreach 'git fetch && git checkout origin/master'

The demand for this 'missing feature' which we call the 'floating submodules' model has been around for some time but until now we could convince people that its not a feature but you are actually loosing history information.

The workflow could always be changed to allow recording revisions. Which is why you use git in the first place right? If you discard revisions for submodules tracking down regression bugs can become a big problem or completely impossible. Try using git bisect on such a history.

Show 5 quoted lines
> >  - In which situations does the developer or maintainer switch between
> >    your attached/detached mode?
> 
> As I told you in the other answer this is voluntary done by the
> developer, as he prefers.
Could you tell me a typical reason?
Show 26 quoted lines
> I came to the conclusion that the
> "--attach|--detach" switches for the "update" command are not that
> useful and can be removed. It's still possible to obtain the switch
> between detached/attached very easily in this way:
> 
> # Attach submodule
> $ git config submodule.<name>.attached "true"
> $ git submodule update
> 
> # Detach submodule
> $ git config submodule.<name>.attached "false"
> $ git submodule update
> 
> # Unset property in both ".gitmodules" and ".git/config" means -> do nothing
> $ git config --unset submodule.<name>.attached
> $ git submodule update
> 
> Also my "submodule.<name>.attached" property at the moment behaves
> like "submodule.<name>.update": it is copied in ".git/config" by "git
> submodule init". This is probably a mistake: the overridden value
> should be stored in ".git/config" only at the developer will, so the
> maintainer has still a chance to modify it in ".gitmodules" and
> propagate the behavior.
> 
> I would send an updated patch but at this point I prefer to wait for a
> full review.

Lets first discuss and figure out what is the real missing feature here and what should be implemented before working further on the code.

Cheers Heiko
W. Trevor King· Jan 6, 2014, 15:58 UTC · re: Heiko Voigt · lore

Re: [PATCH 2/2] Introduce git submodule attached update

On Mon, Jan 06, 2014 at 03:18:05PM +0100, Heiko Voigt wrote:
> If you simply want to always checkout the development tip of some
> project you could do something like this:
> 
> 	git submodule foreach 'git fetch && git checkout origin/master'
Or (respecting submodule.<name>.branch):
  $ git submodule update --remote
You can even:
  $ git submodule update --remote --recursive

whenever you get an itch to upgrade everything everything in one sweeping, hard-to-debug move ;).

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Heiko Voigt· Jan 5, 2014, 20:20 UTC · re: Francesco Pretto · lore

Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:
Show 5 quoted lines
> According to "Documentation/gitmodules.txt", 'checkout' is a valid
> 'submodule.<name>.update' command. Also "git-submodule.sh" refers to
> it and processes it correctly. Reflect commit 'ac1fbb' to support this
> syntax and also validates property values during 'update' command,
> issuing a warning if the value found is unknwon.
s/unknwon/unknown/
Show 31 quoted lines
> ---
>  git-submodule.sh | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 2677f2e..1d041a7 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -622,7 +622,7 @@ cmd_init()
>  		   test -z "$(git config submodule."$name".update)"
>  		then
>  			case "$upd" in
> -			rebase | merge | none)
> +			checkout | rebase | merge | none)
>  				;; # known modes of updating
>  			*)
>  				echo >&2 "warning: unknown update mode '$upd' suggested for submodule '$name'"
> @@ -805,6 +805,18 @@ cmd_update()
>  			update_module=$update
>  		else
>  			update_module=$(git config submodule."$name".update)
> +			case "$update_module" in
> +			'')
> +				;; # Unset update mode
> +			checkout | rebase | merge | none)
> +				;; # Known update modes
> +			!*)
> +				;; # Custom update command
> +			*)
> +				update_module=
> +				echo >&2 "warning: invalid update mode for submodule '$name'"

How about additionally telling the user the current value that is wrong like this:

	echo >&2 "warning: invalid update mode '$update_module' for submodule '$name'"
?
But apart from those minor nits the patch looks good to me.
Cheers Heiko
W. Trevor King· Jan 5, 2014, 20:44 UTC · re: Francesco Pretto · lore

Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:
Show 12 quoted lines
> +			case "$update_module" in
> +			'')
> +				;; # Unset update mode
> +			checkout | rebase | merge | none)
> +				;; # Known update modes
> +			!*)
> +				;; # Custom update command
> +			*)
> +				update_module=
> +				echo >&2 "warning: invalid update mode for submodule '$name'"
> +				;;
> +			esac

I'd prefer `die "…"` to `echo >&2 "…"`. It's hard to know if mapping the user's preferred (unknown) update mechanism to 'checkout' is serious or not.

This commit also makes me think that --rebase, --merge, and --checkout should be replaced with a single --update={rebase|merge|checkout|!…} option, but that's probably food for another commit (and a long finger-breaking deprecation period).

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Junio C Hamano· Jan 6, 2014, 16:20 UTC · re: W. Trevor King · lore

Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

"W. Trevor King" <wking@tremily.us> writes:
Show 22 quoted lines
> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:
>> +			case "$update_module" in
>> +			'')
>> +				;; # Unset update mode
>> +			checkout | rebase | merge | none)
>> +				;; # Known update modes
>> +			!*)
>> +				;; # Custom update command
>> +			*)
>> +				update_module=
>> +				echo >&2 "warning: invalid update mode for submodule '$name'"
>> +				;;
>> +			esac
>
> I'd prefer `die "…"` to `echo >&2 "…"`.  It's hard to know if mapping
> the user's preferred (unknown) update mechanism to 'checkout' is
> serious or not.
>
> This commit also makes me think that --rebase, --merge, and --checkout
> should be replaced with a single --update={rebase|merge|checkout|!…}
> option, but that's probably food for another commit (and a long
> finger-breaking deprecation period).
All of the above points sound sensible to me.
Junio C Hamano· Jan 6, 2014, 17:42 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

Junio C Hamano <gitster@pobox.com> writes:
Show 26 quoted lines
> "W. Trevor King" <wking@tremily.us> writes:
>
>> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:
>>> +			case "$update_module" in
>>> +			'')
>>> +				;; # Unset update mode
>>> +			checkout | rebase | merge | none)
>>> +				;; # Known update modes
>>> +			!*)
>>> +				;; # Custom update command
>>> +			*)
>>> +				update_module=
>>> +				echo >&2 "warning: invalid update mode for submodule '$name'"
>>> +				;;
>>> +			esac
>>
>> I'd prefer `die "…"` to `echo >&2 "…"`.  It's hard to know if mapping
>> the user's preferred (unknown) update mechanism to 'checkout' is
>> serious or not.
>>
>> This commit also makes me think that --rebase, --merge, and --checkout
>> should be replaced with a single --update={rebase|merge|checkout|!…}
>> option, but that's probably food for another commit (and a long
>> finger-breaking deprecation period).
>
> All of the above points sound sensible to me.

I'll tentatively queue this on 'pu' (with the suggested "die" update), with some rewording of the log message. The patch needs to be signed-off, though.

Thanks.
Francesco Pretto· Jan 6, 2014, 17:52 UTC · re: Junio C Hamano · lore

Re: [PATCH 1/2] git-submodule.sh: Support 'checkout' as a valid update command

Ok, applying the suggested modifications and resending shortly.

Thank you, Francesco

2014/1/6 Junio C Hamano <gitster@pobox.com>:
Show 34 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> "W. Trevor King" <wking@tremily.us> writes:
>>
>>> On Sun, Jan 05, 2014 at 03:50:48AM +0100, Francesco Pretto wrote:
>>>> +                   case "$update_module" in
>>>> +                   '')
>>>> +                           ;; # Unset update mode
>>>> +                   checkout | rebase | merge | none)
>>>> +                           ;; # Known update modes
>>>> +                   !*)
>>>> +                           ;; # Custom update command
>>>> +                   *)
>>>> +                           update_module=
>>>> +                           echo >&2 "warning: invalid update mode for submodule '$name'"
>>>> +                           ;;
>>>> +                   esac
>>>
>>> I'd prefer `die "…"` to `echo >&2 "…"`.  It's hard to know if mapping
>>> the user's preferred (unknown) update mechanism to 'checkout' is
>>> serious or not.
>>>
>>> This commit also makes me think that --rebase, --merge, and --checkout
>>> should be replaced with a single --update={rebase|merge|checkout|!…}
>>> option, but that's probably food for another commit (and a long
>>> finger-breaking deprecation period).
>>
>> All of the above points sound sensible to me.
>
> I'll tentatively queue this on 'pu' (with the suggested "die"
> update), with some rewording of the log message.  The patch needs to
> be signed-off, though.
>
> Thanks.

← back to recent threads