threads / patch / 14922

patchgit-submodule - Add 'foreach' subcommand

Subject: [PATCH] git-submodule - Add 'foreach' subcommand

## tl;dr

10 messages between Aug 10, 2008 and Aug 12, 2008. Diffs are folded; open one to read it.

replies: 9people: 7as markdown or json

Mark Levedahl· Aug 10, 2008, 16:44 UTC · lore

submodule foreach <command-list> will execute the list of commands in each currently checked out submodule directory. The list of commands is arbitrary as long as it is acceptable to sh. The variables '$path' and '$sha1' are availble to the command-list, defining the submodule path relative to the superproject and the submodules's commitID as recorded in the superproject (this may be different than HEAD in the submodule).

This utility is inspired by a number of threads on the mailing list looking for ways to better integrate submodules in a tree and work with them as a unit. This could include fetching a new branch in each from a given source, or possibly checking out a given named branch in each. Currently, there is no consensus as to what additional commands should be implemented in the porcelain, requiring all users whose needs exceed that of git-submodule to do their own scripting. The foreach command is intended to support such scripting, and in particular does no error checking and produces no output, thus allowing end users complete control over any information printed out and over what constitutes an error.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   12 ++++++++++++
 git-submodule.sh                |   22 ++++++++++++++++++++--
 2 files changed, 32 insertions(+), 2 deletions(-)
Show changes to 2 files +32 −2

Documentation/git-submodule.txt, git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index bf33b0c..5b52f35 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
+'git submodule' foreach <command-list>
 
 
 DESCRIPTION
@@ -123,6 +124,17 @@ summary::
 	in the submodule between the given super project commit and the
 	index or working tree (switched by --cached) are shown.
 
+foreach::
+	Executes an arbitrary list of commands in each checked out submodule.
+	$path is the name of the submodule directory relative to the
+	superproject, and $sha1 is the commit as recorded in the superproject.
+	And submodules defined in the superproject but not checked out are
+	ignored by this command, and an empty command-list provides no output.
++
+As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
+show the path and currently checked out commit for each submodule.
+
+
 OPTIONS
 -------
 -q::
diff --git a/git-submodule.sh b/git-submodule.sh
index b40f876..abbc7bb 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]"
+[--] [<path>...]|[foreach <command-list>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -199,6 +199,24 @@ cmd_add()
 }
 
 #
+# Execute an arbitrary command sequence in each checked out
+# submodule
+#
+# $@ = command to execute
+#
+cmd_foreach()
+{
+	git ls-files --stage | grep '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if test -e "$path"/.git
+		then
+			(cd "$path" && eval "$@")
+		fi
+	done
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -583,7 +601,7 @@ cmd_status()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | init | update | status | summary)
+	add | foreach | init | update | status | summary)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.6.0.rc2.44.g974eb
Johan Herland· Aug 10, 2008, 22:22 UTC · re: Mark Levedahl · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

On Sunday 10 August 2008, Mark Levedahl wrote:
Show 21 quoted lines
> submodule foreach <command-list> will execute the list of commands in
> each currently checked out submodule directory. The list of commands
> is arbitrary as long as it is acceptable to sh. The variables '$path'
> and '$sha1' are availble to the command-list, defining the submodule
> path relative to the superproject and the submodules's commitID as
> recorded in the superproject (this may be different than HEAD in the
> submodule).
>
> This utility is inspired by a number of threads on the mailing list
> looking for ways to better integrate submodules in a tree and work
> with them as a unit. This could include fetching a new branch in each
> from a given source, or possibly checking out a given named branch in
> each. Currently, there is no consensus as to what additional commands
> should be implemented in the porcelain, requiring all users whose needs
> exceed that of git-submodule to do their own scripting. The foreach
> command is intended to support such scripting, and in particular does
> no error checking and produces no output, thus allowing end users
> complete control over any information printed out and over what
> constitutes an error.
>
> Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
Hi,

I was about to send an equivalent patch, but you beat me to it. Yours is shorter and more elegant as well, so thanks for saving me the embarrasment ;)

So, FWIW:
Liked-by: Johan Herland <johan@herland.net>
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Junio C Hamano· Aug 10, 2008, 22:37 UTC · re: Mark Levedahl · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

Mark Levedahl <mlevedahl@gmail.com> writes:
> This utility is inspired by a number of threads on the mailing list
> looking for ways to better integrate submodules in a tree and work
> with them as a unit.
Thanks for keeping the topic going
Show 9 quoted lines
> +foreach::
> +	Executes an arbitrary list of commands in each checked out submodule.
> +	$path is the name of the submodule directory relative to the
> +	superproject, and $sha1 is the commit as recorded in the superproject.
> +	And submodules defined in the superproject but not checked out are
> +	ignored by this command, and an empty command-list provides no output.
> ++
> +As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
> +show the path and currently checked out commit for each submodule.
I think this is a sensible building block to have.
Show 11 quoted lines
> +cmd_foreach()
> +{
> +	git ls-files --stage | grep '^160000 ' |
> +	while read mode sha1 stage path
> +	do
> +		if test -e "$path"/.git
> +		then
> +			(cd "$path" && eval "$@")
> +		fi
> +	done
> +}

I suspect we may want to stop the loop upon failure from eval (any user that wants to run command in all submodules no matter what could add '||:' to the cmd list), but other than that this is a simple enough and good addition.

Mark Levedahl· Aug 10, 2008, 23:10 UTC · re: Junio C Hamano · lore

submodule foreach <command-list> will execute the list of commands in each currently checked out submodule directory. The list of commands is arbitrary as long as it is acceptable to sh. The variables '$path' and '$sha1' are availble to the command-list, defining the submodule path relative to the superproject and the submodules's commitID as recorded in the superproject (this may be different than HEAD in the submodule).

This utility is inspired by a number of threads on the mailing list looking for ways to better integrate submodules in a tree and work with them as a unit. This could include fetching a new branch in each from a given source, or possibly checking out a given named branch in each. Currently, there is no consensus as to what additional commands should be implemented in the porcelain, requiring all users whose needs exceed that of git-submodule to do their own scripting. The foreach command is intended to support such scripting, and in particular does no error checking and produces no output, thus allowing end users complete control over any information printed out and over what constitutes an error. The processing does terminate if the command-list returns an error, but processing can easily be forced for all submodules be terminating the list with ';true'.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   15 +++++++++++++++
 git-submodule.sh                |   23 +++++++++++++++++++++--
 2 files changed, 36 insertions(+), 2 deletions(-)
Show changes to 2 files +36 −2

Documentation/git-submodule.txt, git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index bf33b0c..1e7d352 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,6 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
+'git submodule' foreach <command-list>
 
 
 DESCRIPTION
@@ -123,6 +124,20 @@ summary::
 	in the submodule between the given super project commit and the
 	index or working tree (switched by --cached) are shown.
 
+foreach::
+	Executes an arbitrary list of commands in each checked out submodule.
+	$path is the name of the submodule directory relative to the
+	superproject, and $sha1 is the commit as recorded in the superproject.
+	Any submodules defined in the superproject but not checked out are
+	ignored by this command, and an empty command-list provides no output.
+	A non-zero return from the command-list in any submodule causes
+	the processing to terminate. This can be overridden by adding '; true'
+	to the end of the command list.
++
+As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
+show the path and currently checked out commit for each submodule.
+
+
 OPTIONS
 -------
 -q::
diff --git a/git-submodule.sh b/git-submodule.sh
index b40f876..39a19f0 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]"
+[--] [<path>...]|[foreach <command-list>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -199,6 +199,25 @@ cmd_add()
 }
 
 #
+# Execute an arbitrary command sequence in each checked out
+# submodule
+#
+# $@ = command to execute
+#
+cmd_foreach()
+{
+	git ls-files --stage | grep '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if test -e "$path"/.git
+		then
+			(cd "$path" && eval "$@") ||
+			die "Error detected evaluating commands in '$path'"
+		fi
+	done
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -583,7 +602,7 @@ cmd_status()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | init | update | status | summary)
+	add | foreach | init | update | status | summary)
 		command=$1
 		;;
 	-q|--quiet)
-- 
1.6.0.rc2.44.g974eb
Balazs Nagy· Aug 11, 2008, 06:28 UTC · re: Mark Levedahl · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

Hi,

Sorry for being a bit late, but what if we would have --all for all submodule commands instead of having another command?

Rgds,
-- 
-jul-
Johannes Schindelin· Aug 11, 2008, 10:08 UTC · re: Balazs Nagy · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

Hi,
On Mon, 11 Aug 2008, Balazs Nagy wrote:
> Sorry for being a bit late, but what if we would have --all for all 
> submodule commands instead of having another command?

and the foreach patch would go into which existing command, exactly? "git --all"? A new "git foreach --all"?

Just kidding, Dscho

Matthias Kestenholz· Aug 11, 2008, 10:11 UTC · re: Balazs Nagy · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

On Mon, 2008-08-11 at 08:28 +0200, Balazs Nagy wrote:
Show 6 quoted lines
> Hi,
> 
> Sorry for being a bit late, but what if we would have --all for all
> submodule commands instead of having another command?
> 
> Rgds,

This would not be very smart. The example in the documentation part of the patch would be impossible only with --all switches (or, you'd need to duplicate all git commands in git-submodule instead of reusing them with git submodule foreach)

Petr Baudis· Aug 11, 2008, 10:23 UTC · re: Mark Levedahl · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

  Hi,
On Sun, Aug 10, 2008 at 07:10:04PM -0400, Mark Levedahl wrote:
Show 7 quoted lines
> submodule foreach <command-list> will execute the list of commands in
> each currently checked out submodule directory. The list of commands
> is arbitrary as long as it is acceptable to sh. The variables '$path'
> and '$sha1' are availble to the command-list, defining the submodule
> path relative to the superproject and the submodules's commitID as
> recorded in the superproject (this may be different than HEAD in the
> submodule).
  in principle, this looks pretty sensible.
Show 12 quoted lines
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index bf33b0c..1e7d352 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -14,6 +14,7 @@ SYNOPSIS
>  'git submodule' [--quiet] init [--] [<path>...]
>  'git submodule' [--quiet] update [--init] [--] [<path>...]
>  'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
> +'git submodule' foreach <command-list>
>  
>  
>  DESCRIPTION
  But as visible here, this is a little bit inconsistent. I think
foreach should be by default verbose about which submodules does it
recurse into - this is what you want in case of casual usage on
command-line. In case you want to have full control on the output within
a script, you can always pass the extra --quiet and it's less obnoxious
this way around.
  I also have a problem with the <command-list> - is this one argument?
Multiple arguments? The semantics is ill-defined. If it is supposed to
be a single argument, please drop the -list bit; silent DWIMmery of
using "$@" internally is acceptable, I guess. If it is supposed to be
multiple arguments, you need to
	(i) Specify that as <command>... instead
	(ii) Either have an eternal annoyance about insane behaviour
here, or use something better than eval "$@". Since
	git submodule foreach cp x\ y z
will simply _not_ work properly.
  So I think it's best to just drop the 'list' part. You're just
evaluating a shell expression passed in a parameter.
Show 6 quoted lines
> @@ -123,6 +124,20 @@ summary::
>  	in the submodule between the given super project commit and the
>  	index or working tree (switched by --cached) are shown.
>  
> +foreach::
> +	Executes an arbitrary list of commands in each checked out submodule.
I think "evaluates" is a better word here, too.
-- 
				Petr "Pasky" Baudis
The next generation of interesting software will be done
on the Macintosh, not the IBM PC.  -- Bill Gates
Junio C Hamano· Aug 11, 2008, 20:40 UTC · re: Petr Baudis · lore

Re: [PATCH] git-submodule - Add 'foreach' subcommand

Petr Baudis <pasky@suse.cz> writes:
>   in principle, this looks pretty sensible.
> ...
> I think "evaluates" is a better word here, too.
I think your refinements all make sense.  Thanks for commenting.
Mark Levedahl· Aug 12, 2008, 01:45 UTC · re: Junio C Hamano · lore

[PATCH] git-submodule foreach - Include output and clarify usage.

Petr Baudis provided good feedback on the list, leading to this patch to modify foreach.

This clarifies that the input to "foreach" is a single shell command, though internally foreach evaluates "$@" so *may* work even if the user fails to enclose the command in quotes. Also, this adds output of each submodule entered unless --quiet is given, providing feedback at the command line but still allowing a script to precisely control the output given.

Signed-off-by: Mark Levedahl <mlevedahl@gmail.com>
---
 Documentation/git-submodule.txt |   12 +++++++-----
 git-submodule.sh                |    3 ++-
 2 files changed, 9 insertions(+), 6 deletions(-)
Show changes to 2 files +9 −6

Documentation/git-submodule.txt, git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 3470795..abbd5b7 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -14,7 +14,7 @@ SYNOPSIS
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] update [--init] [--] [<path>...]
 'git submodule' [--quiet] summary [--summary-limit <n>] [commit] [--] [<path>...]
-'git submodule' foreach <command-list>
+'git submodule' [--quiet] foreach <command>
 
 
 DESCRIPTION
@@ -125,14 +125,16 @@ summary::
 	index or working tree (switched by --cached) are shown.
 
 foreach::
-	Executes an arbitrary list of commands in each checked out submodule.
+	Evaluates an arbitrary shell command in each checked out submodule.
+	The command has access to the variables $path and $sha1:
 	$path is the name of the submodule directory relative to the
 	superproject, and $sha1 is the commit as recorded in the superproject.
 	Any submodules defined in the superproject but not checked out are
-	ignored by this command, and an empty command-list provides no output.
-	A non-zero return from the command-list in any submodule causes
+	ignored by this command. Unless given --quiet, foreach prints the name
+	of each submodule before evaluating the command.
+	A non-zero return from the command in any submodule causes
 	the processing to terminate. This can be overridden by adding '|| :'
-	to the end of the command list.
+	to the end of the command.
 +
 As an example, "git submodule foreach 'echo $path `git rev-parse HEAD`' will
 show the path and currently checked out commit for each submodule.
diff --git a/git-submodule.sh b/git-submodule.sh
index fc41e59..2d57d60 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -6,7 +6,7 @@
 
 USAGE="[--quiet] [--cached] \
 [add <repo> [-b branch] <path>]|[status|init|update [-i|--init]|summary [-n|--summary-limit <n>] [<commit>]] \
-[--] [<path>...]|[foreach <command-list>]"
+[--] [<path>...]|[foreach <command>]"
 OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
@@ -211,6 +211,7 @@ cmd_foreach()
 	do
 		if test -e "$path"/.git
 		then
+			say "Entering '$path'"
 			(cd "$path" && eval "$@") ||
 			die "Stopping at '$path'; script returned non-zero status."
 		fi
-- 
1.6.0.rc2.45.g1363

← back to recent threads