git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC/PATCH 4/6] git submodule foreach: Add --recursive to recurse into nested submodules

From
Johan Herland <johan@herland.net>
Date
Aug 19, 2009, 01:45 UTC
Message-ID
<1250646324-961-5-git-send-email-johan@herland.net>
In-Reply-To
<1250646324-961-1-git-send-email-johan@herland.net>

In very large and hierarchically structured projects, one may encounter nested submodules. In these situations, it is valuable to not only operate on all the submodules in the current repo (which is what is currently done by 'git submodule foreach'), but also to operate on all submodules at all levels (i.e. recursing into nested submodules as well).

This patch teaches the new --recursive option to the 'git submodule foreach' command. The patch also includes documentation and selftests.

Signed-off-by: Johan Herland <johan@herland.net>
---
 Documentation/git-submodule.txt |   10 ++++-
 git-submodule.sh                |   19 ++++++-
 t/t7407-submodule-foreach.sh    |   99 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 124 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index cfa8d73..c604550 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -16,7 +16,7 @@ SYNOPSIS
 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase]
 	      [--reference <repository>] [--merge] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
-'git submodule' [--quiet] foreach <command>
+'git submodule' [--quiet] foreach [--recursive] <command>
 'git submodule' [--quiet] sync [--] [<path>...]
 
 
@@ -142,6 +142,8 @@ foreach::
 	Any submodules defined in the superproject but not checked out are
 	ignored by this command. Unless given --quiet, foreach prints the name
 	of each submodule before evaluating the command.
+	If --recursive is given, submodules are traversed recursively (i.e.
+	the given shell command is evaluated in nested submodules as well).
 	A non-zero return from the command in any submodule causes
 	the processing to terminate. This can be overridden by adding '|| :'
 	to the end of the command.
@@ -219,6 +221,12 @@ OPTIONS
 *NOTE*: Do *not* use this option unless you have read the note
 for linkgit:git-clone[1]'s --reference and --shared options carefully.
 
+--recursive::
+	This option is only valid for the foreach command.
+	Traverse submodules recursively. The operation is performed not
+	only in the submodules of the current repo, but also
+	in any nested submodules inside those submodules (and so on).
+
 <path>...::
 	Paths to submodule(s). When specified this will restrict the command
 	to only operate on the submodules found at the specified paths.
diff --git a/git-submodule.sh b/git-submodule.sh
index 6163d01..dbfc483 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -10,7 +10,7 @@ USAGE="[--quiet] add [-b branch] [--reference <repository>] [--] <repository> <p
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] update [--init] [-N|--no-fetch] [--rebase] [--reference <repository>] [--merge] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
-   or: $dashless [--quiet] foreach <command>
+   or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--] [<path>...]"
 OPTIONS_SPEC=
 . git-sh-setup
@@ -24,6 +24,7 @@ cached=
 files=
 nofetch=
 update=
+prefix=
 
 # Resolve relative url by appending to parent's url
 resolve_relative_url ()
@@ -250,6 +251,9 @@ cmd_foreach()
 		-q|--quiet)
 			GIT_QUIET=1
 			;;
+		--recursive)
+			recursive=1
+			;;
 		-*)
 			usage
 			;;
@@ -265,9 +269,18 @@ cmd_foreach()
 	do
 		if test -e "$path"/.git
 		then
-			say "Entering '$path'"
+			say "Entering '$prefix$path'"
 			name=$(module_name "$path")
-			(cd "$path" && eval "$@") ||
+			(
+				prefix="$prefix$path/"
+				unset GIT_DIR
+				cd "$path" &&
+				eval "$@" &&
+				if test -n "$recursive"
+				then
+					cmd_foreach "--recursive" "$@"
+				fi
+			) ||
 			die "Stopping at '$path'; script returned non-zero status."
 		fi
 	done
diff --git a/t/t7407-submodule-foreach.sh b/t/t7407-submodule-foreach.sh
index 991aa80..be122c7 100755
--- a/t/t7407-submodule-foreach.sh
+++ b/t/t7407-submodule-foreach.sh
@@ -76,4 +76,103 @@ test_expect_success 'test basic "submodule foreach" usage' '
 	test_cmp expect actual
 '
 
+test_expect_success 'setup nested submodules' '
+	git clone submodule nested1 &&
+	git clone submodule nested2 &&
+	git clone submodule nested3 &&
+	(
+		cd nested3 &&
+		git submodule add ../submodule submodule &&
+		test_tick &&
+		git commit -m "submodule" &&
+		git submodule init submodule
+	) &&
+	(
+		cd nested2 &&
+		git submodule add ../nested3 nested3 &&
+		test_tick &&
+		git commit -m "nested3" &&
+		git submodule init nested3
+	) &&
+	(
+		cd nested1 &&
+		git submodule add ../nested2 nested2 &&
+		test_tick &&
+		git commit -m "nested2" &&
+		git submodule init nested2
+	) &&
+	(
+		cd super &&
+		git submodule add ../nested1 nested1 &&
+		test_tick &&
+		git commit -m "nested1" &&
+		git submodule init nested1
+	)
+'
+
+test_expect_success 'use "submodule foreach" to checkout 2nd level submodule' '
+	git clone super clone2 &&
+	(
+		cd clone2 &&
+		test ! -d sub1/.git &&
+		test ! -d sub2/.git &&
+		test ! -d sub3/.git &&
+		test ! -d nested1/.git &&
+		git submodule update --init &&
+		test -d sub1/.git &&
+		test -d sub2/.git &&
+		test -d sub3/.git &&
+		test -d nested1/.git &&
+		test ! -d nested1/nested2/.git &&
+		git submodule foreach "git submodule update --init" &&
+		test -d nested1/nested2/.git &&
+		test ! -d nested1/nested2/nested3/.git
+	)
+'
+
+test_expect_success 'use "foreach --recursive" to checkout all submodules' '
+	(
+		cd clone2 &&
+		git submodule foreach --recursive "git submodule update --init" &&
+		test -d nested1/nested2/nested3/.git &&
+		test -d nested1/nested2/nested3/submodule/.git
+	)
+'
+
+cat > expect <<EOF
+Entering 'nested1'
+Entering 'nested1/nested2'
+Entering 'nested1/nested2/nested3'
+Entering 'nested1/nested2/nested3/submodule'
+Entering 'sub1'
+Entering 'sub2'
+Entering 'sub3'
+EOF
+
+test_expect_success 'test messages from "foreach --recursive"' '
+	(
+		cd clone2 &&
+		git submodule foreach --recursive "true" > ../actual
+	) &&
+	test_cmp expect actual
+'
+
+cat > expect <<EOF
+nested1-nested1
+nested2-nested2
+nested3-nested3
+submodule-submodule
+foo1-sub1
+foo2-sub2
+foo3-sub3
+EOF
+
+test_expect_success 'test "foreach --quiet --recursive"' '
+	(
+		cd clone2 &&
+		git submodule foreach -q --recursive "echo \$name-\$path" > ../actual
+	) &&
+	test_cmp expect actual
+'
+
 test_done
-- 
1.6.4.304.g1365c.dirty
Previous: Johan HerlandNext: Johan Herland
Message 5 of 8 in “Git submodule: 'foreach' enhancements and nested submodule handling”
  1. 0/6 Git submodule: 'foreach' enhancements and nested submodule handlingJohan Herland, Aug 19, 2009
  2. 1/6 git submodule: Cleanup usage string and add option parsing to cmd_foreach()Johan Herland, Aug 19, 2009
  3. 2/6 Add selftest for 'git submodule foreach'Johan Herland, Aug 19, 2009
  4. 3/6 git submodule foreach: Provide access to submodule name, as '$name'Johan Herland, Aug 19, 2009
  5. 4/6 git submodule foreach: Add --recursive to recurse into nested submodulesJohan Herland, Aug 19, 2009
  6. 5/6 git submodule update: Introduce --recursive to update nested submodulesJohan Herland, Aug 19, 2009
  7. 6/6 git submodule status: Add --recursive to recurse into nested submodulesJohan Herland, Aug 19, 2009
  8. 7/6 git clone: Add --recursive to automatically checkout (nested) submodulesJohan Herland, Aug 19, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.