git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 16/16] git-clone: add --submodules for cloning submodules

From
Sskimo@liacs.nl <skimo@liacs.nl>
Date
May 18, 2007, 19:25 UTC
Message-ID
<11795163073175-git-send-email-skimo@liacs.nl>
In-Reply-To
<11795163053812-git-send-email-skimo@liacs.nl>
From: Sven Verdoolaege <skimo@kotnet.org>

When the --submodules option is specified, git-clone will search for submodule.<submodule>.url options in the remote configuration and clone each submodule using the first url that it can use from the local site.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
 Documentation/config.txt    |    7 +++
 Documentation/git-clone.txt |    6 ++-
 git-clone.sh                |   18 +++++++-
 git-fetch.sh                |   90 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 117 insertions(+), 4 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ee1c35e..cee9e40 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -256,6 +256,10 @@ You probably do not need to adjust this value.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
+core.submodules
+	If true, gitlink:git-checkout[1] also checks out submodules.
+	False by default.
+
 alias.*::
 	Command aliases for the gitlink:git[1] command wrapper - e.g.
 	after defining "alias.last = cat-file commit HEAD", the invocation
@@ -606,6 +610,9 @@ showbranch.default::
 	The default set of branches for gitlink:git-show-branch[1].
 	See gitlink:git-show-branch[1].
 
+submodule.<submodule>.url
+	The URL of a submodule.  See gitlink:git-clone[1].
+
 tar.umask::
 	By default, gitlink:git-tar-tree[1] sets file and directories modes
 	to 0666 or 0777. While this is both useful and acceptable for projects
diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt
index 644bf12..565155b 100644
--- a/Documentation/git-clone.txt
+++ b/Documentation/git-clone.txt
@@ -11,7 +11,7 @@ SYNOPSIS
 [verse]
 'git-clone' [--template=<template_directory>] [-l [-s]] [-q] [-n] [--bare]
 	  [-o <name>] [-u <upload-pack>] [--reference <repository>]
-	  [--depth <depth>] <repository> [<directory>]
+	  [--depth <depth>] [--submodules] <repository> [<directory>]
 
 DESCRIPTION
 -----------
@@ -105,6 +105,10 @@ OPTIONS
 	with a long history, and would want to send in a fixes
 	as patches.
 
+--submodules::
+	Clone submodules specified in (remote) configuration parameters
+	submodule.<submodule>.url.
+
 <repository>::
 	The (possibly remote) repository to clone from.  It can
 	be any URL git-fetch supports.
diff --git a/git-clone.sh b/git-clone.sh
index 44387f4..f5a3548 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -14,7 +14,7 @@ die() {
 }
 
 usage() {
-	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] <repo> [<dir>]"
+	die "Usage: $0 [--template=<template_directory>] [--reference <reference-repo>] [--bare] [-l [-s]] [-q] [-u <upload-pack>] [--origin <name>] [--depth <n>] [-n] [--submodules] <repo> [<dir>]"
 }
 
 get_repo_base() {
@@ -88,6 +88,7 @@ origin_override=
 use_separate_remote=t
 depth=
 no_progress=
+submodules=
 test -t 1 || no_progress=--no-progress
 while
 	case "$#,$1" in
@@ -138,6 +139,8 @@ while
 	*,--depth)
 		shift
 		depth="--depth=$1";;
+	*,--su|*,--sub|*,--subm|*,--submo|*,--submod|*,--submodu|*,--submodul|\
+	*,--submodule|*,--submodules) submodules="--submodules" ;;
 	*,-*) usage ;;
 	*) break ;;
 	esac
@@ -156,6 +159,10 @@ then
 	then
 		die '--bare and --origin $origin options are incompatible.'
 	fi
+	if test -n "$submodules"
+	then
+		die '--bare and --submodules origin options are incompatible.'
+	fi
 	no_checkout=yes
 	use_separate_remote=
 fi
@@ -309,7 +316,7 @@ yes,yes)
 		;;
 	*)
 		git-fetch --all -k $quiet "$upload_pack" $depth \
-			$separate_remote_flag "$origin" ||
+			$separate_remote_flag $submodules "$origin" ||
 			die "fetch from '$repo' failed."
 		;;
 	esac
@@ -405,10 +412,15 @@ then
 		git-config branch."$head_points_at".merge "refs/heads/$head_points_at"
 	esac
 
+	if test -n "$submodules"
+	then
+		git-config core.submodules true
+	fi
+
 	case "$no_checkout" in
 	'')
 		test "z$quiet" = z -a "z$no_progress" = z && v=-v || v=
-		git-read-tree -m -u $v HEAD HEAD
+		git-read-tree -m -u $v $submodules HEAD HEAD
 	esac
 fi
 rm -f "$GIT_DIR/CLONE_HEAD" "$GIT_DIR/REMOTE_HEAD"
diff --git a/git-fetch.sh b/git-fetch.sh
index 84c2523..b7ef0c3 100755
--- a/git-fetch.sh
+++ b/git-fetch.sh
@@ -15,6 +15,70 @@ LF='
 '
 IFS="$LF"
 
+local_URL() {
+	# tranforms a "URL" on the remote to a URL that works on the local machine
+	# $1 - remote, $2 - URL on remote
+	echo "$1 $2" >&2
+	case "$1" in
+	https://*|http://*|ftp://*)
+		case "$2" in
+		https://*|http://*|ftp://*)
+			echo $2
+		esac
+		;;
+	ssh://*)
+		case "$2" in
+		https://*|http://*|ftp://*)
+			echo $2
+			;;
+		/*)
+			echo $(echo $1 | sed -e 's/\(ssh:\/\/[^\/]*\)\/.*/\1/')$2
+		esac
+		;;
+	/*)
+		echo $2
+		;;
+	*)
+		case "$2" in
+		https://*|http://*|ftp://*)
+			echo $2
+		esac
+	esac
+}
+
+clone_submodules () {
+	# $1 - remote
+	remote=$1
+	( : subshell because we muck with IFS
+        IFS=" 	$LF"
+	cd "$GIT_DIR/.."
+	git-config --remote="$remote" --get-regexp 'submodule\..*\.url' | \
+	sed -e 's/^submodule\.//' -e 's/\.url / /' |
+	while read submodule URL
+	do
+		previous=$(git-config "submodule.$submodule.url")
+		if test -n "$previous"
+		then
+			continue;
+		fi
+		URL=$(local_URL "$remote" "$URL")
+		if test -z "$URL"
+		then
+			continue;
+		fi
+		# At this point, we don't know if the submodule
+		# appears in the HEAD of the supermodule, so clone it
+		# without a checkout and overwrite HEAD so that a subsequent
+		# checkout won't assume the submodule has already been
+		# checked out.
+		git-clone --submodules -n "$URL" "$submodule"
+		z40=0000000000000000000000000000000000000000
+		GIT_DIR="$submodule/.git" git-update-ref --no-deref HEAD $z40
+		git-config "submodule.$submodule.url" "$URL"
+	done
+	)
+}
+
 all=
 no_tags=
 tags=
@@ -27,6 +91,7 @@ keep=
 shallow_depth=
 no_progress=
 use_separate_remote=
+submodules=
 test -t 1 || no_progress=--no-progress
 quiet=
 while case "$#" in 0) break ;; esac
@@ -84,6 +149,15 @@ do
 		shift
 		shallow_depth="--depth=$1"
 		;;
+	--su|--sub|--subm|--submo|--submod|--submodu|--submodul|\
+	--submodule|--submodules)
+		submodules="yes"
+		;;
+	--no-su|--no-sub|--no-subm|--no-submo|--no-submod|\
+	--no-submodu|--no-submodul|\
+	--no-submodule|--no-submodules)
+		submodules="no"
+		;;
 	-*)
 		usage
 		;;
@@ -149,6 +223,18 @@ case "$tags$no_tags" in
 	esac
 esac
 
+case "$submodules" in
+'')
+	case "$(git-config --bool core.submodules)" in
+	true)
+		submodues=yes
+		;;
+	*)
+		submodules=no
+		;;
+	esac
+esac
+
 # If --tags (and later --heads or --all) is specified, then we are
 # not talking about defaults stored in Pull: line of remotes or
 # branches file, and just fetch those and refspecs explicitly given.
@@ -407,3 +493,7 @@ case "$orig_head" in
 	fi
 	;;
 esac
+
+if test "$submodules" = yes; then
+	clone_submodules "$remote"
+fi

-- 
1.5.2.rc3.783.gc7476-dirty
Previous: skimo@liacs.nlNext: Sven Verdoolaege
Message 62 of 63 in “Second round of support for cloning submodules”
  1. skimo@liacs.nlMay 18, 2007
  2. 01/16 Add dump-configskimo@liacs.nl, May 18, 2007
  3. 02/16 git-config: add --remote option for reading config from remote reposkimo@liacs.nl, May 18, 2007
  4. 03/16 http.h: make fill_active_slots a function pointerskimo@liacs.nl, May 18, 2007
  5. 04/16 git-config: read remote config files over HTTPskimo@liacs.nl, May 18, 2007
  6. 05/16 unpack-trees.c: verify_uptodate: remove dead codeskimo@liacs.nl, May 18, 2007
  7. Junio C HamanoMay 18, 2007
  8. 06/16 unpack-trees.c: pass cache_entry * to verify_absent rather than just the nameskimo@liacs.nl, May 18, 2007
  9. 07/16 git-read-tree: take --submodules optionskimo@liacs.nl, May 18, 2007
  10. Alex RiesenMay 18, 2007
  11. Sven VerdoolaegeMay 18, 2007
  12. Alex RiesenMay 18, 2007
  13. Junio C HamanoMay 19, 2007
  14. Shawn O. PearceMay 19, 2007
  15. Alex RiesenMay 19, 2007
  16. Sven VerdoolaegeMay 19, 2007
  17. Junio C HamanoMay 19, 2007
  18. Jan HudecMay 20, 2007
  19. Junio C HamanoMay 20, 2007
  20. Sven VerdoolaegeMay 20, 2007
  21. Jan HudecMay 21, 2007
  22. Sven VerdoolaegeMay 21, 2007
  23. Junio C HamanoMay 21, 2007
  24. Jan HudecMay 21, 2007
  25. Martin WaitzMay 21, 2007
  26. Jan HudecMay 22, 2007
  27. Martin WaitzMay 24, 2007
  28. Jakub NarebskiMay 25, 2007
  29. Jan HudecMay 25, 2007
  30. Junio C HamanoMay 24, 2007
  31. Sven VerdoolaegeMay 24, 2007
  32. Junio C HamanoMay 24, 2007
  33. Sven VerdoolaegeMay 24, 2007
  34. Junio C HamanoMay 24, 2007
  35. Petr BaudisMay 24, 2007
  36. Junio C HamanoMay 24, 2007
  37. Junio C HamanoMay 24, 2007
  38. Petr BaudisMay 24, 2007
  39. Jan HudecMay 25, 2007
  40. Junio C HamanoMay 25, 2007
  41. Steven GrimmMay 25, 2007
  42. Junio C HamanoMay 25, 2007
  43. Petr BaudisMay 19, 2007
  44. 08/16 unpack-trees.c: assume submodules are cleanskimo@liacs.nl, May 18, 2007
  45. 09/16 entry.c: optionally checkout submodulesskimo@liacs.nl, May 18, 2007
  46. Alex RiesenMay 18, 2007
  47. Sven VerdoolaegeMay 18, 2007
  48. Alex RiesenMay 18, 2007
  49. Alex RiesenMay 18, 2007
  50. Add run_command_v_opt_cd: chdir into a directory before execAlex Riesen, May 18, 2007
  51. Use run_command_v_opt_cd when checking out a submoduleAlex Riesen, May 18, 2007
  52. 10/16 git-checkout: pass --submodules option to git-read-treeskimo@liacs.nl, May 18, 2007
  53. Petr BaudisMay 19, 2007
  54. 11/16 git-fetch: skip empty argumentsskimo@liacs.nl, May 18, 2007
  55. Junio C HamanoMay 18, 2007
  56. 12/16 builtin-fetch--tool: extend "native-store" for use in cloningskimo@liacs.nl, May 18, 2007
  57. Alex RiesenMay 18, 2007
  58. Sven VerdoolaegeMay 19, 2007
  59. 13/16 git-clone: rely on git-fetch for fetching for most protocolsskimo@liacs.nl, May 18, 2007
  60. 14/16 git-clone: rely on git-fetch for non-bare fetching over httpskimo@liacs.nl, May 18, 2007
  61. 15/16 git-read-tree: treat null commit as empty treeskimo@liacs.nl, May 18, 2007
  62. 16/16 git-clone: add --submodules for cloning submodulesskimo@liacs.nl, May 18, 2007
  63. Sven VerdoolaegeMay 18, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.