threads / patch / 8672

patch[RESEND] git-submodule: provide easy way of adding new submodules

Subject: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

## tl;dr

10 messages between Jun 21, 2007 and Jul 4, 2007. Diffs are folded; open one to read it.

replies: 9people: 3as markdown or json

Sven Verdoolaege· Jun 21, 2007, 09:53 UTC · lore

To make a submodule effectively usable, the path and a URL where the submodule can be cloned need to be stored in .gitmodules. This subcommand takes care of setting this information after cloning the new submodule. Only the index is updated, so, if needed, the user may still change the URL or switch to a different branch of the submodule before committing.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
I didn't get any reaction (negative or positive) to this patch.
The only difference with the previous version is that I know
check for the existence of the destination path before calling
module_clone.
skimo
 Documentation/git-submodule.txt |    8 ++++
 git-submodule.sh                |   68 +++++++++++++++++++++++++++++++++++---
 2 files changed, 70 insertions(+), 6 deletions(-)
Show changes to 2 files +70 −6

Documentation/git-submodule.txt, git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index f8fb80f..bf9cbf9 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,11 +8,19 @@ git-submodule - Initialize, update or inspect submodules
 
 SYNOPSIS
 --------
+'git-submodule' [--quiet] add <repository> [<path>]
 'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
 
 
 COMMANDS
 --------
+add::
+	Add the given repository as a submodule at the given path
+	to the changeset to be committed next.  In particular, the
+	repository is cloned at the specified path, added to the
+	changeset and registered in .gitmodules.   If no path is
+	specified, the path is deduced from the repository specification.
+
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
diff --git a/git-submodule.sh b/git-submodule.sh
index 89a3885..3df7121 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1,13 +1,14 @@
 #!/bin/sh
 #
-# git-submodules.sh: init, update or list git submodules
+# git-submodules.sh: add, init, update or list git submodules
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'
 . git-sh-setup
 require_work_tree
 
+add=
 init=
 update=
 status=
@@ -25,6 +26,17 @@ say()
 	fi
 }
 
+get_repo_base() {
+	(
+		cd "`/bin/pwd`" &&
+		cd "$1" || cd "$1.git" &&
+		{
+			cd .git
+			pwd
+		}
+	) 2>/dev/null
+}
+
 #
 # Map submodule path to submodule name
 #
@@ -66,6 +78,44 @@ module_clone()
 }
 
 #
+# Add a new submodule to the working tree, .gitmodules and the index
+#
+# $@ = repo [path]
+#
+module_add()
+{
+	repo=$1
+	path=$2
+
+	# Turn the source into an absolute path if
+	# it is local
+	if base=$(get_repo_base "$repo"); then
+		repo="$base"
+	fi
+
+	# Guess path from repo if not specified or strip trailing slashes
+	if test -z "$path"; then
+		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
+	else
+		path=$(echo "$path" | sed -e 's|/*$||')
+	fi
+
+	test -e "$path" &&
+	die "'$path' already exists"
+
+	module_clone "$path" "$repo" || exit
+	(unset GIT_DIR && cd "$path" && git checkout -q) ||
+	die "Unable to checkout submodule '$path'"
+	git add "$path" ||
+	die "Failed to add submodule '$path'"
+
+	GIT_CONFIG=.gitmodules git config submodule."$path".path "$path" &&
+	GIT_CONFIG=.gitmodules git config submodule."$path".url "$repo" &&
+	git add .gitmodules ||
+	die "Failed to register submodule '$path'"
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -173,6 +223,9 @@ modules_list()
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
+	add)
+		add=1
+		;;
 	init)
 		init=1
 		;;
@@ -201,14 +254,17 @@ do
 	shift
 done
 
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$add,$init,$update,$status,$cached" in
+1,,,,)
+	module_add "$@"
+	;;
+,1,,,)
 	modules_init "$@"
 	;;
-,1,,)
+,,1,,)
 	modules_update "$@"
 	;;
-,,*,*)
+,,,*,*)
 	modules_list "$@"
 	;;
 *)
-- 
1.5.2.2.250.g1cd90-dirty
Lars Hjemli· Jun 23, 2007, 19:26 UTC · re: Sven Verdoolaege · lore

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

On 6/21/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
> I didn't get any reaction (negative or positive) to this patch.

Sorry for the late reply, I've been buried in day-job stuff lately (and still is).

The patch looks sane to me. I'll try to play around with it tomorrow and give some more feedback.

-- larsh

Junio C Hamano· Jun 23, 2007, 20:15 UTC · re: Lars Hjemli · lore

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

"Lars Hjemli" <hjemli@gmail.com> writes:
Show 8 quoted lines
> On 6/21/07, Sven Verdoolaege <skimo@kotnet.org> wrote:
>> I didn't get any reaction (negative or positive) to this patch.
>
> Sorry for the late reply, I've been buried in day-job stuff lately
> (and still is).
>
> The patch looks sane to me. I'll try to play around with it tomorrow
> and give some more feedback.
Thanks.
Junio C Hamano· Jun 23, 2007, 19:58 UTC · re: Sven Verdoolaege · lore

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

Sven Verdoolaege <skimo@kotnet.org> writes:
Show 9 quoted lines
>  COMMANDS
>  --------
> +add::
> +	Add the given repository as a submodule at the given path
> +	to the changeset to be committed next.  In particular, the
> +	repository is cloned at the specified path, added to the
> +	changeset and registered in .gitmodules.   If no path is
> +	specified, the path is deduced from the repository specification.
> +

Somehow "git submodule add $URL $my_subdirectory" feels unnatural, although it certainly is simpler to write the command usage string. Wouldn't a commit on the maintenance branch of cgit.git want to say "Add the 'maint' branch of git.git as my submodule", for example?

The alternatives I can come up with do not feel right either, though.
	git submodule $my_subdirectory $URL [$branch]
	git submodule $URL [--branch $branch] $my_subdirectory
Hmmm...
Show 14 quoted lines
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 89a3885..3df7121 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -1,13 +1,14 @@
>  #!/bin/sh
>  #
> -# git-submodules.sh: init, update or list git submodules
> +# git-submodules.sh: add, init, update or list git submodules
>  #
>  # Copyright (c) 2007 Lars Hjemli
>  
> -USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
> +USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'

Can a single repo added at more than one path with this syntax? I do not see that in the code, but this implies such.

Show 22 quoted lines
>  . git-sh-setup
>  require_work_tree
>  
> +add=
>  init=
>  update=
>  status=
> @@ -25,6 +26,17 @@ say()
>  	fi
>  }
>  
> +get_repo_base() {
> +	(
> +		cd "`/bin/pwd`" &&
> +		cd "$1" || cd "$1.git" &&
> +		{
> +			cd .git
> +			pwd
> +		}
> +	) 2>/dev/null
> +}
> +

I've seen this code before elsewhere. We do not need to refactor right now with this patch, but please mark this copy with something like:

	# NEEDSWORK: identical function exists in get_repo_base
        # in clone.sh
	get_repo_base () {
        	...
as a reminder.
Show 30 quoted lines
> @@ -66,6 +78,44 @@ module_clone()
>  }
>  
>  #
> +# Add a new submodule to the working tree, .gitmodules and the index
> +#
> +# $@ = repo [path]
> +#
> +module_add()
> +{
> +	repo=$1
> +	path=$2
> +
> +	# Turn the source into an absolute path if
> +	# it is local
> +	if base=$(get_repo_base "$repo"); then
> +		repo="$base"
> +	fi
> +
> +	# Guess path from repo if not specified or strip trailing slashes
> +	if test -z "$path"; then
> +		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
> +	else
> +		path=$(echo "$path" | sed -e 's|/*$||')
> +	fi
> +
> +	test -e "$path" &&
> +	die "'$path' already exists"
> +
> +	module_clone "$path" "$repo" || exit
 - module_clone catches the "$path already exists" case; but the
   test is done differently.  One particular case of "an empty
   directory exists" is allowed there, but you are dying early
   to forbid it.  Is that warranted?  My gut feeling is that
   they should share the same check, iow, don't check yourself
   but have module_clone take care of the error case.
 - If $path does not exist in the worktree (because it hasn't
   been checked out), but does exist in the index, what should
   happen?  Should it be flagged as an error (in module_clone,
   not here)?
Sven Verdoolaege· Jun 24, 2007, 15:17 UTC · re: Junio C Hamano · lore

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

On Sat, Jun 23, 2007 at 12:58:03PM -0700, Junio C Hamano wrote:
> Somehow "git submodule add $URL $my_subdirectory" feels
> unnatural, although it certainly is simpler to write the command

The order of the arguments is the same as those of git-clone and you can read it as "git: add submodule $URL (at) $my_subdirectory"

> usage string.  Wouldn't a commit on the maintenance branch of
> cgit.git want to say "Add the 'maint' branch of git.git as my
> submodule", for example?
Sounds plausible
> The alternatives I can come up with do not feel right either, though.
> 
> 	git submodule $my_subdirectory $URL [$branch]
> 	git submodule $URL [--branch $branch] $my_subdirectory

I'll add the latter, although it's not clear if you actually want me to.

Show 16 quoted lines
> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index 89a3885..3df7121 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -1,13 +1,14 @@
> >  #!/bin/sh
> >  #
> > -# git-submodules.sh: init, update or list git submodules
> > +# git-submodules.sh: add, init, update or list git submodules
> >  #
> >  # Copyright (c) 2007 Lars Hjemli
> >  
> > -USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
> > +USAGE='[--quiet] [--cached] [add <repo>|status|init|update] [--] [<path>...]'
> 
> Can a single repo added at more than one path with this syntax?

No. I was trying to be brief. The more correct syntax would be the one in the documentation, but I thought that would be a bit lengthy for USAGE.

Show 8 quoted lines
> I've seen this code before elsewhere.  We do not need to
> refactor right now with this patch, but please mark this copy
> with something like:
> 
> 	# NEEDSWORK: identical function exists in get_repo_base
>         # in clone.sh
> 	get_repo_base () {
>         	...
OK
Show 11 quoted lines
> > +	test -e "$path" &&
> > +	die "'$path' already exists"
> > +
> > +	module_clone "$path" "$repo" || exit
> 
>  - module_clone catches the "$path already exists" case; but the
>    test is done differently.  One particular case of "an empty
>    directory exists" is allowed there, but you are dying early
>    to forbid it.  Is that warranted?  My gut feeling is that
>    they should share the same check, iow, don't check yourself
>    but have module_clone take care of the error case.

They're different because submodule update (which also calls module_clone) is performed on a module that already exists in the repo and was therefore checked out by git as an empty directory. If you add a new submodule, then there is no reason for the subdirectory to exist already.

>  - If $path does not exist in the worktree (because it hasn't
>    been checked out), but does exist in the index, what should
>    happen?  Should it be flagged as an error (in module_clone,
>    not here)?

Good question. It should fail. However, I think this new check does belong here, because when module_clone is called from modules_update, the path _should_ exist in the index.

skimo
Junio C Hamano· Jun 24, 2007, 17:50 UTC · re: Sven Verdoolaege · lore

Re: [PATCH][RESEND] git-submodule: provide easy way of adding new submodules

Sven Verdoolaege <skimo@kotnet.org> writes:
Show 5 quoted lines
> They're different because submodule update (which also calls module_clone)
> is performed on a module that already exists in the repo and
> was therefore checked out by git as an empty directory.  If you
> add a new submodule, then there is no reason for the subdirectory
> to exist already.
Ah, you are absolutely right.  Thanks for the clarification.

By the way, when I have to say "thanks for the clarification", I think it is an indication that the relevant parts of the code need a bit of commenting ;-).

Sven Verdoolaege· Jun 24, 2007, 21:06 UTC · re: Junio C Hamano · lore

[PATCH] git-submodule: provide easy way of adding new submodules

To make a submodule effectively usable, the path and a URL where the submodule can be cloned need to be stored in .gitmodules. This subcommand takes care of setting this information after cloning the new submodule. Only the index is updated, so, if needed, the user may still change the URL or switch to a different branch of the submodule before committing.

Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
On Sun, Jun 24, 2007 at 10:50:48AM -0700, Junio C Hamano wrote:
Show 13 quoted lines
> Sven Verdoolaege <skimo@kotnet.org> writes:
> 
> > They're different because submodule update (which also calls module_clone)
> > is performed on a module that already exists in the repo and
> > was therefore checked out by git as an empty directory.  If you
> > add a new submodule, then there is no reason for the subdirectory
> > to exist already.
> 
> Ah, you are absolutely right.  Thanks for the clarification.
> 
> By the way, when I have to say "thanks for the clarification", I
> think it is an indication that the relevant parts of the code
> need a bit of commenting ;-).
How's this ?
skimo
 Documentation/git-submodule.txt |   11 ++++
 git-submodule.sh                |  102 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 107 insertions(+), 6 deletions(-)
Show changes to 2 files +107 −6

Documentation/git-submodule.txt, git-submodule.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index f8fb80f..7f0904e 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -8,11 +8,19 @@ git-submodule - Initialize, update or inspect submodules
 
 SYNOPSIS
 --------
+'git-submodule' [--quiet] [-b branch] add <repository> [<path>]
 'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
 
 
 COMMANDS
 --------
+add::
+	Add the given repository as a submodule at the given path
+	to the changeset to be committed next.  In particular, the
+	repository is cloned at the specified path, added to the
+	changeset and registered in .gitmodules.   If no path is
+	specified, the path is deduced from the repository specification.
+
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
@@ -39,6 +47,9 @@ OPTIONS
 -q, --quiet::
 	Only print error messages.
 
+-b, --branch::
+	Branch of repository to add as submodule.
+
 --cached::
 	Display the SHA-1 stored in the index, not the SHA-1 of the currently
 	checked out submodule commit. This option is only valid for the
diff --git a/git-submodule.sh b/git-submodule.sh
index 89a3885..67c78f2 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -1,13 +1,15 @@
 #!/bin/sh
 #
-# git-submodules.sh: init, update or list git submodules
+# git-submodules.sh: add, init, update or list git submodules
 #
 # Copyright (c) 2007 Lars Hjemli
 
-USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+USAGE='[--quiet] [--cached] [add <repo> [-b branch]|status|init|update] [--] [<path>...]'
 . git-sh-setup
 require_work_tree
 
+add=
+branch=
 init=
 update=
 status=
@@ -25,6 +27,18 @@ say()
 	fi
 }
 
+# NEEDSWORK: identical function exists in get_repo_base in clone.sh
+get_repo_base() {
+	(
+		cd "`/bin/pwd`" &&
+		cd "$1" || cd "$1.git" &&
+		{
+			cd .git
+			pwd
+		}
+	) 2>/dev/null
+}
+
 #
 # Map submodule path to submodule name
 #
@@ -42,6 +56,11 @@ module_name()
 #
 # Clone a submodule
 #
+# Prior to calling, modules_update checks that a possibly existing
+# path is not a git repository.
+# Likewise, module_add checks that path does not exist at all,
+# since it is the location of a new submodule.
+#
 module_clone()
 {
 	path=$1
@@ -66,6 +85,53 @@ module_clone()
 }
 
 #
+# Add a new submodule to the working tree, .gitmodules and the index
+#
+# $@ = repo [path]
+#
+# optional branch is stored in global branch variable
+#
+module_add()
+{
+	repo=$1
+	path=$2
+
+	if test -z "$repo"; then
+		usage
+	fi
+
+	# Turn the source into an absolute path if
+	# it is local
+	if base=$(get_repo_base "$repo"); then
+		repo="$base"
+	fi
+
+	# Guess path from repo if not specified or strip trailing slashes
+	if test -z "$path"; then
+		path=$(echo "$repo" | sed -e 's|/*$||' -e 's|:*/*\.git$||' -e 's|.*[/:]||g')
+	else
+		path=$(echo "$path" | sed -e 's|/*$||')
+	fi
+
+	test -e "$path" &&
+	die "'$path' already exists"
+
+	git-ls-files --error-unmatch "$path" > /dev/null 2>&1 &&
+	die "'$path' already exists in the index"
+
+	module_clone "$path" "$repo" || exit
+	(unset GIT_DIR && cd "$path" && git checkout -q ${branch:+-b "$branch" "origin/$branch"}) ||
+	die "Unable to checkout submodule '$path'"
+	git add "$path" ||
+	die "Failed to add submodule '$path'"
+
+	GIT_CONFIG=.gitmodules git config submodule."$path".path "$path" &&
+	GIT_CONFIG=.gitmodules git config submodule."$path".url "$repo" &&
+	git add .gitmodules ||
+	die "Failed to register submodule '$path'"
+}
+
+#
 # Register submodules in .git/config
 #
 # $@ = requested paths (default to all)
@@ -173,6 +239,9 @@ modules_list()
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
+	add)
+		add=1
+		;;
 	init)
 		init=1
 		;;
@@ -185,6 +254,14 @@ do
 	-q|--quiet)
 		quiet=1
 		;;
+	-b|--branch)
+		case "$2" in
+		'')
+			usage
+			;;
+		esac
+		branch="$2"; shift
+		;;
 	--cached)
 		cached=1
 		;;
@@ -201,14 +278,27 @@ do
 	shift
 done
 
-case "$init,$update,$status,$cached" in
-1,,,)
+case "$add,$branch" in
+1,*)
+	;;
+,)
+	;;
+,*)
+	usage
+	;;
+esac
+
+case "$add,$init,$update,$status,$cached" in
+1,,,,)
+	module_add "$@"
+	;;
+,1,,,)
 	modules_init "$@"
 	;;
-,1,,)
+,,1,,)
 	modules_update "$@"
 	;;
-,,*,*)
+,,,1,*)
 	modules_list "$@"
 	;;
 *)
-- 
1.5.2.2.1064.g59b4
Junio C Hamano· Jun 25, 2007, 07:09 UTC · re: Sven Verdoolaege · lore

Re: [PATCH] git-submodule: provide easy way of adding new submodules

Sven Verdoolaege <skimo@kotnet.org> writes:
Show 26 quoted lines
> To make a submodule effectively usable, the path and
> a URL where the submodule can be cloned need to be stored
> in .gitmodules.  This subcommand takes care of setting
> this information after cloning the new submodule.
> Only the index is updated, so, if needed, the user may still
> change the URL or switch to a different branch of the submodule
> before committing.
>
> Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
> ---
> On Sun, Jun 24, 2007 at 10:50:48AM -0700, Junio C Hamano wrote:
>> Sven Verdoolaege <skimo@kotnet.org> writes:
>> 
>> > They're different because submodule update (which also calls module_clone)
>> > is performed on a module that already exists in the repo and
>> > was therefore checked out by git as an empty directory.  If you
>> > add a new submodule, then there is no reason for the subdirectory
>> > to exist already.
>> 
>> Ah, you are absolutely right.  Thanks for the clarification.
>> 
>> By the way, when I have to say "thanks for the clarification", I
>> think it is an indication that the relevant parts of the code
>> need a bit of commenting ;-).
>
> How's this ?

I am not sure about the usefulness of "-b branch" thing myself, but other than that, looks very sane to me.

Lars?
Sven Verdoolaege· Jul 3, 2007, 15:47 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-submodule: provide easy way of adding new submodules

On Mon, Jun 25, 2007 at 12:09:09AM -0700, Junio C Hamano wrote:
> I am not sure about the usefulness of "-b branch" thing myself,
> but other than that, looks very sane to me.
So, do you want me to send it in again, without the branch thing?
skimo
Junio C Hamano· Jul 4, 2007, 05:10 UTC · re: Sven Verdoolaege · lore

Re: [PATCH] git-submodule: provide easy way of adding new submodules

Sven Verdoolaege <skimo@kotnet.org> writes:
Show 5 quoted lines
> On Mon, Jun 25, 2007 at 12:09:09AM -0700, Junio C Hamano wrote:
>> I am not sure about the usefulness of "-b branch" thing myself,
>> but other than that, looks very sane to me.
>
> So, do you want me to send it in again, without the branch thing?
It is already scheduled for 1.5.3 and is part of 'master'.

← back to recent threads