threads / patch / 36038

patchsubmodule : Add --no-separate-git-dir option to add and update command.

Subject: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

## tl;dr

16 messages between Mar 3, 2014 and Mar 11, 2014. Diffs are folded; open one to read it.

replies: 15people: 4as markdown or json

Henri GEIST· Mar 3, 2014, 14:47 UTC · lore

This new option prevent git submodule <add|update> to clone the missing submodules with the --separate-git-dir option. Then the submodule will be regular repository and their gitdir will not be placed in the superproject gitdir/modules directory.

Signed-off-by: Henri GEIST <geist.henri@laposte.net>
---
 Documentation/git-submodule.txt |   18 ++++++++++++++++--
 git-submodule.sh                |   22 ++++++++++++++++++++--
 t/t7400-submodule-basic.sh      |   12 ++++++++++++
 3 files changed, 48 insertions(+), 4 deletions(-)
Show changes to 3 files +48 −4

Documentation/git-submodule.txt, git-submodule.sh, t/t7400-submodule-basic.sh

diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
index 21cb59a..303a475 100644
--- a/Documentation/git-submodule.txt
+++ b/Documentation/git-submodule.txt
@@ -10,13 +10,14 @@ SYNOPSIS
 --------
 [verse]
 'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
-	      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
+	      [--reference <repository>] [--depth <depth>] [--no-separate-git-dir]
+	      [--] <repository> [<path>]
 'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
 'git submodule' [--quiet] init [--] [<path>...]
 'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
 'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
 	      [-f|--force] [--rebase|--merge|--checkout] [--reference <repository>]
-	      [--depth <depth>] [--recursive] [--] [<path>...]
+	      [--depth <depth>] [--recursive] [--no-separate-git-dir] [--] [<path>...]
 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
 	      [commit] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
@@ -107,6 +108,10 @@ is the superproject and submodule repositories will be kept
 together in the same relative location, and only the
 superproject's URL needs to be provided: git-submodule will correctly
 locate the submodule using the relative URL in .gitmodules.
++
+If `--no-separate-git-dir` is specified, missing submodules will be cloned
+has normal git repository without the option `--separate-git-dir` pointing
+to the modules directory of the superproject gitdir.
 
 status::
 	Show the status of the submodules. This will print the SHA-1 of the
@@ -185,6 +190,10 @@ If the submodule is not yet initialized, and you just want to use the
 setting as stored in .gitmodules, you can automatically initialize the
 submodule with the `--init` option.
 +
+If `--no-separate-git-dir` is specified, missing submodules will be cloned
+has normal git repository without the option `--separate-git-dir` pointing
+to the modules directory of the superproject gitdir.
++
 If `--recursive` is specified, this command will recurse into the
 registered submodules, and update any nested submodules within.
 +
@@ -363,6 +372,11 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` options carefully.
 	clone with a history truncated to the specified number of revisions.
 	See linkgit:git-clone[1]
 
+--no-separate-git-dir::
+	This option is valid for add and update commands. Specify that missing
+	submodules should be clonned as self contain repository without a
+	separate gitdir placed in the modules directory of the superproject
+	gitdir.
 
 <path>...::
 	Paths to submodule(s). When specified this will restrict the command
diff --git a/git-submodule.sh b/git-submodule.sh
index a33f68d..36eaf31 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -5,11 +5,11 @@
 # Copyright (c) 2007 Lars Hjemli
 
 dashless=$(basename "$0" | sed -e 's/-/ /')
-USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
+USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--no-separate-git-dir] [--] <repository> [<path>]
    or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
    or: $dashless [--quiet] init [--] [<path>...]
    or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
-   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
+   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--no-separate-git-dir] [--] [<path>...]
    or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
    or: $dashless [--quiet] foreach [--recursive] <command>
    or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
@@ -36,6 +36,7 @@ update=
 prefix=
 custom_name=
 depth=
+noseparategitdir=
 
 # The function takes at most 2 arguments. The first argument is the
 # URL that navigates to the submodule origin repo. When relative, this URL
@@ -270,6 +271,17 @@ module_clone()
 		quiet=-q
 	fi
 
+
+	if test -n "$noseparategitdir"
+	then
+		(
+			clear_local_git_env
+			git clone $quiet ${depth:+"$depth"} -n ${reference:+"$reference"} "$url" "$sm_path"
+		) ||
+		die "$(eval_gettext "Clone of '\$url' into submodule path '\$sm_path' failed")"
+		return
+	fi
+
 	gitdir=
 	gitdir_base=
 	base_name=$(dirname "$name")
@@ -359,6 +371,9 @@ cmd_add()
 		-q|--quiet)
 			GIT_QUIET=1
 			;;
+		--no-separate-git-dir)
+			noseparategitdir=1
+			;;
 		--reference)
 			case "$2" in '') usage ;; esac
 			reference_path=$2
@@ -758,6 +773,9 @@ cmd_update()
 		-f|--force)
 			force=$1
 			;;
+		--no-separate-git-dir)
+			noseparategitdir=1
+			;;
 		-r|--rebase)
 			update="rebase"
 			;;
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index c28e8d8..aa2df3d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -81,6 +81,18 @@ inspect() {
 	)
 }
 
+test_expect_success 'submodule add --no-separate-git-dir' '
+	(
+		cd addtest &&
+		rm -rf submod &&
+		git submodule add --no-separate-git-dir -q "$submodurl" submod >actual &&
+		test_must_be_empty actual &&
+		test -d submod/.git &&
+		rm -rf submod &&
+		git reset --hard
+	)
+'
+
 test_expect_success 'submodule add' '
 	echo "refs/heads/master" >expect &&
 	>empty &&
-- 
1.7.9.3.369.gd715.dirty
Jens Lehmann· Mar 3, 2014, 17:45 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Am 03.03.2014 14:47, schrieb Henri GEIST:
> This new option prevent git submodule <add|update> to clone the missing
> submodules with the --separate-git-dir option.
> Then the submodule will be regular repository and their gitdir will not
> be placed in the superproject gitdir/modules directory.

And what is your motivation for this? After all submodules containing a .git directory are second class citizens (because they can never be safely removed by regular git commands).

Show 150 quoted lines
> Signed-off-by: Henri GEIST <geist.henri@laposte.net>
> ---
>  Documentation/git-submodule.txt |   18 ++++++++++++++++--
>  git-submodule.sh                |   22 ++++++++++++++++++++--
>  t/t7400-submodule-basic.sh      |   12 ++++++++++++
>  3 files changed, 48 insertions(+), 4 deletions(-)
> 
> diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> index 21cb59a..303a475 100644
> --- a/Documentation/git-submodule.txt
> +++ b/Documentation/git-submodule.txt
> @@ -10,13 +10,14 @@ SYNOPSIS
>  --------
>  [verse]
>  'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
> -	      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
> +	      [--reference <repository>] [--depth <depth>] [--no-separate-git-dir]
> +	      [--] <repository> [<path>]
>  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
>  'git submodule' [--quiet] init [--] [<path>...]
>  'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
>  'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
>  	      [-f|--force] [--rebase|--merge|--checkout] [--reference <repository>]
> -	      [--depth <depth>] [--recursive] [--] [<path>...]
> +	      [--depth <depth>] [--recursive] [--no-separate-git-dir] [--] [<path>...]
>  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
>  	      [commit] [--] [<path>...]
>  'git submodule' [--quiet] foreach [--recursive] <command>
> @@ -107,6 +108,10 @@ is the superproject and submodule repositories will be kept
>  together in the same relative location, and only the
>  superproject's URL needs to be provided: git-submodule will correctly
>  locate the submodule using the relative URL in .gitmodules.
> ++
> +If `--no-separate-git-dir` is specified, missing submodules will be cloned
> +has normal git repository without the option `--separate-git-dir` pointing
> +to the modules directory of the superproject gitdir.
>  
>  status::
>  	Show the status of the submodules. This will print the SHA-1 of the
> @@ -185,6 +190,10 @@ If the submodule is not yet initialized, and you just want to use the
>  setting as stored in .gitmodules, you can automatically initialize the
>  submodule with the `--init` option.
>  +
> +If `--no-separate-git-dir` is specified, missing submodules will be cloned
> +has normal git repository without the option `--separate-git-dir` pointing
> +to the modules directory of the superproject gitdir.
> ++
>  If `--recursive` is specified, this command will recurse into the
>  registered submodules, and update any nested submodules within.
>  +
> @@ -363,6 +372,11 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` options carefully.
>  	clone with a history truncated to the specified number of revisions.
>  	See linkgit:git-clone[1]
>  
> +--no-separate-git-dir::
> +	This option is valid for add and update commands. Specify that missing
> +	submodules should be clonned as self contain repository without a
> +	separate gitdir placed in the modules directory of the superproject
> +	gitdir.
>  
>  <path>...::
>  	Paths to submodule(s). When specified this will restrict the command
> diff --git a/git-submodule.sh b/git-submodule.sh
> index a33f68d..36eaf31 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -5,11 +5,11 @@
>  # Copyright (c) 2007 Lars Hjemli
>  
>  dashless=$(basename "$0" | sed -e 's/-/ /')
> -USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
> +USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--no-separate-git-dir] [--] <repository> [<path>]
>     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
>     or: $dashless [--quiet] init [--] [<path>...]
>     or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
> -   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> +   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--no-separate-git-dir] [--] [<path>...]
>     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
>     or: $dashless [--quiet] foreach [--recursive] <command>
>     or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
> @@ -36,6 +36,7 @@ update=
>  prefix=
>  custom_name=
>  depth=
> +noseparategitdir=
>  
>  # The function takes at most 2 arguments. The first argument is the
>  # URL that navigates to the submodule origin repo. When relative, this URL
> @@ -270,6 +271,17 @@ module_clone()
>  		quiet=-q
>  	fi
>  
> +
> +	if test -n "$noseparategitdir"
> +	then
> +		(
> +			clear_local_git_env
> +			git clone $quiet ${depth:+"$depth"} -n ${reference:+"$reference"} "$url" "$sm_path"
> +		) ||
> +		die "$(eval_gettext "Clone of '\$url' into submodule path '\$sm_path' failed")"
> +		return
> +	fi
> +
>  	gitdir=
>  	gitdir_base=
>  	base_name=$(dirname "$name")
> @@ -359,6 +371,9 @@ cmd_add()
>  		-q|--quiet)
>  			GIT_QUIET=1
>  			;;
> +		--no-separate-git-dir)
> +			noseparategitdir=1
> +			;;
>  		--reference)
>  			case "$2" in '') usage ;; esac
>  			reference_path=$2
> @@ -758,6 +773,9 @@ cmd_update()
>  		-f|--force)
>  			force=$1
>  			;;
> +		--no-separate-git-dir)
> +			noseparategitdir=1
> +			;;
>  		-r|--rebase)
>  			update="rebase"
>  			;;
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index c28e8d8..aa2df3d 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -81,6 +81,18 @@ inspect() {
>  	)
>  }
>  
> +test_expect_success 'submodule add --no-separate-git-dir' '
> +	(
> +		cd addtest &&
> +		rm -rf submod &&
> +		git submodule add --no-separate-git-dir -q "$submodurl" submod >actual &&
> +		test_must_be_empty actual &&
> +		test -d submod/.git &&
> +		rm -rf submod &&
> +		git reset --hard
> +	)
> +'
> +
>  test_expect_success 'submodule add' '
>  	echo "refs/heads/master" >expect &&
>  	>empty &&
> 
Henri GEIST· Mar 3, 2014, 20:34 UTC · re: Jens Lehmann · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le lundi 03 mars 2014 à 17:45 +0000, Jens Lehmann a écrit :
Show 10 quoted lines
> Am 03.03.2014 14:47, schrieb Henri GEIST:
> > This new option prevent git submodule <add|update> to clone the missing
> > submodules with the --separate-git-dir option.
> > Then the submodule will be regular repository and their gitdir will not
> > be placed in the superproject gitdir/modules directory.
> 
> And what is your motivation for this? After all submodules containing
> a .git directory are second class citizens (because they can never be
> safely removed by regular git commands).
>

I recognize most people will prefer to have the .git directory separate. And I do not intend to make this option the default.

My reasons are:
  - As it is not clearly stated in the doc that the gitdir is separate.
    The first time I have copied one module to an USB key I had a big
    surprise.
  - This will not change anything for people not using it.
  - I use an other patch which I plane to send later which enable multiple
    level of superproject to add a gitlink to the same submodule.
    And in this case the superproject containing the separate gitdir will be
    arbitrary and depend on the processing order of the
    'git submodule update --recursive' command.
  - I have written this for myself and have using it since 2012 and send it in
    the hope it could be useful for someone else even if it is only a few
    people. But if its not the case no problem I will keep using it for myself.
Show 151 quoted lines
> > Signed-off-by: Henri GEIST <geist.henri@laposte.net>
> > ---
> >  Documentation/git-submodule.txt |   18 ++++++++++++++++--
> >  git-submodule.sh                |   22 ++++++++++++++++++++--
> >  t/t7400-submodule-basic.sh      |   12 ++++++++++++
> >  3 files changed, 48 insertions(+), 4 deletions(-)
> > 
> > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
> > index 21cb59a..303a475 100644
> > --- a/Documentation/git-submodule.txt
> > +++ b/Documentation/git-submodule.txt
> > @@ -10,13 +10,14 @@ SYNOPSIS
> >  --------
> >  [verse]
> >  'git submodule' [--quiet] add [-b <branch>] [-f|--force] [--name <name>]
> > -	      [--reference <repository>] [--depth <depth>] [--] <repository> [<path>]
> > +	      [--reference <repository>] [--depth <depth>] [--no-separate-git-dir]
> > +	      [--] <repository> [<path>]
> >  'git submodule' [--quiet] status [--cached] [--recursive] [--] [<path>...]
> >  'git submodule' [--quiet] init [--] [<path>...]
> >  'git submodule' [--quiet] deinit [-f|--force] [--] <path>...
> >  'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch]
> >  	      [-f|--force] [--rebase|--merge|--checkout] [--reference <repository>]
> > -	      [--depth <depth>] [--recursive] [--] [<path>...]
> > +	      [--depth <depth>] [--recursive] [--no-separate-git-dir] [--] [<path>...]
> >  'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) <n>]
> >  	      [commit] [--] [<path>...]
> >  'git submodule' [--quiet] foreach [--recursive] <command>
> > @@ -107,6 +108,10 @@ is the superproject and submodule repositories will be kept
> >  together in the same relative location, and only the
> >  superproject's URL needs to be provided: git-submodule will correctly
> >  locate the submodule using the relative URL in .gitmodules.
> > ++
> > +If `--no-separate-git-dir` is specified, missing submodules will be cloned
> > +has normal git repository without the option `--separate-git-dir` pointing
> > +to the modules directory of the superproject gitdir.
> >  
> >  status::
> >  	Show the status of the submodules. This will print the SHA-1 of the
> > @@ -185,6 +190,10 @@ If the submodule is not yet initialized, and you just want to use the
> >  setting as stored in .gitmodules, you can automatically initialize the
> >  submodule with the `--init` option.
> >  +
> > +If `--no-separate-git-dir` is specified, missing submodules will be cloned
> > +has normal git repository without the option `--separate-git-dir` pointing
> > +to the modules directory of the superproject gitdir.
> > ++
> >  If `--recursive` is specified, this command will recurse into the
> >  registered submodules, and update any nested submodules within.
> >  +
> > @@ -363,6 +372,11 @@ for linkgit:git-clone[1]'s `--reference` and `--shared` options carefully.
> >  	clone with a history truncated to the specified number of revisions.
> >  	See linkgit:git-clone[1]
> >  
> > +--no-separate-git-dir::
> > +	This option is valid for add and update commands. Specify that missing
> > +	submodules should be clonned as self contain repository without a
> > +	separate gitdir placed in the modules directory of the superproject
> > +	gitdir.
> >  
> >  <path>...::
> >  	Paths to submodule(s). When specified this will restrict the command
> > diff --git a/git-submodule.sh b/git-submodule.sh
> > index a33f68d..36eaf31 100755
> > --- a/git-submodule.sh
> > +++ b/git-submodule.sh
> > @@ -5,11 +5,11 @@
> >  # Copyright (c) 2007 Lars Hjemli
> >  
> >  dashless=$(basename "$0" | sed -e 's/-/ /')
> > -USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
> > +USAGE="[--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--no-separate-git-dir] [--] <repository> [<path>]
> >     or: $dashless [--quiet] status [--cached] [--recursive] [--] [<path>...]
> >     or: $dashless [--quiet] init [--] [<path>...]
> >     or: $dashless [--quiet] deinit [-f|--force] [--] <path>...
> > -   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--] [<path>...]
> > +   or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference <repository>] [--merge] [--recursive] [--no-separate-git-dir] [--] [<path>...]
> >     or: $dashless [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
> >     or: $dashless [--quiet] foreach [--recursive] <command>
> >     or: $dashless [--quiet] sync [--recursive] [--] [<path>...]"
> > @@ -36,6 +36,7 @@ update=
> >  prefix=
> >  custom_name=
> >  depth=
> > +noseparategitdir=
> >  
> >  # The function takes at most 2 arguments. The first argument is the
> >  # URL that navigates to the submodule origin repo. When relative, this URL
> > @@ -270,6 +271,17 @@ module_clone()
> >  		quiet=-q
> >  	fi
> >  
> > +
> > +	if test -n "$noseparategitdir"
> > +	then
> > +		(
> > +			clear_local_git_env
> > +			git clone $quiet ${depth:+"$depth"} -n ${reference:+"$reference"} "$url" "$sm_path"
> > +		) ||
> > +		die "$(eval_gettext "Clone of '\$url' into submodule path '\$sm_path' failed")"
> > +		return
> > +	fi
> > +
> >  	gitdir=
> >  	gitdir_base=
> >  	base_name=$(dirname "$name")
> > @@ -359,6 +371,9 @@ cmd_add()
> >  		-q|--quiet)
> >  			GIT_QUIET=1
> >  			;;
> > +		--no-separate-git-dir)
> > +			noseparategitdir=1
> > +			;;
> >  		--reference)
> >  			case "$2" in '') usage ;; esac
> >  			reference_path=$2
> > @@ -758,6 +773,9 @@ cmd_update()
> >  		-f|--force)
> >  			force=$1
> >  			;;
> > +		--no-separate-git-dir)
> > +			noseparategitdir=1
> > +			;;
> >  		-r|--rebase)
> >  			update="rebase"
> >  			;;
> > diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> > index c28e8d8..aa2df3d 100755
> > --- a/t/t7400-submodule-basic.sh
> > +++ b/t/t7400-submodule-basic.sh
> > @@ -81,6 +81,18 @@ inspect() {
> >  	)
> >  }
> >  
> > +test_expect_success 'submodule add --no-separate-git-dir' '
> > +	(
> > +		cd addtest &&
> > +		rm -rf submod &&
> > +		git submodule add --no-separate-git-dir -q "$submodurl" submod >actual &&
> > +		test_must_be_empty actual &&
> > +		test -d submod/.git &&
> > +		rm -rf submod &&
> > +		git reset --hard
> > +	)
> > +'
> > +
> >  test_expect_success 'submodule add' '
> >  	echo "refs/heads/master" >expect &&
> >  	>empty &&
> > 
> 
Jens Lehmann· Mar 5, 2014, 18:13 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Am 03.03.2014 21:34, schrieb Henri GEIST:
Show 20 quoted lines
> Le lundi 03 mars 2014 à 17:45 +0000, Jens Lehmann a écrit :
>> Am 03.03.2014 14:47, schrieb Henri GEIST:
>>> This new option prevent git submodule <add|update> to clone the missing
>>> submodules with the --separate-git-dir option.
>>> Then the submodule will be regular repository and their gitdir will not
>>> be placed in the superproject gitdir/modules directory.
>>
>> And what is your motivation for this? After all submodules containing
>> a .git directory are second class citizens (because they can never be
>> safely removed by regular git commands).
>>
> 
> I recognize most people will prefer to have the .git directory separate.
> And I do not intend to make this option the default.
> 
> My reasons are:
> 
>   - As it is not clearly stated in the doc that the gitdir is separate.
>     The first time I have copied one module to an USB key I had a big
>     surprise.

Oops! Could you please help us by hinting how the documentation could be improved here?

>   - This will not change anything for people not using it.

I do not agree, as they'll be seeing a new option and might use it to "go backward" as Junio explained in his answer.

Show 5 quoted lines
>   - I use an other patch which I plane to send later which enable multiple
>     level of superproject to add a gitlink to the same submodule.
>     And in this case the superproject containing the separate gitdir will be
>     arbitrary and depend on the processing order of the
>     'git submodule update --recursive' command.

I don't understand that. How is that different from using different names (and thus different separate gitdirs) for that duplicated submodule? After all, the .git directory is just moved somewhere else in the superproject's work tree, and as the name defaults to the path of the submodule ...

>   - I have written this for myself and have using it since 2012 and send it in
>     the hope it could be useful for someone else even if it is only a few
>     people. But if its not the case no problem I will keep using it for myself.
Thanks.
Henri GEIST· Mar 6, 2014, 01:25 UTC · re: Jens Lehmann · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le mercredi 05 mars 2014 à 19:13 +0100, Jens Lehmann a écrit :
Show 25 quoted lines
> Am 03.03.2014 21:34, schrieb Henri GEIST:
> > Le lundi 03 mars 2014 à 17:45 +0000, Jens Lehmann a écrit :
> >> Am 03.03.2014 14:47, schrieb Henri GEIST:
> >>> This new option prevent git submodule <add|update> to clone the missing
> >>> submodules with the --separate-git-dir option.
> >>> Then the submodule will be regular repository and their gitdir will not
> >>> be placed in the superproject gitdir/modules directory.
> >>
> >> And what is your motivation for this? After all submodules containing
> >> a .git directory are second class citizens (because they can never be
> >> safely removed by regular git commands).
> >>
> > 
> > I recognize most people will prefer to have the .git directory separate.
> > And I do not intend to make this option the default.
> > 
> > My reasons are:
> > 
> >   - As it is not clearly stated in the doc that the gitdir is separate.
> >     The first time I have copied one module to an USB key I had a big
> >     surprise.
> 
> Oops! Could you please help us by hinting how the documentation
> could be improved here?
> 

Of course. There is nothing in Documentation/git-submodule.txt to inform that submodules clones are different from regular clones. I will write and propose a patch for the documentation. But maybe in a new thread.

Show 17 quoted lines
> >   - This will not change anything for people not using it.
> 
> I do not agree, as they'll be seeing a new option and might use
> it to "go backward" as Junio explained in his answer.
> 
> >   - I use an other patch which I plane to send later which enable multiple
> >     level of superproject to add a gitlink to the same submodule.
> >     And in this case the superproject containing the separate gitdir will be
> >     arbitrary and depend on the processing order of the
> >     'git submodule update --recursive' command.
> 
> I don't understand that. How is that different from using different
> names (and thus different separate gitdirs) for that duplicated
> submodule? After all, the .git directory is just moved somewhere
> else in the superproject's work tree, and as the name defaults to
> the path of the submodule ...
> 

I think I should give an example. If I have a hierarchy like this :

superproject/submodule/subsubmodule
What I often do is:
superproject --> submodule --> subsubmodule
             |                 ^
             '-----------------'
Where '-->' is a gitlink.
That mean .gitmodules and index of the superproject contain both submodule and
submodule/subsubmodule.
And also mean (and that is the point) subsubmodule is a direct 'child' of both
superproject and submodule.
In this case where should the separate gitdir of subsubmodule be placed ?
  - In superproject/modules/submodule/subsubmodule ?
  - In superproject/submodule/modules/subsubmodule ?
  - Depending on the 'git submodule update' command order ?
  - Or both ?
Show 5 quoted lines
> >   - I have written this for myself and have using it since 2012 and send it in
> >     the hope it could be useful for someone else even if it is only a few
> >     people. But if its not the case no problem I will keep using it for myself.
> 
> Thanks.
Jens Lehmann· Mar 6, 2014, 19:48 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Am 06.03.2014 02:25, schrieb Henri GEIST:
Show 32 quoted lines
> Le mercredi 05 mars 2014 à 19:13 +0100, Jens Lehmann a écrit :
>> Am 03.03.2014 21:34, schrieb Henri GEIST:
>>> Le lundi 03 mars 2014 à 17:45 +0000, Jens Lehmann a écrit :
>>>> Am 03.03.2014 14:47, schrieb Henri GEIST:
>>>>> This new option prevent git submodule <add|update> to clone the missing
>>>>> submodules with the --separate-git-dir option.
>>>>> Then the submodule will be regular repository and their gitdir will not
>>>>> be placed in the superproject gitdir/modules directory.
>>>>
>>>> And what is your motivation for this? After all submodules containing
>>>> a .git directory are second class citizens (because they can never be
>>>> safely removed by regular git commands).
>>>>
>>>
>>> I recognize most people will prefer to have the .git directory separate.
>>> And I do not intend to make this option the default.
>>>
>>> My reasons are:
>>>
>>>   - As it is not clearly stated in the doc that the gitdir is separate.
>>>     The first time I have copied one module to an USB key I had a big
>>>     surprise.
>>
>> Oops! Could you please help us by hinting how the documentation
>> could be improved here?
>>
> 
> Of course.
> There is nothing in Documentation/git-submodule.txt to inform that submodules
> clones are different from regular clones.
> I will write and propose a patch for the documentation.
> But maybe in a new thread.
Thanks!
Show 34 quoted lines
>>>   - This will not change anything for people not using it.
>>
>> I do not agree, as they'll be seeing a new option and might use
>> it to "go backward" as Junio explained in his answer.
>>
>>>   - I use an other patch which I plane to send later which enable multiple
>>>     level of superproject to add a gitlink to the same submodule.
>>>     And in this case the superproject containing the separate gitdir will be
>>>     arbitrary and depend on the processing order of the
>>>     'git submodule update --recursive' command.
>>
>> I don't understand that. How is that different from using different
>> names (and thus different separate gitdirs) for that duplicated
>> submodule? After all, the .git directory is just moved somewhere
>> else in the superproject's work tree, and as the name defaults to
>> the path of the submodule ...
>>
> 
> I think I should give an example.
> If I have a hierarchy like this :
> 
> superproject/submodule/subsubmodule
> 
> What I often do is:
> 
> superproject --> submodule --> subsubmodule
>              |                 ^
>              '-----------------'
> 
> Where '-->' is a gitlink.
> 
> 
> That mean .gitmodules and index of the superproject contain both submodule and
> submodule/subsubmodule.

Wow, that shouldn't even work (as everything inside "submodule" shouldn't be part of the superproject but must be contained in the submodule itself). Do the "git submodule" script and other git commands like "git status" work for you in such setups?

> And also mean (and that is the point) subsubmodule is a direct 'child' of both
> superproject and submodule.

Which I think should not be possible. If that works with current Git I suspect we have a bug to fix ... or does your other patch make this work?

Show 5 quoted lines
> In this case where should the separate gitdir of subsubmodule be placed ?
>   - In superproject/modules/submodule/subsubmodule ?
>   - In superproject/submodule/modules/subsubmodule ?
>   - Depending on the 'git submodule update' command order ?
>   - Or both ?

It should be placed in .git/modules/submodule/modules/subsubmodule of the superproject (assuming the subsubmodule is part of the first level submodule). But in your example that would live in .git/modules/submodule/subsubmodule (but as mentioned above, I do not consider this a valid setup because then two repositories would be responsible for the data inside subsubmodule, which will lead to lots of trouble).

Henri GEIST· Mar 6, 2014, 20:15 UTC · re: Jens Lehmann · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le jeudi 06 mars 2014 à 20:48 +0100, Jens Lehmann a écrit :
Show 76 quoted lines
> Am 06.03.2014 02:25, schrieb Henri GEIST:
> > Le mercredi 05 mars 2014 à 19:13 +0100, Jens Lehmann a écrit :
> >> Am 03.03.2014 21:34, schrieb Henri GEIST:
> >>> Le lundi 03 mars 2014 à 17:45 +0000, Jens Lehmann a écrit :
> >>>> Am 03.03.2014 14:47, schrieb Henri GEIST:
> >>>>> This new option prevent git submodule <add|update> to clone the missing
> >>>>> submodules with the --separate-git-dir option.
> >>>>> Then the submodule will be regular repository and their gitdir will not
> >>>>> be placed in the superproject gitdir/modules directory.
> >>>>
> >>>> And what is your motivation for this? After all submodules containing
> >>>> a .git directory are second class citizens (because they can never be
> >>>> safely removed by regular git commands).
> >>>>
> >>>
> >>> I recognize most people will prefer to have the .git directory separate.
> >>> And I do not intend to make this option the default.
> >>>
> >>> My reasons are:
> >>>
> >>>   - As it is not clearly stated in the doc that the gitdir is separate.
> >>>     The first time I have copied one module to an USB key I had a big
> >>>     surprise.
> >>
> >> Oops! Could you please help us by hinting how the documentation
> >> could be improved here?
> >>
> > 
> > Of course.
> > There is nothing in Documentation/git-submodule.txt to inform that submodules
> > clones are different from regular clones.
> > I will write and propose a patch for the documentation.
> > But maybe in a new thread.
> 
> Thanks!
> 
> >>>   - This will not change anything for people not using it.
> >>
> >> I do not agree, as they'll be seeing a new option and might use
> >> it to "go backward" as Junio explained in his answer.
> >>
> >>>   - I use an other patch which I plane to send later which enable multiple
> >>>     level of superproject to add a gitlink to the same submodule.
> >>>     And in this case the superproject containing the separate gitdir will be
> >>>     arbitrary and depend on the processing order of the
> >>>     'git submodule update --recursive' command.
> >>
> >> I don't understand that. How is that different from using different
> >> names (and thus different separate gitdirs) for that duplicated
> >> submodule? After all, the .git directory is just moved somewhere
> >> else in the superproject's work tree, and as the name defaults to
> >> the path of the submodule ...
> >>
> > 
> > I think I should give an example.
> > If I have a hierarchy like this :
> > 
> > superproject/submodule/subsubmodule
> > 
> > What I often do is:
> > 
> > superproject --> submodule --> subsubmodule
> >              |                 ^
> >              '-----------------'
> > 
> > Where '-->' is a gitlink.
> > 
> > 
> > That mean .gitmodules and index of the superproject contain both submodule and
> > submodule/subsubmodule.
> 
> Wow, that shouldn't even work (as everything inside "submodule"
> shouldn't be part of the superproject but must be contained in
> the submodule itself). Do the "git submodule" script and other
> git commands like "git status" work for you in such setups?
>
As I stated above it is the purpose of the other patch that I have not already send
to implement this behavior. And that is why it work.
Everything including 'git submodule' and 'git status' work perfectly.
The intent of this patch is only to permit this for gitlinks. Not for regular files.
 
Show 6 quoted lines
> > and also mean (and that is the point) subsubmodule is a direct 'child' of both
> > superproject and submodule.
> 
> Which I think should not be possible. If that works with current
> Git I suspect we have a bug to fix ... or does your other patch
> make this work?
You have no bug on this point without my modification this is not possible.
Show 14 quoted lines
> 
> > In this case where should the separate gitdir of subsubmodule be placed ?
> >   - In superproject/modules/submodule/subsubmodule ?
> >   - In superproject/submodule/modules/subsubmodule ?
> >   - Depending on the 'git submodule update' command order ?
> >   - Or both ?
> 
> It should be placed in .git/modules/submodule/modules/subsubmodule
> of the superproject (assuming the subsubmodule is part of the first
> level submodule). But in your example that would live in
> .git/modules/submodule/subsubmodule (but as mentioned above, I do
> not consider this a valid setup because then two repositories would
> be responsible for the data inside subsubmodule, which will lead to
> lots of trouble).

That is why a had proposed an option '--no-separate-git-dir' for 'git submodule <add|update>' then no repository is responsible for the data in subsubmodule except subsubmodule itself.

Jens Lehmann· Mar 6, 2014, 20:51 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Am 06.03.2014 21:15, schrieb Henri GEIST:
Show 40 quoted lines
> Le jeudi 06 mars 2014 à 20:48 +0100, Jens Lehmann a écrit :
>> Am 06.03.2014 02:25, schrieb Henri GEIST:
>>> Le mercredi 05 mars 2014 à 19:13 +0100, Jens Lehmann a écrit :
>>>> Am 03.03.2014 21:34, schrieb Henri GEIST:
>>>>>   - I use an other patch which I plane to send later which enable multiple
>>>>>     level of superproject to add a gitlink to the same submodule.
>>>>>     And in this case the superproject containing the separate gitdir will be
>>>>>     arbitrary and depend on the processing order of the
>>>>>     'git submodule update --recursive' command.
>>>>
>>>> I don't understand that. How is that different from using different
>>>> names (and thus different separate gitdirs) for that duplicated
>>>> submodule? After all, the .git directory is just moved somewhere
>>>> else in the superproject's work tree, and as the name defaults to
>>>> the path of the submodule ...
>>>
>>> I think I should give an example.
>>> If I have a hierarchy like this :
>>>
>>> superproject/submodule/subsubmodule
>>>
>>> What I often do is:
>>>
>>> superproject --> submodule --> subsubmodule
>>>              |                 ^
>>>              '-----------------'
>>>
>>> Where '-->' is a gitlink.
>>>
>>>
>>> That mean .gitmodules and index of the superproject contain both submodule and
>>> submodule/subsubmodule.
>>
>> Wow, that shouldn't even work (as everything inside "submodule"
>> shouldn't be part of the superproject but must be contained in
>> the submodule itself). Do the "git submodule" script and other
>> git commands like "git status" work for you in such setups?
> 
> As I stated above it is the purpose of the other patch that I have not already send
> to implement this behavior. And that is why it work.
Ok.
> Everything including 'git submodule' and 'git status' work perfectly.
> The intent of this patch is only to permit this for gitlinks. Not for regular files.

But I still believe that this shouldn't be permitted at all, no matter if files or submodules are concerned. The pitfalls files face in such a scenario are pretty much the same for submodules too.

Show 8 quoted lines
>>> and also mean (and that is the point) subsubmodule is a direct 'child' of both
>>> superproject and submodule.
>>
>> Which I think should not be possible. If that works with current
>> Git I suspect we have a bug to fix ... or does your other patch
>> make this work?
> 
> You have no bug on this point without my modification this is not possible.
Glad to hear that.
Show 17 quoted lines
>>> In this case where should the separate gitdir of subsubmodule be placed ?
>>>   - In superproject/modules/submodule/subsubmodule ?
>>>   - In superproject/submodule/modules/subsubmodule ?
>>>   - Depending on the 'git submodule update' command order ?
>>>   - Or both ?
>>
>> It should be placed in .git/modules/submodule/modules/subsubmodule
>> of the superproject (assuming the subsubmodule is part of the first
>> level submodule). But in your example that would live in
>> .git/modules/submodule/subsubmodule (but as mentioned above, I do
>> not consider this a valid setup because then two repositories would
>> be responsible for the data inside subsubmodule, which will lead to
>> lots of trouble).
> 
> That is why a had proposed an option '--no-separate-git-dir'
> for 'git submodule <add|update>' then no repository is responsible for the data
> in subsubmodule except subsubmodule itself.

As I mentioned in my other email, it doesn't matter at all for the setup you're describing if the git directory lives under .git/modules of the superproject or in a .git directory in the submodule. The problem you're creating with your future patch is related to the work tree, not the GIT_DIR: "subsubmodule" could also be added to and tracked by "submodule" (as that is completely unaware of "subsubmodule" already being tracked by the superproject). Then you would end up in real trouble, as "superproject" and "submodule" could have differing SHA-1s recorded for subsubmodule. Don't go there, for just the same reasons we do not allow that for files.

What is the use case you are trying to solve and why can that not be handled by adding "subsubmodule" inside "submodule"?

Henri GEIST· Mar 6, 2014, 22:20 UTC · re: Jens Lehmann · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le jeudi 06 mars 2014 à 21:51 +0100, Jens Lehmann a écrit :
Show 52 quoted lines
> Am 06.03.2014 21:15, schrieb Henri GEIST:
> > Le jeudi 06 mars 2014 à 20:48 +0100, Jens Lehmann a écrit :
> >> Am 06.03.2014 02:25, schrieb Henri GEIST:
> >>> Le mercredi 05 mars 2014 à 19:13 +0100, Jens Lehmann a écrit :
> >>>> Am 03.03.2014 21:34, schrieb Henri GEIST:
> >>>>>   - I use an other patch which I plane to send later which enable multiple
> >>>>>     level of superproject to add a gitlink to the same submodule.
> >>>>>     And in this case the superproject containing the separate gitdir will be
> >>>>>     arbitrary and depend on the processing order of the
> >>>>>     'git submodule update --recursive' command.
> >>>>
> >>>> I don't understand that. How is that different from using different
> >>>> names (and thus different separate gitdirs) for that duplicated
> >>>> submodule? After all, the .git directory is just moved somewhere
> >>>> else in the superproject's work tree, and as the name defaults to
> >>>> the path of the submodule ...
> >>>
> >>> I think I should give an example.
> >>> If I have a hierarchy like this :
> >>>
> >>> superproject/submodule/subsubmodule
> >>>
> >>> What I often do is:
> >>>
> >>> superproject --> submodule --> subsubmodule
> >>>              |                 ^
> >>>              '-----------------'
> >>>
> >>> Where '-->' is a gitlink.
> >>>
> >>>
> >>> That mean .gitmodules and index of the superproject contain both submodule and
> >>> submodule/subsubmodule.
> >>
> >> Wow, that shouldn't even work (as everything inside "submodule"
> >> shouldn't be part of the superproject but must be contained in
> >> the submodule itself). Do the "git submodule" script and other
> >> git commands like "git status" work for you in such setups?
> > 
> > As I stated above it is the purpose of the other patch that I have not already send
> > to implement this behavior. And that is why it work.
> 
> Ok.
> 
> > Everything including 'git submodule' and 'git status' work perfectly.
> > The intent of this patch is only to permit this for gitlinks. Not for regular files.
> 
> But I still believe that this shouldn't be permitted at all,
> no matter if files or submodules are concerned. The pitfalls
> files face in such a scenario are pretty much the same for
> submodules too.
>
May be you have a good argument for this belief ?

As for the difference between submodules and regular files the only difference is in the meaning. Technically directory are just a special kind of file. But there day to day use is drastically different of the use of files which are not directories. I am not against enabling it for files as well. I am just unable to imagine a case where it make sens. A possible solution when someone try to do it is to issue a warning. "We are not able to see any good reason to do this are sure (y/n) ?"

Show 41 quoted lines
> >>> and also mean (and that is the point) subsubmodule is a direct 'child' of both
> >>> superproject and submodule.
> >>
> >> Which I think should not be possible. If that works with current
> >> Git I suspect we have a bug to fix ... or does your other patch
> >> make this work?
> > 
> > You have no bug on this point without my modification this is not possible.
> 
> Glad to hear that.
> 
> >>> In this case where should the separate gitdir of subsubmodule be placed ?
> >>>   - In superproject/modules/submodule/subsubmodule ?
> >>>   - In superproject/submodule/modules/subsubmodule ?
> >>>   - Depending on the 'git submodule update' command order ?
> >>>   - Or both ?
> >>
> >> It should be placed in .git/modules/submodule/modules/subsubmodule
> >> of the superproject (assuming the subsubmodule is part of the first
> >> level submodule). But in your example that would live in
> >> .git/modules/submodule/subsubmodule (but as mentioned above, I do
> >> not consider this a valid setup because then two repositories would
> >> be responsible for the data inside subsubmodule, which will lead to
> >> lots of trouble).
> > 
> > That is why a had proposed an option '--no-separate-git-dir'
> > for 'git submodule <add|update>' then no repository is responsible for the data
> > in subsubmodule except subsubmodule itself.
> 
> As I mentioned in my other email, it doesn't matter at all for
> the setup you're describing if the git directory lives under
> .git/modules of the superproject or in a .git directory in the
> submodule. The problem you're creating with your future patch
> is related to the work tree, not the GIT_DIR: "subsubmodule"
> could also be added to and tracked by "submodule" (as that is
> completely unaware of "subsubmodule" already being tracked by
> the superproject). Then you would end up in real trouble, as
> "superproject" and "submodule" could have differing SHA-1s
> recorded for subsubmodule. Don't go there, for just the same
> reasons we do not allow that for files.
> 

In fact it mater. Because multiples checkout of superproject and submodules in versions where subsubmodule is present and not. subsubmodule could have been clone one time by submodule and one time by superproject. And then if they are cloned with --separate-gitdir subsubmodule can have two gitdirs in superproject/modules/submodule/subsubmodule and in superproject/submodule/modules/subsubmodule. Only one is active at a given time but they are two and not synchronized.

> What is the use case you are trying to solve and why can that
> not be handled by adding "subsubmodule" inside "submodule"?
The problem is access rights.

Imagine you have 2 people Pierre and Paul. Each with different access write on the server. Pierre has full access on every things. Paul has full access on superproject and subsubmodule but no read/write access to submodule only execution on the directory.

I want all user to get every things they are allowed to have with the command 'git submodule update --init --recursive'. Then as Paul can not clone submodule he can not get subsubmodule recursively through it. And I need superproject to add also submodule/subsubmodule.

Jens Lehmann· Mar 7, 2014, 23:00 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Am 06.03.2014 23:20, schrieb Henri GEIST:
Show 23 quoted lines
> Le jeudi 06 mars 2014 à 21:51 +0100, Jens Lehmann a écrit :
>> Am 06.03.2014 21:15, schrieb Henri GEIST:
>>> Le jeudi 06 mars 2014 à 20:48 +0100, Jens Lehmann a écrit :
>>>> Am 06.03.2014 02:25, schrieb Henri GEIST:
>>>> Wow, that shouldn't even work (as everything inside "submodule"
>>>> shouldn't be part of the superproject but must be contained in
>>>> the submodule itself). Do the "git submodule" script and other
>>>> git commands like "git status" work for you in such setups?
>>>
>>> As I stated above it is the purpose of the other patch that I have not already send
>>> to implement this behavior. And that is why it work.
>>
>> Ok.
>>
>>> Everything including 'git submodule' and 'git status' work perfectly.
>>> The intent of this patch is only to permit this for gitlinks. Not for regular files.
>>
>> But I still believe that this shouldn't be permitted at all,
>> no matter if files or submodules are concerned. The pitfalls
>> files face in such a scenario are pretty much the same for
>> submodules too.
> 
> May be you have a good argument for this belief ?
Sure, I stated it further down:
Show 8 quoted lines
>> The problem you're creating with your future patch
>> is related to the work tree, not the GIT_DIR: "subsubmodule"
>> could also be added to and tracked by "submodule" (as that is
>> completely unaware of "subsubmodule" already being tracked by
>> the superproject). Then you would end up in real trouble, as
>> "superproject" and "submodule" could have differing SHA-1s
>> recorded for subsubmodule. Don't go there, for just the same
>> reasons we do not allow that for files.
Show 7 quoted lines
> As for the difference between submodules and regular files
> the only difference is in the meaning.
> Technically directory are just a special kind of file.
> But there day to day use is drastically different of
> the use of files which are not directories.
> I am not against enabling it for files as well.
> I am just unable to imagine a case where it make sens.
It doesn't make sense for both files and submodules.
> A possible solution when someone try to do it is to issue a warning.
> "We are not able to see any good reason to do this are sure (y/n) ?"

No, the only possible solution I see is not to allow that at all.

Show 35 quoted lines
>>>>> In this case where should the separate gitdir of subsubmodule be placed ?
>>>>>   - In superproject/modules/submodule/subsubmodule ?
>>>>>   - In superproject/submodule/modules/subsubmodule ?
>>>>>   - Depending on the 'git submodule update' command order ?
>>>>>   - Or both ?
>>>>
>>>> It should be placed in .git/modules/submodule/modules/subsubmodule
>>>> of the superproject (assuming the subsubmodule is part of the first
>>>> level submodule). But in your example that would live in
>>>> .git/modules/submodule/subsubmodule (but as mentioned above, I do
>>>> not consider this a valid setup because then two repositories would
>>>> be responsible for the data inside subsubmodule, which will lead to
>>>> lots of trouble).
>>>
>>> That is why a had proposed an option '--no-separate-git-dir'
>>> for 'git submodule <add|update>' then no repository is responsible for the data
>>> in subsubmodule except subsubmodule itself.
>>
>> As I mentioned in my other email, it doesn't matter at all for
>> the setup you're describing if the git directory lives under
>> .git/modules of the superproject or in a .git directory in the
>> submodule. The problem you're creating with your future patch
>> is related to the work tree, not the GIT_DIR: "subsubmodule"
>> could also be added to and tracked by "submodule" (as that is
>> completely unaware of "subsubmodule" already being tracked by
>> the superproject). Then you would end up in real trouble, as
>> "superproject" and "submodule" could have differing SHA-1s
>> recorded for subsubmodule. Don't go there, for just the same
>> reasons we do not allow that for files.
> 
> In fact it mater.
> Because multiples checkout of superproject and submodules in versions
> where subsubmodule is present and not.
> subsubmodule could have been clone one time by submodule and one time
> by superproject.

And each would have a different .git directory. Where is the problem with that? Size? Different refs?

> And then if they are cloned with --separate-gitdir subsubmodule can
> have two gitdirs in superproject/modules/submodule/subsubmodule and
> in superproject/submodule/modules/subsubmodule.
> Only one is active at a given time but they are two and not synchronized.
But the synchronization is done via the superproject, no?
Show 10 quoted lines
>> What is the use case you are trying to solve and why can that
>> not be handled by adding "subsubmodule" inside "submodule"?
> 
> The problem is access rights.
> 
> Imagine you have 2 people Pierre and Paul.
> Each with different access write on the server.
> Pierre has full access on every things.
> Paul has full access on superproject and subsubmodule but no read/write
> access to submodule only execution on the directory.
Ok, I think I'm slowly beginning to understand your setup.
> I want all user to get every things they are allowed to have with the
> command 'git submodule update --init --recursive'.
> Then as Paul can not clone submodule he can not get subsubmodule
> recursively through it.

Sure, that's how it should work. Paul could only work on a branch where "submodule" is an empty directory containing "subsubmodule", as he doesn't have the rights to clone "submodule".

> And I need superproject to add also submodule/subsubmodule.

No. Never let the same file/directory be tracked by two git repositories at the same time. Give Paul a branch to work on where "submodule" is just an empty directory, and everything will be fine. Or move "subsubmodule" outside of "submodule" (and let a symbolic link point to the new location if the path cannot be easily changed). Would that work for you?

Henri GEIST· Mar 10, 2014, 09:08 UTC · re: Jens Lehmann · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
Show 91 quoted lines
> Am 06.03.2014 23:20, schrieb Henri GEIST:
> > Le jeudi 06 mars 2014 à 21:51 +0100, Jens Lehmann a écrit :
> >> Am 06.03.2014 21:15, schrieb Henri GEIST:
> >>> Le jeudi 06 mars 2014 à 20:48 +0100, Jens Lehmann a écrit :
> >>>> Am 06.03.2014 02:25, schrieb Henri GEIST:
> >>>> Wow, that shouldn't even work (as everything inside "submodule"
> >>>> shouldn't be part of the superproject but must be contained in
> >>>> the submodule itself). Do the "git submodule" script and other
> >>>> git commands like "git status" work for you in such setups?
> >>>
> >>> As I stated above it is the purpose of the other patch that I have not already send
> >>> to implement this behavior. And that is why it work.
> >>
> >> Ok.
> >>
> >>> Everything including 'git submodule' and 'git status' work perfectly.
> >>> The intent of this patch is only to permit this for gitlinks. Not for regular files.
> >>
> >> But I still believe that this shouldn't be permitted at all,
> >> no matter if files or submodules are concerned. The pitfalls
> >> files face in such a scenario are pretty much the same for
> >> submodules too.
> > 
> > May be you have a good argument for this belief ?
> 
> Sure, I stated it further down:
> 
> >> The problem you're creating with your future patch
> >> is related to the work tree, not the GIT_DIR: "subsubmodule"
> >> could also be added to and tracked by "submodule" (as that is
> >> completely unaware of "subsubmodule" already being tracked by
> >> the superproject). Then you would end up in real trouble, as
> >> "superproject" and "submodule" could have differing SHA-1s
> >> recorded for subsubmodule. Don't go there, for just the same
> >> reasons we do not allow that for files.
> 
> > As for the difference between submodules and regular files
> > the only difference is in the meaning.
> > Technically directory are just a special kind of file.
> > But there day to day use is drastically different of
> > the use of files which are not directories.
> > I am not against enabling it for files as well.
> > I am just unable to imagine a case where it make sens.
> 
> It doesn't make sense for both files and submodules.
> 
> > A possible solution when someone try to do it is to issue a warning.
> > "We are not able to see any good reason to do this are sure (y/n) ?"
> 
> No, the only possible solution I see is not to allow that at
> all.
> 
> >>>>> In this case where should the separate gitdir of subsubmodule be placed ?
> >>>>>   - In superproject/modules/submodule/subsubmodule ?
> >>>>>   - In superproject/submodule/modules/subsubmodule ?
> >>>>>   - Depending on the 'git submodule update' command order ?
> >>>>>   - Or both ?
> >>>>
> >>>> It should be placed in .git/modules/submodule/modules/subsubmodule
> >>>> of the superproject (assuming the subsubmodule is part of the first
> >>>> level submodule). But in your example that would live in
> >>>> .git/modules/submodule/subsubmodule (but as mentioned above, I do
> >>>> not consider this a valid setup because then two repositories would
> >>>> be responsible for the data inside subsubmodule, which will lead to
> >>>> lots of trouble).
> >>>
> >>> That is why a had proposed an option '--no-separate-git-dir'
> >>> for 'git submodule <add|update>' then no repository is responsible for the data
> >>> in subsubmodule except subsubmodule itself.
> >>
> >> As I mentioned in my other email, it doesn't matter at all for
> >> the setup you're describing if the git directory lives under
> >> .git/modules of the superproject or in a .git directory in the
> >> submodule. The problem you're creating with your future patch
> >> is related to the work tree, not the GIT_DIR: "subsubmodule"
> >> could also be added to and tracked by "submodule" (as that is
> >> completely unaware of "subsubmodule" already being tracked by
> >> the superproject). Then you would end up in real trouble, as
> >> "superproject" and "submodule" could have differing SHA-1s
> >> recorded for subsubmodule. Don't go there, for just the same
> >> reasons we do not allow that for files.
> > 
> > In fact it mater.
> > Because multiples checkout of superproject and submodules in versions
> > where subsubmodule is present and not.
> > subsubmodule could have been clone one time by submodule and one time
> > by superproject.
> 
> And each would have a different .git directory. Where is the
> problem with that? Size? Different refs?
>
The problem is having two gitdir for one worktree.
with the .git file in the worktree pointing sometime on one and sometime on
the other.
 
Show 7 quoted lines
> > And then if they are cloned with --separate-gitdir subsubmodule can
> > have two gitdirs in superproject/modules/submodule/subsubmodule and
> > in superproject/submodule/modules/subsubmodule.
> > Only one is active at a given time but they are two and not synchronized.
> 
> But the synchronization is done via the superproject, no?
> 

Only lot of careful manual command by the user could keep them synchronize. But it is big wast of time. For no added value. It is quicker to make subsubmodule a regular clone without a separate gitdir then there is nothing needing a synchronization.

Show 21 quoted lines
> >> What is the use case you are trying to solve and why can that
> >> not be handled by adding "subsubmodule" inside "submodule"?
> > 
> > The problem is access rights.
> > 
> > Imagine you have 2 people Pierre and Paul.
> > Each with different access write on the server.
> > Pierre has full access on every things.
> > Paul has full access on superproject and subsubmodule but no read/write
> > access to submodule only execution on the directory.
> 
> Ok, I think I'm slowly beginning to understand your setup.
> 
> > I want all user to get every things they are allowed to have with the
> > command 'git submodule update --init --recursive'.
> > Then as Paul can not clone submodule he can not get subsubmodule
> > recursively through it.
> 
> Sure, that's how it should work. Paul could only work on a branch
> where "submodule" is an empty directory containing "subsubmodule",
> as he doesn't have the rights to clone "submodule".

I will not redundantly create a branch for each user on the server. When users clone the server it already create a special branch for them 'master' which track 'origin/master'. And if each user have its own branch on the server it will completely defeat the goal of the server "collaboration". And transform the git server in simple rsync server.

Show 9 quoted lines
> 
> > And I need superproject to add also submodule/subsubmodule.
> 
> No. Never let the same file/directory be tracked by two git
> repositories at the same time. Give Paul a branch to work on
> where "submodule" is just an empty directory, and everything
> will be fine. Or move "subsubmodule" outside of "submodule"
> (and let a symbolic link point to the new location if the
> path cannot be easily changed). Would that work for you?
If I use symbolic links it will just as gitlink enable to use the
same subsubmodule clone by more than one superproject but with two
major problems :
  - symbolic links do not work under Windows and some of my users do
    not even know something else could exist.
  - symbolic links will not store the SHA-1 of the subsubmodule.
    And a 'git status' in the repository containing the symbolic link
    will say nothing about subsubmodule state.

I think where we diverge is in the way we are looking gitlinks. Where you see a hierarchic tree, I see a web. And I use gitlinks just like multiplatform symbolic links storing the SHA-1 of there destination and pointing exclusively on git repositories.

Heiko Voigt· Mar 10, 2014, 20:32 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
Show 29 quoted lines
> Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > >> What is the use case you are trying to solve and why can that
> > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > 
> > > The problem is access rights.
> > > 
> > > Imagine you have 2 people Pierre and Paul.
> > > Each with different access write on the server.
> > > Pierre has full access on every things.
> > > Paul has full access on superproject and subsubmodule but no read/write
> > > access to submodule only execution on the directory.
> > 
> > Ok, I think I'm slowly beginning to understand your setup.
> > 
> > > I want all user to get every things they are allowed to have with the
> > > command 'git submodule update --init --recursive'.
> > > Then as Paul can not clone submodule he can not get subsubmodule
> > > recursively through it.
> > 
> > Sure, that's how it should work. Paul could only work on a branch
> > where "submodule" is an empty directory containing "subsubmodule",
> > as he doesn't have the rights to clone "submodule".
> 
> I will not redundantly create a branch for each user on the server.
> When users clone the server it already create a special branch for them
> 'master' which track 'origin/master'. And if each user have its own branch
> on the server it will completely defeat the goal of the server "collaboration".
> And transform the git server in simple rsync server.

I do not think that is what Jens was suggesting. It does not matter in which branch they work, they can directly use master if you like. What he was suggesting is that they create their repository structure like this:

git clone git@somewhere.net:superproject.git cd superproject/submodule git clone git@somehwere.net:subsubmodule.git cd subsubmodule ... work, commit, work, commit ...

The same applies for the superproject. Now only someone with access to the submodule has to update the registered sha1 once the work is pushed to submodule.

Show 17 quoted lines
> > > And I need superproject to add also submodule/subsubmodule.
> > 
> > No. Never let the same file/directory be tracked by two git
> > repositories at the same time. Give Paul a branch to work on
> > where "submodule" is just an empty directory, and everything
> > will be fine. Or move "subsubmodule" outside of "submodule"
> > (and let a symbolic link point to the new location if the
> > path cannot be easily changed). Would that work for you?
> 
> If I use symbolic links it will just as gitlink enable to use the
> same subsubmodule clone by more than one superproject but with two
> major problems :
>   - symbolic links do not work under Windows and some of my users do
>     not even know something else could exist.
>   - symbolic links will not store the SHA-1 of the subsubmodule.
>     And a 'git status' in the repository containing the symbolic link
>     will say nothing about subsubmodule state.

Here you are also missing something. What Jens was suggesting was that you move your subsubmodule directly underneath the superproject and from the old location you create a link to the new location for a quick transition. But you can also change all paths in your project to point to the new location. But in the new location you will have subsubmodule registered as a submodule only that it is now directly linked (as submodule) from the superproject instead of the submodule.

> I think where we diverge is in the way we are looking gitlinks.
> Where you see a hierarchic tree, I see a web.
> And I use gitlinks just like multiplatform symbolic links storing
> the SHA-1 of there destination and pointing exclusively on git repositories.

Well but the problem with a web is that it will introduce a lot of problems that need to be solved. Some repository has to have the authority about a file (or link). If you have a file in multiple repositories overlayed how do you know who is in charge and when?

There is a reason why it is designed like this: simplicity. I currently do not see how your web idea can be simple without introducing a lot of user interface questions.

Cheers Heiko
Henri GEIST· Mar 11, 2014, 09:55 UTC · re: Heiko Voigt · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le lundi 10 mars 2014 à 21:32 +0100, Heiko Voigt a écrit :
Show 45 quoted lines
> On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
> > Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > > >> What is the use case you are trying to solve and why can that
> > > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > > 
> > > > The problem is access rights.
> > > > 
> > > > Imagine you have 2 people Pierre and Paul.
> > > > Each with different access write on the server.
> > > > Pierre has full access on every things.
> > > > Paul has full access on superproject and subsubmodule but no read/write
> > > > access to submodule only execution on the directory.
> > > 
> > > Ok, I think I'm slowly beginning to understand your setup.
> > > 
> > > > I want all user to get every things they are allowed to have with the
> > > > command 'git submodule update --init --recursive'.
> > > > Then as Paul can not clone submodule he can not get subsubmodule
> > > > recursively through it.
> > > 
> > > Sure, that's how it should work. Paul could only work on a branch
> > > where "submodule" is an empty directory containing "subsubmodule",
> > > as he doesn't have the rights to clone "submodule".
> > 
> > I will not redundantly create a branch for each user on the server.
> > When users clone the server it already create a special branch for them
> > 'master' which track 'origin/master'. And if each user have its own branch
> > on the server it will completely defeat the goal of the server "collaboration".
> > And transform the git server in simple rsync server.
> 
> I do not think that is what Jens was suggesting. It does not matter in
> which branch they work, they can directly use master if you like. What
> he was suggesting is that they create their repository structure like
> this:
> 
> git clone git@somewhere.net:superproject.git
> cd superproject/submodule
> git clone git@somehwere.net:subsubmodule.git
> cd subsubmodule
> ... work, commit, work, commit ...
> 
> The same applies for the superproject. Now only someone with access to
> the submodule has to update the registered sha1 once the work is pushed
> to submodule.

I am not sure to understand everything. But if you suggest to clone manually subsubmodule because it could not be clone recursively by submodule due to the lake of access write to get submodule.

It is not practical in my use cases. Two of the superprojects I have in charge contains hundreds of submodules or subsubmodules and I have too much users with disparate computer skills.

Getting all what a user has access on should be just a recursive clone.
Show 27 quoted lines
> 
> > > > And I need superproject to add also submodule/subsubmodule.
> > > 
> > > No. Never let the same file/directory be tracked by two git
> > > repositories at the same time. Give Paul a branch to work on
> > > where "submodule" is just an empty directory, and everything
> > > will be fine. Or move "subsubmodule" outside of "submodule"
> > > (and let a symbolic link point to the new location if the
> > > path cannot be easily changed). Would that work for you?
> > 
> > If I use symbolic links it will just as gitlink enable to use the
> > same subsubmodule clone by more than one superproject but with two
> > major problems :
> >   - symbolic links do not work under Windows and some of my users do
> >     not even know something else could exist.
> >   - symbolic links will not store the SHA-1 of the subsubmodule.
> >     And a 'git status' in the repository containing the symbolic link
> >     will say nothing about subsubmodule state.
> 
> Here you are also missing something. What Jens was suggesting was that
> you move your subsubmodule directly underneath the superproject and from
> the old location you create a link to the new location for a quick
> transition. But you can also change all paths in your project to point
> to the new location. But in the new location you will have subsubmodule
> registered as a submodule only that it is now directly linked (as
> submodule) from the superproject instead of the submodule.
> 

Ok but in this case what happen to someone cloning only submodule but not superproject ? He will not get subsubmodule which is part of it. Just a dead symbolic link with no hint on what is missing behind.

Each of my submodules (at any level) should be usable superprojects by them self having a gitlink to each subsubmodules they needs.

Show 10 quoted lines
> > I think where we diverge is in the way we are looking gitlinks.
> > Where you see a hierarchic tree, I see a web.
> > And I use gitlinks just like multiplatform symbolic links storing
> > the SHA-1 of there destination and pointing exclusively on git repositories.
> 
> Well but the problem with a web is that it will introduce a lot of
> problems that need to be solved. Some repository has to have the
> authority about a file (or link). If you have a file in multiple
> repositories overlayed how do you know who is in charge and when?
> 

It will not introduce a lot of problems. Me and my teams are using gitlinks this way every days for 2 years know. With a web far more complex than the example I give above. And the problem you are speaking about and which we solve with the --no-separate-git-dir option is the only one we encounter until now.

This solve the question of who is in charge ? and when ? subsubmodule is in charge of itself. Always.

I know there is some good reasons for the separate gitdir. But none of them bother me in my day to day use. That is why I came with this simple solution

Another solution to combine all advantages. Is deciding to always make superproject in charge and place a link $SUBMODULE_GIT_DIR/modules/link_to_subsubmodule_gitdir pointing to $SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule.

I think I can write a 3 step patch doing just that :
  1) A little change in the $GIT_DIR/modules layout making the
$SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/.git instead of
$SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule

Then it will be possible to have $SUBSUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule/.git and so on without risk of name collision in case subsubmodule have a name like 'branches', 'hooks', 'refs' or anything like that.

  2) Making submodules aware that they have been cloned has submodule and
where is at least one of their superproject that is the non trivial part.
Maybe choosing the one which actually clone it.
Another solution is to simply found it by the fact the $SUBMODULE_GIT_DIR
is supposed to be in the $SUPERPROJECT_GIT_DIR.
  3) Making 'git submodule <add|update>' search recursively through its
superprojects or directly for the top one and place adequately its own
$SUBSUBMODULE_GIT_DIR.
> There is a reason why it is designed like this: simplicity. I currently
> do not see how your web idea can be simple without introducing a lot of
> user interface questions.
> 

Working with the web idea for several time now I can ensure you that Git is so well designed that it is ready to use with this concept. I have no user interface problem except the one we are speaking about.

> Cheers Heiko
Heiko Voigt· Mar 11, 2014, 20:11 UTC · re: Henri GEIST · lore

Re: Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

On Tue, Mar 11, 2014 at 10:55:03AM +0100, Henri GEIST wrote:
Show 57 quoted lines
> Le lundi 10 mars 2014 à 21:32 +0100, Heiko Voigt a écrit :
> > On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
> > > Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > > > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > > > >> What is the use case you are trying to solve and why can that
> > > > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > > > 
> > > > > The problem is access rights.
> > > > > 
> > > > > Imagine you have 2 people Pierre and Paul.
> > > > > Each with different access write on the server.
> > > > > Pierre has full access on every things.
> > > > > Paul has full access on superproject and subsubmodule but no read/write
> > > > > access to submodule only execution on the directory.
> > > > 
> > > > Ok, I think I'm slowly beginning to understand your setup.
> > > > 
> > > > > I want all user to get every things they are allowed to have with the
> > > > > command 'git submodule update --init --recursive'.
> > > > > Then as Paul can not clone submodule he can not get subsubmodule
> > > > > recursively through it.
> > > > 
> > > > Sure, that's how it should work. Paul could only work on a branch
> > > > where "submodule" is an empty directory containing "subsubmodule",
> > > > as he doesn't have the rights to clone "submodule".
> > > 
> > > I will not redundantly create a branch for each user on the server.
> > > When users clone the server it already create a special branch for them
> > > 'master' which track 'origin/master'. And if each user have its own branch
> > > on the server it will completely defeat the goal of the server "collaboration".
> > > And transform the git server in simple rsync server.
> > 
> > I do not think that is what Jens was suggesting. It does not matter in
> > which branch they work, they can directly use master if you like. What
> > he was suggesting is that they create their repository structure like
> > this:
> > 
> > git clone git@somewhere.net:superproject.git
> > cd superproject/submodule
> > git clone git@somehwere.net:subsubmodule.git
> > cd subsubmodule
> > ... work, commit, work, commit ...
> > 
> > The same applies for the superproject. Now only someone with access to
> > the submodule has to update the registered sha1 once the work is pushed
> > to submodule.
> 
> I am not sure to understand everything.
> But if you suggest to clone manually subsubmodule because it could
> not be clone recursively by submodule due to the lake of access write
> to get submodule.
> 
> It is not practical in my use cases.
> Two of the superprojects I have in charge contains hundreds of submodules
> or subsubmodules and I have too much users with disparate computer skills.
> 
> Getting all what a user has access on should be just a recursive clone.

Then I would think about getting rid of the recursion part as it seems you have interdependencies which can only be solved by a package management system. I would see the superproject as this package management system, but it requires you to have all the submodules next to each other instead of contained in each other.

I think in terms of combining libraries that is actually the correct solution because there can be modules that need each other. Some submodule A might evolve and add a dependency to a subsubmodule B that is itself contained in another submodule C. Then it just does not feel correct anymore that B is contained in C. You want to have one instance that is in charge of all the dependencies, that is IMO directly the superproject and not something that reaches through another submodule to record a dependency to a subsubmodule.

Show 33 quoted lines
> > > > > And I need superproject to add also submodule/subsubmodule.
> > > > 
> > > > No. Never let the same file/directory be tracked by two git
> > > > repositories at the same time. Give Paul a branch to work on
> > > > where "submodule" is just an empty directory, and everything
> > > > will be fine. Or move "subsubmodule" outside of "submodule"
> > > > (and let a symbolic link point to the new location if the
> > > > path cannot be easily changed). Would that work for you?
> > > 
> > > If I use symbolic links it will just as gitlink enable to use the
> > > same subsubmodule clone by more than one superproject but with two
> > > major problems :
> > >   - symbolic links do not work under Windows and some of my users do
> > >     not even know something else could exist.
> > >   - symbolic links will not store the SHA-1 of the subsubmodule.
> > >     And a 'git status' in the repository containing the symbolic link
> > >     will say nothing about subsubmodule state.
> > 
> > Here you are also missing something. What Jens was suggesting was that
> > you move your subsubmodule directly underneath the superproject and from
> > the old location you create a link to the new location for a quick
> > transition. But you can also change all paths in your project to point
> > to the new location. But in the new location you will have subsubmodule
> > registered as a submodule only that it is now directly linked (as
> > submodule) from the superproject instead of the submodule.
> > 
> 
> Ok but in this case what happen to someone cloning only submodule but
> not superproject ? He will not get subsubmodule which is part of it.
> Just a dead symbolic link with no hint on what is missing behind.
> 
> Each of my submodules (at any level) should be usable superprojects by
> them self having a gitlink to each subsubmodules they needs.

Then you will end up duplicating many submodules if everything should contain everything it needs. Think about the example I described above. In case of libraries, that will become a management nightmare.

Show 59 quoted lines
> > > I think where we diverge is in the way we are looking gitlinks.
> > > Where you see a hierarchic tree, I see a web.
> > > And I use gitlinks just like multiplatform symbolic links storing
> > > the SHA-1 of there destination and pointing exclusively on git repositories.
> > 
> > Well but the problem with a web is that it will introduce a lot of
> > problems that need to be solved. Some repository has to have the
> > authority about a file (or link). If you have a file in multiple
> > repositories overlayed how do you know who is in charge and when?
> > 
> 
> It will not introduce a lot of problems.
> Me and my teams are using gitlinks this way every days for 2 years know.
> With a web far more complex than the example I give above.
> And the problem you are speaking about and which we solve with the
> --no-separate-git-dir option is the only one we encounter until now.
> 
> This solve the question of who is in charge ? and when ?
> subsubmodule is in charge of itself. Always.
> 
> I know there is some good reasons for the separate gitdir.
> But none of them bother me in my day to day use.
> That is why I came with this simple solution
> 
> Another solution to combine all advantages.
> Is deciding to always make superproject in charge and place a link
> $SUBMODULE_GIT_DIR/modules/link_to_subsubmodule_gitdir
> pointing to $SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule.
> 
> I think I can write a 3 step patch doing just that :
> 
>   1) A little change in the $GIT_DIR/modules layout making the
> $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/.git instead of
> $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule
> 
> Then it will be possible to have
> $SUBSUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule/.git
> and so on without risk of name collision in case subsubmodule have a name like
> 'branches', 'hooks', 'refs' or anything like that.
> 
>   2) Making submodules aware that they have been cloned has submodule and
> where is at least one of their superproject that is the non trivial part.
> Maybe choosing the one which actually clone it.
> Another solution is to simply found it by the fact the $SUBMODULE_GIT_DIR
> is supposed to be in the $SUPERPROJECT_GIT_DIR.
> 
>   3) Making 'git submodule <add|update>' search recursively through its
> superprojects or directly for the top one and place adequately its own
> $SUBSUBMODULE_GIT_DIR.
> 
> 
> > There is a reason why it is designed like this: simplicity. I currently
> > do not see how your web idea can be simple without introducing a lot of
> > user interface questions.
> > 
> 
> Working with the web idea for several time now I can ensure you that
> Git is so well designed that it is ready to use with this concept.
> I have no user interface problem except the one we are speaking about.

If you have multiple superprojects for a submodule in which superproject do you commit a change on a gitlink?

Don't you end up having a lot of commits if you have to commit in multiple superprojects?

Suppose you have a directory layout like this:
super\
   submoduleA\
   	submoduleB

What happens if submoduleA has no knowledge about submoduleB but super decides to track it? Then on one point submoduleA decides to put files into the directory named 'submoduleB'. What happens?

Just a few questions that immediately pop into my mind.
Cheers Heiko
Henri GEIST· Mar 11, 2014, 22:07 UTC · re: Heiko Voigt · lore

Re: Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

Le mardi 11 mars 2014 à 21:11 +0100, Heiko Voigt a écrit :
Show 65 quoted lines
> On Tue, Mar 11, 2014 at 10:55:03AM +0100, Henri GEIST wrote:
> > Le lundi 10 mars 2014 à 21:32 +0100, Heiko Voigt a écrit :
> > > On Mon, Mar 10, 2014 at 10:08:06AM +0100, Henri GEIST wrote:
> > > > Le samedi 08 mars 2014 à 00:00 +0100, Jens Lehmann a écrit :
> > > > > Am 06.03.2014 23:20, schrieb Henri GEIST:
> > > > > >> What is the use case you are trying to solve and why can that
> > > > > >> not be handled by adding "subsubmodule" inside "submodule"?
> > > > > > 
> > > > > > The problem is access rights.
> > > > > > 
> > > > > > Imagine you have 2 people Pierre and Paul.
> > > > > > Each with different access write on the server.
> > > > > > Pierre has full access on every things.
> > > > > > Paul has full access on superproject and subsubmodule but no read/write
> > > > > > access to submodule only execution on the directory.
> > > > > 
> > > > > Ok, I think I'm slowly beginning to understand your setup.
> > > > > 
> > > > > > I want all user to get every things they are allowed to have with the
> > > > > > command 'git submodule update --init --recursive'.
> > > > > > Then as Paul can not clone submodule he can not get subsubmodule
> > > > > > recursively through it.
> > > > > 
> > > > > Sure, that's how it should work. Paul could only work on a branch
> > > > > where "submodule" is an empty directory containing "subsubmodule",
> > > > > as he doesn't have the rights to clone "submodule".
> > > > 
> > > > I will not redundantly create a branch for each user on the server.
> > > > When users clone the server it already create a special branch for them
> > > > 'master' which track 'origin/master'. And if each user have its own branch
> > > > on the server it will completely defeat the goal of the server "collaboration".
> > > > And transform the git server in simple rsync server.
> > > 
> > > I do not think that is what Jens was suggesting. It does not matter in
> > > which branch they work, they can directly use master if you like. What
> > > he was suggesting is that they create their repository structure like
> > > this:
> > > 
> > > git clone git@somewhere.net:superproject.git
> > > cd superproject/submodule
> > > git clone git@somehwere.net:subsubmodule.git
> > > cd subsubmodule
> > > ... work, commit, work, commit ...
> > > 
> > > The same applies for the superproject. Now only someone with access to
> > > the submodule has to update the registered sha1 once the work is pushed
> > > to submodule.
> > 
> > I am not sure to understand everything.
> > But if you suggest to clone manually subsubmodule because it could
> > not be clone recursively by submodule due to the lake of access write
> > to get submodule.
> > 
> > It is not practical in my use cases.
> > Two of the superprojects I have in charge contains hundreds of submodules
> > or subsubmodules and I have too much users with disparate computer skills.
> > 
> > Getting all what a user has access on should be just a recursive clone.
> 
> Then I would think about getting rid of the recursion part as it seems
> you have interdependencies which can only be solved by a package
> management system. I would see the superproject as this package
> management system, but it requires you to have all the submodules next
> to each other instead of contained in each other.
>
You put the finger on a key point.

I use the submodule system exactly as a package management system. It is even the only use I have of it. I am not able to imagine another use. (My imagination is limited). I really use 'git clone --recursive' as 'apt-get install'. And I am pretty sure you also.

And in fact for the case where the submodules/packages should be side by
side, I have a third patch witch enable just this by enabling '../' to be
part of a gitlink.
Much of my submodules/packages make use of this feature but I also have the
case where the dependency make them contained in each others.
 
Show 8 quoted lines
> I think in terms of combining libraries that is actually the correct
> solution because there can be modules that need each other. Some
> submodule A might evolve and add a dependency to a subsubmodule B that
> is itself contained in another submodule C. Then it just does not feel
> correct anymore that B is contained in C. You want to have one instance
> that is in charge of all the dependencies, that is IMO directly the
> superproject and not something that reaches through another submodule
> to record a dependency to a subsubmodule.

Right. But each module need to know by its own gitlinks which are its dependency to be able to track version compatibility and not rely on an hypothetic superproject which may or may not do it as a submodule do not even know if it is part of a superproject. And could be include in totally different superprojects.

Show 39 quoted lines
> 
> > > > > > And I need superproject to add also submodule/subsubmodule.
> > > > > 
> > > > > No. Never let the same file/directory be tracked by two git
> > > > > repositories at the same time. Give Paul a branch to work on
> > > > > where "submodule" is just an empty directory, and everything
> > > > > will be fine. Or move "subsubmodule" outside of "submodule"
> > > > > (and let a symbolic link point to the new location if the
> > > > > path cannot be easily changed). Would that work for you?
> > > > 
> > > > If I use symbolic links it will just as gitlink enable to use the
> > > > same subsubmodule clone by more than one superproject but with two
> > > > major problems :
> > > >   - symbolic links do not work under Windows and some of my users do
> > > >     not even know something else could exist.
> > > >   - symbolic links will not store the SHA-1 of the subsubmodule.
> > > >     And a 'git status' in the repository containing the symbolic link
> > > >     will say nothing about subsubmodule state.
> > > 
> > > Here you are also missing something. What Jens was suggesting was that
> > > you move your subsubmodule directly underneath the superproject and from
> > > the old location you create a link to the new location for a quick
> > > transition. But you can also change all paths in your project to point
> > > to the new location. But in the new location you will have subsubmodule
> > > registered as a submodule only that it is now directly linked (as
> > > submodule) from the superproject instead of the submodule.
> > > 
> > 
> > Ok but in this case what happen to someone cloning only submodule but
> > not superproject ? He will not get subsubmodule which is part of it.
> > Just a dead symbolic link with no hint on what is missing behind.
> > 
> > Each of my submodules (at any level) should be usable superprojects by
> > them self having a gitlink to each subsubmodules they needs.
> 
> Then you will end up duplicating many submodules if everything should
> contain everything it needs. Think about the example I described above.
> In case of libraries, that will become a management nightmare.
> 

As stated above in this case my other patch enable me to put the submodules sides by sides and address this issue. Then I have no duplication.

Show 62 quoted lines
> > > > I think where we diverge is in the way we are looking gitlinks.
> > > > Where you see a hierarchic tree, I see a web.
> > > > And I use gitlinks just like multiplatform symbolic links storing
> > > > the SHA-1 of there destination and pointing exclusively on git repositories.
> > > 
> > > Well but the problem with a web is that it will introduce a lot of
> > > problems that need to be solved. Some repository has to have the
> > > authority about a file (or link). If you have a file in multiple
> > > repositories overlayed how do you know who is in charge and when?
> > > 
> > 
> > It will not introduce a lot of problems.
> > Me and my teams are using gitlinks this way every days for 2 years know.
> > With a web far more complex than the example I give above.
> > And the problem you are speaking about and which we solve with the
> > --no-separate-git-dir option is the only one we encounter until now.
> > 
> > This solve the question of who is in charge ? and when ?
> > subsubmodule is in charge of itself. Always.
> > 
> > I know there is some good reasons for the separate gitdir.
> > But none of them bother me in my day to day use.
> > That is why I came with this simple solution
> > 
> > Another solution to combine all advantages.
> > Is deciding to always make superproject in charge and place a link
> > $SUBMODULE_GIT_DIR/modules/link_to_subsubmodule_gitdir
> > pointing to $SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule.
> > 
> > I think I can write a 3 step patch doing just that :
> > 
> >   1) A little change in the $GIT_DIR/modules layout making the
> > $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/.git instead of
> > $SUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule
> > 
> > Then it will be possible to have
> > $SUBSUBMODULE_GIT_DIR=$SUPERPROJECT_GIT_DIR/modules/submodule/subsubmodule/.git
> > and so on without risk of name collision in case subsubmodule have a name like
> > 'branches', 'hooks', 'refs' or anything like that.
> > 
> >   2) Making submodules aware that they have been cloned has submodule and
> > where is at least one of their superproject that is the non trivial part.
> > Maybe choosing the one which actually clone it.
> > Another solution is to simply found it by the fact the $SUBMODULE_GIT_DIR
> > is supposed to be in the $SUPERPROJECT_GIT_DIR.
> > 
> >   3) Making 'git submodule <add|update>' search recursively through its
> > superprojects or directly for the top one and place adequately its own
> > $SUBSUBMODULE_GIT_DIR.
> > 
> > 
> > > There is a reason why it is designed like this: simplicity. I currently
> > > do not see how your web idea can be simple without introducing a lot of
> > > user interface questions.
> > > 
> > 
> > Working with the web idea for several time now I can ensure you that
> > Git is so well designed that it is ready to use with this concept.
> > I have no user interface problem except the one we are speaking about.
> 
> If you have multiple superprojects for a submodule in which superproject
> do you commit a change on a gitlink?

Each superproject impacted by the changes. But only when I need it to be updated.

> 
> Don't you end up having a lot of commits if you have to commit in
> multiple superprojects?
> 

I have only commit in project which have a real change which conceptually need a commit. A careful design of projects modularization and dependency will not lead to unneeded commits. But if you create unneeded dependency between your submodules you can have a lot of meaningless commit. It is only a project design problem not a Git problem.

But in counterpart it add some capital value (at least to my eyes), traceability of the dependency between submodules/packages through their SHA-1.

Show 10 quoted lines
> Suppose you have a directory layout like this:
> 
> super\
>    submoduleA\
>    	submoduleB
> 
> What happens if submoduleA has no knowledge about submoduleB but super
> decides to track it? Then on one point submoduleA decides to put files
> into the directory named 'submoduleB'. What happens?
> 

It is a general design question not a Git question. If super have put a submoduleB in the submoduleA directory it is intentional and should be sound in your design. And as designer I see no good reason to place submoduleB in a submoduleA already containing a submoduleA directory or track file in submoduleB by submoduleA. And in fact Git prevent this. My patch only enable it for gitlinks.

But maybe you have in mind a case where the maintainer of super will do this for a good reason in this case I can enable it.

> Just a few questions that immediately pop into my mind.
> 
> Cheers Heiko
Junio C Hamano· Mar 3, 2014, 19:22 UTC · re: Henri GEIST · lore

Re: [PATCH] submodule : Add --no-separate-git-dir option to add and update command.

[CC'ing the submodule area experts.]
Henri GEIST <geist.henri@laposte.net> writes:
Show 7 quoted lines
> This new option prevent git submodule <add|update> to clone the missing
> submodules with the --separate-git-dir option.
> Then the submodule will be regular repository and their gitdir will not
> be placed in the superproject gitdir/modules directory.
>
> Signed-off-by: Henri GEIST <geist.henri@laposte.net>
> ---
Thanks.

The above describes what the new option does, but does not explain why the new option is a good idea in the first place.

Given that we used to directly clone into the superproject's working tree like this patch does, realized that it was a very bad idea and are trying to move to the direction of keeping it in modules/ subdirectory of the superproject's .git directory, there needs to be a very good explanation to justify why this "going backwards" is sometimes a desirable thing.

← back to recent threads