threads / patch / 24044

patch, 2 partsAdd optional parameters to the diff option "--ignore-submodules"

Subject: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

## tl;dr

14 messages between Jun 8, 2010 and Jun 24, 2010. Diffs are folded; open one to read it.

replies: 13people: 5as markdown or json

Jens Lehmann· Jun 8, 2010, 16:30 UTC · lore

After thinking some time about peoples expectations and troubles with the recursive scanning of submodules, I came up with this:

What about expanding the "--ignore-submodules" option of the git diff family with three parameters:

--ignore-submodules=all : Same behavior as "--ignore-submodules",
  submodules show never up as modified.
--ignore-submodules=untracked : Don't consider submodules as modified
  when they only contain untracked files, but do if the commits in the
  superproject are different or tracked content is modified.
--ignore-submodules=dirty : Don't consider submodules as modified
  when their work tree is dirty, no matter why. This is the pre 1.7.0
  behavior and doesn't recurse into submodules at all.

The first patch is just a resend of a test case rename, the second contains the implementation.

To make that more useful the default could be controlled by the .git/config or .gitmodules file. So you could have two submodules:

[submodule "sub1"]
	path = sub1
	url = /home/me/sub1.git/
	ignore = dirty
[submodule "sub2"]
	path = sub2
	url = /home/me/sub2.git/
	ignore = untracked

Where sub1 will not be scanned for work tree modifications by "git diff" and "git status" and for sub2 any untracked files inside the submodule will be ignored. And then "git status" should learn the "--ignore-submodule" option too. With a fourth parameter "none" it would be possible to override the defaults from the configuration anyway you want.

Opinions?
Jens Lehmann (2):
  git diff: rename test that had a conflicting name
  Add optional parameters to the diff option "--ignore-submodules"
 Documentation/diff-options.txt                     |   10 ++-
 diff-lib.c                                         |    1 +
 diff.c                                             |   11 +++-
 diff.h                                             |    1 +
 t/t4027-diff-submodule.sh                          |   20 ++++-
 ...submodule.sh => t4041-diff-submodule-option.sh} |   81 ++++++++++++++++++++
 6 files changed, 118 insertions(+), 6 deletions(-)
 rename t/{t4041-diff-submodule.sh => t4041-diff-submodule-option.sh} (74%)
Jens Lehmann· Jun 8, 2010, 16:31 UTC · re: Jens Lehmann · lore

[PATCH 1/2] git diff: rename test that had a conflicting name

In 86140d5 the new test t4041-diff-submodule.sh was introduced although t4027-diff-submodule.sh already existed. Rename the newer test to t4041-diff-submodule-option.sh to fix that.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
 ...submodule.sh => t4041-diff-submodule-option.sh} |    0
 1 files changed, 0 insertions(+), 0 deletions(-)
 rename t/{t4041-diff-submodule.sh => t4041-diff-submodule-option.sh} (100%)
Show changes to t/t4041-diff-submodule.sh +0 −0
diff --git a/t/t4041-diff-submodule.sh b/t/t4041-diff-submodule-option.sh
similarity index 100%
rename from t/t4041-diff-submodule.sh
rename to t/t4041-diff-submodule-option.sh
-- 
1.7.1.447.g4d763
Jens Lehmann· Jun 8, 2010, 16:31 UTC · re: Jens Lehmann · lore

[PATCH 2/2] Add optional parameters to the diff option "--ignore-submodules"

In some use cases it is not desirable that the diff family considers submodules that only contain untracked content as dirty. This may happen e.g. when the submodule is not under the developers control and not all build generated files have been added to .gitignore by the upstream developers. Using the "untracked" parameter for the "--ignore-submodules" option disables checking for untracked content and lets git diff report them as changed only when they have new commits or modified content.

Sometimes it is not wanted to have submodules show up as changed when they just contain changes to their work tree. An example for that are scripts which just want to check for submodule commits while ignoring any changes to the work tree. Also users having large submodules known not to change might want to use this option, as the - sometimes substantial - time it takes to scan the submodule work tree(s) is saved.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
 Documentation/diff-options.txt   |   10 ++++-
 diff-lib.c                       |    1 +
 diff.c                           |   11 +++++-
 diff.h                           |    1 +
 t/t4027-diff-submodule.sh        |   20 ++++++++-
 t/t4041-diff-submodule-option.sh |   81 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 118 insertions(+), 6 deletions(-)
Show changes to 6 files +118 −5

Documentation/diff-options.txt, diff-lib.c, diff.c, diff.h, t/t4027-diff-submodule.sh, t/t4041-diff-submodule-option.sh

diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt
index e745a3c..2371262 100644
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@ -328,8 +328,14 @@ endif::git-format-patch[]
 --no-ext-diff::
 	Disallow external diff drivers.

---ignore-submodules::
-	Ignore changes to submodules in the diff generation.
+--ignore-submodules[=<when>]::
+	Ignore changes to submodules in the diff generation. <when> can be
+	either "untracked", "dirty" or "all", which is the default. When
+	"untracked" is used submodules are not considered dirty when they only
+	contain untracked content (but they are still scanned for modified
+	content). Using "dirty" ignores all changes to the work tree of submodules,
+	only changes to the commits stored in the superproject are shown (this was
+	the behavior until 1.7.0). Using "all" hides all changes to submodules.

 --src-prefix=<prefix>::
 	Show the given source prefix instead of "a/".
diff --git a/diff-lib.c b/diff-lib.c
index c9f6e05..8b8978a 100644
--- a/diff-lib.c
+++ b/diff-lib.c
@@ -70,6 +70,7 @@ static int match_stat_with_submodule(struct diff_options *diffopt,
 	int changed = ce_match_stat(ce, st, ce_option);
 	if (S_ISGITLINK(ce->ce_mode)
 	    && !DIFF_OPT_TST(diffopt, IGNORE_SUBMODULES)
+	    && !DIFF_OPT_TST(diffopt, IGNORE_DIRTY_SUBMODULES)
 	    && (!changed || DIFF_OPT_TST(diffopt, DIRTY_SUBMODULES))) {
 		*dirty_submodule = is_submodule_modified(ce->name, DIFF_OPT_TST(diffopt, IGNORE_UNTRACKED_IN_SUBMODULES));
 	}
diff --git a/diff.c b/diff.c
index 2327cea..71fff7e 100644
--- a/diff.c
+++ b/diff.c
@@ -3174,7 +3174,16 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)
 		DIFF_OPT_CLR(options, ALLOW_TEXTCONV);
 	else if (!strcmp(arg, "--ignore-submodules"))
 		DIFF_OPT_SET(options, IGNORE_SUBMODULES);
-	else if (!strcmp(arg, "--submodule"))
+	else if (!prefixcmp(arg, "--ignore-submodules=")) {
+		if (!strcmp(arg + 20, "all"))
+			DIFF_OPT_SET(options, IGNORE_SUBMODULES);
+		else if (!strcmp(arg + 20, "untracked"))
+			DIFF_OPT_SET(options, IGNORE_UNTRACKED_IN_SUBMODULES);
+		else if (!strcmp(arg + 20, "dirty"))
+			DIFF_OPT_SET(options, IGNORE_DIRTY_SUBMODULES);
+		else
+			die("bad --ignore-submodules argument: %s", arg + 20);
+	} else if (!strcmp(arg, "--submodule"))
 		DIFF_OPT_SET(options, SUBMODULE_LOG);
 	else if (!prefixcmp(arg, "--submodule=")) {
 		if (!strcmp(arg + 12, "log"))
diff --git a/diff.h b/diff.h
index 48abe7a..5248cd8 100644
--- a/diff.h
+++ b/diff.h
@@ -74,6 +74,7 @@ typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data)
 #define DIFF_OPT_SUBMODULE_LOG       (1 << 23)
 #define DIFF_OPT_DIRTY_SUBMODULES    (1 << 24)
 #define DIFF_OPT_IGNORE_UNTRACKED_IN_SUBMODULES (1 << 25)
+#define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)

 #define DIFF_OPT_TST(opts, flag)    ((opts)->flags & DIFF_OPT_##flag)
 #define DIFF_OPT_SET(opts, flag)    ((opts)->flags |= DIFF_OPT_##flag)
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index 83c1914..559b41e 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -103,9 +103,17 @@ test_expect_success 'git diff HEAD with dirty submodule (work tree, refs match)'
 	git diff HEAD >actual &&
 	sed -e "1,/^@@/d" actual >actual.body &&
 	expect_from_to >expect.body $subprev $subprev-dirty &&
-	test_cmp expect.body actual.body
+	test_cmp expect.body actual.body &&
+	git diff --ignore-submodules HEAD >actual2 &&
+	echo -n "" | test_cmp - actual2 &&
+	git diff --ignore-submodules=untracked HEAD >actual3 &&
+	sed -e "1,/^@@/d" actual3 >actual3.body &&
+	expect_from_to >expect.body $subprev $subprev-dirty &&
+	test_cmp expect.body actual3.body &&
+	git diff --ignore-submodules=dirty HEAD >actual4 &&
+	echo -n "" | test_cmp - actual4
 '
-
+test_done
 test_expect_success 'git diff HEAD with dirty submodule (index, refs match)' '
 	(
 		cd sub &&
@@ -129,7 +137,13 @@ test_expect_success 'git diff HEAD with dirty submodule (untracked, refs match)'
 	git diff HEAD >actual &&
 	sed -e "1,/^@@/d" actual >actual.body &&
 	expect_from_to >expect.body $subprev $subprev-dirty &&
-	test_cmp expect.body actual.body
+	test_cmp expect.body actual.body &&
+	git diff --ignore-submodules=all HEAD >actual2 &&
+	echo -n "" | test_cmp - actual2 &&
+	git diff --ignore-submodules=untracked HEAD >actual3 &&
+	echo -n "" | test_cmp - actual3 &&
+	git diff --ignore-submodules=dirty HEAD >actual4 &&
+	echo -n "" | test_cmp - actual4
 '

 test_expect_success 'git diff (empty submodule dir)' '
diff --git a/t/t4041-diff-submodule-option.sh b/t/t4041-diff-submodule-option.sh
index 019acb9..f44b906 100755
--- a/t/t4041-diff-submodule-option.sh
+++ b/t/t4041-diff-submodule-option.sh
@@ -205,6 +205,21 @@ Submodule sm1 contains untracked content
 EOF
 "

+test_expect_success 'submodule contains untracked content (untracked ignored)' "
+	git diff-index -p --ignore-submodules=untracked --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
+test_expect_success 'submodule contains untracked content (dirty ignored)' "
+	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
+test_expect_success 'submodule contains untracked content (all ignored)' "
+	git diff-index -p --ignore-submodules=all --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
 test_expect_success 'submodule contains untracked and modifed content' "
 	echo new > sm1/foo6 &&
 	git diff-index -p --submodule=log HEAD >actual &&
@@ -214,6 +229,26 @@ Submodule sm1 contains modified content
 EOF
 "

+test_expect_success 'submodule contains untracked and modifed content (untracked ignored)' "
+	echo new > sm1/foo6 &&
+	git diff-index -p --ignore-submodules=untracked --submodule=log HEAD >actual &&
+	diff actual - <<-EOF
+Submodule sm1 contains modified content
+EOF
+"
+
+test_expect_success 'submodule contains untracked and modifed content (dirty ignored)' "
+	echo new > sm1/foo6 &&
+	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
+test_expect_success 'submodule contains untracked and modifed content (all ignored)' "
+	echo new > sm1/foo6 &&
+	git diff-index -p --ignore-submodules --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
 test_expect_success 'submodule contains modifed content' "
 	rm -f sm1/new-file &&
 	git diff-index -p --submodule=log HEAD >actual &&
@@ -242,6 +277,27 @@ Submodule sm1 $head6..$head8:
 EOF
 "

+test_expect_success 'modified submodule contains untracked content (untracked ignored)' "
+	git diff-index -p --ignore-submodules=untracked --submodule=log HEAD >actual &&
+	diff actual - <<-EOF
+Submodule sm1 $head6..$head8:
+  > change
+EOF
+"
+
+test_expect_success 'modified submodule contains untracked content (dirty ignored)' "
+	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
+	diff actual - <<-EOF
+Submodule sm1 $head6..$head8:
+  > change
+EOF
+"
+
+test_expect_success 'modified submodule contains untracked content (all ignored)' "
+	git diff-index -p --ignore-submodules=all --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
 test_expect_success 'modified submodule contains untracked and modifed content' "
 	echo modification >> sm1/foo6 &&
 	git diff-index -p --submodule=log HEAD >actual &&
@@ -253,6 +309,31 @@ Submodule sm1 $head6..$head8:
 EOF
 "

+test_expect_success 'modified submodule contains untracked and modifed content (untracked ignored)' "
+	echo modification >> sm1/foo6 &&
+	git diff-index -p --ignore-submodules=untracked --submodule=log HEAD >actual &&
+	diff actual - <<-EOF
+Submodule sm1 contains modified content
+Submodule sm1 $head6..$head8:
+  > change
+EOF
+"
+
+test_expect_success 'modified submodule contains untracked and modifed content (dirty ignored)' "
+	echo modification >> sm1/foo6 &&
+	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
+	diff actual - <<-EOF
+Submodule sm1 $head6..$head8:
+  > change
+EOF
+"
+
+test_expect_success 'modified submodule contains untracked and modifed content (all ignored)' "
+	echo modification >> sm1/foo6 &&
+	git diff-index -p --ignore-submodules --submodule=log HEAD >actual &&
+	echo -n '' | diff actual -
+"
+
 test_expect_success 'modified submodule contains modifed content' "
 	rm -f sm1/new-file &&
 	git diff-index -p --submodule=log HEAD >actual &&
-- 
1.7.1.447.g4d763
Brandon Casey· Jun 24, 2010, 14:23 UTC · re: Jens Lehmann · lore

Re: [PATCH 2/2] Add optional parameters to the diff option "--ignore-submodules"

On 06/08/2010 11:31 AM, Jens Lehmann wrote:
Two things...
Show 12 quoted lines
> diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
> index 83c1914..559b41e 100755
> --- a/t/t4027-diff-submodule.sh
> +++ b/t/t4027-diff-submodule.sh
> @@ -103,9 +103,17 @@ test_expect_success 'git diff HEAD with dirty submodule (work tree, refs match)'
>  	git diff HEAD >actual &&
>  	sed -e "1,/^@@/d" actual >actual.body &&
>  	expect_from_to >expect.body $subprev $subprev-dirty &&
> -	test_cmp expect.body actual.body
> +	test_cmp expect.body actual.body &&
> +	git diff --ignore-submodules HEAD >actual2 &&
> +	echo -n "" | test_cmp - actual2 &&

'echo -n' is not portable (here, and below). Please use printf instead. Ditto for t4041

Show 9 quoted lines
> +	git diff --ignore-submodules=untracked HEAD >actual3 &&
> +	sed -e "1,/^@@/d" actual3 >actual3.body &&
> +	expect_from_to >expect.body $subprev $subprev-dirty &&
> +	test_cmp expect.body actual3.body &&
> +	git diff --ignore-submodules=dirty HEAD >actual4 &&
> +	echo -n "" | test_cmp - actual4
>  '
> -
> +test_done
   ^^^^^^^^^

Why is this test_done here? There are additional tests after this point, and an additional call to test_done.

Show 22 quoted lines
>  test_expect_success 'git diff HEAD with dirty submodule (index, refs match)' '
>  	(
>  		cd sub &&
> @@ -129,7 +137,13 @@ test_expect_success 'git diff HEAD with dirty submodule (untracked, refs match)'
>  	git diff HEAD >actual &&
>  	sed -e "1,/^@@/d" actual >actual.body &&
>  	expect_from_to >expect.body $subprev $subprev-dirty &&
> -	test_cmp expect.body actual.body
> +	test_cmp expect.body actual.body &&
> +	git diff --ignore-submodules=all HEAD >actual2 &&
> +	echo -n "" | test_cmp - actual2 &&
> +	git diff --ignore-submodules=untracked HEAD >actual3 &&
> +	echo -n "" | test_cmp - actual3 &&
> +	git diff --ignore-submodules=dirty HEAD >actual4 &&
> +	echo -n "" | test_cmp - actual4
>  '
> 
>  test_expect_success 'git diff (empty submodule dir)' '
> diff --git a/t/t4041-diff-submodule-option.sh b/t/t4041-diff-submodule-option.sh
> index 019acb9..f44b906 100755
> --- a/t/t4041-diff-submodule-option.sh
> +++ b/t/t4041-diff-submodule-option.sh
Jens Lehmann· Jun 24, 2010, 17:11 UTC · re: Brandon Casey · lore

Re: [PATCH 2/2] Add optional parameters to the diff option "--ignore-submodules"

Am 24.06.2010 16:23, schrieb Brandon Casey:
Show 19 quoted lines
> On 06/08/2010 11:31 AM, Jens Lehmann wrote:
> 
> Two things...
> 
>> diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
>> index 83c1914..559b41e 100755
>> --- a/t/t4027-diff-submodule.sh
>> +++ b/t/t4027-diff-submodule.sh
>> @@ -103,9 +103,17 @@ test_expect_success 'git diff HEAD with dirty submodule (work tree, refs match)'
>>  	git diff HEAD >actual &&
>>  	sed -e "1,/^@@/d" actual >actual.body &&
>>  	expect_from_to >expect.body $subprev $subprev-dirty &&
>> -	test_cmp expect.body actual.body
>> +	test_cmp expect.body actual.body &&
>> +	git diff --ignore-submodules HEAD >actual2 &&
>> +	echo -n "" | test_cmp - actual2 &&
> 
> 'echo -n' is not portable (here, and below).  Please use printf
> instead.  Ditto for t4041
Thanks, I haven't been aware of that.
Show 13 quoted lines
>> +	git diff --ignore-submodules=untracked HEAD >actual3 &&
>> +	sed -e "1,/^@@/d" actual3 >actual3.body &&
>> +	expect_from_to >expect.body $subprev $subprev-dirty &&
>> +	test_cmp expect.body actual3.body &&
>> +	git diff --ignore-submodules=dirty HEAD >actual4 &&
>> +	echo -n "" | test_cmp - actual4
>>  '
>> -
>> +test_done
>    ^^^^^^^^^
> 
> Why is this test_done here?  There are additional tests after this point,
> and an additional call to test_done.

Grmph, this is a leftover from debugging :-( Thank you for spotting this one too!

As the patch introducing this is already in next, here is another one:
--------------------8<--------------------------
From: Jens Lehmann <Jens.Lehmann@web.de>
Date: Thu, 24 Jun 2010 19:01:27 +0200
Subject: [PATCH] t4027 & t4041: Fix use of "echo -n" and a premature "test_done"

Replace the non portable "echo -n" with "printf" and remove a premature "test_done".

Reported-by: Brandon Casey <brandon.casey.ctr@nrlssc.navy.mil>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
---
 t/t4027-diff-submodule.sh        |   12 ++++++------
 t/t4041-diff-submodule-option.sh |   14 +++++++-------
 2 files changed, 13 insertions(+), 13 deletions(-)
Show changes to 2 files +13 −13

t/t4027-diff-submodule.sh, t/t4041-diff-submodule-option.sh

diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index 559b41e..3f2e591 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -105,15 +105,15 @@ test_expect_success 'git diff HEAD with dirty submodule (work tree, refs match)'
 	expect_from_to >expect.body $subprev $subprev-dirty &&
 	test_cmp expect.body actual.body &&
 	git diff --ignore-submodules HEAD >actual2 &&
-	echo -n "" | test_cmp - actual2 &&
+	printf "" | test_cmp - actual2 &&
 	git diff --ignore-submodules=untracked HEAD >actual3 &&
 	sed -e "1,/^@@/d" actual3 >actual3.body &&
 	expect_from_to >expect.body $subprev $subprev-dirty &&
 	test_cmp expect.body actual3.body &&
 	git diff --ignore-submodules=dirty HEAD >actual4 &&
-	echo -n "" | test_cmp - actual4
+	printf "" | test_cmp - actual4
 '
-test_done
+
 test_expect_success 'git diff HEAD with dirty submodule (index, refs match)' '
 	(
 		cd sub &&
@@ -139,11 +139,11 @@ test_expect_success 'git diff HEAD with dirty submodule (untracked, refs match)'
 	expect_from_to >expect.body $subprev $subprev-dirty &&
 	test_cmp expect.body actual.body &&
 	git diff --ignore-submodules=all HEAD >actual2 &&
-	echo -n "" | test_cmp - actual2 &&
+	printf "" | test_cmp - actual2 &&
 	git diff --ignore-submodules=untracked HEAD >actual3 &&
-	echo -n "" | test_cmp - actual3 &&
+	printf "" | test_cmp - actual3 &&
 	git diff --ignore-submodules=dirty HEAD >actual4 &&
-	echo -n "" | test_cmp - actual4
+	printf "" | test_cmp - actual4
 '

 test_expect_success 'git diff (empty submodule dir)' '
diff --git a/t/t4041-diff-submodule-option.sh b/t/t4041-diff-submodule-option.sh
index f44b906..7600ed1 100755
--- a/t/t4041-diff-submodule-option.sh
+++ b/t/t4041-diff-submodule-option.sh
@@ -207,17 +207,17 @@ EOF

 test_expect_success 'submodule contains untracked content (untracked ignored)' "
 	git diff-index -p --ignore-submodules=untracked --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'submodule contains untracked content (dirty ignored)' "
 	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'submodule contains untracked content (all ignored)' "
 	git diff-index -p --ignore-submodules=all --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'submodule contains untracked and modifed content' "
@@ -240,13 +240,13 @@ EOF
 test_expect_success 'submodule contains untracked and modifed content (dirty ignored)' "
 	echo new > sm1/foo6 &&
 	git diff-index -p --ignore-submodules=dirty --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'submodule contains untracked and modifed content (all ignored)' "
 	echo new > sm1/foo6 &&
 	git diff-index -p --ignore-submodules --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'submodule contains modifed content' "
@@ -295,7 +295,7 @@ EOF

 test_expect_success 'modified submodule contains untracked content (all ignored)' "
 	git diff-index -p --ignore-submodules=all --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'modified submodule contains untracked and modifed content' "
@@ -331,7 +331,7 @@ EOF
 test_expect_success 'modified submodule contains untracked and modifed content (all ignored)' "
 	echo modification >> sm1/foo6 &&
 	git diff-index -p --ignore-submodules --submodule=log HEAD >actual &&
-	echo -n '' | diff actual -
+	printf '' | diff actual -
 "

 test_expect_success 'modified submodule contains modifed content' "
-- 
1.7.1.716.g6c3d8
Junio C Hamano· Jun 8, 2010, 17:28 UTC · re: Jens Lehmann · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Jens Lehmann <Jens.Lehmann@web.de> writes:
Show 41 quoted lines
> After thinking some time about peoples expectations and troubles
> with the recursive scanning of submodules, I came up with this:
>
> What about expanding the "--ignore-submodules" option of the git diff
> family with three parameters:
>
> --ignore-submodules=all : Same behavior as "--ignore-submodules",
>   submodules show never up as modified.
>
> --ignore-submodules=untracked : Don't consider submodules as modified
>   when they only contain untracked files, but do if the commits in the
>   superproject are different or tracked content is modified.
>
> --ignore-submodules=dirty : Don't consider submodules as modified
>   when their work tree is dirty, no matter why. This is the pre 1.7.0
>   behavior and doesn't recurse into submodules at all.
>
> The first patch is just a resend of a test case rename, the second
> contains the implementation.
>
>
> To make that more useful the default could be controlled by the
> .git/config or .gitmodules file. So you could have two submodules:
>
> [submodule "sub1"]
> 	path = sub1
> 	url = /home/me/sub1.git/
> 	ignore = dirty
> [submodule "sub2"]
> 	path = sub2
> 	url = /home/me/sub2.git/
> 	ignore = untracked
>
> Where sub1 will not be scanned for work tree modifications by "git
> diff" and "git status" and for sub2 any untracked files inside the
> submodule will be ignored. And then "git status" should learn the
> "--ignore-submodule" option too. With a fourth parameter "none" it
> would be possible to override the defaults from the configuration
> anyway you want.
>
> Opinions?

The above outline (I haven't looked at the code) sounds sane. We might even want to make untracked (or dirty) the default, if nothing is given from the command line nor configuration.

I don't see a reason for patch 1/2, though.
Thanks.
Jens Lehmann· Jun 8, 2010, 20:41 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Am 08.06.2010 19:28, schrieb Junio C Hamano:
> The above outline (I haven't looked at the code) sounds sane.  We might
> even want to make untracked (or dirty) the default, if nothing is given
> from the command line nor configuration.

Hm, as far as I interpreted the postings here, "none" should be a reasonable default for most users because it protects them from errors that are easy to make and hard to detect.

IIRC there are two main exceptions which want a different setting:
 - Huge submodules which take a substantial portion of time to scan may
   want "dirty" to avoid spending the extra time (at least until we have
   an inotfiy daemon ;-).
 - Submodules where the provider doesn't care or is not able to provide
   a proper .gitignore for other reasons want "untracked" to avoid the
   noise.
(Anything I missed?)

And these seem to fall into the category: If you are unlucky enough to have one of these included in your superproject, configure these to use "dirty" or "untracked" as appropriate. And AFAICT they are more the exception than the rule so I came up with this proposal of a per submodule configuration, so people won't have to turn off submodule checking for all submodules because of a single special one.

> I don't see a reason for patch 1/2, though.
Ok, I will drop that from the next round.
Johannes Schindelin· Jun 8, 2010, 21:02 UTC · re: Jens Lehmann · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Hi,
On Tue, 8 Jun 2010, Jens Lehmann wrote:
Show 14 quoted lines
> Am 08.06.2010 19:28, schrieb Junio C Hamano:
> > The above outline (I haven't looked at the code) sounds sane.  We might
> > even want to make untracked (or dirty) the default, if nothing is given
> > from the command line nor configuration.
> 
> Hm, as far as I interpreted the postings here, "none" should be a
> reasonable default for most users because it protects them from errors
> that are easy to make and hard to detect.
> 
> IIRC there are two main exceptions which want a different setting:
> 
>  - Huge submodules which take a substantial portion of time to scan may 
>    want "dirty" to avoid spending the extra time (at least until we have 
>    an inotfiy daemon ;-).

Not only huge submodules. Thrashing the disk cache (e.g. by the sheer number of submodules) is another reason. In my main project, the time for a simple "git status" went up from unnoticable (<0.1 seconds AFAICT) to 45 seconds.

Let me repeat that. Fourty-five seconds. For a simple "git status".

According to me, the fact that nobody noticed can only be explained by the lack of real submodule users. In particular, the people who invented the submodules, and pushed them into "official" Git, never used it, but precluded more appropriate solutions. I am not complaining, I am just stating observations.

It comes back to my argument that developers should not have beefy machines, lest they lose touch with the users and uses, and develop things that closely skirt reality just so.

>  - Submodules where the provider doesn't care or is not able to provide 
>    a proper .gitignore for other reasons want "untracked" to avoid the 
>    noise.

While this is a valid scenario, I do not think that any user was hurt by that case.

Show 6 quoted lines
> And these seem to fall into the category: If you are unlucky enough to 
> have one of these included in your superproject, configure these to use 
> "dirty" or "untracked" as appropriate. And AFAICT they are more the 
> exception than the rule so I came up with this proposal of a per 
> submodule configuration, so people won't have to turn off submodule 
> checking for all submodules because of a single special one.

I agree that the basic reason for the default to check for dirty and untracked files is sound.

It is not your (Jens') fault that this does not integrate well into the Git context, and that users of the submodule feature were punished that heavily (just to reiterate, if anybody missed that number, the time on "git status" -- or for that matter, "git diff" -- went up by >4500%. In English words, that is four-thousand five hundred percent, and that is just a lower bound).

So I would actually argue (being a real submodule user, not just an imaginary one) that the default for dirty checking in the submodules should stay.

> > I don't see a reason for patch 1/2, though.
> 
> Ok, I will drop that from the next round.

If I had not stopped caring about consistency in Git, I would strongly contest this dropping.

Ciao, Dscho

Junio C Hamano· Jun 8, 2010, 23:26 UTC · re: Johannes Schindelin · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 13 quoted lines
> I agree that the basic reason for the default to check for dirty and 
> untracked files is sound.
>
> It is not your (Jens') fault that this does not integrate well into the 
> Git context, and that users of the submodule feature were punished that 
> heavily (just to reiterate, if anybody missed that number, the time on 
> "git status" -- or for that matter, "git diff" -- went up by >4500%. In 
> English words, that is four-thousand five hundred percent, and that is 
> just a lower bound).
>
> So I would actually argue (being a real submodule user, not just an 
> imaginary one) that the default for dirty checking in the submodules 
> should stay.

Let me make sure I understand. You repeated three times that you suffered big time spending too many cycles due to extra checks (compared with older behaviour of not checking submodule working tree at all), but you would recommend the default to be the expensive one?

I am not trying to point out that these two statements are inconsistent. I am guessing that you are suggesting to "default to safe-but-expensive, let people who know what they are doing to tweak for performance", but you are as always a bit too terse for me to be sure, hence this question.

Johannes Schindelin· Jun 8, 2010, 23:36 UTC · re: Junio C Hamano · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Hi,
On Tue, 8 Jun 2010, Junio C Hamano wrote:
Show 20 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I agree that the basic reason for the default to check for dirty and 
> > untracked files is sound.
> >
> > It is not your (Jens') fault that this does not integrate well into the 
> > Git context, and that users of the submodule feature were punished that 
> > heavily (just to reiterate, if anybody missed that number, the time on 
> > "git status" -- or for that matter, "git diff" -- went up by >4500%. In 
> > English words, that is four-thousand five hundred percent, and that is 
> > just a lower bound).
> >
> > So I would actually argue (being a real submodule user, not just an 
> > imaginary one) that the default for dirty checking in the submodules 
> > should stay.
> 
> Let me make sure I understand.  You repeated three times that you suffered
> big time spending too many cycles due to extra checks (compared with older
> behaviour of not checking submodule working tree at all), but you would
> recommend the default to be the expensive one?
I also pointed out that the suffering came from power usage of submodules.

If you only have a handful of submodules, and if they are relatively small, the current default makes much more sense than --ignore-submodules=dirty

Hth, Johannes

Johan Herland· Jun 8, 2010, 22:11 UTC · re: Jens Lehmann · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

On Tuesday 08 June 2010, Jens Lehmann wrote:
Show 16 quoted lines
> After thinking some time about peoples expectations and troubles
> with the recursive scanning of submodules, I came up with this:
> 
> What about expanding the "--ignore-submodules" option of the git diff
> family with three parameters:
> 
> --ignore-submodules=all : Same behavior as "--ignore-submodules",
>   submodules show never up as modified.
> 
> --ignore-submodules=untracked : Don't consider submodules as modified
>   when they only contain untracked files, but do if the commits in the
>   superproject are different or tracked content is modified.
> 
> --ignore-submodules=dirty : Don't consider submodules as modified
>   when their work tree is dirty, no matter why. This is the pre 1.7.0
>   behavior and doesn't recurse into submodules at all.

Pardon my ignorance: Does this make "dirty" a superset of "untracked"? Or are they orthogonal. And how does "all" compare to "dirty"? Are they synonyms, or is "all" a superset of "dirty"?

Show 15 quoted lines
> To make that more useful the default could be controlled by the
> .git/config or .gitmodules file. So you could have two submodules:
> 
> [submodule "sub1"]
> 	path = sub1
> 	url = /home/me/sub1.git/
> 	ignore = dirty
> [submodule "sub2"]
> 	path = sub2
> 	url = /home/me/sub2.git/
> 	ignore = untracked
> 
> [...]
> 
> Opinions?

I agree with adding support for this in .gitmodules, to allow customizing on a per-submodule level.

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Jens Lehmann· Jun 8, 2010, 22:19 UTC · re: Johan Herland · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Am 09.06.2010 00:11, schrieb Johan Herland:
Show 21 quoted lines
> On Tuesday 08 June 2010, Jens Lehmann wrote:
>> After thinking some time about peoples expectations and troubles
>> with the recursive scanning of submodules, I came up with this:
>>
>> What about expanding the "--ignore-submodules" option of the git diff
>> family with three parameters:
>>
>> --ignore-submodules=all : Same behavior as "--ignore-submodules",
>>   submodules show never up as modified.
>>
>> --ignore-submodules=untracked : Don't consider submodules as modified
>>   when they only contain untracked files, but do if the commits in the
>>   superproject are different or tracked content is modified.
>>
>> --ignore-submodules=dirty : Don't consider submodules as modified
>>   when their work tree is dirty, no matter why. This is the pre 1.7.0
>>   behavior and doesn't recurse into submodules at all.
> 
> Pardon my ignorance: Does this make "dirty" a superset of "untracked"? Or 
> are they orthogonal. And how does "all" compare to "dirty"? Are they 
> synonyms, or is "all" a superset of "dirty"?

Sorry I didn't make that clear enough: "all" is a superset of "dirty" and "dirty" is a superset of "untracked".

There are currently (since 1.7.0) three reasons a submodule is considered dirty:

1) It contains untracked content
2) It contains modified tracked content
3) It contains newer commits than those committed in the superproject

"all" would ignore 1), 2) & 3) "dirty" would ignore 1) & 2) "untracked" would ignore 1)

> I agree with adding support for this in .gitmodules, to allow customizing on 
> a per-submodule level.
Yup, and different branches could handle that differently that way too.
Johan Herland· Jun 8, 2010, 23:49 UTC · re: Jens Lehmann · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

On Wednesday 09 June 2010, Jens Lehmann wrote:
Show 35 quoted lines
> Am 09.06.2010 00:11, schrieb Johan Herland:
> > On Tuesday 08 June 2010, Jens Lehmann wrote:
> >> After thinking some time about peoples expectations and troubles
> >> with the recursive scanning of submodules, I came up with this:
> >> 
> >> What about expanding the "--ignore-submodules" option of the git diff
> >> family with three parameters:
> >> 
> >> --ignore-submodules=all : Same behavior as "--ignore-submodules",
> >> 
> >>   submodules show never up as modified.
> >> 
> >> --ignore-submodules=untracked : Don't consider submodules as modified
> >> 
> >>   when they only contain untracked files, but do if the commits in the
> >>   superproject are different or tracked content is modified.
> >> 
> >> --ignore-submodules=dirty : Don't consider submodules as modified
> >> 
> >>   when their work tree is dirty, no matter why. This is the pre 1.7.0
> >>   behavior and doesn't recurse into submodules at all.
> > 
> > Pardon my ignorance: Does this make "dirty" a superset of "untracked"?
> > Or are they orthogonal. And how does "all" compare to "dirty"? Are
> > they synonyms, or is "all" a superset of "dirty"?
> 
> Sorry I didn't make that clear enough: "all" is a superset of "dirty" and
> "dirty" is a superset of "untracked".
> 
> There are currently (since 1.7.0) three reasons a submodule is considered
> dirty:
> 
> 1) It contains untracked content
> 2) It contains modified tracked content
> 3) It contains newer commits than those committed in the superproject

I guess 3) really means that the submodule's HEAD points to a _different_ (not necessarily _newer_) commit than what's referenced in the superproject.

> "all" would ignore 1), 2) & 3)
> "dirty" would ignore 1) & 2)
> "untracked" would ignore 1)

...and just to complete my understanding of this, 3) requires only checking the submodule's current HEAD, while 1) and 2) require traversing its work tree (i.e. the equivalent of a 'git status'), hence the potential expensiveness.

Also, I guess 2) includes both staged and unstaged modifications to tracked content?

Thanks for your help. All the ideas in your cover letter seem good to me.
...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net
Jens Lehmann· Jun 9, 2010, 06:23 UTC · re: Johan Herland · lore

Re: [PATCH 0/2] Add optional parameters to the diff option "--ignore-submodules"

Am 09.06.2010 01:49, schrieb Johan Herland:
Show 10 quoted lines
> On Wednesday 09 June 2010, Jens Lehmann wrote:
>> There are currently (since 1.7.0) three reasons a submodule is considered
>> dirty:
>>
>> 1) It contains untracked content
>> 2) It contains modified tracked content
>> 3) It contains newer commits than those committed in the superproject
> 
> I guess 3) really means that the submodule's HEAD points to a _different_ 
> (not necessarily _newer_) commit than what's referenced in the superproject.
Sure, please replace my inaccurate description with yours ;-)
Show 8 quoted lines
>> "all" would ignore 1), 2) & 3)
>> "dirty" would ignore 1) & 2)
>> "untracked" would ignore 1)
> 
> ...and just to complete my understanding of this, 3) requires only checking 
> the submodule's current HEAD, while 1) and 2) require traversing its work 
> tree (i.e. the equivalent of a 'git status'), hence the potential 
> expensiveness.
Thats correct.
> Also, I guess 2) includes both staged and unstaged modifications to tracked 
> content?

Yes (as it doesn't make a difference to the superproject if modifications inside the submodule are staged or not there is no distinction made between those two).

> Thanks for your help. All the ideas in your cover letter seem good to me.
Thank you for your comments, I'll start hacking to show some more patches.

← back to recent threads