threads / patch / 57932

patchrange-diff: show submodule changes irrespective of diff.submodule

Subject: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

## tl;dr

9 messages between May 30, 2022 and Jun 7, 2022. Diffs are folded; open one to read it.

replies: 8people: 4as markdown or json

Philippe Blain via GitGitGadget· May 30, 2022, 13:09 UTC · lore
From: Philippe Blain <levraiphilippeblain@gmail.com>

After generating diffs for each range to be compared using a 'git log' invocation, range-diff.c::read_patches looks for the "diff --git" header in those diffs to recognize the beginning of a new change.

In a project with submodules, and with 'diff.submodule=log' set in the config, this header is missing for the diff of a changed submodule, so any submodule changes are quietly ignored in the range-diff.

When 'diff.submodule=diff' is set in the config, the "diff --git" header is also missing for the submodule itself, but is shown for submodule content changes, which can easily confuse 'git range-diff' and lead to errors such as:

    error: git apply: bad git-diff - inconsistent old filename on line 1
    error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
    '
    error: could not parse log for '@{u}..@{1}'

Force the submodule diff format to its default ("short") when invoking 'git log' to generate the patches for each range, such that submodule changes are always shown.

Note that the test must use '--creation-factor=100' to force the second commit in the range not to be considered a complete rewrite.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
---
    range-diff: show submodule changes irrespective of diff.submodule
    
    This fixes a bug that I reported last summer [1].
    
    [1]
    https://lore.kernel.org/git/e469038c-d78c-cd4b-0214-7094746b9281@gmail.com/
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1244
 range-diff.c          |  2 +-
 t/t3206-range-diff.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 1 deletion(-)
Show changes to 2 files +45 −1

range-diff.c, t/t3206-range-diff.sh

diff --git a/range-diff.c b/range-diff.c
index b72eb9fdbee..068bf214544 100644
--- a/range-diff.c
+++ b/range-diff.c
@@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
 
 	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
 		     "--reverse", "--date-order", "--decorate=no",
-		     "--no-prefix",
+		     "--no-prefix", "--submodule=short",
 		     /*
 		      * Choose indicators that are not used anywhere
 		      * else in diffs, but still look reasonable
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e30bc48a290..ac848c42536 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -772,4 +772,48 @@ test_expect_success '--left-only/--right-only' '
 	test_cmp expect actual
 '
 
+test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
+	git init sub-repo &&
+	test_commit -C sub-repo sub-first &&
+	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
+	test_commit -C sub-repo sub-second &&
+	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
+	test_commit -C sub-repo sub-third &&
+	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
+
+	git checkout -b main-sub topic &&
+	git submodule add ./sub-repo sub &&
+	git -C sub checkout --detach sub-first &&
+	git add sub &&
+	git commit -m "add sub" &&
+	sup_oid1=$(git rev-parse --short HEAD) &&
+	git checkout -b topic-sub &&
+	git -C sub checkout sub-second &&
+	git add sub &&
+	git commit -m "change sub" &&
+	sup_oid2=$(git rev-parse --short HEAD) &&
+	git checkout -b modified-sub main-sub &&
+	git -C sub checkout sub-third &&
+	git add sub &&
+	git commit -m "change sub" &&
+	sup_oid3=$(git rev-parse --short HEAD) &&
+
+	test_config diff.submodule log &&
+	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
+	cat >expect <<-EOF &&
+	1:  $sup_oid1 = 1:  $sup_oid1 add sub
+	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
+	    @@ Commit message
+	      ## sub ##
+	     @@
+	     -Subproject commit $sub_oid1
+	    -+Subproject commit $sub_oid2
+	    ++Subproject commit $sub_oid3
+	EOF
+	test_cmp expect actual &&
+	test_config diff.submodule diff &&
+	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
+	test_cmp expect actual
+'
+
 test_done

base-commit: 7a3eb286977746bc09a5de7682df0e5a7085e17c
-- 
gitgitgadget
Ævar Arnfjörð Bjarmason· May 30, 2022, 13:46 UTC · re: Philippe Blain via GitGitGadget · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

On Mon, May 30 2022, Philippe Blain via GitGitGadget wrote:
Show 43 quoted lines
> From: Philippe Blain <levraiphilippeblain@gmail.com>
>
> After generating diffs for each range to be compared using a 'git log'
> invocation, range-diff.c::read_patches looks for the "diff --git" header
> in those diffs to recognize the beginning of a new change.
>
> In a project with submodules, and with 'diff.submodule=log' set in the
> config, this header is missing for the diff of a changed submodule, so
> any submodule changes are quietly ignored in the range-diff.
>
> When 'diff.submodule=diff' is set in the config, the "diff --git" header
> is also missing for the submodule itself, but is shown for submodule
> content changes, which can easily confuse 'git range-diff' and lead to
> errors such as:
>
>     error: git apply: bad git-diff - inconsistent old filename on line 1
>     error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
>     '
>     error: could not parse log for '@{u}..@{1}'
>
> Force the submodule diff format to its default ("short") when invoking
> 'git log' to generate the patches for each range, such that submodule
> changes are always shown.
>
> Note that the test must use '--creation-factor=100' to force the second
> commit in the range not to be considered a complete rewrite.
>
> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>     range-diff: show submodule changes irrespective of diff.submodule
>     
>     This fixes a bug that I reported last summer [1].
>     
>     [1]
>     https://lore.kernel.org/git/e469038c-d78c-cd4b-0214-7094746b9281@gmail.com/
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1244
>
>  range-diff.c          |  2 +-
>  t/t3206-range-diff.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 45 insertions(+), 1 deletion(-)

Thanks for picking this up again, and nice to have a test on this iteration!

Show 65 quoted lines
> diff --git a/range-diff.c b/range-diff.c
> index b72eb9fdbee..068bf214544 100644
> --- a/range-diff.c
> +++ b/range-diff.c
> @@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
>  
>  	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
>  		     "--reverse", "--date-order", "--decorate=no",
> -		     "--no-prefix",
> +		     "--no-prefix", "--submodule=short",
>  		     /*
>  		      * Choose indicators that are not used anywhere
>  		      * else in diffs, but still look reasonable
> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> index e30bc48a290..ac848c42536 100755
> --- a/t/t3206-range-diff.sh
> +++ b/t/t3206-range-diff.sh
> @@ -772,4 +772,48 @@ test_expect_success '--left-only/--right-only' '
>  	test_cmp expect actual
>  '
>  
> +test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
> +	git init sub-repo &&
> +	test_commit -C sub-repo sub-first &&
> +	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
> +	test_commit -C sub-repo sub-second &&
> +	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
> +	test_commit -C sub-repo sub-third &&
> +	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
> +
> +	git checkout -b main-sub topic &&
> +	git submodule add ./sub-repo sub &&
> +	git -C sub checkout --detach sub-first &&
> +	git add sub &&
> +	git commit -m "add sub" &&
> +	sup_oid1=$(git rev-parse --short HEAD) &&
> +	git checkout -b topic-sub &&
> +	git -C sub checkout sub-second &&
> +	git add sub &&
> +	git commit -m "change sub" &&
> +	sup_oid2=$(git rev-parse --short HEAD) &&
> +	git checkout -b modified-sub main-sub &&
> +	git -C sub checkout sub-third &&
> +	git add sub &&
> +	git commit -m "change sub" &&
> +	sup_oid3=$(git rev-parse --short HEAD) &&
> +
> +	test_config diff.submodule log &&
> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
> +	cat >expect <<-EOF &&
> +	1:  $sup_oid1 = 1:  $sup_oid1 add sub
> +	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
> +	    @@ Commit message
> +	      ## sub ##
> +	     @@
> +	     -Subproject commit $sub_oid1
> +	    -+Subproject commit $sub_oid2
> +	    ++Subproject commit $sub_oid3
> +	EOF
> +	test_cmp expect actual &&
> +	test_config diff.submodule diff &&
> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
> +	test_cmp expect actual
> +'
> +

I'd find this much easier to follow if this were a two-part where we do most of this test code in the 1st commit, and assert the current (failing) behavior with a test_expect_failure.

Then this commit would narrowly be the bugfix itself.

I also see that the --creation-factor=100 isn't necessary and seems somewhat orthagonal, i.e. we'd like to test this *without* that option and see how we behave, i.e. we'll emit the "full replacement".

Why not compare the output without --creation-factor=100, and then just have another --creation-factor=100 test to show what we emit if we "look into" those commits and diff their contents?

Philippe Blain· May 31, 2022, 02:22 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Hi Ævar,
Le 2022-05-30 à 09:46, Ævar Arnfjörð Bjarmason a écrit :
Show 49 quoted lines
> 
> On Mon, May 30 2022, Philippe Blain via GitGitGadget wrote:
> 
>> From: Philippe Blain <levraiphilippeblain@gmail.com>
>>
>> After generating diffs for each range to be compared using a 'git log'
>> invocation, range-diff.c::read_patches looks for the "diff --git" header
>> in those diffs to recognize the beginning of a new change.
>>
>> In a project with submodules, and with 'diff.submodule=log' set in the
>> config, this header is missing for the diff of a changed submodule, so
>> any submodule changes are quietly ignored in the range-diff.
>>
>> When 'diff.submodule=diff' is set in the config, the "diff --git" header
>> is also missing for the submodule itself, but is shown for submodule
>> content changes, which can easily confuse 'git range-diff' and lead to
>> errors such as:
>>
>>     error: git apply: bad git-diff - inconsistent old filename on line 1
>>     error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
>>     '
>>     error: could not parse log for '@{u}..@{1}'
>>
>> Force the submodule diff format to its default ("short") when invoking
>> 'git log' to generate the patches for each range, such that submodule
>> changes are always shown.
>>
>> Note that the test must use '--creation-factor=100' to force the second
>> commit in the range not to be considered a complete rewrite.
>>
>> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
>> ---
>>     range-diff: show submodule changes irrespective of diff.submodule
>>     
>>     This fixes a bug that I reported last summer [1].
>>     
>>     [1]
>>     https://lore.kernel.org/git/e469038c-d78c-cd4b-0214-7094746b9281@gmail.com/
>>
>> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v1
>> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v1
>> Pull-Request: https://github.com/gitgitgadget/git/pull/1244
>>
>>  range-diff.c          |  2 +-
>>  t/t3206-range-diff.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 45 insertions(+), 1 deletion(-)
> 
> Thanks for picking this up again, and nice to have a test on this
> iteration!

Well, what I sent last summer was really just a bug report with a "I think this should fix it" diff, in case anybody wanted to pick it up. It was not sent as a patch, I would not send a patch fixing a bug without correponding tests ;)

Show 72 quoted lines
> 
>> diff --git a/range-diff.c b/range-diff.c
>> index b72eb9fdbee..068bf214544 100644
>> --- a/range-diff.c
>> +++ b/range-diff.c
>> @@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
>>  
>>  	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
>>  		     "--reverse", "--date-order", "--decorate=no",
>> -		     "--no-prefix",
>> +		     "--no-prefix", "--submodule=short",
>>  		     /*
>>  		      * Choose indicators that are not used anywhere
>>  		      * else in diffs, but still look reasonable
>> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> index e30bc48a290..ac848c42536 100755
>> --- a/t/t3206-range-diff.sh
>> +++ b/t/t3206-range-diff.sh
>> @@ -772,4 +772,48 @@ test_expect_success '--left-only/--right-only' '
>>  	test_cmp expect actual
>>  '
>>  
>> +test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
>> +	git init sub-repo &&
>> +	test_commit -C sub-repo sub-first &&
>> +	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
>> +	test_commit -C sub-repo sub-second &&
>> +	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
>> +	test_commit -C sub-repo sub-third &&
>> +	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
>> +
>> +	git checkout -b main-sub topic &&
>> +	git submodule add ./sub-repo sub &&
>> +	git -C sub checkout --detach sub-first &&
>> +	git add sub &&
>> +	git commit -m "add sub" &&
>> +	sup_oid1=$(git rev-parse --short HEAD) &&
>> +	git checkout -b topic-sub &&
>> +	git -C sub checkout sub-second &&
>> +	git add sub &&
>> +	git commit -m "change sub" &&
>> +	sup_oid2=$(git rev-parse --short HEAD) &&
>> +	git checkout -b modified-sub main-sub &&
>> +	git -C sub checkout sub-third &&
>> +	git add sub &&
>> +	git commit -m "change sub" &&
>> +	sup_oid3=$(git rev-parse --short HEAD) &&
>> +
>> +	test_config diff.submodule log &&
>> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
>> +	cat >expect <<-EOF &&
>> +	1:  $sup_oid1 = 1:  $sup_oid1 add sub
>> +	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
>> +	    @@ Commit message
>> +	      ## sub ##
>> +	     @@
>> +	     -Subproject commit $sub_oid1
>> +	    -+Subproject commit $sub_oid2
>> +	    ++Subproject commit $sub_oid3
>> +	EOF
>> +	test_cmp expect actual &&
>> +	test_config diff.submodule diff &&
>> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
>> +	test_cmp expect actual
>> +'
>> +
> 
> I'd find this much easier to follow if this were a two-part where we do
> most of this test code in the 1st commit, and assert the current
> (failing) behavior with a test_expect_failure.
> 
> Then this commit would narrowly be the bugfix itself.

I agree that I would prefer if it were the norm to do it this way in this project. This way you can check that the test you are adding actually fails without the fix, not only that it passes with the fix. This is what I usually do locally before sending.

However Junio has expressed his view about that approach before [1] and he prefers to see regression / bug fixes as a single commit with code and tests. So that's why I sent this as a single commit.

Show 9 quoted lines
> 
> I also see that the --creation-factor=100 isn't necessary and seems
> somewhat orthagonal, i.e. we'd like to test this *without* that option
> and see how we behave, i.e. we'll emit the "full replacement".
> 
> Why not compare the output without --creation-factor=100, and then just
> have another --creation-factor=100 test to show what we emit if we "look
> into" those commits and diff their contents?
> 

OK, I think this makes sense. It's true that even without '--creation-factor=100', with '-c diff.submodule=log' we end up showing the second commit as an exact match in both ranges, which is wrong. I'll add that to the test.

Thanks for taking a look,
Philippe.
[1] https://lore.kernel.org/git/37DD13D4-FBE4-4DB7-85F5-824E850BA9AE@gmail.com/
Johannes Schindelin· Jun 2, 2022, 15:36 UTC · re: Philippe Blain via GitGitGadget · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Hi Philippe,
On Mon, 30 May 2022, Philippe Blain via GitGitGadget wrote:
Show 9 quoted lines
> From: Philippe Blain <levraiphilippeblain@gmail.com>
>
> After generating diffs for each range to be compared using a 'git log'
> invocation, range-diff.c::read_patches looks for the "diff --git" header
> in those diffs to recognize the beginning of a new change.
>
> In a project with submodules, and with 'diff.submodule=log' set in the
> config, this header is missing for the diff of a changed submodule, so
> any submodule changes are quietly ignored in the range-diff.

This means that we can go two ways here: either we explicitly disable `diff.submodule` for the invocation that is spawned from `range-diff`, or we allow it but then handle the diff header as expected.

Show 14 quoted lines
>
> When 'diff.submodule=diff' is set in the config, the "diff --git" header
> is also missing for the submodule itself, but is shown for submodule
> content changes, which can easily confuse 'git range-diff' and lead to
> errors such as:
>
>     error: git apply: bad git-diff - inconsistent old filename on line 1
>     error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
>     '
>     error: could not parse log for '@{u}..@{1}'
>
> Force the submodule diff format to its default ("short") when invoking
> 'git log' to generate the patches for each range, such that submodule
> changes are always shown.

Full disclosure: I do not see much value in range-diffs in the presence of submodules. Nothing in the design of range-diffs is prepared for submodules.

But since `--submodules=short` does not change anything when running `range-diff` in repositories without submodules, I don't mind this change.

>
> Note that the test must use '--creation-factor=100' to force the second
> commit in the range not to be considered a complete rewrite.
Thank you for this considerate note!
Show 28 quoted lines
>
> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
> ---
>     range-diff: show submodule changes irrespective of diff.submodule
>
>     This fixes a bug that I reported last summer [1].
>
>     [1]
>     https://lore.kernel.org/git/e469038c-d78c-cd4b-0214-7094746b9281@gmail.com/
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1244
>
>  range-diff.c          |  2 +-
>  t/t3206-range-diff.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 45 insertions(+), 1 deletion(-)
>
> diff --git a/range-diff.c b/range-diff.c
> index b72eb9fdbee..068bf214544 100644
> --- a/range-diff.c
> +++ b/range-diff.c
> @@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
>
>  	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
>  		     "--reverse", "--date-order", "--decorate=no",
> -		     "--no-prefix",
> +		     "--no-prefix", "--submodule=short",

As I mentioned above, since this does not change anything in the intended scenarios (i.e. without submodules), I am fine with it.

Show 25 quoted lines
>  		     /*
>  		      * Choose indicators that are not used anywhere
>  		      * else in diffs, but still look reasonable
> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
> index e30bc48a290..ac848c42536 100755
> --- a/t/t3206-range-diff.sh
> +++ b/t/t3206-range-diff.sh
> @@ -772,4 +772,48 @@ test_expect_success '--left-only/--right-only' '
>  	test_cmp expect actual
>  '
>
> +test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
> +	git init sub-repo &&
> +	test_commit -C sub-repo sub-first &&
> +	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
> +	test_commit -C sub-repo sub-second &&
> +	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
> +	test_commit -C sub-repo sub-third &&
> +	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
> +
> +	git checkout -b main-sub topic &&
> +	git submodule add ./sub-repo sub &&
> +	git -C sub checkout --detach sub-first &&
> +	git add sub &&
> +	git commit -m "add sub" &&

Just a suggestion: use `git commit -m sub-first sub` instead (one `git` invocation instead of two).

Show 7 quoted lines
> +	sup_oid1=$(git rev-parse --short HEAD) &&
> +	git checkout -b topic-sub &&
> +	git -C sub checkout sub-second &&
> +	git add sub &&
> +	git commit -m "change sub" &&
> +	sup_oid2=$(git rev-parse --short HEAD) &&
> +	git checkout -b modified-sub main-sub &&

Another suggestion: instead of naming the branches, use the `sup_oid*` variables directly.

Show 22 quoted lines
> +	git -C sub checkout sub-third &&
> +	git add sub &&
> +	git commit -m "change sub" &&
> +	sup_oid3=$(git rev-parse --short HEAD) &&
> +
> +	test_config diff.submodule log &&
> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
> +	cat >expect <<-EOF &&
> +	1:  $sup_oid1 = 1:  $sup_oid1 add sub
> +	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
> +	    @@ Commit message
> +	      ## sub ##
> +	     @@
> +	     -Subproject commit $sub_oid1
> +	    -+Subproject commit $sub_oid2
> +	    ++Subproject commit $sub_oid3
> +	EOF
> +	test_cmp expect actual &&
> +	test_config diff.submodule diff &&
> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
> +	test_cmp expect actual
> +'

This test case is very clear and concise, even without my suggested changes. Therefore, if you want to keep the patch as-is, I am fine with that, too.

Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Thank you, Dscho

Show 7 quoted lines
> +
>  test_done
>
> base-commit: 7a3eb286977746bc09a5de7682df0e5a7085e17c
> --
> gitgitgadget
>
Junio C Hamano· Jun 2, 2022, 17:36 UTC · re: Johannes Schindelin · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 10 quoted lines
>> Force the submodule diff format to its default ("short") when invoking
>> 'git log' to generate the patches for each range, such that submodule
>> changes are always shown.
>
> Full disclosure: I do not see much value in range-diffs in the presence of
> submodules. Nothing in the design of range-diffs is prepared for
> submodules.
>
> But since `--submodules=short` does not change anything when running
> `range-diff` in repositories without submodules, I don't mind this change.

IOW, "I wrote it for the purpose of doing X, I do not care those who have been using it for doing Y, I am OK with changing behaviour on them".

Philippe, do you have a good guess on other users and workflows that may benefit from the current behaviour? I suspect in the longer term this might have to become configurable, and I am having a hard time judging if (1) a temporary regression (to them) is acceptable or (2) the new feature to also show submodule changes is not urgent enough that it may be better to make it configurable from day one, instead of using a different hardcoded and only setting like this patch does.

Show 5 quoted lines
> This test case is very clear and concise, even without my suggested
> changes. Therefore, if you want to keep the patch as-is, I am fine with
> that, too.
>
> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Thanks for a review.
Will queue.
Philippe Blain· Jun 6, 2022, 20:33 UTC · re: Junio C Hamano · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Hi Junio,
Le 2022-06-02 à 13:36, Junio C Hamano a écrit :
Show 25 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
>>> Force the submodule diff format to its default ("short") when invoking
>>> 'git log' to generate the patches for each range, such that submodule
>>> changes are always shown.
>>
>> Full disclosure: I do not see much value in range-diffs in the presence of
>> submodules. Nothing in the design of range-diffs is prepared for
>> submodules.
>>
>> But since `--submodules=short` does not change anything when running
>> `range-diff` in repositories without submodules, I don't mind this change.
> 
> IOW, "I wrote it for the purpose of doing X, I do not care those who
> have been using it for doing Y, I am OK with changing behaviour on
> them".
> 
> Philippe, do you have a good guess on other users and workflows that
> may benefit from the current behaviour?  I suspect in the longer term
> this might have to become configurable, and I am having a hard time
> judging if (1) a temporary regression (to them) is acceptable or (2)
> the new feature to also show submodule changes is not urgent enough
> that it may be better to make it configurable from day one, instead
> of using a different hardcoded and only setting like this patch does.
> 

Just to be clear: the "out of the box" behaviour (i.e. nothing in the config) is correct, i.e. submodule changes are detected and shown by 'git range-diff'.

It's only if someone has 'diff.submodule={log,diff}' in their config that submodule changes are quietly ignored (log) or might crash 'git range-diff' (diff). So I do not think of any user or workflow that benefit from the current behaviour, no. If you have diff.submodule={log,diff} set in your config, it's most probably because you work in projects that involve submodules and you do care about submodule changes. So having these changes "hidden" by range-diff (or having range-diff crash) just because the output format of 'git -c diff.submodule={log,diff} log' does not use a 'diff --git' header for submodules is really not expected. So I do not think we need to make that configurable. I think hardcoding '--submodule=short' is an easy fix and a good first step in making 'git range-diff' more useful for submodule users.

Thanks,
Philippe.
Junio C Hamano· Jun 7, 2022, 01:26 UTC · re: Philippe Blain · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Philippe Blain <levraiphilippeblain@gmail.com> writes:
Show 16 quoted lines
> Just to be clear: the "out of the box" behaviour (i.e. nothing in the config)
> is correct, i.e. submodule changes are detected and shown by 'git range-diff'.
>
> It's only if someone has 'diff.submodule={log,diff}' in their
> config that submodule changes are quietly ignored (log) or might
> crash 'git range-diff' (diff). So I do not think of any user or
> workflow that benefit from the current behaviour, no. If you have
> diff.submodule={log,diff} set in your config, it's most probably
> because you work in projects that involve submodules and you do
> care about submodule changes. So having these changes "hidden" by
> range-diff (or having range-diff crash) just because the output
> format of 'git -c diff.submodule={log,diff} log' does not use a
> 'diff --git' header for submodules is really not expected. So I do
> not think we need to make that configurable. I think hardcoding
> '--submodule=short' is an easy fix and a good first step in making
> 'git range-diff' more useful for submodule users.

OK. As "diff.submodule=none" does not exist, hardcoding "short" would not hurt anybody, I agree.

Thanks.
Philippe Blain· Jun 6, 2022, 20:18 UTC · re: Johannes Schindelin · lore

Re: [PATCH] range-diff: show submodule changes irrespective of diff.submodule

Hi Dscho,
Le 2022-06-02 à 11:36, Johannes Schindelin a écrit :
Show 106 quoted lines
> Hi Philippe,
> 
> On Mon, 30 May 2022, Philippe Blain via GitGitGadget wrote:
> 
>> From: Philippe Blain <levraiphilippeblain@gmail.com>
>>
>> After generating diffs for each range to be compared using a 'git log'
>> invocation, range-diff.c::read_patches looks for the "diff --git" header
>> in those diffs to recognize the beginning of a new change.
>>
>> In a project with submodules, and with 'diff.submodule=log' set in the
>> config, this header is missing for the diff of a changed submodule, so
>> any submodule changes are quietly ignored in the range-diff.
> 
> This means that we can go two ways here: either we explicitly disable
> `diff.submodule` for the invocation that is spawned from `range-diff`, or
> we allow it but then handle the diff header as expected.
> 
>>
>> When 'diff.submodule=diff' is set in the config, the "diff --git" header
>> is also missing for the submodule itself, but is shown for submodule
>> content changes, which can easily confuse 'git range-diff' and lead to
>> errors such as:
>>
>>     error: git apply: bad git-diff - inconsistent old filename on line 1
>>     error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
>>     '
>>     error: could not parse log for '@{u}..@{1}'
>>
>> Force the submodule diff format to its default ("short") when invoking
>> 'git log' to generate the patches for each range, such that submodule
>> changes are always shown.
> 
> Full disclosure: I do not see much value in range-diffs in the presence of
> submodules. Nothing in the design of range-diffs is prepared for
> submodules.
> 
> But since `--submodules=short` does not change anything when running
> `range-diff` in repositories without submodules, I don't mind this change.
> 
>>
>> Note that the test must use '--creation-factor=100' to force the second
>> commit in the range not to be considered a complete rewrite.
> 
> Thank you for this considerate note!
> 
>>
>> Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
>> ---
>>     range-diff: show submodule changes irrespective of diff.submodule
>>
>>     This fixes a bug that I reported last summer [1].
>>
>>     [1]
>>     https://lore.kernel.org/git/e469038c-d78c-cd4b-0214-7094746b9281@gmail.com/
>>
>> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v1
>> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v1
>> Pull-Request: https://github.com/gitgitgadget/git/pull/1244
>>
>>  range-diff.c          |  2 +-
>>  t/t3206-range-diff.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 45 insertions(+), 1 deletion(-)
>>
>> diff --git a/range-diff.c b/range-diff.c
>> index b72eb9fdbee..068bf214544 100644
>> --- a/range-diff.c
>> +++ b/range-diff.c
>> @@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
>>
>>  	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
>>  		     "--reverse", "--date-order", "--decorate=no",
>> -		     "--no-prefix",
>> +		     "--no-prefix", "--submodule=short",
> 
> As I mentioned above, since this does not change anything in the intended
> scenarios (i.e. without submodules), I am fine with it.
> 
>>  		     /*
>>  		      * Choose indicators that are not used anywhere
>>  		      * else in diffs, but still look reasonable
>> diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
>> index e30bc48a290..ac848c42536 100755
>> --- a/t/t3206-range-diff.sh
>> +++ b/t/t3206-range-diff.sh
>> @@ -772,4 +772,48 @@ test_expect_success '--left-only/--right-only' '
>>  	test_cmp expect actual
>>  '
>>
>> +test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
>> +	git init sub-repo &&
>> +	test_commit -C sub-repo sub-first &&
>> +	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
>> +	test_commit -C sub-repo sub-second &&
>> +	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
>> +	test_commit -C sub-repo sub-third &&
>> +	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
>> +
>> +	git checkout -b main-sub topic &&
>> +	git submodule add ./sub-repo sub &&
>> +	git -C sub checkout --detach sub-first &&
>> +	git add sub &&
>> +	git commit -m "add sub" &&
> 
> Just a suggestion: use `git commit -m sub-first sub` instead (one `git`
> invocation instead of two).
OK, good idea. I'll tweak that.
Show 12 quoted lines
> 
>> +	sup_oid1=$(git rev-parse --short HEAD) &&
>> +	git checkout -b topic-sub &&
>> +	git -C sub checkout sub-second &&
>> +	git add sub &&
>> +	git commit -m "change sub" &&
>> +	sup_oid2=$(git rev-parse --short HEAD) &&
>> +	git checkout -b modified-sub main-sub &&
> 
> Another suggestion: instead of naming the branches, use the `sup_oid*`
> variables directly.
> 

I think I like the branch names, they make the test closer to a "real-life" scenario (in my opinion). So I think I'll keep them, since you write later in your reply that you do not mind that much.

Show 30 quoted lines
>> +	git -C sub checkout sub-third &&
>> +	git add sub &&
>> +	git commit -m "change sub" &&
>> +	sup_oid3=$(git rev-parse --short HEAD) &&
>> +
>> +	test_config diff.submodule log &&
>> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
>> +	cat >expect <<-EOF &&
>> +	1:  $sup_oid1 = 1:  $sup_oid1 add sub
>> +	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
>> +	    @@ Commit message
>> +	      ## sub ##
>> +	     @@
>> +	     -Subproject commit $sub_oid1
>> +	    -+Subproject commit $sub_oid2
>> +	    ++Subproject commit $sub_oid3
>> +	EOF
>> +	test_cmp expect actual &&
>> +	test_config diff.submodule diff &&
>> +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
>> +	test_cmp expect actual
>> +'
> 
> This test case is very clear and concise, even without my suggested
> changes. Therefore, if you want to keep the patch as-is, I am fine with
> that, too.
> 
> Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> Thank you,
Thanks,
Philippe.	
Philippe Blain via GitGitGadget· Jun 6, 2022, 20:59 UTC · re: Philippe Blain via GitGitGadget · lore

[PATCH v2] range-diff: show submodule changes irrespective of diff.submodule

From: Philippe Blain <levraiphilippeblain@gmail.com>

After generating diffs for each range to be compared using a 'git log' invocation, range-diff.c::read_patches looks for the "diff --git" header in those diffs to recognize the beginning of a new change.

In a project with submodules, and with 'diff.submodule=log' set in the config, this header is missing for the diff of a changed submodule, so any submodule changes are quietly ignored in the range-diff.

When 'diff.submodule=diff' is set in the config, the "diff --git" header is also missing for the submodule itself, but is shown for submodule content changes, which can easily confuse 'git range-diff' and lead to errors such as:

    error: git apply: bad git-diff - inconsistent old filename on line 1
    error: could not parse git header 'diff --git path/to/submodule/and/some/file/within
    '
    error: could not parse log for '@{u}..@{1}'

Force the submodule diff format to its default ("short") when invoking 'git log' to generate the patches for each range, such that submodule changes are always detected.

Add a test, including an invocation with '--creation-factor=100' to force the second commit in the range not to be considered a complete rewrite, in order to verify we do indeed get the "short" format.

Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
---
    range-diff: show submodule changes irrespective of diff.submodule
    
    Changes since v1:
    
     * added a comparison without '--creation-factor' to the test, as
       suggested by Ævar
     * remove separate 'git add sub' invocations in favor of 'git commit -m
       msg sub', as suggested by Dscho
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1244%2Fphil-blain%2Frange-diff-submodule-diff-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1244/phil-blain/range-diff-submodule-diff-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1244
Range-diff vs v1:
 1:  4ae11694ac6 ! 1:  1982b720a04 range-diff: show submodule changes irrespective of diff.submodule
     @@ Commit message
      
          Force the submodule diff format to its default ("short") when invoking
          'git log' to generate the patches for each range, such that submodule
     -    changes are always shown.
     +    changes are always detected.
      
     -    Note that the test must use '--creation-factor=100' to force the second
     -    commit in the range not to be considered a complete rewrite.
     +    Add a test, including an invocation with '--creation-factor=100' to
     +    force the second commit in the range not to be considered a complete
     +    rewrite, in order to verify we do indeed get the "short" format.
      
          Signed-off-by: Philippe Blain <levraiphilippeblain@gmail.com>
      
     @@ t/t3206-range-diff.sh: test_expect_success '--left-only/--right-only' '
      +	git checkout -b main-sub topic &&
      +	git submodule add ./sub-repo sub &&
      +	git -C sub checkout --detach sub-first &&
     -+	git add sub &&
     -+	git commit -m "add sub" &&
     ++	git commit -m "add sub" sub &&
      +	sup_oid1=$(git rev-parse --short HEAD) &&
      +	git checkout -b topic-sub &&
      +	git -C sub checkout sub-second &&
     -+	git add sub &&
     -+	git commit -m "change sub" &&
     ++	git commit -m "change sub" sub &&
      +	sup_oid2=$(git rev-parse --short HEAD) &&
      +	git checkout -b modified-sub main-sub &&
      +	git -C sub checkout sub-third &&
     -+	git add sub &&
     -+	git commit -m "change sub" &&
     ++	git commit -m "change sub" sub &&
      +	sup_oid3=$(git rev-parse --short HEAD) &&
     ++	sup_oid0=$(test_oid __) &&
      +
      +	test_config diff.submodule log &&
     ++	git range-diff topic topic-sub modified-sub >actual &&
     ++	cat >expect <<-EOF &&
     ++	1:  $sup_oid1 = 1:  $sup_oid1 add sub
     ++	2:  $sup_oid2 < -:  $sup_oid0 change sub
     ++	-:  $sup_oid0 > 2:  $sup_oid3 change sub
     ++	EOF
     ++	test_cmp expect actual &&
     ++	test_config diff.submodule diff &&
     ++	git range-diff topic topic-sub modified-sub >actual &&
      +	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
      +	cat >expect <<-EOF &&
      +	1:  $sup_oid1 = 1:  $sup_oid1 add sub
 range-diff.c          |  2 +-
 t/t3206-range-diff.sh | 51 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 52 insertions(+), 1 deletion(-)
Show changes to 2 files +52 −1

range-diff.c, t/t3206-range-diff.sh

diff --git a/range-diff.c b/range-diff.c
index b72eb9fdbee..068bf214544 100644
--- a/range-diff.c
+++ b/range-diff.c
@@ -44,7 +44,7 @@ static int read_patches(const char *range, struct string_list *list,
 
 	strvec_pushl(&cp.args, "log", "--no-color", "-p", "--no-merges",
 		     "--reverse", "--date-order", "--decorate=no",
-		     "--no-prefix",
+		     "--no-prefix", "--submodule=short",
 		     /*
 		      * Choose indicators that are not used anywhere
 		      * else in diffs, but still look reasonable
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index e30bc48a290..d12e4e4cc6c 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -772,4 +772,55 @@ test_expect_success '--left-only/--right-only' '
 	test_cmp expect actual
 '
 
+test_expect_success 'submodule changes are shown irrespective of diff.submodule' '
+	git init sub-repo &&
+	test_commit -C sub-repo sub-first &&
+	sub_oid1=$(git -C sub-repo rev-parse HEAD) &&
+	test_commit -C sub-repo sub-second &&
+	sub_oid2=$(git -C sub-repo rev-parse HEAD) &&
+	test_commit -C sub-repo sub-third &&
+	sub_oid3=$(git -C sub-repo rev-parse HEAD) &&
+
+	git checkout -b main-sub topic &&
+	git submodule add ./sub-repo sub &&
+	git -C sub checkout --detach sub-first &&
+	git commit -m "add sub" sub &&
+	sup_oid1=$(git rev-parse --short HEAD) &&
+	git checkout -b topic-sub &&
+	git -C sub checkout sub-second &&
+	git commit -m "change sub" sub &&
+	sup_oid2=$(git rev-parse --short HEAD) &&
+	git checkout -b modified-sub main-sub &&
+	git -C sub checkout sub-third &&
+	git commit -m "change sub" sub &&
+	sup_oid3=$(git rev-parse --short HEAD) &&
+	sup_oid0=$(test_oid __) &&
+
+	test_config diff.submodule log &&
+	git range-diff topic topic-sub modified-sub >actual &&
+	cat >expect <<-EOF &&
+	1:  $sup_oid1 = 1:  $sup_oid1 add sub
+	2:  $sup_oid2 < -:  $sup_oid0 change sub
+	-:  $sup_oid0 > 2:  $sup_oid3 change sub
+	EOF
+	test_cmp expect actual &&
+	test_config diff.submodule diff &&
+	git range-diff topic topic-sub modified-sub >actual &&
+	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
+	cat >expect <<-EOF &&
+	1:  $sup_oid1 = 1:  $sup_oid1 add sub
+	2:  $sup_oid2 ! 2:  $sup_oid3 change sub
+	    @@ Commit message
+	      ## sub ##
+	     @@
+	     -Subproject commit $sub_oid1
+	    -+Subproject commit $sub_oid2
+	    ++Subproject commit $sub_oid3
+	EOF
+	test_cmp expect actual &&
+	test_config diff.submodule diff &&
+	git range-diff --creation-factor=100 topic topic-sub modified-sub >actual &&
+	test_cmp expect actual
+'
+
 test_done

base-commit: 7a3eb286977746bc09a5de7682df0e5a7085e17c
-- 
gitgitgadget

← back to recent threads