threads / discuss / 47099

Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

Subject: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

## tl;dr

9 messages between Nov 2, 2017 and Nov 6, 2017.

replies: 8people: 4as markdown or json

Orgad Shaneh· Nov 2, 2017, 08:30 UTC · lore
I can't reproduce this with a minimal example, but it happens in my project.

What I tried to do for reproducing is: rm -rf super sub mkdir sub; cd sub; git init git commit --allow-empty -m 'Initial commit' mkdir ../super; cd ../super git init git submodule add ../sub touch foo; git add foo sub git commit -m 'Initial commit' touch a; git add a; git commit -m 'a' touch b; git add b; git commit -m 'b' cd sub; git commit --allow-empty -m 'New commit'; cd .. git rebase -i HEAD^^

Then drop a.

In my project I get: error: cannot rebase: You have unstaged changes.

This works fine with 2.14.3.
- Orgad
Stefan Beller· Nov 2, 2017, 18:34 UTC · re: Orgad Shaneh · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On Thu, Nov 2, 2017 at 1:30 AM, Orgad Shaneh <orgads@gmail.com> wrote:
Show 22 quoted lines
> I can't reproduce this with a minimal example, but it happens in my project.
>
> What I tried to do for reproducing is:
> rm -rf super sub
> mkdir sub; cd sub; git init
> git commit --allow-empty -m 'Initial commit'
> mkdir ../super; cd ../super
> git init
> git submodule add ../sub
> touch foo; git add foo sub
> git commit -m 'Initial commit'
> touch a; git add a; git commit -m 'a'
> touch b; git add b; git commit -m 'b'
> cd sub; git commit --allow-empty -m 'New commit'; cd ..
> git rebase -i HEAD^^
>
> Then drop a.
>
> In my project I get:
> error: cannot rebase: You have unstaged changes.
>
> This works fine with 2.14.3.
  git log --oneline v2.14.3..v2.15.0 -- submodule.c
doesn't give any promising hints (i.e. I don't think one of a
submodule related series introduced this either by chance or
on purpose)

"rebase -i" was rewritten into C in 570676e011, though that series was extensively tested by DScho, so I wouldn't want to point fingers here quickly.

Would you be willing to bisect this behavior?
Orgad Shaneh· Nov 2, 2017, 21:45 UTC · re: Stefan Beller · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On Thu, Nov 2, 2017 at 8:34 PM, Stefan Beller <sbeller@google.com> wrote:
Show 34 quoted lines
> On Thu, Nov 2, 2017 at 1:30 AM, Orgad Shaneh <orgads@gmail.com> wrote:
>> I can't reproduce this with a minimal example, but it happens in my project.
>>
>> What I tried to do for reproducing is:
>> rm -rf super sub
>> mkdir sub; cd sub; git init
>> git commit --allow-empty -m 'Initial commit'
>> mkdir ../super; cd ../super
>> git init
>> git submodule add ../sub
>> touch foo; git add foo sub
>> git commit -m 'Initial commit'
>> touch a; git add a; git commit -m 'a'
>> touch b; git add b; git commit -m 'b'
>> cd sub; git commit --allow-empty -m 'New commit'; cd ..
>> git rebase -i HEAD^^
>>
>> Then drop a.
>>
>> In my project I get:
>> error: cannot rebase: You have unstaged changes.
>>
>> This works fine with 2.14.3.
>
>   git log --oneline v2.14.3..v2.15.0 -- submodule.c
> doesn't give any promising hints (i.e. I don't think one of a
> submodule related series introduced this either by chance or
> on purpose)
>
> "rebase -i" was rewritten into C in 570676e011, though
> that series was extensively tested by DScho, so I wouldn't
> want to point fingers here quickly.
>
> Would you be willing to bisect this behavior?

Bisected to ff6f1f564c48def1f8e1852826bab58af5044b06: submodule-config: lazy-load a repository's .gitmodules file

- Orgad
Johannes Schindelin· Nov 3, 2017, 16:06 UTC · re: Orgad Shaneh · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

Hi Orgad,
On Thu, 2 Nov 2017, Orgad Shaneh wrote:
Show 22 quoted lines
> I can't reproduce this with a minimal example, but it happens in my project.
> 
> What I tried to do for reproducing is:
> rm -rf super sub
> mkdir sub; cd sub; git init
> git commit --allow-empty -m 'Initial commit'
> mkdir ../super; cd ../super
> git init
> git submodule add ../sub
> touch foo; git add foo sub
> git commit -m 'Initial commit'
> touch a; git add a; git commit -m 'a'
> touch b; git add b; git commit -m 'b'
> cd sub; git commit --allow-empty -m 'New commit'; cd ..
> git rebase -i HEAD^^
> 
> Then drop a.
> 
> In my project I get:
> error: cannot rebase: You have unstaged changes.
> 
> This works fine with 2.14.3.
I tried to turn this into a regression test, but I cannot make it fail:
-- snip --
diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
index ebf4f5e4b2c..55aebe53191 100755
--- a/t/t3426-rebase-submodule.sh
+++ b/t/t3426-rebase-submodule.sh
@@ -20,7 +20,7 @@ git_rebase () {
 	git rebase "$1"
 }
 
-test_submodule_switch "git_rebase"
+#test_submodule_switch "git_rebase"
 
 git_rebase_interactive () {
 	git status -su >expect &&
@@ -38,6 +38,27 @@ git_rebase_interactive () {
 	git rebase -i "$1"
 }
 
-test_submodule_switch "git_rebase_interactive"
+#test_submodule_switch "git_rebase_interactive"
+
+test_expect_success '123' '
+	git init sub &&
+	test_commit -C sub init-submodule &&
+	git init super &&
+	git -C super submodule add ../sub &&
+	(
+		cd super &&
+		test_tick &&
+		touch foo &&
+		git add sub foo &&
+		git commit -m initial &&
+		test_commit a &&
+		test_commit b &&
+		test_commit -C sub new &&
+		set_fake_editor &&
+		FAKE_LINES=2 &&
+		export FAKE_LINES &&
+		git rebase -i HEAD^^
+	)
+'
 
 test_done
-- snap --

Can you help me spot what I did wrong?

Ciao,
Dscho
Johannes Schindelin· Nov 3, 2017, 16:20 UTC · re: Johannes Schindelin · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

Hi Orgad,
On Fri, 3 Nov 2017, Johannes Schindelin wrote:
> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
> 
> > I can't reproduce this with a minimal example, but it happens in my project.
Whoa, I somehow overlooked the "can't". Sorry.
Show 61 quoted lines
> > What I tried to do for reproducing is:
> > rm -rf super sub
> > mkdir sub; cd sub; git init
> > git commit --allow-empty -m 'Initial commit'
> > mkdir ../super; cd ../super
> > git init
> > git submodule add ../sub
> > touch foo; git add foo sub
> > git commit -m 'Initial commit'
> > touch a; git add a; git commit -m 'a'
> > touch b; git add b; git commit -m 'b'
> > cd sub; git commit --allow-empty -m 'New commit'; cd ..
> > git rebase -i HEAD^^
> > 
> > Then drop a.
> > 
> > In my project I get:
> > error: cannot rebase: You have unstaged changes.
> > 
> > This works fine with 2.14.3.
> 
> I tried to turn this into a regression test, but I cannot make it fail:
> 
> -- snip --
> diff --git a/t/t3426-rebase-submodule.sh b/t/t3426-rebase-submodule.sh
> index ebf4f5e4b2c..55aebe53191 100755
> --- a/t/t3426-rebase-submodule.sh
> +++ b/t/t3426-rebase-submodule.sh
> @@ -20,7 +20,7 @@ git_rebase () {
>  	git rebase "$1"
>  }
>  
> -test_submodule_switch "git_rebase"
> +#test_submodule_switch "git_rebase"
>  
>  git_rebase_interactive () {
>  	git status -su >expect &&
> @@ -38,6 +38,27 @@ git_rebase_interactive () {
>  	git rebase -i "$1"
>  }
>  
> -test_submodule_switch "git_rebase_interactive"
> +#test_submodule_switch "git_rebase_interactive"
> +
> +test_expect_success '123' '
> +	git init sub &&
> +	test_commit -C sub init-submodule &&
> +	git init super &&
> +	git -C super submodule add ../sub &&
> +	(
> +		cd super &&
> +		test_tick &&
> +		touch foo &&
> +		git add sub foo &&
> +		git commit -m initial &&
> +		test_commit a &&
> +		test_commit b &&
> +		test_commit -C sub new &&
> +		set_fake_editor &&
> +		FAKE_LINES=2 &&
> +		export FAKE_LINES &&

I inserted a `git diff-files` here, and it printed exactly what I expected:

++ git diff-files :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M sub

> +		git rebase -i HEAD^^
> +	)
> +'

There must be something else going wrong that we did not replicate here. Maybe the `error: cannot rebase: You have unstaged changes.` message was caused not by a change in the submodule? Could you run `git diff-files` before the rebase?

This does *not* refresh the index, but maybe that is what is going wrong; you could call `git update-index --refresh` before the rebase and see whether that works around the issue?

Ciao, Dscho

Orgad Shaneh· Nov 4, 2017, 18:04 UTC · re: Johannes Schindelin · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 24 quoted lines
> Hi Orgad,
>
> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
>
>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
>>
>> > I can't reproduce this with a minimal example, but it happens in my project.
>
> Whoa, I somehow overlooked the "can't". Sorry.
>
> I inserted a `git diff-files` here, and it printed exactly what I
> expected:
>
> ++ git diff-files
> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
>
>> +             git rebase -i HEAD^^
>> +     )
>> +'
>
> There must be something else going wrong that we did not replicate here.
> Maybe the `error: cannot rebase: You have unstaged changes.` message was
> caused not by a change in the submodule? Could you run `git diff-files`
> before the rebase?

It's the same before and during the rebase: $ git diff-files :160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8 0000000000000000000000000000000000000000 M sub

>
> This does *not* refresh the index, but maybe that is what is going wrong;
> you could call `git update-index --refresh` before the rebase and see
> whether that works around the issue?
Nope.

If I run git submodule update, then rebase --continue works fine, so it's definitely somehow caused by the submodule.

- Orgad
Orgad Shaneh· Nov 5, 2017, 07:30 UTC · re: Orgad Shaneh · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On Sat, Nov 4, 2017 at 8:04 PM, Orgad Shaneh <orgads@gmail.com> wrote:
Show 42 quoted lines
> On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
>> Hi Orgad,
>>
>> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
>>
>>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
>>>
>>> > I can't reproduce this with a minimal example, but it happens in my project.
>>
>> Whoa, I somehow overlooked the "can't". Sorry.
>>
>> I inserted a `git diff-files` here, and it printed exactly what I
>> expected:
>>
>> ++ git diff-files
>> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
>>
>>> +             git rebase -i HEAD^^
>>> +     )
>>> +'
>>
>> There must be something else going wrong that we did not replicate here.
>> Maybe the `error: cannot rebase: You have unstaged changes.` message was
>> caused not by a change in the submodule? Could you run `git diff-files`
>> before the rebase?
>
> It's the same before and during the rebase:
> $ git diff-files
> :160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8
> 0000000000000000000000000000000000000000 M      sub
>
>
>>
>> This does *not* refresh the index, but maybe that is what is going wrong;
>> you could call `git update-index --refresh` before the rebase and see
>> whether that works around the issue?
>
> Nope.
>
> If I run git submodule update, then rebase --continue works fine, so
> it's definitely somehow caused by the submodule.

I just checked out v2.15.0.windows.1 and reverted ff6f1f564c - it solves the problem. I still have no idea how to minimally reproduce (in my project it's easily reproducible) :)

- Orgad
Brandon Williams· Nov 6, 2017, 18:37 UTC · re: Orgad Shaneh · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On 11/05, Orgad Shaneh wrote:
Show 48 quoted lines
> On Sat, Nov 4, 2017 at 8:04 PM, Orgad Shaneh <orgads@gmail.com> wrote:
> > On Fri, Nov 3, 2017 at 6:20 PM, Johannes Schindelin
> > <Johannes.Schindelin@gmx.de> wrote:
> >> Hi Orgad,
> >>
> >> On Fri, 3 Nov 2017, Johannes Schindelin wrote:
> >>
> >>> On Thu, 2 Nov 2017, Orgad Shaneh wrote:
> >>>
> >>> > I can't reproduce this with a minimal example, but it happens in my project.
> >>
> >> Whoa, I somehow overlooked the "can't". Sorry.
> >>
> >> I inserted a `git diff-files` here, and it printed exactly what I
> >> expected:
> >>
> >> ++ git diff-files
> >> :160000 160000 62cab94c8d8cf047bbb60c12def559339300efa4 0000000000000000000000000000000000000000 M      sub
> >>
> >>> +             git rebase -i HEAD^^
> >>> +     )
> >>> +'
> >>
> >> There must be something else going wrong that we did not replicate here.
> >> Maybe the `error: cannot rebase: You have unstaged changes.` message was
> >> caused not by a change in the submodule? Could you run `git diff-files`
> >> before the rebase?
> >
> > It's the same before and during the rebase:
> > $ git diff-files
> > :160000 160000 c840225a7cf6bb2ec64da9d35d2c29210bc5e5e8
> > 0000000000000000000000000000000000000000 M      sub
> >
> >
> >>
> >> This does *not* refresh the index, but maybe that is what is going wrong;
> >> you could call `git update-index --refresh` before the rebase and see
> >> whether that works around the issue?
> >
> > Nope.
> >
> > If I run git submodule update, then rebase --continue works fine, so
> > it's definitely somehow caused by the submodule.
> 
> I just checked out v2.15.0.windows.1 and reverted ff6f1f564c - it
> solves the problem. I still have no idea how to minimally reproduce
> (in my project it's easily reproducible) :)
> 

After reading your bug report and the fact that you weren't able to reproduce it outside of your project I think i figured out what is happening. Before ff6f1f564c the gitmodules file wasn't being loaded unless a codepath explicitly wanted to work with submodules. Now they are being lazy-loaded so if you call into the submodule config subsystem it'll work without having to have initialized it before. I suspect that the submodule which is causing the failure has a "submodule.<name>.ignore" entry in the .gitmodules file or somewhere in your repositories config (I actually suspect the latter based on the code path).

When rebase calls into the diff machinery to see if there are unstaged changes it explicitly requests that submodule's be ignored, but this desired gets overridden by your repository's config, clearing the ignored flag and making rebase actually pay attention to the fact that the submodule has changes in it.

I don't have a patch available to for you to test just yet (but I'll have some time later today to write one up) but could you verify that (1) you have an ignore entry for the submodule in question in your config and (2) removing it from your config avoids the failure? If that's the case then we would be able to put together a reproducible recipe for this failure.

-- 
Brandon Williams
Orgad Shaneh· Nov 6, 2017, 20:36 UTC · re: Brandon Williams · lore

Re: Regression[2.14.3->2.15]: Interactive rebase fails if submodule is modified

On Mon, Nov 6, 2017 at 8:37 PM, Brandon Williams <bmwill@google.com> wrote:
Show 23 quoted lines
> After reading your bug report and the fact that you weren't able to
> reproduce it outside of your project I think i figured out what is
> happening.  Before ff6f1f564c the gitmodules file wasn't being loaded
> unless a codepath explicitly wanted to work with submodules.  Now they
> are being lazy-loaded so if you call into the submodule config subsystem
> it'll work without having to have initialized it before.  I suspect
> that the submodule which is causing the failure has a
> "submodule.<name>.ignore" entry in the .gitmodules file or somewhere in
> your repositories config (I actually suspect the latter based on the
> code path).
>
> When rebase calls into the diff machinery to see if there are unstaged
> changes it explicitly requests that submodule's be ignored, but this
> desired gets overridden by your repository's config, clearing the
> ignored flag and making rebase actually pay attention to the fact that
> the submodule has changes in it.
>
> I don't have a patch available to for you to test just yet (but I'll
> have some time later today to write one up) but could you verify that
> (1) you have an ignore entry for the submodule in question in your
> config and (2) removing it from your config avoids the failure?  If
> that's the case then we would be able to put together a reproducible
> recipe for this failure.

You're right. Thanks for the info. I have ignore = dirty, and removing it from the config solves the problem indeed.

The following script reproduces the bug (it's the same as before, only added git config):

rm -rf super sub mkdir sub; cd sub; git init git commit --allow-empty -m 'Initial commit' mkdir ../super; cd ../super git init git submodule add ../sub git config submodule.sub.ignore dirty touch foo; git add foo sub git commit -m 'Initial commit' touch a; git add a; git commit -m 'a' touch b; git add b; git commit -m 'b' cd sub; git commit --allow-empty -m 'New commit'; cd .. git rebase -i HEAD^^

- Orgad

← back to recent threads