threads / discuss / 43985

If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

Subject: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

## tl;dr

7 messages between Sep 2, 2016 and Sep 10, 2016.

replies: 6people: 2as markdown or json

Dakota Hawkins· Sep 2, 2016, 19:22 UTC · lore
Below is a simple reproduction of the issue.

The _real_ problem is that this is how our pull request merges work, they're not allowed to do fast-forward merges. To work around this we are having to split this up into two pull requests/merges: One that copies the submodules to the new location and includes any fixes required to support the move, and a second that removes the old locations.

## Setup steps
git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git
cd submodule-move-merge-bug-main-repo
    ## How it was initially constructed
    # git submodule add ../submodule-move-merge-bug-submodule-repo.git
./submodule-location-1
    # git commit -m "Added submodule in its initial location"
    # git push
    # git checkout -b move-submodule
    # git mv ./submodule-location-1 ./submodule-location-2
    # git commit -m "Moved submodule"
    # git push --set-upstream origin move-submodule
git branch move-submodule origin/move-submodule

## Test fast-forward merge, this will work git checkout -b merge-ff-test master # warning: unable to rmdir submodule-location-2: Directory not empty rm -rf ./submodule-location-2 git merge --ff-only move-submodule

## Test no-fast-forward merge, this will fail with conflicts:
git checkout -b merge-no-ff-test master
git merge --no-ff move-submodule
    # Auto-merging submodule-location-2
    # Adding as submodule-location-2~move-submodule instead
    # Automatic merge failed; fix conflicts and then commit the result.
git status
    # On branch merge-no-ff-test
    # You have unmerged paths.
    #   (fix conflicts and run "git commit")
    #   (use "git merge --abort" to abort the merge)
    #
    # Changes to be committed:
    #
    #         modified:   .gitmodules
    #         deleted:    submodule-location-1
    #
    # Unmerged paths:
    #   (use "git add <file>..." to mark resolution)
    #
    #         added by us:     submodule-location-2
    #
    # fatal: Not a git repository: 'submodule-location-1/.git'
    # Submodule changes to be committed:
    #
    # * submodule-location-1 07fec24...0000000:
Dakota Hawkins· Sep 6, 2016, 18:40 UTC · re: Dakota Hawkins · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

Is there any additional information I could provide that would be helpful?
Dakota
On Fri, Sep 2, 2016 at 3:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
Show 55 quoted lines
> Below is a simple reproduction of the issue.
>
> The _real_ problem is that this is how our pull request merges work,
> they're not allowed to do fast-forward merges. To work around this we
> are having to split this up into two pull requests/merges: One that
> copies the submodules to the new location and includes any fixes
> required to support the move, and a second that removes the old
> locations.
>
> ## Setup steps
> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git
> cd submodule-move-merge-bug-main-repo
>     ## How it was initially constructed
>     # git submodule add ../submodule-move-merge-bug-submodule-repo.git
> ./submodule-location-1
>     # git commit -m "Added submodule in its initial location"
>     # git push
>     # git checkout -b move-submodule
>     # git mv ./submodule-location-1 ./submodule-location-2
>     # git commit -m "Moved submodule"
>     # git push --set-upstream origin move-submodule
> git branch move-submodule origin/move-submodule
>
> ## Test fast-forward merge, this will work
> git checkout -b merge-ff-test master # warning: unable to rmdir
> submodule-location-2: Directory not empty
> rm -rf ./submodule-location-2
> git merge --ff-only move-submodule
>
> ## Test no-fast-forward merge, this will fail with conflicts:
> git checkout -b merge-no-ff-test master
> git merge --no-ff move-submodule
>     # Auto-merging submodule-location-2
>     # Adding as submodule-location-2~move-submodule instead
>     # Automatic merge failed; fix conflicts and then commit the result.
> git status
>     # On branch merge-no-ff-test
>     # You have unmerged paths.
>     #   (fix conflicts and run "git commit")
>     #   (use "git merge --abort" to abort the merge)
>     #
>     # Changes to be committed:
>     #
>     #         modified:   .gitmodules
>     #         deleted:    submodule-location-1
>     #
>     # Unmerged paths:
>     #   (use "git add <file>..." to mark resolution)
>     #
>     #         added by us:     submodule-location-2
>     #
>     # fatal: Not a git repository: 'submodule-location-1/.git'
>     # Submodule changes to be committed:
>     #
>     # * submodule-location-1 07fec24...0000000:
Stefan Beller· Sep 6, 2016, 19:00 UTC · re: Dakota Hawkins · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
> Below is a simple reproduction of the issue.
>
> The _real_ problem is that this is how our pull request merges work,

So your workflow is the problem or is the actual bug just exposed in your workflow?

Show 26 quoted lines
> they're not allowed to do fast-forward merges. To work around this we
> are having to split this up into two pull requests/merges: One that
> copies the submodules to the new location and includes any fixes
> required to support the move, and a second that removes the old
> locations.
>
> ## Setup steps
> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git
> cd submodule-move-merge-bug-main-repo
>     ## How it was initially constructed
>     # git submodule add ../submodule-move-merge-bug-submodule-repo.git
> ./submodule-location-1
>     # git commit -m "Added submodule in its initial location"
>     # git push
>     # git checkout -b move-submodule
>     # git mv ./submodule-location-1 ./submodule-location-2
>     # git commit -m "Moved submodule"
>     # git push --set-upstream origin move-submodule
> git branch move-submodule origin/move-submodule
>
> ## Test fast-forward merge, this will work
> git checkout -b merge-ff-test master # warning: unable to rmdir
> submodule-location-2: Directory not empty
> rm -rf ./submodule-location-2
> git merge --ff-only move-submodule
>

And no reset/rm in between, i.e. we still have submodule-location-2 from merge-ff-test still around?

Show 26 quoted lines
> ## Test no-fast-forward merge, this will fail with conflicts:
> git checkout -b merge-no-ff-test master
> git merge --no-ff move-submodule
>     # Auto-merging submodule-location-2
>     # Adding as submodule-location-2~move-submodule instead
>     # Automatic merge failed; fix conflicts and then commit the result.
> git status
>     # On branch merge-no-ff-test
>     # You have unmerged paths.
>     #   (fix conflicts and run "git commit")
>     #   (use "git merge --abort" to abort the merge)
>     #
>     # Changes to be committed:
>     #
>     #         modified:   .gitmodules
>     #         deleted:    submodule-location-1
>     #
>     # Unmerged paths:
>     #   (use "git add <file>..." to mark resolution)
>     #
>     #         added by us:     submodule-location-2
>     #
>     # fatal: Not a git repository: 'submodule-location-1/.git'
>     # Submodule changes to be committed:
>     #
>     # * submodule-location-1 07fec24...0000000:
Dakota Hawkins· Sep 6, 2016, 20:02 UTC · re: Stefan Beller · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

On Tue, Sep 6, 2016 at 3:00 PM, Stefan Beller <sbeller@google.com> wrote:
Show 7 quoted lines
> On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
>> Below is a simple reproduction of the issue.
>>
>> The _real_ problem is that this is how our pull request merges work,
>
> So your workflow is the problem or is the actual bug just exposed in
> your workflow?

I believe the workflow just exposes the problem, which is that generically for this case a fast-forward merge works without conflicts while a non-fast-forward merge fails with conflicts. Sorry if this confuses the issue, it's just how we experienced it so I wanted to add that background.

Show 9 quoted lines
>> ## Test fast-forward merge, this will work
>> git checkout -b merge-ff-test master # warning: unable to rmdir
>> submodule-location-2: Directory not empty
>> rm -rf ./submodule-location-2
>> git merge --ff-only move-submodule
>>
>
> And no reset/rm in between, i.e. we still have
> submodule-location-2 from  merge-ff-test still around?

That is true in that example, but somewhat immaterial. The first merge was only to demonstrate that a fast-forward merge works without conflict. The simplest reproduction is to skip that and get straight to the failure case:

## Clone and setup branches git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git cd submodule-move-merge-bug-main-repo git branch move-submodule origin/move-submodule git checkout -b merge-no-ff-test master

## This will fail git merge --no-ff move-submodule # Auto-merging submodule-location-2 # Adding as submodule-location-2~move-submodule instead # Automatic merge failed; fix conflicts and then commit the result.

Does that make things a little bit clearer?
Dakota
Dakota Hawkins· Sep 9, 2016, 23:37 UTC · re: Dakota Hawkins · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

Any ideas on this anywhere?

In my opinion if a merge can fast-forward without conflict it should trivially be able to non-fast-forward without conflict.

Also, I'm not too familiar/comfortable with mailing list etiquette, and I don't want to be a bother by continuing to ping this thread.

Thanks,
Dakota
On Tue, Sep 6, 2016 at 4:02 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
Show 45 quoted lines
> On Tue, Sep 6, 2016 at 3:00 PM, Stefan Beller <sbeller@google.com> wrote:
>> On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
>>> Below is a simple reproduction of the issue.
>>>
>>> The _real_ problem is that this is how our pull request merges work,
>>
>> So your workflow is the problem or is the actual bug just exposed in
>> your workflow?
>
> I believe the workflow just exposes the problem, which is that
> generically for this case a fast-forward merge works without conflicts
> while a non-fast-forward merge fails with conflicts. Sorry if this
> confuses the issue, it's just how we experienced it so I wanted to add
> that background.
>
>>> ## Test fast-forward merge, this will work
>>> git checkout -b merge-ff-test master # warning: unable to rmdir
>>> submodule-location-2: Directory not empty
>>> rm -rf ./submodule-location-2
>>> git merge --ff-only move-submodule
>>>
>>
>> And no reset/rm in between, i.e. we still have
>> submodule-location-2 from  merge-ff-test still around?
>
> That is true in that example, but somewhat immaterial. The first merge
> was only to demonstrate that a fast-forward merge works without
> conflict. The simplest reproduction is to skip that and get straight
> to the failure case:
>
> ## Clone and setup branches
> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git
> cd submodule-move-merge-bug-main-repo
> git branch move-submodule origin/move-submodule
> git checkout -b merge-no-ff-test master
>
> ## This will fail
> git merge --no-ff move-submodule
> # Auto-merging submodule-location-2
> # Adding as submodule-location-2~move-submodule instead
> # Automatic merge failed; fix conflicts and then commit the result.
>
> Does that make things a little bit clearer?
>
> Dakota
Stefan Beller· Sep 9, 2016, 23:57 UTC · re: Dakota Hawkins · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

On Fri, Sep 9, 2016 at 4:37 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:
> Any ideas on this anywhere?
>
> In my opinion if a merge can fast-forward without conflict it should
> trivially be able to non-fast-forward without conflict.

Yes, I agree. Though the submodules still have a lot of sharp edges that you can injure yourself with.

I am sorry to let you down here, but I did not have time and mental capacity to actually dive into the issue further this week. I hoped someone else would have picked up this thread and have a good idea.

As you know currently the checkout doesn't touch submodules, i.e. you have to run "git submodule update" whenever the submodule changes. So when you checkout a different part of history, that moved a submodule, this will fail as the submodule still resides at the old place (and may have path name conflict with another thing) and is not there at the new place.

I plan to fix that in the near future (read: teach git checkout to know about submodules), mind that there is already a patch series for exactly that out there in one of the branches of https://github.com/jlehmann/git-submod-enhancements

This seems to be not an easy fix; Someone tried already and getting it polished enough to include it upstream was not successful.

>
> Also, I'm not too familiar/comfortable with mailing list etiquette,
> and I don't want to be a bother by continuing to ping this thread.

Pinging is fine, as it is rather easy to ignore mails on a mailing list. ;) I just don't know if it increases likelihood of someone responding.

Thanks, Stefan

Dakota Hawkins· Sep 10, 2016, 00:17 UTC · re: Stefan Beller · lore

Re: If a branch moves a submodule, "merge --ff[-only]" succeeds while "merge --no-ff" fails with conflicts

Show 6 quoted lines
> As you know currently the checkout doesn't touch submodules, i.e.
> you have to run "git submodule update" whenever the submodule
> changes. So when you checkout a different part of history, that moved
> a submodule, this will fail as the submodule still resides at the old
> place (and may have path name conflict with another thing)
> and is not there at the new place.

I _think_ (may be wrong) that you can reproduce this without ever updating the submodules at all. I may be missing/forgetting something in my config: I normally have global post-checkout and post-merge hooks to run a submodule update, however I realized that and (I think) disabled them for the second more concise reproduction I sent. I moved them to a "no" subdirectory in my hook directory -- git doesn't recursively look for hooks, does it? Anyway, I don't think that's a difference in my case. It may be, I agree, if you happen to be reproducing this issue locally in your working repository for some reason.

That thought process makes sense to me because as I mentioned initially these failures presented on our remote (bitbucket server, if it matters) for a pull-request merge. I don't know for certain but I don't believe the server's copy of the repo would need to update the submodules at all, ever.

Show 6 quoted lines
>>
>> Also, I'm not too familiar/comfortable with mailing list etiquette,
>> and I don't want to be a bother by continuing to ping this thread.
>
> Pinging is fine, as it is rather easy to ignore mails on a mailing list. ;)
> I just don't know if it increases likelihood of someone responding.
Apparently it got you on the hook! Thanks for taking the bait :)
Dakota

← back to recent threads