{"thread":{"id":"43985","subject":"If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","startedAt":"2016-09-02T19:22:32Z","lastAt":"2016-09-10T00:17:29Z","messageCount":7,"participants":["Dakota Hawkins","Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"300904","messageId":"CAG0BQX=wvpkJ=PQWV-NbmhuPV8yzvd_KYKzJmsfWq9xStZ2bnQ@mail.gmail.com","threadId":"43985","inReplyTo":null,"subject":"If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Dakota Hawkins","fromEmail":"dakotahawkins@gmail.com","sentAt":"2016-09-02T19:22:26Z","receivedAt":"2016-09-02T19:22:32Z","isPatch":false,"sender":{"key":"dakotahawkins@gmail.com","avatar":"https://gravatar.com/avatar/6283c977565cfcecb284e62d5713a91a6a8df0a3b06f27439edaab4220ae143d?d=mp&s=160"},"body":"Below is a simple reproduction of the issue.\n\nThe _real_ problem is that this is how our pull request merges work,\nthey're not allowed to do fast-forward merges. To work around this we\nare having to split this up into two pull requests/merges: One that\ncopies the submodules to the new location and includes any fixes\nrequired to support the move, and a second that removes the old\nlocations.\n\n## Setup steps\ngit clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git\ncd submodule-move-merge-bug-main-repo\n    ## How it was initially constructed\n    # git submodule add ../submodule-move-merge-bug-submodule-repo.git\n./submodule-location-1\n    # git commit -m \"Added submodule in its initial location\"\n    # git push\n    # git checkout -b move-submodule\n    # git mv ./submodule-location-1 ./submodule-location-2\n    # git commit -m \"Moved submodule\"\n    # git push --set-upstream origin move-submodule\ngit branch move-submodule origin/move-submodule\n\n## Test fast-forward merge, this will work\ngit checkout -b merge-ff-test master # warning: unable to rmdir\nsubmodule-location-2: Directory not empty\nrm -rf ./submodule-location-2\ngit merge --ff-only move-submodule\n\n## Test no-fast-forward merge, this will fail with conflicts:\ngit checkout -b merge-no-ff-test master\ngit merge --no-ff move-submodule\n    # Auto-merging submodule-location-2\n    # Adding as submodule-location-2~move-submodule instead\n    # Automatic merge failed; fix conflicts and then commit the result.\ngit status\n    # On branch merge-no-ff-test\n    # You have unmerged paths.\n    #   (fix conflicts and run \"git commit\")\n    #   (use \"git merge --abort\" to abort the merge)\n    #\n    # Changes to be committed:\n    #\n    #         modified:   .gitmodules\n    #         deleted:    submodule-location-1\n    #\n    # Unmerged paths:\n    #   (use \"git add <file>...\" to mark resolution)\n    #\n    #         added by us:     submodule-location-2\n    #\n    # fatal: Not a git repository: 'submodule-location-1/.git'\n    # Submodule changes to be committed:\n    #\n    # * submodule-location-1 07fec24...0000000:\n"},{"id":"301220","messageId":"CAG0BQX=i6gcA0ba5T7vcnaDPucZDcTOZUEGr=fhgpcDrbvQ1+g@mail.gmail.com","threadId":"43985","inReplyTo":"CAG0BQX=wvpkJ=PQWV-NbmhuPV8yzvd_KYKzJmsfWq9xStZ2bnQ@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Dakota Hawkins","fromEmail":"dakotahawkins@gmail.com","sentAt":"2016-09-06T18:40:35Z","receivedAt":"2016-09-06T18:41:13Z","isPatch":false,"sender":{"key":"dakotahawkins@gmail.com","avatar":"https://gravatar.com/avatar/6283c977565cfcecb284e62d5713a91a6a8df0a3b06f27439edaab4220ae143d?d=mp&s=160"},"body":"Is there any additional information I could provide that would be helpful?\n\nDakota\n\nOn Fri, Sep 2, 2016 at 3:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n> Below is a simple reproduction of the issue.\n>\n> The _real_ problem is that this is how our pull request merges work,\n> they're not allowed to do fast-forward merges. To work around this we\n> are having to split this up into two pull requests/merges: One that\n> copies the submodules to the new location and includes any fixes\n> required to support the move, and a second that removes the old\n> locations.\n>\n> ## Setup steps\n> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git\n> cd submodule-move-merge-bug-main-repo\n>     ## How it was initially constructed\n>     # git submodule add ../submodule-move-merge-bug-submodule-repo.git\n> ./submodule-location-1\n>     # git commit -m \"Added submodule in its initial location\"\n>     # git push\n>     # git checkout -b move-submodule\n>     # git mv ./submodule-location-1 ./submodule-location-2\n>     # git commit -m \"Moved submodule\"\n>     # git push --set-upstream origin move-submodule\n> git branch move-submodule origin/move-submodule\n>\n> ## Test fast-forward merge, this will work\n> git checkout -b merge-ff-test master # warning: unable to rmdir\n> submodule-location-2: Directory not empty\n> rm -rf ./submodule-location-2\n> git merge --ff-only move-submodule\n>\n> ## Test no-fast-forward merge, this will fail with conflicts:\n> git checkout -b merge-no-ff-test master\n> git merge --no-ff move-submodule\n>     # Auto-merging submodule-location-2\n>     # Adding as submodule-location-2~move-submodule instead\n>     # Automatic merge failed; fix conflicts and then commit the result.\n> git status\n>     # On branch merge-no-ff-test\n>     # You have unmerged paths.\n>     #   (fix conflicts and run \"git commit\")\n>     #   (use \"git merge --abort\" to abort the merge)\n>     #\n>     # Changes to be committed:\n>     #\n>     #         modified:   .gitmodules\n>     #         deleted:    submodule-location-1\n>     #\n>     # Unmerged paths:\n>     #   (use \"git add <file>...\" to mark resolution)\n>     #\n>     #         added by us:     submodule-location-2\n>     #\n>     # fatal: Not a git repository: 'submodule-location-1/.git'\n>     # Submodule changes to be committed:\n>     #\n>     # * submodule-location-1 07fec24...0000000:\n"},{"id":"301226","messageId":"CAGZ79kb=LyusFH6tGirGP9qK1k-cov2UubEbKPfeyPRThUsa-Q@mail.gmail.com","threadId":"43985","inReplyTo":"CAG0BQX=wvpkJ=PQWV-NbmhuPV8yzvd_KYKzJmsfWq9xStZ2bnQ@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-09-06T19:00:14Z","receivedAt":"2016-09-06T19:00:43Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n> Below is a simple reproduction of the issue.\n>\n> The _real_ problem is that this is how our pull request merges work,\n\nSo your workflow is the problem or is the actual bug just exposed in\nyour workflow?\n\n> they're not allowed to do fast-forward merges. To work around this we\n> are having to split this up into two pull requests/merges: One that\n> copies the submodules to the new location and includes any fixes\n> required to support the move, and a second that removes the old\n> locations.\n>\n> ## Setup steps\n> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git\n> cd submodule-move-merge-bug-main-repo\n>     ## How it was initially constructed\n>     # git submodule add ../submodule-move-merge-bug-submodule-repo.git\n> ./submodule-location-1\n>     # git commit -m \"Added submodule in its initial location\"\n>     # git push\n>     # git checkout -b move-submodule\n>     # git mv ./submodule-location-1 ./submodule-location-2\n>     # git commit -m \"Moved submodule\"\n>     # git push --set-upstream origin move-submodule\n> git branch move-submodule origin/move-submodule\n>\n> ## Test fast-forward merge, this will work\n> git checkout -b merge-ff-test master # warning: unable to rmdir\n> submodule-location-2: Directory not empty\n> rm -rf ./submodule-location-2\n> git merge --ff-only move-submodule\n>\n\nAnd no reset/rm in between, i.e. we still have\nsubmodule-location-2 from  merge-ff-test still around?\n\n> ## Test no-fast-forward merge, this will fail with conflicts:\n> git checkout -b merge-no-ff-test master\n> git merge --no-ff move-submodule\n>     # Auto-merging submodule-location-2\n>     # Adding as submodule-location-2~move-submodule instead\n>     # Automatic merge failed; fix conflicts and then commit the result.\n> git status\n>     # On branch merge-no-ff-test\n>     # You have unmerged paths.\n>     #   (fix conflicts and run \"git commit\")\n>     #   (use \"git merge --abort\" to abort the merge)\n>     #\n>     # Changes to be committed:\n>     #\n>     #         modified:   .gitmodules\n>     #         deleted:    submodule-location-1\n>     #\n>     # Unmerged paths:\n>     #   (use \"git add <file>...\" to mark resolution)\n>     #\n>     #         added by us:     submodule-location-2\n>     #\n>     # fatal: Not a git repository: 'submodule-location-1/.git'\n>     # Submodule changes to be committed:\n>     #\n>     # * submodule-location-1 07fec24...0000000:\n"},{"id":"301231","messageId":"CAG0BQX=FFWqR=45g6buujzK2jknx8aZnQoiV_m-QZhcx4Dd52g@mail.gmail.com","threadId":"43985","inReplyTo":"CAGZ79kb=LyusFH6tGirGP9qK1k-cov2UubEbKPfeyPRThUsa-Q@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Dakota Hawkins","fromEmail":"dakotahawkins@gmail.com","sentAt":"2016-09-06T20:02:11Z","receivedAt":"2016-09-06T20:02:27Z","isPatch":false,"sender":{"key":"dakotahawkins@gmail.com","avatar":"https://gravatar.com/avatar/6283c977565cfcecb284e62d5713a91a6a8df0a3b06f27439edaab4220ae143d?d=mp&s=160"},"body":"On Tue, Sep 6, 2016 at 3:00 PM, Stefan Beller <sbeller@google.com> wrote:\n> On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n>> Below is a simple reproduction of the issue.\n>>\n>> The _real_ problem is that this is how our pull request merges work,\n>\n> So your workflow is the problem or is the actual bug just exposed in\n> your workflow?\n\nI believe the workflow just exposes the problem, which is that\ngenerically for this case a fast-forward merge works without conflicts\nwhile a non-fast-forward merge fails with conflicts. Sorry if this\nconfuses the issue, it's just how we experienced it so I wanted to add\nthat background.\n\n>> ## Test fast-forward merge, this will work\n>> git checkout -b merge-ff-test master # warning: unable to rmdir\n>> submodule-location-2: Directory not empty\n>> rm -rf ./submodule-location-2\n>> git merge --ff-only move-submodule\n>>\n>\n> And no reset/rm in between, i.e. we still have\n> submodule-location-2 from  merge-ff-test still around?\n\nThat is true in that example, but somewhat immaterial. The first merge\nwas only to demonstrate that a fast-forward merge works without\nconflict. The simplest reproduction is to skip that and get straight\nto the failure case:\n\n## Clone and setup branches\ngit clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git\ncd submodule-move-merge-bug-main-repo\ngit branch move-submodule origin/move-submodule\ngit checkout -b merge-no-ff-test master\n\n## This will fail\ngit merge --no-ff move-submodule\n# Auto-merging submodule-location-2\n# Adding as submodule-location-2~move-submodule instead\n# Automatic merge failed; fix conflicts and then commit the result.\n\nDoes that make things a little bit clearer?\n\nDakota\n"},{"id":"301583","messageId":"CAG0BQXmyW-Hzdyj4rTviToEVbJ73d94P9GcfFVR6P2XCsA2t-g@mail.gmail.com","threadId":"43985","inReplyTo":"CAG0BQX=FFWqR=45g6buujzK2jknx8aZnQoiV_m-QZhcx4Dd52g@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Dakota Hawkins","fromEmail":"dakotahawkins@gmail.com","sentAt":"2016-09-09T23:37:31Z","receivedAt":"2016-09-09T23:37:46Z","isPatch":false,"sender":{"key":"dakotahawkins@gmail.com","avatar":"https://gravatar.com/avatar/6283c977565cfcecb284e62d5713a91a6a8df0a3b06f27439edaab4220ae143d?d=mp&s=160"},"body":"Any ideas on this anywhere?\n\nIn my opinion if a merge can fast-forward without conflict it should\ntrivially be able to non-fast-forward without conflict.\n\nAlso, I'm not too familiar/comfortable with mailing list etiquette,\nand I don't want to be a bother by continuing to ping this thread.\n\nThanks,\n\nDakota\n\nOn Tue, Sep 6, 2016 at 4:02 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n> On Tue, Sep 6, 2016 at 3:00 PM, Stefan Beller <sbeller@google.com> wrote:\n>> On Fri, Sep 2, 2016 at 12:22 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n>>> Below is a simple reproduction of the issue.\n>>>\n>>> The _real_ problem is that this is how our pull request merges work,\n>>\n>> So your workflow is the problem or is the actual bug just exposed in\n>> your workflow?\n>\n> I believe the workflow just exposes the problem, which is that\n> generically for this case a fast-forward merge works without conflicts\n> while a non-fast-forward merge fails with conflicts. Sorry if this\n> confuses the issue, it's just how we experienced it so I wanted to add\n> that background.\n>\n>>> ## Test fast-forward merge, this will work\n>>> git checkout -b merge-ff-test master # warning: unable to rmdir\n>>> submodule-location-2: Directory not empty\n>>> rm -rf ./submodule-location-2\n>>> git merge --ff-only move-submodule\n>>>\n>>\n>> And no reset/rm in between, i.e. we still have\n>> submodule-location-2 from  merge-ff-test still around?\n>\n> That is true in that example, but somewhat immaterial. The first merge\n> was only to demonstrate that a fast-forward merge works without\n> conflict. The simplest reproduction is to skip that and get straight\n> to the failure case:\n>\n> ## Clone and setup branches\n> git clone https://github.com/dakotahawkins/submodule-move-merge-bug-main-repo.git\n> cd submodule-move-merge-bug-main-repo\n> git branch move-submodule origin/move-submodule\n> git checkout -b merge-no-ff-test master\n>\n> ## This will fail\n> git merge --no-ff move-submodule\n> # Auto-merging submodule-location-2\n> # Adding as submodule-location-2~move-submodule instead\n> # Automatic merge failed; fix conflicts and then commit the result.\n>\n> Does that make things a little bit clearer?\n>\n> Dakota\n"},{"id":"301585","messageId":"CAGZ79kapz7PR7vO2oyWE8WygXr+hTSJeZfZgpyeERv+Z1jTjCQ@mail.gmail.com","threadId":"43985","inReplyTo":"CAG0BQXmyW-Hzdyj4rTviToEVbJ73d94P9GcfFVR6P2XCsA2t-g@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-09-09T23:57:02Z","receivedAt":"2016-09-09T23:57:08Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Sep 9, 2016 at 4:37 PM, Dakota Hawkins <dakotahawkins@gmail.com> wrote:\n> Any ideas on this anywhere?\n>\n> In my opinion if a merge can fast-forward without conflict it should\n> trivially be able to non-fast-forward without conflict.\n\nYes, I agree. Though the submodules still have a lot of sharp edges\nthat you can injure yourself with.\n\nI am sorry to let you down here, but I did not have time and mental\ncapacity to actually dive into the issue further this week.\nI hoped someone else would have picked up this thread and have\na good idea.\n\nAs you know currently the checkout doesn't touch submodules, i.e.\nyou have to run \"git submodule update\" whenever the submodule\nchanges. So when you checkout a different part of history, that moved\na submodule, this will fail as the submodule still resides at the old\nplace (and may have path name conflict with another thing)\nand is not there at the new place.\n\nI plan to fix that in the near future (read: teach git checkout to know\nabout submodules), mind that there is already a patch series for\nexactly that out there in one of the branches of\nhttps://github.com/jlehmann/git-submod-enhancements\n\nThis seems to be not an easy fix; Someone tried already and getting\nit polished enough to include it upstream was not successful.\n\n>\n> Also, I'm not too familiar/comfortable with mailing list etiquette,\n> and I don't want to be a bother by continuing to ping this thread.\n\nPinging is fine, as it is rather easy to ignore mails on a mailing list. ;)\nI just don't know if it increases likelihood of someone responding.\n\nThanks,\nStefan\n"},{"id":"301586","messageId":"CAG0BQX=ym399NzJZWk=emKfsPqKztDBeYS4+GXpYKhcTmtkJTQ@mail.gmail.com","threadId":"43985","inReplyTo":"CAGZ79kapz7PR7vO2oyWE8WygXr+hTSJeZfZgpyeERv+Z1jTjCQ@mail.gmail.com","subject":"Re: If a branch moves a submodule, \"merge --ff[-only]\" succeeds while \"merge --no-ff\" fails with conflicts","fromName":"Dakota Hawkins","fromEmail":"dakotahawkins@gmail.com","sentAt":"2016-09-10T00:17:18Z","receivedAt":"2016-09-10T00:17:29Z","isPatch":false,"sender":{"key":"dakotahawkins@gmail.com","avatar":"https://gravatar.com/avatar/6283c977565cfcecb284e62d5713a91a6a8df0a3b06f27439edaab4220ae143d?d=mp&s=160"},"body":"> As you know currently the checkout doesn't touch submodules, i.e.\n> you have to run \"git submodule update\" whenever the submodule\n> changes. So when you checkout a different part of history, that moved\n> a submodule, this will fail as the submodule still resides at the old\n> place (and may have path name conflict with another thing)\n> and is not there at the new place.\n\nI _think_ (may be wrong) that you can reproduce this without ever\nupdating the submodules at all. I may be missing/forgetting something\nin my config: I normally have global post-checkout and post-merge\nhooks to run a submodule update, however I realized that and (I think)\ndisabled them for the second more concise reproduction I sent. I moved\nthem to a \"no\" subdirectory in my hook directory -- git doesn't\nrecursively look for hooks, does it? Anyway, I don't think that's a\ndifference in my case. It may be, I agree, if you happen to be\nreproducing this issue locally in your working repository for some\nreason.\n\nThat thought process makes sense to me because as I mentioned\ninitially these failures presented on our remote (bitbucket server, if\nit matters) for a pull-request merge. I don't know for certain but I\ndon't believe the server's copy of the repo would need to update the\nsubmodules at all, ever.\n\n>>\n>> Also, I'm not too familiar/comfortable with mailing list etiquette,\n>> and I don't want to be a bother by continuing to ping this thread.\n>\n> Pinging is fine, as it is rather easy to ignore mails on a mailing list. ;)\n> I just don't know if it increases likelihood of someone responding.\n\nApparently it got you on the hook! Thanks for taking the bait :)\n\nDakota\n"}]}