{"thread":{"id":"55496","subject":"possible bug due to directory rename detection","startedAt":"2021-04-15T00:16:13Z","lastAt":"2021-04-15T04:21:36Z","messageCount":2,"participants":["Matt Neuburg","Elijah Newren"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"421996","messageId":"C0E64A34-AAA0-4ECE-8EC5-045815B3413D@tidbits.com","threadId":"55496","inReplyTo":null,"subject":"possible bug due to directory rename detection","fromName":"Matt Neuburg","fromEmail":"matt@tidbits.com","sentAt":"2021-04-15T00:17:14Z","receivedAt":"2021-04-15T00:16:13Z","isPatch":false,"sender":{"key":"matt@tidbits.com","avatar":null},"body":"git version 2.24.3 (Apple Git-128) running under Big Sur on a MacBook Pro.\n\nPlease see\n\n  https://stackoverflow.com/questions/67067686/git-file-level-merge-conflict-caused-by-git-suggesting-the-file-should-perhap \n\nfor a complete description of what I saw and why I think it's wrong.\n\nHowever, I will also reproduce the question here (just copy and paste, in GitHub Markdown format). Notice that I provide a complete `ls` for both branch tips and their merge-base, and since the problem has to do with a file-level merge conflict that I contend is generated by Git itself due to its overly aggressive directory rename detection behavior, that should be sufficient to describe the entire matter.\n\nAs I say in a comment on SO:\n\n`git diff --find-renames` from merge-base to master reveals that (as one can also see by eye) the vast preponderance is entirely new stuff in _wordpressed_ or stuff that was moved (renamed) from top level to _wordpressed_. There is exactly one existing rename where an article was moved from _dan_ to _wordpressed_: \n\n    diff --git a/dan/bdd.md b/wordpressed/bdd.md\n\nAnd yet on just that basis alone, Git assumes that this new article that popped up in _dan_ should have popped up in _wordpressed_ instead?? I'm sorry, I regard this as a Git bug. There needs at the very least to be a way to turn off this \"feature\", surely.\n\nThanks for listening. m.  \n\n========\n\nI've got a Git repo of Markdown articles, where people create pull requests (one per Markdown article) that eventually get merged to `master` and then, working directly on `master`, I move that article into a folder called _wordpressed_ and commit and push to `master`.\n\nAnd I've got a very old pull request that is still sitting there in a branch called `home-base`, consisting of a file _dan/homebase.md_ which I'm thinking of editing into shape with some more commits and then merging to `master`.\n\nThis `home-base` branch is _so_ old that the repo, seen from its point of view, is totally different from how it looks today in `master`. So I thought it might be good to reverse merge `master` into `home-base` just to bring it more up to date and move the merge base up a lot.\n\nBut when I try to do that, I get a merge conflict that I can't understand.\n\n----\n\nTo show you the situation, I'm going to show what a `ls` of files and folders looks like in each branch. I'll start with `master`:\n\n```\n$ git status\nOn branch master\nYour branch is up to date with 'origin/master'.\nnothing to commit, working tree clean\n\n$ ls -1 -R\nREADME.md\ndan\ninstructions.md\nwordpressed\n\n./dan:\n\n./wordpressed:\nAppDelegateRefactor.md\nBareListContentView.md\nCellContentConfiguration.md\nControlTargetActioniOS14.md\nGetStartedWithPods.md\nMultipleTrailingClosures.md\nStatesAndBackgrounds.md\nSwift52notes1.md\nSwift52notes2.md\nSwift52notes3.md\nUserInteractiveCells.md\nbdd-common-pitfalls.md\nbdd.md\nbrantsMenuBar.md\ncollectionViewContentConfig.md\ncollectionViewLists.md\ncollectionViewLists2.md\ncollectionViewOutline.md\ndebuggingLinks.md\ndiffableDataSources.md\ndiffableDataSources2.md\nforwardCompatibity.md\nhaptics.md\niOS13deprecations.md\nimages\ninAppPurchases.md\nlogging.md\nmiscSwiftTricks1.md\nmiscSwiftTricks2.md\nmiscSwiftTricks3.md\nmiscSwiftTricks4.md\nmultipleSelection.md\nmysteriesOfLayout.md\nnewInIOS14.md\npackages.md\nphotoPicker.md\nprotocolParadox.md\nsplitViewControllers.md\nsplitViewControllers2.md\nswiftTrickTwoClasses.md\ntargetActionRant.md\ntypescript-shape-of-things.md\nuntappableButton.md\nwhat-vs-how.md\nxcode12Editing.md\nxcode12testing.md\nxcodeWhereAmI.md\nxcodeWorkInTwoPlaces.md\n\n./wordpressed/images:\nargumentsInScheme.png\nbigFlags.png\ncallHierarchy.png\ncallersMenu.png\nchooser.png\nconfig.png\ncreatedTester.png\ndataOrUrl.png\nipadportraitoverlay.png\nlistOfPepBoys.png\nlistWithSectionHeaders.png\nlittleFlags.png\nlooksLikeADuck.png\nnewTester.png\noutlineIndicatorsWrong.png\noutlineIndicatorsWrong2.png\nperfectAfterHack.png\npickAPeppa.png\nrootOnly.png\nsandboxAccountOnDevice.png\nselfSizingStringDrawer.png\nsimpleList.png\nsplit1.png\nsplit2.png\nsplitViewControllerPadLandscape.png\ntableViewAsChoice.png\nthreecolumns.png\ntwoPossibilities.png\ntwoscreensphone.png\nvariableRowHeights.png\nworkingOutline.png\n```\n\nAs you can see, the main folder is itself mostly empty. All the articles are in _wordpressed_, along with their images which are in _wordpressed/images_.\n\n----\n\nOkay, now here is `home-base`:\n\n```\n$ git switch home-base\nSwitched to branch 'home-base'\nYour branch is up to date with 'origin/home-base'.\n\n$ ls -1 -R\nGetStartedWithPods.md\nREADME.md\nSwift52notes1.md\nSwift52notes2.md\nSwift52notes3.md\ndan\ndebuggingLinks.md\nimages\nlogging.md\nmiscSwiftTricks1.md\nmiscSwiftTricks2.md\nmiscSwiftTricks3.md\nmiscSwiftTricks4.md\npackages.md\nwordpressed\nxcode12Editing.md\nxcode12testing.md\nxcodeWhereAmI.md\nxcodeWorkInTwoPlaces.md\n\n./dan:\nbdd.md\nhomebase.md\n\n./images:\nchooser.png\nconfig.png\nsplit1.png\nsplit2.png\n\n./wordpressed:\nimages\n\n./wordpressed/images:\n```\n\nAs you can see, that's a much earlier situation. Nothing had been moved to _wordpressed_ yet. There were a few articles floating around the top level, and the _dan_ folder had two articles, including the one I'm interested in here, _homebase.md_.\n\n----\n\nFinally, for the sake of completeness, I'm going to show you the merge base between `master` and `home-base`:\n\n```\n$ git merge-base master home-base\nb5d7355fe42eddad96beb200df2cba65381c288a\n$ git checkout b5d7355fe\n$ ls -1 -R\nGetStartedWithPods.md\nREADME.md\nSwift52notes1.md\nSwift52notes2.md\nSwift52notes3.md\ndan\ndebuggingLinks.md\nimages\nlogging.md\nmiscSwiftTricks1.md\nmiscSwiftTricks2.md\nmiscSwiftTricks3.md\nmiscSwiftTricks4.md\npackages.md\nwordpressed\nxcode12Editing.md\nxcode12testing.md\nxcodeWhereAmI.md\nxcodeWorkInTwoPlaces.md\n\n./dan:\nbdd.md\n\n./images:\nchooser.png\nconfig.png\nsplit1.png\nsplit2.png\n\n./wordpressed:\nimages\n\n./wordpressed/images:\n```\n\nWell, it looks an awful lot like `home-base` looks now, doesn't it? The main difference is that that's before _homebase.md_ appeared in the _dan_ folder.\n\n----\n\nNow then. Ask yourself, please, what should happen when I try to merge `master` into `home-base`. What are contributions from both sides of the merge? In my view, Git should realize that in `master` a lot of new files appeared in _wordpressed_, and that some of them are renames of files that used to be at the top level. Plus, of course, in `home-base`, a new file _homebase.md_ has appeared in _dan_.\n\nThose contributions should not conflict (in my view).\n\nOkay, let's try it.\n\n```\n$ git switch home-base\n$ git merge master\nCONFLICT (file location): dan/homebase.md added in HEAD inside a directory that was renamed in master, suggesting it should perhaps be moved to wordpressed/homebase.md.\nAutomatic merge failed; fix conflicts and then commit the result.\n\n$ git status\nOn branch home-base\nYour branch is up to date with 'origin/home-base'.\n\nYou have unmerged paths.\n  (fix conflicts and run \"git commit\")\n  (use \"git merge --abort\" to abort the merge)\n\nChanges to be committed:\n\tdeleted:    dan/homebase.md\n\tnew file:   instructions.md\n\tnew file:   wordpressed/AppDelegateRefactor.md\n\tnew file:   wordpressed/BareListContentView.md\n\tnew file:   wordpressed/CellContentConfiguration.md\n\tnew file:   wordpressed/ControlTargetActioniOS14.md\n\trenamed:    GetStartedWithPods.md -> wordpressed/GetStartedWithPods.md\n\tnew file:   wordpressed/MultipleTrailingClosures.md\n\tnew file:   wordpressed/StatesAndBackgrounds.md\n\trenamed:    Swift52notes1.md -> wordpressed/Swift52notes1.md\n\trenamed:    Swift52notes2.md -> wordpressed/Swift52notes2.md\n\trenamed:    Swift52notes3.md -> wordpressed/Swift52notes3.md\n\tnew file:   wordpressed/UserInteractiveCells.md\n\tnew file:   wordpressed/bdd-common-pitfalls.md\n\trenamed:    dan/bdd.md -> wordpressed/bdd.md\n\tnew file:   wordpressed/brantsMenuBar.md\n\tnew file:   wordpressed/collectionViewContentConfig.md\n\tnew file:   wordpressed/collectionViewLists.md\n\tnew file:   wordpressed/collectionViewLists2.md\n\tnew file:   wordpressed/collectionViewOutline.md\n\trenamed:    debuggingLinks.md -> wordpressed/debuggingLinks.md\n\tnew file:   wordpressed/diffableDataSources.md\n\tnew file:   wordpressed/diffableDataSources2.md\n\tnew file:   wordpressed/forwardCompatibity.md\n\tnew file:   wordpressed/haptics.md\n\tnew file:   wordpressed/iOS13deprecations.md\n\tnew file:   wordpressed/images/argumentsInScheme.png\n\tnew file:   wordpressed/images/bigFlags.png\n\tnew file:   wordpressed/images/callHierarchy.png\n\tnew file:   wordpressed/images/callersMenu.png\n\trenamed:    images/chooser.png -> wordpressed/images/chooser.png\n\trenamed:    images/config.png -> wordpressed/images/config.png\n\tnew file:   wordpressed/images/createdTester.png\n\tnew file:   wordpressed/images/dataOrUrl.png\n\tnew file:   wordpressed/images/ipadportraitoverlay.png\n\tnew file:   wordpressed/images/listOfPepBoys.png\n\tnew file:   wordpressed/images/listWithSectionHeaders.png\n\tnew file:   wordpressed/images/littleFlags.png\n\tnew file:   wordpressed/images/looksLikeADuck.png\n\tnew file:   wordpressed/images/newTester.png\n\tnew file:   wordpressed/images/outlineIndicatorsWrong.png\n\tnew file:   wordpressed/images/outlineIndicatorsWrong2.png\n\tnew file:   wordpressed/images/perfectAfterHack.png\n\tnew file:   wordpressed/images/pickAPeppa.png\n\tnew file:   wordpressed/images/rootOnly.png\n\tnew file:   wordpressed/images/sandboxAccountOnDevice.png\n\tnew file:   wordpressed/images/selfSizingStringDrawer.png\n\tnew file:   wordpressed/images/simpleList.png\n\trenamed:    images/split1.png -> wordpressed/images/split1.png\n\trenamed:    images/split2.png -> wordpressed/images/split2.png\n\tnew file:   wordpressed/images/splitViewControllerPadLandscape.png\n\tnew file:   wordpressed/images/tableViewAsChoice.png\n\tnew file:   wordpressed/images/threecolumns.png\n\tnew file:   wordpressed/images/twoPossibilities.png\n\tnew file:   wordpressed/images/twoscreensphone.png\n\tnew file:   wordpressed/images/variableRowHeights.png\n\tnew file:   wordpressed/images/workingOutline.png\n\tnew file:   wordpressed/inAppPurchases.md\n\trenamed:    logging.md -> wordpressed/logging.md\n\trenamed:    miscSwiftTricks1.md -> wordpressed/miscSwiftTricks1.md\n\trenamed:    miscSwiftTricks2.md -> wordpressed/miscSwiftTricks2.md\n\trenamed:    miscSwiftTricks3.md -> wordpressed/miscSwiftTricks3.md\n\trenamed:    miscSwiftTricks4.md -> wordpressed/miscSwiftTricks4.md\n\tnew file:   wordpressed/multipleSelection.md\n\tnew file:   wordpressed/mysteriesOfLayout.md\n\tnew file:   wordpressed/newInIOS14.md\n\trenamed:    packages.md -> wordpressed/packages.md\n\tnew file:   wordpressed/photoPicker.md\n\tnew file:   wordpressed/protocolParadox.md\n\tnew file:   wordpressed/splitViewControllers.md\n\tnew file:   wordpressed/splitViewControllers2.md\n\tnew file:   wordpressed/swiftTrickTwoClasses.md\n\tnew file:   wordpressed/targetActionRant.md\n\tnew file:   wordpressed/typescript-shape-of-things.md\n\tnew file:   wordpressed/untappableButton.md\n\tnew file:   wordpressed/what-vs-how.md\n\trenamed:    xcode12Editing.md -> wordpressed/xcode12Editing.md\n\trenamed:    xcode12testing.md -> wordpressed/xcode12testing.md\n\trenamed:    xcodeWhereAmI.md -> wordpressed/xcodeWhereAmI.md\n\trenamed:    xcodeWorkInTwoPlaces.md -> wordpressed/xcodeWorkInTwoPlaces.md\n\nUnmerged paths:\n  (use \"git add <file>...\" to mark resolution)\n\tadded by us:     wordpressed/homebase.md \n```\n\n----\n\nOkay, so that last line is the one I don't understand. What does it mean to say that \"added by us\" is _wordpressed/homebase.md_? _NO!_ That's not where we added it. We added it in _dan_.\n\nNeither `master` nor `home-base` has a file _wordpressed/homebase.md_. So where is Git getting the idea that there is one??? It seems to have something to do with the same reason why it thinks there is a merge conflict at all:\n\n> _dan/homebase.md_ added in HEAD inside a directory that was renamed in master, suggesting it should perhaps be moved to wordpressed/homebase.md.\n\nSo: _What is Git talking about here?_ Why does Git think that _dan/homebase.md_ should be _wordpressed/homebase.md_, on no evidence whatever that I can find? How can I prevent this merge conflict and get Git to be more simplistic about this? Could I perhaps use a different merge strategy? Or how can I simply resolve the conflict in such a way as to say, No, Git, just leave _dan/homebase.md_ alone, please?\n\nAgain, apologies for the length of the detail in the question.\n\n========\n\n\n--\nmatt neuburg, phd = http://www.apeth.net/matt/\npantes anthropoi tou eidenai oregontai phusei\nProgramming iOS 14! https://www.oreilly.com/library/view/programming-ios-14/9781492092162/\niOS 14 Fundamentals! https://www.oreilly.com/library/view/ios-14-programming/9781492092087/\nRubyFrontier! http://www.apeth.com/RubyFrontierDocs/default.html\n\n"},{"id":"422001","messageId":"CABPp-BFm12ao-M9TVLpde3B7tkuX746t7Qs5V8-hEbshig0ZTw@mail.gmail.com","threadId":"55496","inReplyTo":"C0E64A34-AAA0-4ECE-8EC5-045815B3413D@tidbits.com","subject":"Re: possible bug due to directory rename detection","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-04-15T04:21:21Z","receivedAt":"2021-04-15T04:21:36Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Wed, Apr 14, 2021 at 5:51 PM Matt Neuburg <matt@tidbits.com> wrote:\n>\n> git version 2.24.3 (Apple Git-128) running under Big Sur on a MacBook Pro.\n>\n> Please see\n>\n>   https://stackoverflow.com/questions/67067686/git-file-level-merge-conflict-caused-by-git-suggesting-the-file-should-perhap\n>\n> for a complete description of what I saw and why I think it's wrong.\n>\n> However, I will also reproduce the question here (just copy and paste, in GitHub Markdown format). Notice that I provide a complete `ls` for both branch tips and their merge-base, and since the problem has to do with a file-level merge conflict that I contend is generated by Git itself due to its overly aggressive directory rename detection behavior, that should be sufficient to describe the entire matter.\n>\n> As I say in a comment on SO:\n>\n> `git diff --find-renames` from merge-base to master reveals that (as one can also see by eye) the vast preponderance is entirely new stuff in _wordpressed_ or stuff that was moved (renamed) from top level to _wordpressed_. There is exactly one existing rename where an article was moved from _dan_ to _wordpressed_:\n>\n>     diff --git a/dan/bdd.md b/wordpressed/bdd.md\n>\n> And yet on just that basis alone, Git assumes that this new article that popped up in _dan_ should have popped up in _wordpressed_ instead?? I'm sorry, I regard this as a Git bug. There needs at the very least to be a way to turn off this \"feature\", surely.\n\nYes, it's certainly possible to avoid the conflict.  There's two ways.\nFirst, you can just tell it to automatically accept the directory\nrenames without conflicting:\n  git config merge.directoryRenames true\nOr, you can tell it to never use these detected directory renames:\n  git config merge.directoryRenames false\nThe default behavior is equivalent to\n  git config merge.directoryRenames conflict\n\nDifferent people like different behaviors, there's no way to know\nwhich one _you_ prefer.  I know a lot of people running with\nmerge.directoryRenames set to true.  I once saw someone on this list\nwho wanted merge.directoryRenames to be false, and it sounds like you\nmay be the second.  There are almost certainly more folks.  But the\nfact that people have different preferences and there's no way to tell\nwhat would be wanted from looking at the history suggests that\nreporting a conflict is the only sane thing to do by default.\n\nIn case it helps, here's the parts from your output that are relevant...\n\n> Thanks for listening. m.\n\n> To show you the situation, I'm going to show what a `ls` of files and folders looks like in each branch. I'll start with `master`:\n>\n> ```\n> $ git status\n> On branch master\n> Your branch is up to date with 'origin/master'.\n> nothing to commit, working tree clean\n>\n> $ ls -1 -R\n...\n> ./dan:\n...doesn't have any files, thus isn't tracked by git...\n>\n> ./wordpressed:\n...snipped lots of other files...\n> bdd.md\n\n\n> Okay, now here is `home-base`:\n>\n> ```\n> $ git switch home-base\n> Switched to branch 'home-base'\n> Your branch is up to date with 'origin/home-base'.\n>\n> $ ls -1 -R\n...\n>\n> ./dan:\n> bdd.md\n> homebase.md\n...So two files in `dan`; bdd.md and homebase.md\n\n\n> Finally, for the sake of completeness, I'm going to show you the merge base between `master` and `home-base`:\n\nYes, it's critical to look at this too.\n\n> ```\n> $ git merge-base master home-base\n> b5d7355fe42eddad96beb200df2cba65381c288a\n> $ git checkout b5d7355fe\n> $ ls -1 -R\n...\n> ./dan:\n> bdd.md\n\nSo `dan` only had one file: bdd.md\n\n> Now then. Ask yourself, please, what should happen when I try to merge `master` into `home-base`. What are contributions from both sides of the merge? In my view, Git should realize that in `master` a lot of new files appeared in _wordpressed_, and that some of them are renames of files that used to be at the top level. Plus, of course, in `home-base`, a new file _homebase.md_ has appeared in _dan_.\n\nYes, and in addition every single file in the dan/ directory was moved\ninto the wordpressed/ directory in master (all one of them in this\ncase), while home-base added a new file to the old dan/ directory.\nOften, folks put files together in a directory because they are\nrelated and want to keep those files together.  When every file in a\ndirectory is moved to some new directory by one side of history, and\nthe other side of history adds a new file to the old directory,\nshouldn't the merge realize that and move the new file to the new\ndirectory?\n\nSome may say 'no', it shouldn't.  Many people say 'yes', it should.\nHow is git supposed to know?  It can't be based solely on looking at\nthe history; the relatedness of files in the same directory is some\nkind of semantic meaning that people don't record.  So, by default,\ngit marks it as a conflict and asks you to decide like this:\n\n> $ git switch home-base\n> $ git merge master\n> CONFLICT (file location): dan/homebase.md added in HEAD inside a directory that was renamed in master, suggesting it should perhaps be moved to wordpressed/homebase.md.\n> Automatic merge failed; fix conflicts and then commit the result.\n\nAs I mentioned above, you can set merge.directoryRenames if you want\nto avoid these conflicts, picking whichever of the two options other\nthan 'conflict' that you prefer.  Perhaps we should mention the\nmerge.directoryRenames config variable when one or more warnings of\nthis type are involved in a merge; otherwise it's not that\ndiscoverable.\n\n\nHope that helps,\nElijah\n"}]}