{"thread":{"id":"56439","subject":"Aborting 'rebase main feat' removes unversioned files","startedAt":"2021-09-03T20:33:19Z","lastAt":"2021-09-08T00:41:16Z","messageCount":15,"participants":["Fedor Biryukov","Bagas Sanjaya","Jeff King","Elijah Newren","Ævar Arnfjörð Bjarmason","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"434671","messageId":"C357A648-8B13-45C3-9388-C0C7F7D40DAE@gmail.com","threadId":"56439","inReplyTo":"CAG2t84Uaw-Kdp+EXU8CY1QYfykFQj-hGLJnTSH8MYO8Vi_yqgA@mail.gmail.com","subject":"Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-03T20:33:06Z","receivedAt":"2021-09-03T20:33:19Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"Looks like a bug in git rebase main feat.\n\nTo reproduce:\ngit init\ngit commit -m 'init' --allow-empty\ngit checkout -b feat\necho 123 > readme.txt\ngit add readme.txt\ngit commit -m 'txt=123'\ngit checkout main\necho 012 > readme.txt\ngit rebase main feat\ngit rebase --abort\n\nExpected result:\nreadme.txt contains 012\n\nActual result:\nreadme.txt contains 123\n\nAccording to the docs, git rebase main feat is a shorthand for git checkout feat followed by git rebase main. I have checked that doing checkout and rebase separately instead of using the shorthand does not have the same issue.\n\nCheers!\n\nFedor\n"},{"id":"434678","messageId":"dbe7d88d-4174-e080-03df-e35d0ac6004f@gmail.com","threadId":"56439","inReplyTo":"C357A648-8B13-45C3-9388-C0C7F7D40DAE@gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-09-04T06:57:11Z","receivedAt":"2021-09-04T06:57:21Z","isPatch":false,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"On 04/09/21 03.33, Fedor Biryukov wrote:\n> Looks like a bug in git rebase main feat.\n> \n> To reproduce:\n> git init\n> git commit -m 'init' --allow-empty\n> git checkout -b feat\n> echo 123 > readme.txt\n> git add readme.txt\n> git commit -m 'txt=123'\n> git checkout main\n> echo 012 > readme.txt\n> git rebase main feat\n> git rebase --abort\n> \n\nDid you forget committing?\n\n> Expected result:\n> readme.txt contains 012\n> \n> Actual result:\n> readme.txt contains 123\n> \n> According to the docs, git rebase main feat is a shorthand for git checkout feat followed by git rebase main. I have checked that doing checkout and rebase separately instead of using the shorthand does not have the same issue.\n\nI think this is non-issue (behavior as intended).\n\nSo when you say `git rebase main feat`, Git will rebase your commits in \nfeat on top of main. If any conflicts occur and you abort rebasing (`git \nrebase --abort`), your feat branch just looks like before rebasing.\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"434684","messageId":"YTNA6Qo6Yj5o9NmQ@coredump.intra.peff.net","threadId":"56439","inReplyTo":"dbe7d88d-4174-e080-03df-e35d0ac6004f@gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-09-04T09:48:25Z","receivedAt":"2021-09-04T09:48:31Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 04, 2021 at 01:57:11PM +0700, Bagas Sanjaya wrote:\n\n> On 04/09/21 03.33, Fedor Biryukov wrote:\n> > Looks like a bug in git rebase main feat.\n> > \n> > To reproduce:\n> > git init\n> > git commit -m 'init' --allow-empty\n> > git checkout -b feat\n> > echo 123 > readme.txt\n> > git add readme.txt\n> > git commit -m 'txt=123'\n> > git checkout main\n> > echo 012 > readme.txt\n> > git rebase main feat\n> > git rebase --abort\n> > \n> \n> Did you forget committing?\n\nI don't think so.\n\nThe point is that \"readme.txt\" is not a tracked file on the main branch,\nand thus Git should consider it precious.\n\nI don't think the \"rebase --abort\" is the problem here, though. It's the\ncommand before:\n\n  git rebase main feat\n\nThe \"feat\" branch is already ahead of \"main\" (which has no new commits),\nand so it just says:\n\n  Current branch feat is up to date.\n\nand leaves us on the \"feat\" branch. But in doing so, it overwrites the\nprecious untracked content in the working tree.\n\nThe \"git rebase --abort\" command then does nothing, because there's no\nrebase in progress.\n\n-Peff\n"},{"id":"434685","messageId":"CAG2t84Xe2XwdwdAK42bRrwAeNaB3-A+WhgSsJGOoW9-rG1S9Xg@mail.gmail.com","threadId":"56439","inReplyTo":"YTNA6Qo6Yj5o9NmQ@coredump.intra.peff.net","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-04T09:51:19Z","receivedAt":"2021-09-04T09:51:33Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"There is no way this could be the intended behavior, but the good news\nis that I cannot reproduce it...\nLooks like it occurs only in one git version (2.31.0.windows.1, IIRC).\nAnd it does not occur in the latest git version: git version 2.33.0.windows.2.\n\n-Fedor\n\nOn Sat, Sep 4, 2021 at 11:48 AM Jeff King <peff@peff.net> wrote:\n>\n> On Sat, Sep 04, 2021 at 01:57:11PM +0700, Bagas Sanjaya wrote:\n>\n> > On 04/09/21 03.33, Fedor Biryukov wrote:\n> > > Looks like a bug in git rebase main feat.\n> > >\n> > > To reproduce:\n> > > git init\n> > > git commit -m 'init' --allow-empty\n> > > git checkout -b feat\n> > > echo 123 > readme.txt\n> > > git add readme.txt\n> > > git commit -m 'txt=123'\n> > > git checkout main\n> > > echo 012 > readme.txt\n> > > git rebase main feat\n> > > git rebase --abort\n> > >\n> >\n> > Did you forget committing?\n>\n> I don't think so.\n>\n> The point is that \"readme.txt\" is not a tracked file on the main branch,\n> and thus Git should consider it precious.\n>\n> I don't think the \"rebase --abort\" is the problem here, though. It's the\n> command before:\n>\n>   git rebase main feat\n>\n> The \"feat\" branch is already ahead of \"main\" (which has no new commits),\n> and so it just says:\n>\n>   Current branch feat is up to date.\n>\n> and leaves us on the \"feat\" branch. But in doing so, it overwrites the\n> precious untracked content in the working tree.\n>\n> The \"git rebase --abort\" command then does nothing, because there's no\n> rebase in progress.\n>\n> -Peff\n"},{"id":"434686","messageId":"CAG2t84WY35RvieOc5mhRJExwHdXuaiBafB4S4APkrnGiBM_t_g@mail.gmail.com","threadId":"56439","inReplyTo":"CAG2t84Xe2XwdwdAK42bRrwAeNaB3-A+WhgSsJGOoW9-rG1S9Xg@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-04T09:58:03Z","receivedAt":"2021-09-04T09:58:18Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"Just tested it on my MacBook. To my surprise, the file got deleted by\n'git rebase main feat':\n\n550$ git --version\ngit version 2.33.0\n\n551$ git init -b main\nInitialized empty Git repository in\n/Users/ted/workspace/git-abort-bug/test/.git/\n\n552$ git commit -m 'init' --allow-empty\n[main (root-commit) ede7bdf] init\n\n553$ git checkout -b feat\nSwitched to a new branch 'feat'\n\n554$ echo 123 > readme.txt\n\n555$ git add readme.txt\n\n556$ git commit -m 'txt=123'\n[feat 29ac3de] txt=123\n 1 file changed, 1 insertion(+)\n create mode 100644 readme.txt\n\n557$ git checkout main\nSwitched to branch 'main'\n\n558$ echo 012 > readme.txt\n\n559$ git rebase main feat\nCurrent branch feat is up to date.\n\n560$ git rebase --abort\nfatal: No rebase in progress?\n\nOn Sat, Sep 4, 2021 at 11:51 AM Fedor Biryukov <fedor.birjukov@gmail.com> wrote:\n>\n> There is no way this could be the intended behavior, but the good news\n> is that I cannot reproduce it...\n> Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> And it does not occur in the latest git version: git version 2.33.0.windows.2.\n>\n> -Fedor\n>\n> On Sat, Sep 4, 2021 at 11:48 AM Jeff King <peff@peff.net> wrote:\n> >\n> > On Sat, Sep 04, 2021 at 01:57:11PM +0700, Bagas Sanjaya wrote:\n> >\n> > > On 04/09/21 03.33, Fedor Biryukov wrote:\n> > > > Looks like a bug in git rebase main feat.\n> > > >\n> > > > To reproduce:\n> > > > git init\n> > > > git commit -m 'init' --allow-empty\n> > > > git checkout -b feat\n> > > > echo 123 > readme.txt\n> > > > git add readme.txt\n> > > > git commit -m 'txt=123'\n> > > > git checkout main\n> > > > echo 012 > readme.txt\n> > > > git rebase main feat\n> > > > git rebase --abort\n> > > >\n> > >\n> > > Did you forget committing?\n> >\n> > I don't think so.\n> >\n> > The point is that \"readme.txt\" is not a tracked file on the main branch,\n> > and thus Git should consider it precious.\n> >\n> > I don't think the \"rebase --abort\" is the problem here, though. It's the\n> > command before:\n> >\n> >   git rebase main feat\n> >\n> > The \"feat\" branch is already ahead of \"main\" (which has no new commits),\n> > and so it just says:\n> >\n> >   Current branch feat is up to date.\n> >\n> > and leaves us on the \"feat\" branch. But in doing so, it overwrites the\n> > precious untracked content in the working tree.\n> >\n> > The \"git rebase --abort\" command then does nothing, because there's no\n> > rebase in progress.\n> >\n> > -Peff\n"},{"id":"434687","messageId":"CAG2t84Ud9AFfHhCxyKWvcqVUKTrkRufdMsQL93cdUUvVAyNpzA@mail.gmail.com","threadId":"56439","inReplyTo":"CAG2t84WY35RvieOc5mhRJExwHdXuaiBafB4S4APkrnGiBM_t_g@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-04T10:03:49Z","receivedAt":"2021-09-04T10:04:03Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"Here's the output from Windows, where everything works as expected.\n\nPS> git --version\ngit version 2.33.0.windows.2\nPS> rm -re -fo *\nPS> git init -b main\nInitialized empty Git repository in C:/Users/ted/test/.git/\nPS> git commit -m 'init' --allow-empty\n[main (root-commit) ea4422d] init\nPS> git checkout -b feat\nSwitched to a new branch 'feat'\nPS> echo 123 > readme.txt\nPS> git add readme.txt\nPS> git commit -m 'txt=123'\n[feat 82fe34b] txt=123\n 1 file changed, 0 insertions(+), 0 deletions(-)\n create mode 100644 readme.txt\nPS> git checkout main\nSwitched to branch 'main'\nPS> echo 012 > readme.txt\nPS> git status\nOn branch main\nUntracked files:\n  (use \"git add <file>...\" to include in what will be committed)\n        readme.txt\n\nnothing added to commit but untracked files present (use \"git add\" to track)\nPS> git rebase main feat\nerror: The following untracked working tree files would be overwritten\nby checkout:\n        readme.txt\nPlease move or remove them before you switch branches.\nAborting\nerror: could not detach HEAD\nPS> git rebase --abort\nfatal: No rebase in progress?\n\nOn Sat, Sep 4, 2021 at 11:58 AM Fedor Biryukov <fedor.birjukov@gmail.com> wrote:\n>\n> Just tested it on my MacBook. To my surprise, the file got deleted by\n> 'git rebase main feat':\n>\n> 550$ git --version\n> git version 2.33.0\n>\n> 551$ git init -b main\n> Initialized empty Git repository in\n> /Users/ted/workspace/git-abort-bug/test/.git/\n>\n> 552$ git commit -m 'init' --allow-empty\n> [main (root-commit) ede7bdf] init\n>\n> 553$ git checkout -b feat\n> Switched to a new branch 'feat'\n>\n> 554$ echo 123 > readme.txt\n>\n> 555$ git add readme.txt\n>\n> 556$ git commit -m 'txt=123'\n> [feat 29ac3de] txt=123\n>  1 file changed, 1 insertion(+)\n>  create mode 100644 readme.txt\n>\n> 557$ git checkout main\n> Switched to branch 'main'\n>\n> 558$ echo 012 > readme.txt\n>\n> 559$ git rebase main feat\n> Current branch feat is up to date.\n>\n> 560$ git rebase --abort\n> fatal: No rebase in progress?\n>\n> On Sat, Sep 4, 2021 at 11:51 AM Fedor Biryukov <fedor.birjukov@gmail.com> wrote:\n> >\n> > There is no way this could be the intended behavior, but the good news\n> > is that I cannot reproduce it...\n> > Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> > And it does not occur in the latest git version: git version 2.33.0.windows.2.\n> >\n> > -Fedor\n> >\n> > On Sat, Sep 4, 2021 at 11:48 AM Jeff King <peff@peff.net> wrote:\n> > >\n> > > On Sat, Sep 04, 2021 at 01:57:11PM +0700, Bagas Sanjaya wrote:\n> > >\n> > > > On 04/09/21 03.33, Fedor Biryukov wrote:\n> > > > > Looks like a bug in git rebase main feat.\n> > > > >\n> > > > > To reproduce:\n> > > > > git init\n> > > > > git commit -m 'init' --allow-empty\n> > > > > git checkout -b feat\n> > > > > echo 123 > readme.txt\n> > > > > git add readme.txt\n> > > > > git commit -m 'txt=123'\n> > > > > git checkout main\n> > > > > echo 012 > readme.txt\n> > > > > git rebase main feat\n> > > > > git rebase --abort\n> > > > >\n> > > >\n> > > > Did you forget committing?\n> > >\n> > > I don't think so.\n> > >\n> > > The point is that \"readme.txt\" is not a tracked file on the main branch,\n> > > and thus Git should consider it precious.\n> > >\n> > > I don't think the \"rebase --abort\" is the problem here, though. It's the\n> > > command before:\n> > >\n> > >   git rebase main feat\n> > >\n> > > The \"feat\" branch is already ahead of \"main\" (which has no new commits),\n> > > and so it just says:\n> > >\n> > >   Current branch feat is up to date.\n> > >\n> > > and leaves us on the \"feat\" branch. But in doing so, it overwrites the\n> > > precious untracked content in the working tree.\n> > >\n> > > The \"git rebase --abort\" command then does nothing, because there's no\n> > > rebase in progress.\n> > >\n> > > -Peff\n"},{"id":"434688","messageId":"YTNH2vMPuEW4SBMo@coredump.intra.peff.net","threadId":"56439","inReplyTo":"CAG2t84Xe2XwdwdAK42bRrwAeNaB3-A+WhgSsJGOoW9-rG1S9Xg@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-09-04T10:18:02Z","receivedAt":"2021-09-04T10:18:04Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 04, 2021 at 11:51:19AM +0200, Fedor Biryukov wrote:\n\n> There is no way this could be the intended behavior, but the good news\n> is that I cannot reproduce it...\n> Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> And it does not occur in the latest git version: git version 2.33.0.windows.2.\n\nI think it is a bug, and it seems to reproduce easily for me (with both\nthe current tip of master, and with v2.33.0). Here's the recipe you\nshowed, with a little debugging at the end:\n\n  set -x\n  git init repo\n  cd repo\n  git commit -m base --allow-empty\n  git checkout -b feat\n  echo feat >readme.txt\n  git add readme.txt\n  git commit -m txt=feat\n  git checkout main\n  echo precious >readme.txt\n\n  cat readme.txt\n  git checkout feat\n  cat readme.txt\n  git rebase main feat\n  cat readme.txt\n\nThis produces:\n\n  + cat readme.txt\n  precious\n  + git checkout feat\n  error: The following untracked working tree files would be overwritten by checkout:\n  \treadme.txt\n  Please move or remove them before you switch branches.\n  Aborting\n  + cat readme.txt\n  precious\n  + git rebase main feat\n  Current branch feat is up to date.\n  + cat readme.txt\n  feat\n\nSo git-checkout was not willing to overwrite the untracked content, but\nrebase was happy to obliterate it.\n\nIt does the right thing in very old versions. Bisecting, it looks like\nthe problem arrived in 5541bd5b8f (rebase: default to using the builtin\nrebase, 2018-08-08). So the bug is in the conversion from the legacy\nshell script to C (which makes sense; the shell version was just calling\n\"git checkout\", which we know does the right thing).\n\n-Peff\n"},{"id":"434689","messageId":"YTNJeY2U6tcOQiCh@coredump.intra.peff.net","threadId":"56439","inReplyTo":"CAG2t84Ud9AFfHhCxyKWvcqVUKTrkRufdMsQL93cdUUvVAyNpzA@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-09-04T10:24:57Z","receivedAt":"2021-09-04T10:25:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 04, 2021 at 12:03:49PM +0200, Fedor Biryukov wrote:\n\n> Here's the output from Windows, where everything works as expected.\n> [...]\n> PS> git rebase main feat\n> error: The following untracked working tree files would be overwritten\n> by checkout:\n>         readme.txt\n> Please move or remove them before you switch branches.\n> Aborting\n> error: could not detach HEAD\n\nInteresting that it behaves differently than the mac and linux versions.\nThere are some changes between Git for Windows and regular upstream Git,\nbut looking over the diff between the v2.33.0 releases, I don't see\nanything that might cause this discrepancy.\n\nIf the problem used to occur on v2.31 and now doesn't, it might be\ninteresting to bisect it.\n\n-Peff\n"},{"id":"434703","messageId":"CAG2t84WrV7OZ_0KqOXsNCevEt046hyd-CAHCbWFBY4uBCFjLDg@mail.gmail.com","threadId":"56439","inReplyTo":"YTNJeY2U6tcOQiCh@coredump.intra.peff.net","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-04T18:32:09Z","receivedAt":"2021-09-04T18:32:22Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"Actually it's the same on all operating systems. I just forgot to add\na new base commit in main in the reproduction scenario - there was\nnothing to rebase...\nHere is the correct scenario along with the output.\n\n+ git init -b main repo\nInitialized empty Git repository in\n/Users/ted/workspace/git-abort-bug/repo/.git/\n+ cd repo\n+ git commit -m base --allow-empty\n[main (root-commit) 3299694] base\n+ git checkout -b feat\nSwitched to a new branch 'feat'\n+ echo feat >readme.txt\n+ git add readme.txt\n+ git commit -m txt=feat\n[feat c9dc9b4] txt=feat\n 1 file changed, 1 insertion(+)\n create mode 100644 readme.txt\n+ git checkout main\nSwitched to branch 'main'\n+ git commit -m new-base --allow-empty\n[main c5c292c] new-base\n+ echo precious > readme.txt\n+ git rebase main feat\nerror: The following untracked working tree files would be overwritten by merge:\nreadme.txt\nPlease move or remove them before you merge.\nAborting\nhint: Could not execute the todo command\nhint:\nhint:     pick c9dc9b4d5aca461f1a1bedd3b99b5c8533d5ef10 txt=feat\nhint:\nhint: It has been rescheduled; To edit the command before continuing, please\nhint: edit the todo list first:\nhint:\nhint:     git rebase --edit-todo\nhint:     git rebase --continue\nCould not apply c9dc9b4... txt=feat\n+ cat readme.txt\nprecious\n+ git rebase --abort\n+ cat readme.txt\nfeat\n\n-Fedor\n\nOn Sat, Sep 4, 2021 at 12:24 PM Jeff King <peff@peff.net> wrote:\n>\n> On Sat, Sep 04, 2021 at 12:03:49PM +0200, Fedor Biryukov wrote:\n>\n> > Here's the output from Windows, where everything works as expected.\n> > [...]\n> > PS> git rebase main feat\n> > error: The following untracked working tree files would be overwritten\n> > by checkout:\n> >         readme.txt\n> > Please move or remove them before you switch branches.\n> > Aborting\n> > error: could not detach HEAD\n>\n> Interesting that it behaves differently than the mac and linux versions.\n> There are some changes between Git for Windows and regular upstream Git,\n> but looking over the diff between the v2.33.0 releases, I don't see\n> anything that might cause this discrepancy.\n>\n> If the problem used to occur on v2.31 and now doesn't, it might be\n> interesting to bisect it.\n>\n> -Peff\n"},{"id":"434710","messageId":"CABPp-BFyR19ch71W10oJDFuRX1OHzQ3si971pMn6dPtHKxJDXQ@mail.gmail.com","threadId":"56439","inReplyTo":"YTNH2vMPuEW4SBMo@coredump.intra.peff.net","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-09-05T05:32:57Z","receivedAt":"2021-09-05T05:42:27Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sat, Sep 4, 2021 at 3:19 AM Jeff King <peff@peff.net> wrote:\n>\n> On Sat, Sep 04, 2021 at 11:51:19AM +0200, Fedor Biryukov wrote:\n>\n> > There is no way this could be the intended behavior, but the good news\n> > is that I cannot reproduce it...\n> > Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> > And it does not occur in the latest git version: git version 2.33.0.windows.2.\n>\n> I think it is a bug, and it seems to reproduce easily for me (with both\n> the current tip of master, and with v2.33.0). Here's the recipe you\n> showed, with a little debugging at the end:\n>\n>   set -x\n>   git init repo\n>   cd repo\n>   git commit -m base --allow-empty\n>   git checkout -b feat\n>   echo feat >readme.txt\n>   git add readme.txt\n>   git commit -m txt=feat\n>   git checkout main\n>   echo precious >readme.txt\n>\n>   cat readme.txt\n>   git checkout feat\n>   cat readme.txt\n>   git rebase main feat\n>   cat readme.txt\n>\n> This produces:\n>\n>   + cat readme.txt\n>   precious\n>   + git checkout feat\n>   error: The following untracked working tree files would be overwritten by checkout:\n>         readme.txt\n>   Please move or remove them before you switch branches.\n>   Aborting\n>   + cat readme.txt\n>   precious\n>   + git rebase main feat\n>   Current branch feat is up to date.\n>   + cat readme.txt\n>   feat\n>\n> So git-checkout was not willing to overwrite the untracked content, but\n> rebase was happy to obliterate it.\n>\n> It does the right thing in very old versions. Bisecting, it looks like\n> the problem arrived in 5541bd5b8f (rebase: default to using the builtin\n> rebase, 2018-08-08). So the bug is in the conversion from the legacy\n> shell script to C (which makes sense; the shell version was just calling\n> \"git checkout\", which we know does the right thing).\n>\n> -Peff\n\nTurns out this is quite a mess.  It's also related to the \"don't\nremove empty working directories\" discussion we had earlier this\nweek[1], because we assumed all relevant codepaths correctly avoided\ndeleting untracked files and directories in the way.  But they don't.\nAnd rebase isn't the only offender, because this is buried in\nunpack_trees.  In fact, it traces back to (and before)\n    fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\nworking tree files.\", 2006-05-17)\nwhich has additional commentary over at\nhttps://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\nIt appears that before this time, git happily nuked untracked files\nand considered them expendable, in basically all cases.  However, this\npatch continued considering them as expendable whenever opts->reset\nwas true.  There wasn't much comment about it at the time for the\nreasoning of how opts->reset was handled, though trying to read\nbetween the lines perhaps Junio was trying to limit the backward\ncompatibility concerns of introducing new errors to fewer code paths?\nAnyway, Junio did mention `read-tree --reset` explicitly, but this\nopts->reset usage also occurs in am, checkout, reset -- and anything\nthat calls the reset_head() function including: rebase, stash,\nsequencer.c, and add-patch.c.\n\nSo, then...should we preserve untracked (and non-ignored) files in all\nthese cases?  This rebase case seems clear, but others might be less\nclear.  For example, should \"git reset --hard\" nuke untracked files\n(what if it's a directory of untracked files getting nuked just to\nplace a single file in the location of the directory)?  The\ndocumentation isn't explicit, but after reading it I would assume that\nuntracked files should be preserved.  Since we've had bugs in \"git\nreset --hard\" before, such as requiring two invocations in order to\nclear out unmerged entries (see [2] and [3]), that also suggests that\nthis is just another bug in the same area.  But the bug has been\naround so long that people might be expecting it; our testsuite has\nseveral cases that incidentally do.  Granted, it's better to throw an\nerror and require explicit extra steps than to nuke potentially\nimportant work, but some folks might be unhappy with a change here.\nSimilarly with \"git checkout -f\".\n\nAnd stash.c, which operates in that edge case area has tests with\nfiles nuked from the cache without nuking it from the working tree\n(causing the working tree file to be considered untracked), and then\nattempts to have multiple tests operate on that kind of case.  Those\ncases look a bit buggy to me for other reasons (I'm still digging),\nbut those bugs are kind of hidden by the untracked file nuking bugs,\nso fixing the latter requires fixing the former.  And stash.c is a\nmess of shelling out to subcommands.  Ick.\n\nI have some partial patches, but don't know if I'll have anything to\npost until I figure out the stash mess...\n\n[1] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/\n[2] 25c200a700 (\"t1015: demonstrate directory/file conflict recovery\nfailures\", 2018-07-31)\n[3] ad3762042a (\"read-cache: fix directory/file conflict handling in\nread_index_unmerged()\", 2018-07-31)\n"},{"id":"434717","messageId":"87sfyjpiu3.fsf@evledraar.gmail.com","threadId":"56439","inReplyTo":"CABPp-BFyR19ch71W10oJDFuRX1OHzQ3si971pMn6dPtHKxJDXQ@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-09-05T07:43:08Z","receivedAt":"2021-09-05T07:44:42Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Sat, Sep 04 2021, Elijah Newren wrote:\n\n> On Sat, Sep 4, 2021 at 3:19 AM Jeff King <peff@peff.net> wrote:\n>>\n>> On Sat, Sep 04, 2021 at 11:51:19AM +0200, Fedor Biryukov wrote:\n>>\n>> > There is no way this could be the intended behavior, but the good news\n>> > is that I cannot reproduce it...\n>> > Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n>> > And it does not occur in the latest git version: git version 2.33.0.windows.2.\n>>\n>> I think it is a bug, and it seems to reproduce easily for me (with both\n>> the current tip of master, and with v2.33.0). Here's the recipe you\n>> showed, with a little debugging at the end:\n>>\n>>   set -x\n>>   git init repo\n>>   cd repo\n>>   git commit -m base --allow-empty\n>>   git checkout -b feat\n>>   echo feat >readme.txt\n>>   git add readme.txt\n>>   git commit -m txt=feat\n>>   git checkout main\n>>   echo precious >readme.txt\n>>\n>>   cat readme.txt\n>>   git checkout feat\n>>   cat readme.txt\n>>   git rebase main feat\n>>   cat readme.txt\n>>\n>> This produces:\n>>\n>>   + cat readme.txt\n>>   precious\n>>   + git checkout feat\n>>   error: The following untracked working tree files would be overwritten by checkout:\n>>         readme.txt\n>>   Please move or remove them before you switch branches.\n>>   Aborting\n>>   + cat readme.txt\n>>   precious\n>>   + git rebase main feat\n>>   Current branch feat is up to date.\n>>   + cat readme.txt\n>>   feat\n>>\n>> So git-checkout was not willing to overwrite the untracked content, but\n>> rebase was happy to obliterate it.\n>>\n>> It does the right thing in very old versions. Bisecting, it looks like\n>> the problem arrived in 5541bd5b8f (rebase: default to using the builtin\n>> rebase, 2018-08-08). So the bug is in the conversion from the legacy\n>> shell script to C (which makes sense; the shell version was just calling\n>> \"git checkout\", which we know does the right thing).\n>>\n>> -Peff\n>\n> Turns out this is quite a mess.  It's also related to the \"don't\n> remove empty working directories\" discussion we had earlier this\n> week[1], because we assumed all relevant codepaths correctly avoided\n> deleting untracked files and directories in the way.  But they don't.\n> And rebase isn't the only offender, because this is buried in\n> unpack_trees.  In fact, it traces back to (and before)\n>     fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\n> working tree files.\", 2006-05-17)\n> which has additional commentary over at\n> https://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\n> It appears that before this time, git happily nuked untracked files\n> and considered them expendable, in basically all cases.  However, this\n> patch continued considering them as expendable whenever opts->reset\n> was true.  There wasn't much comment about it at the time for the\n> reasoning of how opts->reset was handled, though trying to read\n> between the lines perhaps Junio was trying to limit the backward\n> compatibility concerns of introducing new errors to fewer code paths?\n> Anyway, Junio did mention `read-tree --reset` explicitly, but this\n> opts->reset usage also occurs in am, checkout, reset -- and anything\n> that calls the reset_head() function including: rebase, stash,\n> sequencer.c, and add-patch.c.\n>\n> So, then...should we preserve untracked (and non-ignored) files in all\n> these cases?  This rebase case seems clear, but others might be less\n> clear.  For example, should \"git reset --hard\" nuke untracked files\n> (what if it's a directory of untracked files getting nuked just to\n> place a single file in the location of the directory)?  The\n> documentation isn't explicit, but after reading it I would assume that\n> untracked files should be preserved.  Since we've had bugs in \"git\n> reset --hard\" before, such as requiring two invocations in order to\n> clear out unmerged entries (see [2] and [3]), that also suggests that\n> this is just another bug in the same area.  But the bug has been\n> around so long that people might be expecting it; our testsuite has\n> several cases that incidentally do.  Granted, it's better to throw an\n> error and require explicit extra steps than to nuke potentially\n> important work, but some folks might be unhappy with a change here.\n> Similarly with \"git checkout -f\".\n>\n> And stash.c, which operates in that edge case area has tests with\n> files nuked from the cache without nuking it from the working tree\n> (causing the working tree file to be considered untracked), and then\n> attempts to have multiple tests operate on that kind of case.  Those\n> cases look a bit buggy to me for other reasons (I'm still digging),\n> but those bugs are kind of hidden by the untracked file nuking bugs,\n> so fixing the latter requires fixing the former.  And stash.c is a\n> mess of shelling out to subcommands.  Ick.\n>\n> I have some partial patches, but don't know if I'll have anything to\n> post until I figure out the stash mess...\n\nI'd just like to applaud this effort, and also suggest that the most\nuseful result of any such findings would be for us to produce some new\ntest in t/ showing these various cases of nuking/clobbering and other\n\"non-precious\" edge cases in this logic. See[1] and its linked [2] for\nreferences to some of the past discussions around these cases.\n\n1. https://lore.kernel.org/git/87a6q9kacx.fsf@evledraar.gmail.com/\n2. https://lore.kernel.org/git/87ftsi68ke.fsf@evledraar.gmail.com/\n\n> [1] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/\n> [2] 25c200a700 (\"t1015: demonstrate directory/file conflict recovery\n> failures\", 2018-07-31)\n> [3] ad3762042a (\"read-cache: fix directory/file conflict handling in\n> read_index_unmerged()\", 2018-07-31)\n\n"},{"id":"434723","messageId":"CAG2t84VNc=y2exEmq9UufBoeSSKqW45oXrjvtNfd7jBrsc0xtg@mail.gmail.com","threadId":"56439","inReplyTo":"87sfyjpiu3.fsf@evledraar.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Fedor Biryukov","fromEmail":"fedor.birjukov@gmail.com","sentAt":"2021-09-05T10:05:22Z","receivedAt":"2021-09-05T10:05:36Z","isPatch":false,"sender":{"key":"fedor.birjukov@gmail.com","avatar":null},"body":"Had a brief look at the code.\nbuiltin/rebase.c is using rebase_head helper function defined in\nrebase.c/h to do a checkout.\nbuiltin/checkout.c does not use the rebase_head helper function.\nI would expect both to use the same code defined in checkout.c/h, but\nthere is no such file.\n\nOn Sun, Sep 5, 2021 at 9:44 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:\n>\n>\n> On Sat, Sep 04 2021, Elijah Newren wrote:\n>\n> > On Sat, Sep 4, 2021 at 3:19 AM Jeff King <peff@peff.net> wrote:\n> >>\n> >> On Sat, Sep 04, 2021 at 11:51:19AM +0200, Fedor Biryukov wrote:\n> >>\n> >> > There is no way this could be the intended behavior, but the good news\n> >> > is that I cannot reproduce it...\n> >> > Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> >> > And it does not occur in the latest git version: git version 2.33.0.windows.2.\n> >>\n> >> I think it is a bug, and it seems to reproduce easily for me (with both\n> >> the current tip of master, and with v2.33.0). Here's the recipe you\n> >> showed, with a little debugging at the end:\n> >>\n> >>   set -x\n> >>   git init repo\n> >>   cd repo\n> >>   git commit -m base --allow-empty\n> >>   git checkout -b feat\n> >>   echo feat >readme.txt\n> >>   git add readme.txt\n> >>   git commit -m txt=feat\n> >>   git checkout main\n> >>   echo precious >readme.txt\n> >>\n> >>   cat readme.txt\n> >>   git checkout feat\n> >>   cat readme.txt\n> >>   git rebase main feat\n> >>   cat readme.txt\n> >>\n> >> This produces:\n> >>\n> >>   + cat readme.txt\n> >>   precious\n> >>   + git checkout feat\n> >>   error: The following untracked working tree files would be overwritten by checkout:\n> >>         readme.txt\n> >>   Please move or remove them before you switch branches.\n> >>   Aborting\n> >>   + cat readme.txt\n> >>   precious\n> >>   + git rebase main feat\n> >>   Current branch feat is up to date.\n> >>   + cat readme.txt\n> >>   feat\n> >>\n> >> So git-checkout was not willing to overwrite the untracked content, but\n> >> rebase was happy to obliterate it.\n> >>\n> >> It does the right thing in very old versions. Bisecting, it looks like\n> >> the problem arrived in 5541bd5b8f (rebase: default to using the builtin\n> >> rebase, 2018-08-08). So the bug is in the conversion from the legacy\n> >> shell script to C (which makes sense; the shell version was just calling\n> >> \"git checkout\", which we know does the right thing).\n> >>\n> >> -Peff\n> >\n> > Turns out this is quite a mess.  It's also related to the \"don't\n> > remove empty working directories\" discussion we had earlier this\n> > week[1], because we assumed all relevant codepaths correctly avoided\n> > deleting untracked files and directories in the way.  But they don't.\n> > And rebase isn't the only offender, because this is buried in\n> > unpack_trees.  In fact, it traces back to (and before)\n> >     fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\n> > working tree files.\", 2006-05-17)\n> > which has additional commentary over at\n> > https://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\n> > It appears that before this time, git happily nuked untracked files\n> > and considered them expendable, in basically all cases.  However, this\n> > patch continued considering them as expendable whenever opts->reset\n> > was true.  There wasn't much comment about it at the time for the\n> > reasoning of how opts->reset was handled, though trying to read\n> > between the lines perhaps Junio was trying to limit the backward\n> > compatibility concerns of introducing new errors to fewer code paths?\n> > Anyway, Junio did mention `read-tree --reset` explicitly, but this\n> > opts->reset usage also occurs in am, checkout, reset -- and anything\n> > that calls the reset_head() function including: rebase, stash,\n> > sequencer.c, and add-patch.c.\n> >\n> > So, then...should we preserve untracked (and non-ignored) files in all\n> > these cases?  This rebase case seems clear, but others might be less\n> > clear.  For example, should \"git reset --hard\" nuke untracked files\n> > (what if it's a directory of untracked files getting nuked just to\n> > place a single file in the location of the directory)?  The\n> > documentation isn't explicit, but after reading it I would assume that\n> > untracked files should be preserved.  Since we've had bugs in \"git\n> > reset --hard\" before, such as requiring two invocations in order to\n> > clear out unmerged entries (see [2] and [3]), that also suggests that\n> > this is just another bug in the same area.  But the bug has been\n> > around so long that people might be expecting it; our testsuite has\n> > several cases that incidentally do.  Granted, it's better to throw an\n> > error and require explicit extra steps than to nuke potentially\n> > important work, but some folks might be unhappy with a change here.\n> > Similarly with \"git checkout -f\".\n> >\n> > And stash.c, which operates in that edge case area has tests with\n> > files nuked from the cache without nuking it from the working tree\n> > (causing the working tree file to be considered untracked), and then\n> > attempts to have multiple tests operate on that kind of case.  Those\n> > cases look a bit buggy to me for other reasons (I'm still digging),\n> > but those bugs are kind of hidden by the untracked file nuking bugs,\n> > so fixing the latter requires fixing the former.  And stash.c is a\n> > mess of shelling out to subcommands.  Ick.\n> >\n> > I have some partial patches, but don't know if I'll have anything to\n> > post until I figure out the stash mess...\n>\n> I'd just like to applaud this effort, and also suggest that the most\n> useful result of any such findings would be for us to produce some new\n> test in t/ showing these various cases of nuking/clobbering and other\n> \"non-precious\" edge cases in this logic. See[1] and its linked [2] for\n> references to some of the past discussions around these cases.\n>\n> 1. https://lore.kernel.org/git/87a6q9kacx.fsf@evledraar.gmail.com/\n> 2. https://lore.kernel.org/git/87ftsi68ke.fsf@evledraar.gmail.com/\n>\n> > [1] https://lore.kernel.org/git/YS8eEtwQvF7TaLCb@coredump.intra.peff.net/\n> > [2] 25c200a700 (\"t1015: demonstrate directory/file conflict recovery\n> > failures\", 2018-07-31)\n> > [3] ad3762042a (\"read-cache: fix directory/file conflict handling in\n> > read_index_unmerged()\", 2018-07-31)\n>\n"},{"id":"434737","messageId":"xmqqr1e2ejs9.fsf@gitster.g","threadId":"56439","inReplyTo":"CABPp-BFyR19ch71W10oJDFuRX1OHzQ3si971pMn6dPtHKxJDXQ@mail.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-09-05T22:31:50Z","receivedAt":"2021-09-05T22:31:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren <newren@gmail.com> writes:\n\n> unpack_trees.  In fact, it traces back to (and before)\n>     fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\n> working tree files.\", 2006-05-17)\n> which has additional commentary over at\n> https://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\n> It appears that before this time, git happily nuked untracked files\n> and considered them expendable, in basically all cases.  However, this\n> patch continued considering them as expendable whenever opts->reset\n> was true.\n\nThanks for digging.  Yes, the 'reset' bit was treated as the license\nto kill untracked working tree files and directories that get in the\nway in order to carry out the unpack_trees operation.\n\n> So, then...should we preserve untracked (and non-ignored) files in all\n> these cases?  This rebase case seems clear, but others might be less\n> clear....\n\nIn short, the guiding principle ought to be that \"checkout --force\"\nand anything that is given \"force\" as a stronger override should be\nallowed to do whatever minimally necessary to match the end result\nin the working tree to what the command wants to show in the absense\nof these untracked paths.  And without being forced, untracked and\nunignored paths are precious and should cause commands to fail, if\nthey need to be touched for the commands to complete what they are\nasked to do [*].\n\n\"reset --hard HEAD\" is an oddball.\n\nNaïvely, because it is often used as the way to tell Git to \"no\nmatter what, match the working tree to HEAD\", even though it does\nnot have an explicit \"--force\" on the command line, it feels that it\nalso should be allowed to do whatever necessary to the working tree\nfiles.  And historically, that is what we wanted to implement.  If\nwe suddenly made it \"safer\", I am sure a lot of existing things will\nbreak.\n\nBut unfortunately, \"--hard\" means a bit more than that in the\ncontext of \"reset\", in that we want to reset in a way that is\ndifferent from \"--mixed\" (reset the index only without touching the\nworking tree) and \"--soft\" (do not touch the index or the working\ntree), more specifically, with \"--hard\", we want to reset both the\nindex and the working tree to match the given committish (often\n\"HEAD\").  From that point of view, \"reset --hard\" that tries to\npreserve untracked and unignored paths, and \"reset --force --hard\"\nthat does whatever necessary to untracked and unignored paths to\nmatch the working tree files, when they reset the index and the\nworking tree to the named committish, may have made sense.  If we\nwere designing the feature without any existing users, it is no\nbrainer to imagine that our design would: (1) call the three 'reset'\nmodes as \"both\", \"index-only\" and \"neither\", instead of \"hard\",\n\"mixed\" and \"soft\", and (2) require \"--force\" to touch untracked and\nunignored paths.\n\nAnd I think that may be a reasonable longer-term goal, but since we\nhave existing users and scripts, we cannot go there overnight without\ndevising a migration path.\n\nThanks.\n\n\n[Footnote]\n\n* We sometimes talk about adding a precious category of paths that\n  are \"ignored\", but this discussion is orthogonal to that.  Things\n  that are not tracked and not ignored ought to be precious in\n  principle.\n"},{"id":"434979","messageId":"CABPp-BFh2wGaDNjQD08bipE1t+B=0m0OBzXpYKR+17HADTHwCg@mail.gmail.com","threadId":"56439","inReplyTo":"87sfyjpiu3.fsf@evledraar.gmail.com","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-09-08T00:40:27Z","receivedAt":"2021-09-08T00:40:41Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sun, Sep 5, 2021 at 12:44 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n> On Sat, Sep 04 2021, Elijah Newren wrote:\n>\n> > On Sat, Sep 4, 2021 at 3:19 AM Jeff King <peff@peff.net> wrote:\n> >>\n> >> On Sat, Sep 04, 2021 at 11:51:19AM +0200, Fedor Biryukov wrote:\n> >>\n> >> > There is no way this could be the intended behavior, but the good news\n> >> > is that I cannot reproduce it...\n> >> > Looks like it occurs only in one git version (2.31.0.windows.1, IIRC).\n> >> > And it does not occur in the latest git version: git version 2.33.0.windows.2.\n> >>\n> >> I think it is a bug, and it seems to reproduce easily for me (with both\n> >> the current tip of master, and with v2.33.0). Here's the recipe you\n> >> showed, with a little debugging at the end:\n> >>\n> >>   set -x\n> >>   git init repo\n> >>   cd repo\n> >>   git commit -m base --allow-empty\n> >>   git checkout -b feat\n> >>   echo feat >readme.txt\n> >>   git add readme.txt\n> >>   git commit -m txt=feat\n> >>   git checkout main\n> >>   echo precious >readme.txt\n> >>\n> >>   cat readme.txt\n> >>   git checkout feat\n> >>   cat readme.txt\n> >>   git rebase main feat\n> >>   cat readme.txt\n> >>\n> >> This produces:\n> >>\n> >>   + cat readme.txt\n> >>   precious\n> >>   + git checkout feat\n> >>   error: The following untracked working tree files would be overwritten by checkout:\n> >>         readme.txt\n> >>   Please move or remove them before you switch branches.\n> >>   Aborting\n> >>   + cat readme.txt\n> >>   precious\n> >>   + git rebase main feat\n> >>   Current branch feat is up to date.\n> >>   + cat readme.txt\n> >>   feat\n> >>\n> >> So git-checkout was not willing to overwrite the untracked content, but\n> >> rebase was happy to obliterate it.\n> >>\n> >> It does the right thing in very old versions. Bisecting, it looks like\n> >> the problem arrived in 5541bd5b8f (rebase: default to using the builtin\n> >> rebase, 2018-08-08). So the bug is in the conversion from the legacy\n> >> shell script to C (which makes sense; the shell version was just calling\n> >> \"git checkout\", which we know does the right thing).\n> >>\n> >> -Peff\n> >\n> > Turns out this is quite a mess.  It's also related to the \"don't\n> > remove empty working directories\" discussion we had earlier this\n> > week[1], because we assumed all relevant codepaths correctly avoided\n> > deleting untracked files and directories in the way.  But they don't.\n> > And rebase isn't the only offender, because this is buried in\n> > unpack_trees.  In fact, it traces back to (and before)\n> >     fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\n> > working tree files.\", 2006-05-17)\n> > which has additional commentary over at\n> > https://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\n> > It appears that before this time, git happily nuked untracked files\n> > and considered them expendable, in basically all cases.  However, this\n> > patch continued considering them as expendable whenever opts->reset\n> > was true.  There wasn't much comment about it at the time for the\n> > reasoning of how opts->reset was handled, though trying to read\n> > between the lines perhaps Junio was trying to limit the backward\n> > compatibility concerns of introducing new errors to fewer code paths?\n> > Anyway, Junio did mention `read-tree --reset` explicitly, but this\n> > opts->reset usage also occurs in am, checkout, reset -- and anything\n> > that calls the reset_head() function including: rebase, stash,\n> > sequencer.c, and add-patch.c.\n> >\n> > So, then...should we preserve untracked (and non-ignored) files in all\n> > these cases?  This rebase case seems clear, but others might be less\n> > clear.  For example, should \"git reset --hard\" nuke untracked files\n> > (what if it's a directory of untracked files getting nuked just to\n> > place a single file in the location of the directory)?  The\n> > documentation isn't explicit, but after reading it I would assume that\n> > untracked files should be preserved.  Since we've had bugs in \"git\n> > reset --hard\" before, such as requiring two invocations in order to\n> > clear out unmerged entries (see [2] and [3]), that also suggests that\n> > this is just another bug in the same area.  But the bug has been\n> > around so long that people might be expecting it; our testsuite has\n> > several cases that incidentally do.  Granted, it's better to throw an\n> > error and require explicit extra steps than to nuke potentially\n> > important work, but some folks might be unhappy with a change here.\n> > Similarly with \"git checkout -f\".\n> >\n> > And stash.c, which operates in that edge case area has tests with\n> > files nuked from the cache without nuking it from the working tree\n> > (causing the working tree file to be considered untracked), and then\n> > attempts to have multiple tests operate on that kind of case.  Those\n> > cases look a bit buggy to me for other reasons (I'm still digging),\n> > but those bugs are kind of hidden by the untracked file nuking bugs,\n> > so fixing the latter requires fixing the former.  And stash.c is a\n> > mess of shelling out to subcommands.  Ick.\n> >\n> > I have some partial patches, but don't know if I'll have anything to\n> > post until I figure out the stash mess...\n>\n> I'd just like to applaud this effort, and also suggest that the most\n> useful result of any such findings would be for us to produce some new\n> test in t/ showing these various cases of nuking/clobbering and other\n> \"non-precious\" edge cases in this logic. See[1] and its linked [2] for\n> references to some of the past discussions around these cases.\n\nYeah, I discovered the problems by making some testcases for the\navoid-removing-current-working-directory case, and then when I was\nsurprised by some behaviors I saw I started adding tests for the\n(mistaken?) nuking of untracked files...and then saw this bug report.\nWhile I've also been trying to add fixes for the issues I've found,\nmaking the testcases (trying to figure out which commands had bugs and\nhow to trigger them) was definitely the hardest part of the effort so\nfar, and likely the most valuable.\n\n> 1. https://lore.kernel.org/git/87a6q9kacx.fsf@evledraar.gmail.com/\n> 2. https://lore.kernel.org/git/87ftsi68ke.fsf@evledraar.gmail.com/\n\nThe first item, preserving ignored files, is related but different (I\nwas specifically discussing non-ignored untracked files above).\nHowever on the topic of precious ignored files...both checkout and\nmerge have a --[no-]overwriite-ignore flag to select this behavior.\nUnfortunately, the merge command only passes that flag along to the\nfast-forwarding code path and ignores it for all other merge backends.\nBut merge-ort could trivially be made to handle it (search for the\nline of code that reads \"if (1/* FIXME: opts->overwrite_ignore*/)\" and\nnote opts->overwrite_ignore isn't a flag that exists, yet), and then\nit'd just need to be threaded through merge/rebase/cherry-pick/revert\ntoplevel commands down to that line of code.  Also, my preliminary\npatches touch on the ignore handling for other codepaths and make it\nclear how one could support such a flag for those other cases; I'll\nleave some comments in the cover letter when I submit it.  Anyway, if\nwe were to get those other codepaths all hooked up, perhaps we could\nadd a core.overwrite_ignored option defaulting to true and giving\npeople a place to just configure this once.  I would much rather a\nglobal option than attempting to provide another class of file that\nusers have to configure (ignored but precious), especially as we have\nhad plenty of bugs and problems just dealing with the two classes of\n'untracked' and 'ignored' already.\n"},{"id":"434980","messageId":"CABPp-BEvSFWN+sGA_0SSqH3sYKtHWs5YpOJSoCGQ_=e6rpfpyg@mail.gmail.com","threadId":"56439","inReplyTo":"xmqqr1e2ejs9.fsf@gitster.g","subject":"Re: Aborting 'rebase main feat' removes unversioned files","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2021-09-08T00:41:02Z","receivedAt":"2021-09-08T00:41:16Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sun, Sep 5, 2021 at 3:31 PM Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Elijah Newren <newren@gmail.com> writes:\n>\n> > unpack_trees.  In fact, it traces back to (and before)\n> >     fcc387db9b (\"read-tree -m -u: do not overwrite or remove untracked\n> > working tree files.\", 2006-05-17)\n> > which has additional commentary over at\n> > https://lore.kernel.org/git/7v8xp1jc9h.fsf_-_@assigned-by-dhcp.cox.net/.\n> > It appears that before this time, git happily nuked untracked files\n> > and considered them expendable, in basically all cases.  However, this\n> > patch continued considering them as expendable whenever opts->reset\n> > was true.\n>\n> Thanks for digging.  Yes, the 'reset' bit was treated as the license\n> to kill untracked working tree files and directories that get in the\n> way in order to carry out the unpack_trees operation.\n>\n> > So, then...should we preserve untracked (and non-ignored) files in all\n> > these cases?  This rebase case seems clear, but others might be less\n> > clear....\n>\n> In short, the guiding principle ought to be that \"checkout --force\"\n> and anything that is given \"force\" as a stronger override should be\n> allowed to do whatever minimally necessary to match the end result\n> in the working tree to what the command wants to show in the absense\n> of these untracked paths.  And without being forced, untracked and\n> unignored paths are precious and should cause commands to fail, if\n> they need to be touched for the commands to complete what they are\n> asked to do [*].\n>\n> \"reset --hard HEAD\" is an oddball.\n>\n> Naïvely, because it is often used as the way to tell Git to \"no\n> matter what, match the working tree to HEAD\", even though it does\n> not have an explicit \"--force\" on the command line, it feels that it\n> also should be allowed to do whatever necessary to the working tree\n> files.  And historically, that is what we wanted to implement.  If\n> we suddenly made it \"safer\", I am sure a lot of existing things will\n> break.\n>\n> But unfortunately, \"--hard\" means a bit more than that in the\n> context of \"reset\", in that we want to reset in a way that is\n> different from \"--mixed\" (reset the index only without touching the\n> working tree) and \"--soft\" (do not touch the index or the working\n> tree), more specifically, with \"--hard\", we want to reset both the\n> index and the working tree to match the given committish (often\n> \"HEAD\").  From that point of view, \"reset --hard\" that tries to\n> preserve untracked and unignored paths, and \"reset --force --hard\"\n> that does whatever necessary to untracked and unignored paths to\n> match the working tree files, when they reset the index and the\n> working tree to the named committish, may have made sense.  If we\n> were designing the feature without any existing users, it is no\n> brainer to imagine that our design would: (1) call the three 'reset'\n> modes as \"both\", \"index-only\" and \"neither\", instead of \"hard\",\n> \"mixed\" and \"soft\", and (2) require \"--force\" to touch untracked and\n> unignored paths.\n>\n> And I think that may be a reasonable longer-term goal, but since we\n> have existing users and scripts, we cannot go there overnight without\n> devising a migration path.\n\nSounds reasonable.  I'm presuming that `read-tree --reset` is also an\noddball, but as plumbing it's better to keep its behavior as-is, i.e.\nlet it nuke untracked files/directories too.  Let me know if you would\nprefer otherwise.\n\nI have a bunch of relatively small patches\n(https://github.com/git/git/pull/1085, if you want a preview), which\nfix most of the problems I found.  I'm going to split it up into five\npatch series, the first three of which are independent and shorter\nthan the remaining two.\n"}]}