{"thread":{"id":"50924","subject":"[BUG] rebase --interactive silently overwrites ignored files","startedAt":"2019-04-11T23:56:22Z","lastAt":"2019-05-07T14:03:34Z","messageCount":8,"participants":["wh","Phillip Wood","Duy Nguyen","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"373695","messageId":"CAL_tzDGRQ4BzJ4c6QypXfBXQNQYocbKbJSBOHhBBB2TwQQPCGA@mail.gmail.com","threadId":"50924","inReplyTo":null,"subject":"[BUG] rebase --interactive silently overwrites ignored files","fromName":"wh","fromEmail":"microrffr@gmail.com","sentAt":"2019-04-11T23:56:09Z","receivedAt":"2019-04-11T23:56:22Z","isPatch":false,"sender":{"key":"microrffr@gmail.com","avatar":null},"body":"I'm using git 2.20.1 from Debian. Git is usually careful not to\noverwrite untracked files, including ignored files. But interactive\nrebase doesn't detect this (non-interactive rebase works fine).\n\nReproduction:\n-----\n\n#!/bin/sh\nmkdir upstream\ncd upstream\ngit init\necho 1 >feature-1\ngit add feature-1\ngit commit -m \"feature 1\"\n\ncd ..\ngit clone upstream local\ncd local\n# write some tools for our own convenience\necho ours >tools\necho /tools >>.git/info/exclude\n# start working on a feature\ngit checkout -b f2\necho wip >feature-2\ngit add feature-2\ngit commit -m \"wip\"\n\ncd ../upstream\n# official tools are available\necho theirs >tools\ngit add tools\ngit commit -m \"tools\"\n\ncd ../local\ngit fetch ../upstream master\n\n# this would be okay\n#git rebase FETCH_HEAD\n\n# problem: overwrites tools silently\nGIT_EDITOR=true git rebase -i FETCH_HEAD\n\ncat tools\n\n-----\n\nExpected: `git rebase -i` fails because it would have to overwrite the\nuntracked \"tools\" file. Contents of tools file remains `ours`.\n\nActual: Contents of tools file becomes `theirs`.\n"},{"id":"373751","messageId":"2643a200-5356-f3bc-1715-3f34b5f19a5b@gmail.com","threadId":"50924","inReplyTo":"CAL_tzDGRQ4BzJ4c6QypXfBXQNQYocbKbJSBOHhBBB2TwQQPCGA@mail.gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2019-04-12T16:30:35Z","receivedAt":"2019-04-12T16:30:41Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi\n\nOn 12/04/2019 00:56, wh wrote:\n> I'm using git 2.20.1 from Debian. Git is usually careful not to\n> overwrite untracked files, including ignored files.\n\nGit normally overwrites ignored files, so I think in your example rebase \n-i is working as expected, I'm surprised that the am based rebase does \nnot overwrite the ignored file. There has been some discussion about \nintroducing 'precious' files that are ignored but protected in the same \nway as untracked files [1].\n\nBest Wishes\n\nPhillip\n\n[1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/\n\n  But interactive\n> rebase doesn't detect this (non-interactive rebase works fine).\n> \n> Reproduction:\n> -----\n> \n> #!/bin/sh\n> mkdir upstream\n> cd upstream\n> git init\n> echo 1 >feature-1\n> git add feature-1\n> git commit -m \"feature 1\"\n> \n> cd ..\n> git clone upstream local\n> cd local\n> # write some tools for our own convenience\n> echo ours >tools\n> echo /tools >>.git/info/exclude\n> # start working on a feature\n> git checkout -b f2\n> echo wip >feature-2\n> git add feature-2\n> git commit -m \"wip\"\n> \n> cd ../upstream\n> # official tools are available\n> echo theirs >tools\n> git add tools\n> git commit -m \"tools\"\n> \n> cd ../local\n> git fetch ../upstream master\n> \n> # this would be okay\n> #git rebase FETCH_HEAD\n> \n> # problem: overwrites tools silently\n> GIT_EDITOR=true git rebase -i FETCH_HEAD\n> \n> cat tools\n> \n> -----\n> \n> Expected: `git rebase -i` fails because it would have to overwrite the\n> untracked \"tools\" file. Contents of tools file remains `ours`.\n> \n> Actual: Contents of tools file becomes `theirs`.\n> \n"},{"id":"373807","messageId":"CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com","threadId":"50924","inReplyTo":"2643a200-5356-f3bc-1715-3f34b5f19a5b@gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"wh","fromEmail":"microrffr@gmail.com","sentAt":"2019-04-14T01:59:36Z","receivedAt":"2019-04-14T01:59:50Z","isPatch":false,"sender":{"key":"microrffr@gmail.com","avatar":null},"body":"Thanks for the info about the upcoming \"precious\" attribute. Looks useful.\n\nI didn't get the impression that Git normally overwrites ignored\nfiles. I ran some more experiments:\n\ngit rebase FETCH_HEAD        # bails\ngit rebase -i FETCH_HEAD     # overwrites\ngit merge FETCH_HEAD         # bails\ngit reset --keep FETCH_HEAD  # bails\ngit reset --merge FETCH_HEAD # bails\ngit checkout FETCH_HEAD      # overwrites\n# without feature 2 locally:\ngit merge FETCH_HEAD         # overwrites, fast-forwards\ngit merge --no-ff FETCH_HEAD # bails\n\nOn Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:\n>\n> Hi\n>\n> On 12/04/2019 00:56, wh wrote:\n> > I'm using git 2.20.1 from Debian. Git is usually careful not to\n> > overwrite untracked files, including ignored files.\n>\n> Git normally overwrites ignored files, so I think in your example rebase\n> -i is working as expected, I'm surprised that the am based rebase does\n> not overwrite the ignored file. There has been some discussion about\n> introducing 'precious' files that are ignored but protected in the same\n> way as untracked files [1].\n>\n> Best Wishes\n>\n> Phillip\n>\n> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/\n>\n>   But interactive\n> > rebase doesn't detect this (non-interactive rebase works fine).\n> >\n> > Reproduction:\n> > -----\n> >\n> > #!/bin/sh\n> > mkdir upstream\n> > cd upstream\n> > git init\n> > echo 1 >feature-1\n> > git add feature-1\n> > git commit -m \"feature 1\"\n> >\n> > cd ..\n> > git clone upstream local\n> > cd local\n> > # write some tools for our own convenience\n> > echo ours >tools\n> > echo /tools >>.git/info/exclude\n> > # start working on a feature\n> > git checkout -b f2\n> > echo wip >feature-2\n> > git add feature-2\n> > git commit -m \"wip\"\n> >\n> > cd ../upstream\n> > # official tools are available\n> > echo theirs >tools\n> > git add tools\n> > git commit -m \"tools\"\n> >\n> > cd ../local\n> > git fetch ../upstream master\n> >\n> > # this would be okay\n> > #git rebase FETCH_HEAD\n> >\n> > # problem: overwrites tools silently\n> > GIT_EDITOR=true git rebase -i FETCH_HEAD\n> >\n> > cat tools\n> >\n> > -----\n> >\n> > Expected: `git rebase -i` fails because it would have to overwrite the\n> > untracked \"tools\" file. Contents of tools file remains `ours`.\n> >\n> > Actual: Contents of tools file becomes `theirs`.\n> >\n"},{"id":"373820","messageId":"a3c544e2-2087-1877-38e3-c85651db32f5@gmail.com","threadId":"50924","inReplyTo":"CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2019-04-14T13:24:31Z","receivedAt":"2019-04-14T13:24:37Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 14/04/2019 02:59, wh wrote:\n> Thanks for the info about the upcoming \"precious\" attribute. Looks useful.\n\nHmm, unfortunately it's not looking so hopeful now [1]\n\n> I didn't get the impression that Git normally overwrites ignored\n> files. I ran some more experiments:\n> \n> git rebase FETCH_HEAD        # bails\n> git rebase -i FETCH_HEAD     # overwrites\n> git merge FETCH_HEAD         # bails\n> git reset --keep FETCH_HEAD  # bails\n> git reset --merge FETCH_HEAD # bails\n> git checkout FETCH_HEAD      # overwrites\n> # without feature 2 locally:\n> git merge FETCH_HEAD         # overwrites, fast-forwards\n> git merge --no-ff FETCH_HEAD # bails\n\nThanks for sharing that, I was assuming the other commands behaved like \ncheckout, I'm intrigued that merge behaves differently to rebase -i as \nthey use the same underlying machinery, I'm a bit snowed under at the \nmoment but I'll try and take a look at what's going sometime in the not \ntoo distant future.\n\nBest Wishes\n\nPhillip\n\n[1] \nhttps://public-inbox.org/git/CACsJy8AEZ-Lz6zgEsuNukvphB9TTa9FAC1gK05fhnie2xtfc9w@mail.gmail.com/T/#u\n> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:\n>>\n>> Hi\n>>\n>> On 12/04/2019 00:56, wh wrote:\n>>> I'm using git 2.20.1 from Debian. Git is usually careful not to\n>>> overwrite untracked files, including ignored files.\n>>\n>> Git normally overwrites ignored files, so I think in your example rebase\n>> -i is working as expected, I'm surprised that the am based rebase does\n>> not overwrite the ignored file. There has been some discussion about\n>> introducing 'precious' files that are ignored but protected in the same\n>> way as untracked files [1].\n>>\n>> Best Wishes\n>>\n>> Phillip\n>>\n>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/\n>>\n>>    But interactive\n>>> rebase doesn't detect this (non-interactive rebase works fine).\n>>>\n>>> Reproduction:\n>>> -----\n>>>\n>>> #!/bin/sh\n>>> mkdir upstream\n>>> cd upstream\n>>> git init\n>>> echo 1 >feature-1\n>>> git add feature-1\n>>> git commit -m \"feature 1\"\n>>>\n>>> cd ..\n>>> git clone upstream local\n>>> cd local\n>>> # write some tools for our own convenience\n>>> echo ours >tools\n>>> echo /tools >>.git/info/exclude\n>>> # start working on a feature\n>>> git checkout -b f2\n>>> echo wip >feature-2\n>>> git add feature-2\n>>> git commit -m \"wip\"\n>>>\n>>> cd ../upstream\n>>> # official tools are available\n>>> echo theirs >tools\n>>> git add tools\n>>> git commit -m \"tools\"\n>>>\n>>> cd ../local\n>>> git fetch ../upstream master\n>>>\n>>> # this would be okay\n>>> #git rebase FETCH_HEAD\n>>>\n>>> # problem: overwrites tools silently\n>>> GIT_EDITOR=true git rebase -i FETCH_HEAD\n>>>\n>>> cat tools\n>>>\n>>> -----\n>>>\n>>> Expected: `git rebase -i` fails because it would have to overwrite the\n>>> untracked \"tools\" file. Contents of tools file remains `ours`.\n>>>\n>>> Actual: Contents of tools file becomes `theirs`.\n>>>\n"},{"id":"374833","messageId":"3bf0f04d-83d4-ef13-8e19-a679b4153793@gmail.com","threadId":"50924","inReplyTo":"CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2019-05-02T15:45:46Z","receivedAt":"2019-05-02T15:45:52Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi wh\n\nOn 14/04/2019 02:59, wh wrote:\n> Thanks for the info about the upcoming \"precious\" attribute. Looks useful.\n> \n> I didn't get the impression that Git normally overwrites ignored\n> files. I ran some more experiments:\n> \n> git rebase FETCH_HEAD        # bails\n> git rebase -i FETCH_HEAD     # overwrites\n> git merge FETCH_HEAD         # bails\n> git reset --keep FETCH_HEAD  # bails\n> git reset --merge FETCH_HEAD # bails\n> git checkout FETCH_HEAD      # overwrites\n> # without feature 2 locally:\n> git merge FETCH_HEAD         # overwrites, fast-forwards\n> git merge --no-ff FETCH_HEAD # bails\n\nI've had a look at the rebase -i code and I think it only overwrites \nignored files when it is fast-forwarding. This matches what merge does \nwhen fast-forwarding but I'm not convinced either of them should be \ndoing this by default (I think checkout doing it is probably asking for \ntrouble), as you point out most operations preserve ignored files. When \npull fast-forwards it does not overwrite ignored files.\n\nBoth checkout and merge have an undocumented --overwrite-ignore option \nwhich is on by default. Checkout and fast-forward merges overwriting \nignored files dates back a long way to commit 1127148089 (\"Loosen \n\"working file will be lost\" check in Porcelain-ish\", 2006-12-04). There \ndoes not seem to have been much discussion of changing the semantics of \nignored files around the patches that implemented this [1,2]. There is a \nbrief mention of doing it in this thread [3] but no one seems to comment \non the idea as far as I can see. Before this change ignored files were \njust ignored by status and add but otherwise treated like any other \nuntracked file.\n\nI'll put a patch together to fix rebase -i, I'd like to see the defaults \nfor merge and checkout changed but I'm not sure that would be popular. \nIt does seem like surprising behavior though when most operations try to \npreserve untracked files.\n\nBest Wishes\n\nPhillip\n\n[1] \nhttps://public-inbox.org/git/7vlklnkv39.fsf@assigned-by-dhcp.cox.net/T/#u\n[2] \nhttps://public-inbox.org/git/7vbqmjkuzd.fsf@assigned-by-dhcp.cox.net/T/#u\n[3]https://public-inbox.org/git/Pine.LNX.4.64.0610081657400.3952@g5.osdl.org/\n\n> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:\n>>\n>> Hi\n>>\n>> On 12/04/2019 00:56, wh wrote:\n>>> I'm using git 2.20.1 from Debian. Git is usually careful not to\n>>> overwrite untracked files, including ignored files.\n>>\n>> Git normally overwrites ignored files, so I think in your example rebase\n>> -i is working as expected, I'm surprised that the am based rebase does\n>> not overwrite the ignored file. There has been some discussion about\n>> introducing 'precious' files that are ignored but protected in the same\n>> way as untracked files [1].\n>>\n>> Best Wishes\n>>\n>> Phillip\n>>\n>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/\n>>\n>>    But interactive\n>>> rebase doesn't detect this (non-interactive rebase works fine).\n>>>\n>>> Reproduction:\n>>> -----\n>>>\n>>> #!/bin/sh\n>>> mkdir upstream\n>>> cd upstream\n>>> git init\n>>> echo 1 >feature-1\n>>> git add feature-1\n>>> git commit -m \"feature 1\"\n>>>\n>>> cd ..\n>>> git clone upstream local\n>>> cd local\n>>> # write some tools for our own convenience\n>>> echo ours >tools\n>>> echo /tools >>.git/info/exclude\n>>> # start working on a feature\n>>> git checkout -b f2\n>>> echo wip >feature-2\n>>> git add feature-2\n>>> git commit -m \"wip\"\n>>>\n>>> cd ../upstream\n>>> # official tools are available\n>>> echo theirs >tools\n>>> git add tools\n>>> git commit -m \"tools\"\n>>>\n>>> cd ../local\n>>> git fetch ../upstream master\n>>>\n>>> # this would be okay\n>>> #git rebase FETCH_HEAD\n>>>\n>>> # problem: overwrites tools silently\n>>> GIT_EDITOR=true git rebase -i FETCH_HEAD\n>>>\n>>> cat tools\n>>>\n>>> -----\n>>>\n>>> Expected: `git rebase -i` fails because it would have to overwrite the\n>>> untracked \"tools\" file. Contents of tools file remains `ours`.\n>>>\n>>> Actual: Contents of tools file becomes `theirs`.\n>>>\n"},{"id":"374834","messageId":"CACsJy8DLUfpL9Rpne2qpWWpRr-myF28LS8MW6zeO8_WJWZoLmg@mail.gmail.com","threadId":"50924","inReplyTo":"3bf0f04d-83d4-ef13-8e19-a679b4153793@gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2019-05-02T16:22:34Z","receivedAt":"2019-05-02T16:23:03Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Thu, May 2, 2019 at 10:45 PM Phillip Wood <phillip.wood123@gmail.com> wrote:\n>\n> Hi wh\n>\n> On 14/04/2019 02:59, wh wrote:\n> > Thanks for the info about the upcoming \"precious\" attribute. Looks useful.\n> >\n> > I didn't get the impression that Git normally overwrites ignored\n> > files.\n\nPlease do not include me in these discussions again. Thank you.\n-- \nDuy\n"},{"id":"374902","messageId":"xmqq4l69tupt.fsf@gitster-ct.c.googlers.com","threadId":"50924","inReplyTo":"3bf0f04d-83d4-ef13-8e19-a679b4153793@gmail.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-05-05T04:02:06Z","receivedAt":"2019-05-05T04:02:13Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood <phillip.wood123@gmail.com> writes:\n\n> I've had a look at the rebase -i code and I think it only overwrites\n> ignored files when it is fast-forwarding. This matches what merge does\n> when fast-forwarding but I'm not convinced either of them should be\n> doing this by default (I think checkout doing it is probably asking\n> for trouble)...\n\n> I'll put a patch together to fix rebase -i, I'd like to see the\n> defaults for merge and checkout changed but I'm not sure that would be\n> popular.\n\nI suspect that such a change in behaviour for \"rebase -i\" is not a\nfix but a regression.  Shouldn't it be consistent with checkout and\nmerge?\n\n> It does seem like surprising behavior though when most\n> operations try to preserve untracked files.\n\nAre you conflating untracked and ignored?\n\nBecause we haven't adopted 'precious' (or whatever the final name\nwould be), which is \"ignored but not expendable\", ignored files are\nby definition \"ignored and expendable\".  When checkout (and merge\nthat is its more general form) needs to match the working tree to\nthe index contents and an ingored file is in the way, it should\noverwrite it.  \n\nUntil we introduce \"ignored but not expendable\" class, that is.\n"},{"id":"375030","messageId":"76bb6944-d448-66b9-dd4e-ed3e262e9a96@gmail.com","threadId":"50924","inReplyTo":"xmqq4l69tupt.fsf@gitster-ct.c.googlers.com","subject":"Re: [BUG] rebase --interactive silently overwrites ignored files","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2019-05-07T14:03:28Z","receivedAt":"2019-05-07T14:03:34Z","isPatch":false,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Junio\n\nOn 05/05/2019 05:02, Junio C Hamano wrote:\n> Phillip Wood <phillip.wood123@gmail.com> writes:\n> \n>> I've had a look at the rebase -i code and I think it only overwrites\n>> ignored files when it is fast-forwarding. This matches what merge does\n>> when fast-forwarding but I'm not convinced either of them should be\n>> doing this by default (I think checkout doing it is probably asking\n>> for trouble)...\n> \n>> I'll put a patch together to fix rebase -i, I'd like to see the\n>> defaults for merge and checkout changed but I'm not sure that would be\n>> popular.\n> \n> I suspect that such a change in behaviour for \"rebase -i\" is not a\n> fix but a regression.  Shouldn't it be consistent with checkout and\n> merge?\n\nMy problem is that I don't really understand why some operations should \noverwrite ignored files and others shouldn't. Having nearly all rebase \noperations preserve ignored files and only overwriting them for a \nfast-forward seems confusing and probably surprising to users. Is it a \nbug that pull does not overwrite ignored files when fast-forwarding but \nmerge does? Why is it ok to overwrite ignored files when merge \nfast-forwards but not otherwise?\n\n>> It does seem like surprising behavior though when most\n>> operations try to preserve untracked files.\n> \n> Are you conflating untracked and ignored?\n> \n> Because we haven't adopted 'precious' (or whatever the final name\n> would be), which is \"ignored but not expendable\", ignored files are\n> by definition \"ignored and expendable\".  \n\nMy confusion is that they are only \"ignored and expendable\" in certain \ncircumstances. Many git operations actually fail if they are going to \noverwrite an ignored file [1] - I don't understand if that is \nintentional or not.\n\nBest Wishes\n\nPhillip\n\n[1] \nhttps://public-inbox.org/git/CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com/\n\n> When checkout (and merge\n> that is its more general form) needs to match the working tree to\n> the index contents and an ingored file is in the way, it should\n> overwrite it.\n> \n> Until we introduce \"ignored but not expendable\" class, that is.\n> \n"}]}