{"thread":{"id":"45412","subject":"Bug with .gitignore and branch switching","startedAt":"2017-03-17T21:09:42Z","lastAt":"2017-03-18T04:37:53Z","messageCount":8,"participants":["Nevada Sanchez","Junio C Hamano","Jonathan Nieder","Stefan Beller","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"314387","messageId":"CAPUVn2u0Uos2mT5+4ejj8m0okNK6XwerL6ce2miHfhtuEs-ZnQ@mail.gmail.com","threadId":"45412","inReplyTo":null,"subject":"Bug with .gitignore and branch switching","fromName":"Nevada Sanchez","fromEmail":"sanchez.nevada@gmail.com","sentAt":"2017-03-17T20:42:35Z","receivedAt":"2017-03-17T21:09:42Z","isPatch":false,"sender":{"key":"sanchez.nevada@gmail.com","avatar":null},"body":"Here's an easy to reproduce bug. It's the only example I know of where\ngit legitimately loses data in a way that is unrecoverable,\nunexpected, and without warning.\n\n```\ngit --version\n# git version 2.12.0\n\nmkdir git-demo\ncd git-demo\n\ngit init\n\n# Commit a file that will end up in .gitignore\necho 'original settings' > mine.conf\ngit add mine.conf\ngit commit -m \"Unknowingly committed my settings.\"\n\necho '*.conf' > .gitignore\ngit add .gitignore\ngit commit -m \"Users shouldn't commit their settings\"\n\n# Spin off a feature branch here (but don't check it out)\ngit branch feature\n\n# Realize that we don't want that file committed\ngit rm mine.conf\ngit commit -m \"Delete mine.conf\"\n\necho 'Lots of laboriously tuned settings' > mine.conf\n\n# Hop on the feature branch to do some work\ngit checkout feature\n\n# Hmmm... My settings are gone\ncat mine.conf\n# original settings\n\n# Lemme hop back\ngit checkout master\n\n# Wait... they are gone for good!\ncat mine.conf\n# cat: mine.conf: No such file or directory\n```\n"},{"id":"314393","messageId":"xmqqmvcj61j2.fsf@gitster.mtv.corp.google.com","threadId":"45412","inReplyTo":"CAPUVn2u0Uos2mT5+4ejj8m0okNK6XwerL6ce2miHfhtuEs-ZnQ@mail.gmail.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-17T21:23:13Z","receivedAt":"2017-03-17T21:38:54Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nevada Sanchez writes:\n\n> Here's an easy to reproduce bug. It's the only example I know of where\n> git legitimately loses data in a way that is unrecoverable,\n> unexpected, and without warning.\n\nThis is an example of a user explicitly telling git to discard data\nand git performing as it is told.\n\nThere is no \"untracked but precious\" vs \"untracked and expendable\"\ndifference in the current system. An untracked file that matches\npatterns listed in .gitignore is treated as the latter.\n\nWhen you have an untracked file that .gitignore knows about in the\nworking tree while you are on \"feature\", if switching to another\nbranch requires to remove that file, the content there is deemed\nexpendable, because the user said so by listing it in .gitignore.\n\nWe've discussed the lack of \"untracked but precious\" class a few\ntimes on the list in the past, but I do not recall the topic came up\nin the recent past. It perhaps is because nobody found that class\nuseful enough so far.\n"},{"id":"314394","messageId":"20170317215442.GN26789@aiede.mtv.corp.google.com","threadId":"45412","inReplyTo":"CAPUVn2u0Uos2mT5+4ejj8m0okNK6XwerL6ce2miHfhtuEs-ZnQ@mail.gmail.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-03-17T21:54:42Z","receivedAt":"2017-03-17T21:54:51Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Nevada,\n\nNevada Sanchez wrote:\n\n> # Commit a file that will end up in .gitignore\n> echo 'original settings' > mine.conf\n> git add mine.conf\n> git commit -m \"Unknowingly committed my settings.\"\n>\n> echo '*.conf' > .gitignore\n> git add .gitignore\n> git commit -m \"Users shouldn't commit their settings\"\n\nNaming a file in .gitignore tells git that you do not want to track it\nand are giving git permission to write over it. This commonly happens\nwhen people check in build products. For example:\n\n\tgit rm -f my-build-product\n\techo /my-build-product >>.gitignore\n\tgit commit -m \"Remove generated my-build-product file\"\n\tmake my-build-product\n\n\tgit checkout HEAD^\n\nWithout that rule, this 'git checkout' command would fail.\n\nThat said, there are some cases (e.g. the .conf file case you mention)\nwhere a person would want git not to track a file but do not want to\ngive git permission to write over it. As you've seen, .gitignore does\nnot work well for this. :/\n\nIdeas for next steps:\n\n 1. The gitignore(5) manpage does not do a good job of emphasizing\n that files named there are not precious and can be overwritten by\n git. Do you have ideas for wording that would help with that?\n This would be especially welcome if you can phrase them in the\n form of a patch against Documentation/gitignore.txt.\n\n 2. Occasionally people have mentioned the idea of a .gitprecious file\n listing precious files that git should not track and not overwrite\n (e.g., keys and other configuration files, IDE state, or metadata\n for another version control system being used in parallel). Would\n you be interested in working on that?\n\nThanks and hope that helps,\nJonathan\n"},{"id":"314396","messageId":"20170317220213.GO26789@aiede.mtv.corp.google.com","threadId":"45412","inReplyTo":"xmqqmvcj61j2.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-03-17T22:02:13Z","receivedAt":"2017-03-17T22:02:21Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Junio C Hamano wrote:\n\n> There is no \"untracked but precious\" vs \"untracked and expendable\"\n> difference in the current system. An untracked file that matches\n> patterns listed in .gitignore is treated as the latter.\n[...]\n> We've discussed the lack of \"untracked but precious\" class a few\n> times on the list in the past, but I do not recall the topic came up\n> in the recent past. It perhaps is because nobody found that class\n> useful enough so far.\n\nThe most recent example I can find is 2010:\nhttp://public-inbox.org/git/4C6A1C5B.4030304@workspacewhiz.com/.\n\nIt also came up in 2007:\nhttp://public-inbox.org/git/C0E9F681E68D48EB8989022D11FEE3D1@ntdev.corp.microsoft.com/\nEarlier in that year it even made the \"What's not in 1.5.2\" list.\nhttp://public-inbox.org/git/11793556383977-git-send-email-junkio@cox.net/\n\nPerhaps those references could be a useful starting point for an\ninterested person's thinking.\n\nJonathan\n"},{"id":"314398","messageId":"CAGZ79kZQsauBfoTjyqm+-+LjyyEc2Ykj5exUY5KdErEzFH0GMA@mail.gmail.com","threadId":"45412","inReplyTo":"xmqqmvcj61j2.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2017-03-17T21:58:48Z","receivedAt":"2017-03-17T22:22:17Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Mar 17, 2017 at 2:23 PM, Junio C Hamano wrote:\n> We've discussed the lack of \"untracked but precious\" class a few\n> times on the list in the past, but I do not recall the topic came up\n> in the recent past. It perhaps is because nobody found that class\n> useful enough so far.\n\nMy gut reaction on reading the bug report was that the root cause is\ngit-checkout doing the wrong thing by default. (cf. Git-Merge-2017,\n\"What’s Wrong With Git?\", I am not sure if the video is yet available)\n\nOne argument in that talk was that Git promises to do \"work on multiple\nbranches in parallel (context-switched, single threaded)\", and git-checkout\nis the apparent command to switch to another context (branch).\nHowever by putting away only tracked content, we miss\ndoing a proper context switch for untracked and ignored files.\n\nThat partial switch has advantages in the typical use case, e.g.\n* compiled objects in the worktree may not need to be recompiled.\n* no need to do work for the untracked files (e.g. move to a special\n location).\n\nBoth these reasons argue for performance, instead of \"correctness\"\nin the sense of \"easy-to-understand commands for top level principles\".\n\nAnd in that talk the presenter concluded that git-stash was only invented\nto circumvent these \"correctness\" problems, such that if git-checkout\nwere to also (de)populate the untracked and ignored files on branch\nswitch we would not need git-stash, because git-checkout did it for you\nalready. And by the omission of git-stash and an apparent\neasier-to-understand git-checkout the whole git suite would become\neasier for users.\n\nI further conclude that when git-checkout were to behave \"correct\" as\noutlined above, then this class of bug reports would not occur.\n\nJust food for thought.\n\nThanks,\nStefan\n"},{"id":"314404","messageId":"xmqqefxv5y57.fsf@gitster.mtv.corp.google.com","threadId":"45412","inReplyTo":"20170317220213.GO26789@aiede.mtv.corp.google.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-17T22:36:20Z","receivedAt":"2017-03-17T22:36:28Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder writes:\n\n> The most recent example I can find is 2010:\n> http://public-inbox.org/git/4C6A1C5B.4030304@workspacewhiz.com/.\n>\n> It also came up in 2007:\n> http://public-inbox.org/git/C0E9F681E68D48EB8989022D11FEE3D1@ntdev.corp.microsoft.com/\n> Earlier in that year it even made the \"What's not in 1.5.2\" list.\n> http://public-inbox.org/git/11793556383977-git-send-email-junkio@cox.net/\n>\n> Perhaps those references could be a useful starting point for an\n> interested person's thinking.\n\nThanks for links. It seems that my thinking back in 1.5.3 timeperiod\nwas to introduce \"precious\" attribute.\n\nI noticed that among the four-message \"What's not in 1.5.2\" series,\n3/4 has a large discussion that may be relevant to Brandon's\n\"submodule is-active\" thing.\n"},{"id":"314466","messageId":"CACsJy8DYx_vEyUBJWQ=-rOGvWG2BBGQKbxYTVS18NPqfk02r9g@mail.gmail.com","threadId":"45412","inReplyTo":"20170317220213.GO26789@aiede.mtv.corp.google.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2017-03-18T03:40:45Z","receivedAt":"2017-03-18T03:50:49Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Mar 18, 2017 at 5:02 AM, Jonathan Nieder wrote:\n> Junio C Hamano wrote:\n>\n>> There is no \"untracked but precious\" vs \"untracked and expendable\"\n>> difference in the current system. An untracked file that matches\n>> patterns listed in .gitignore is treated as the latter.\n> [...]\n>> We've discussed the lack of \"untracked but precious\" class a few\n>> times on the list in the past, but I do not recall the topic came up\n>> in the recent past. It perhaps is because nobody found that class\n>> useful enough so far.\n>\n> The most recent example I can find is 2010:\n> http://public-inbox.org/git/4C6A1C5B.4030304@workspacewhiz.com/.\n>\n> It also came up in 2007:\n> http://public-inbox.org/git/C0E9F681E68D48EB8989022D11FEE3D1@ntdev.corp.microsoft.com/\n> Earlier in that year it even made the \"What's not in 1.5.2\" list.\n> http://public-inbox.org/git/11793556383977-git-send-email-junkio@cox.net/\n>\n> Perhaps those references could be a useful starting point for an\n> interested person's thinking.\n\nI think I made it work in 2014 [1] using new \"precious\" attribute, but\nnever submitted it, probably because I was worried about the\ninteraction with untracked cache (adding .gitattributes as a new\ndependency) though maybe we can avoid that by always checking for\npreciousness after all the tree walking/filtering is done, either with\nor without untracked cache. But I never addressed that loose end. Then\nagain, it could also be another useful starting point for interested\nperson's thinking ;-)\n\n[1] https://github.com/pclouds/git/commit/0e7f7afa1879b055369ebd3f1224311c43c8a32b\n-- \nDuy\n"},{"id":"314467","messageId":"CAPUVn2tXxq4Ab0aDKPmRpojnWuTdJBTuYskJLcgNgEemNJ3FnQ@mail.gmail.com","threadId":"45412","inReplyTo":"xmqqmvcj61j2.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug with .gitignore and branch switching","fromName":"Nevada Sanchez","fromEmail":"sanchez.nevada@gmail.com","sentAt":"2017-03-18T04:30:40Z","receivedAt":"2017-03-18T04:37:53Z","isPatch":false,"sender":{"key":"sanchez.nevada@gmail.com","avatar":null},"body":"On Fri, Mar 17, 2017 at 5:23 PM, Junio C Hamano wrote:\n> Nevada Sanchez writes:\n>\n>> Here's an easy to reproduce bug. It's the only example I know of where\n>> git legitimately loses data in a way that is unrecoverable,\n>> unexpected, and without warning.\n>\n> This is an example of a user explicitly telling git to discard data\n> and git performing as it is told.\n>\n> There is no \"untracked but precious\" vs \"untracked and expendable\"\n> difference in the current system. An untracked file that matches\n> patterns listed in .gitignore is treated as the latter.\n>\n> When you have an untracked file that .gitignore knows about in the\n> working tree while you are on \"feature\", if switching to another\n> branch requires to remove that file, the content there is deemed\n> expendable, because the user said so by listing it in .gitignore.\n>\n> We've discussed the lack of \"untracked but precious\" class a few\n> times on the list in the past, but I do not recall the topic came up\n> in the recent past. It perhaps is because nobody found that class\n> useful enough so far.\n\nI must admit that I missed that attribute of .gitignore (i.e.\nuntracked and **expendable**). I have grown accustomed to Git being\nrather conservative and erring on the side of not losing data unless\nthe user is doing something deliberate (for example, 'git clean' won't\nwork unless you force it, checkouts fail if they do anything that\nmight lose data... unless it is in .gitignore, as I just learned).\nWhen I saw this behavior, I assumed that it was a bug.\n\nThis isn't necessarily a situation I need to have fixed--it is not\npart of my workflow and since that fateful commit, all feature\nbranches checked out after the change to .gitignore will not have any\nproblems as I switch branches. It was an unfortunate surprise to one\nof my co-workers, not long after I reassured him that git was\nconservative and will almost never accidentally lose data (even if it\nmeans going to 'git reflog').\n\nIn keeping with this spirit, I would tend to lean towards having\n\"untracked but precious\" being the default behavior (more\nconservative), and if a user wants \"untracked but expendable\"\nbehavior, then that case requires special effort from the user (like\nlearning about and using a new type of ignore file). My guess is that\nif the user is both ignoring and committing something to their\nrepository, it is probably a mistake, and as that user, I would rather\ndiscover that mistake early with loud warning messages (and/or a\nsuggestion to use an alternate ignore strategy, or config flag), than\nlearn about it by losing data.\n\nIn summary, I do not need this fixed for my workflow, but want to\nbring it to light in case other users are being similarly surprised. I\nstruggle to guess how far reaching of an impact it would have on\nexisting users to change the default behavior, but it would probably\nbe less than that of the push default behavior change that happened\nnot too long ago.\n\nA quick and easy immediate step is to make note of this behavior in\nthe very first sentence of gitignore(5):\n\n> A gitignore file specifies intentionally untracked files that Git should ignore *and that Git is allowed to overwrite without warning*.\n\nMore details about untracked but expendable can be placed in the NOTES\nsection, but the last part of that sentence would be quite helpful.\n\nThank you,\n-Nevada\n"}]}