{"thread":{"id":"49573","subject":"Ignored files being silently overwritten when switching branches","startedAt":"2018-10-15T13:01:59Z","lastAt":"2018-10-18T01:55:45Z","messageCount":5,"participants":["Per Lundberg","Jeff King","Ævar Arnfjörð Bjarmason","Duy Nguyen","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"360519","messageId":"7d6858c8-aa84-aa05-6c69-22dbbff7dfaa@hibox.tv","threadId":"49573","inReplyTo":null,"subject":"Ignored files being silently overwritten when switching branches","fromName":"Per Lundberg","fromEmail":"per.lundberg@hibox.tv","sentAt":"2018-10-15T13:01:50Z","receivedAt":"2018-10-15T13:01:59Z","isPatch":false,"sender":{"key":"per.lundberg@hibox.tv","avatar":"https://gravatar.com/avatar/037abe105bde31ba7f4fe80a1436ef76a06c9ca8f2c0c153e6ca66a7527cd637?d=mp&s=160"},"body":"Hi,\n\nSorry if this question has been asked before; I skimmed through the list \narchives and the FAQ but couldn't immediately find it - please point me \nin the right direction if it has indeed been discussed before.\n\nWe were renaming some previously-included configuration files (foo.conf) \nin one of our repos, instead providing a \"default\" configuration \n(foo.conf.default) that can easily be copied over to foo.conf by \nindividual developers. This all works fine, and the *.conf are now added \nto the .gitignore list.\n\n_However_, when switching back to our previous release branches (which \nincludes the foo.conf file in the tree), we have noticed that git \nsilently overwrites the locally-modified foo.conf file with the upstream \nfoo.conf file from that branch. When switching back to master, the file \ncontents is therefore perpetually lost, which is a bit unfortunate.\n\nI did a quick repro case here: https://github.com/perlun/git-test, and \nit seems easy to reproduce this behavior using the following steps (also \ndocumented in that git repo):\n\n$ git init\n$ touch foo.txt\n$ nano foo.txt\n$ git add foo.txt\n$ git commit -m 'Add foo.txt'\n[master (root-commit) 8ef05cb] Add foo.txt\n  1 file changed, 1 insertion(+)\n  create mode 100644 foo.txt\n$ git checkout -b dev\nSwitched to a new branch 'dev'\n$ git mv foo.txt foo.bar\n$ git commit -m \"Rename foo.txt -> foo.bar\"\n[dev 4c55c9b] Rename foo.txt -> foo.bar\n  1 file changed, 0 insertions(+), 0 deletions(-)\n  rename foo.txt => foo.bar (100%)\n$ echo 'my local foo.txt' > foo.txt\n$ echo foo.txt > .gitignore\n$ git commit -m \"Add .gitignore\"\n[dev 4c16acb] Add .gitignore\n  1 file changed, 2 insertions(+)\n  create mode 100644 .gitignore\n$ git checkout master # This will silently overwrite the local foo.txt\n\nSo my question is: is this by design or should this be considered a bug \nin git? Of course, it depends largely on what .gitignore is being used \nfor - if we are talking about files which can easily be regenerated \n(build artifacts, node_modules folders etc.) I can totally understand \nthe current behavior, but when dealing with more sensitive & important \ncontent it's a bit inconvenient.\n\n\nWhat I would have expected would be for git to complain, with this message:\n\nerror: The following untracked working tree files would be overwritten \nby checkout:\n\tfoo.txt\nPlease move or remove them before you switch branches.\nAborting\n\nThis is normally the message you get when a _non-ignored_ file is being \noverwritten. But apparently not so when an ignored file is being \noverwritten. If this can be tweaked in the local repo settings somehow, \nplease let me know.\n--\nBest regards,\nPer\n"},{"id":"360588","messageId":"20181016064049.GB25933@sigill.intra.peff.net","threadId":"49573","inReplyTo":"7d6858c8-aa84-aa05-6c69-22dbbff7dfaa@hibox.tv","subject":"Re: Ignored files being silently overwritten when switching branches","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-16T06:40:49Z","receivedAt":"2018-10-16T06:40:53Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 15, 2018 at 01:01:50PM +0000, Per Lundberg wrote:\n\n> Sorry if this question has been asked before; I skimmed through the list \n> archives and the FAQ but couldn't immediately find it - please point me \n> in the right direction if it has indeed been discussed before.\n\nIt is a frequently asked question, but it doesn't seem to be in any FAQ\nthat I could find. The behavior you're seeing is intended. See this\nmessage (and the rest of the thread) for discussion:\n\n  https://public-inbox.org/git/7viq39avay.fsf@alter.siamese.dyndns.org/\n\n> So my question is: is this by design or should this be considered a bug \n> in git? Of course, it depends largely on what .gitignore is being used \n> for - if we are talking about files which can easily be regenerated \n> (build artifacts, node_modules folders etc.) I can totally understand \n> the current behavior, but when dealing with more sensitive & important \n> content it's a bit inconvenient.\n\nBasically: yes. It would be nice to have that \"do not track this, but do\nnot trash it either\" state for a file, but Git does not currently\nsupport that.\n\n-Peff\n"},{"id":"360600","messageId":"871s8qdzph.fsf@evledraar.gmail.com","threadId":"49573","inReplyTo":"20181016064049.GB25933@sigill.intra.peff.net","subject":"Re: Ignored files being silently overwritten when switching branches","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-10-16T09:10:18Z","receivedAt":"2018-10-16T09:10:24Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, Oct 16 2018, Jeff King wrote:\n\n> On Mon, Oct 15, 2018 at 01:01:50PM +0000, Per Lundberg wrote:\n>\n>> Sorry if this question has been asked before; I skimmed through the list\n>> archives and the FAQ but couldn't immediately find it - please point me\n>> in the right direction if it has indeed been discussed before.\n>\n> It is a frequently asked question, but it doesn't seem to be in any FAQ\n> that I could find. The behavior you're seeing is intended. See this\n> message (and the rest of the thread) for discussion:\n>\n>   https://public-inbox.org/git/7viq39avay.fsf@alter.siamese.dyndns.org/\n>\n>> So my question is: is this by design or should this be considered a bug\n>> in git? Of course, it depends largely on what .gitignore is being used\n>> for - if we are talking about files which can easily be regenerated\n>> (build artifacts, node_modules folders etc.) I can totally understand\n>> the current behavior, but when dealing with more sensitive & important\n>> content it's a bit inconvenient.\n>\n> Basically: yes. It would be nice to have that \"do not track this, but do\n> not trash it either\" state for a file, but Git does not currently\n> support that.\n\nThere's some patches in that thread that could be picked up by someone\ninterested. I think the approach mentioned by Matthieu Moy here makes\nthe most sense:\nhttps://public-inbox.org/git/vpqd3t9656k.fsf@bauges.imag.fr/\n\nI don't think the rationale mentioned by Junio in\nhttps://public-inbox.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/ is\nvery convincing.\n\nThe question is not whether .gitignore is intended to be used in some\nspecific way, e.g. only ignoring *.o files, but whether we can\nreasonably suspect that users use the combination of the features we\nexpose in such a way that their precious data gets destroyed. User data\nshould get the benefit of the doubt.\n\nOff the top of my head, I can imagine many ways in which this'll go\nwrong:\n\n 1. Even if you're using .gitignore only for \"trashable\" as as Junio\n    mentions, git not trashing your data depends on everyone who\n    modifies .gitignore in your project having enough situational\n    awareness not to inadvertently add a glob to the file which\n    *accidentally* ignores existing files, and *nothing warns about\n    this*.\n\n    Between the caveat noted in \"It is not possible to re-include[...]\"\n    in gitignore(5) and negative pathspecs it can be really easy to get\n    this wrong.\n\n    So e.g. in git.git I can add a line with \"*\" to .gitignore, and\n    nothing will complain or look unusual as long as I'm not introducing\n    new files, and I'll only find out when some-new-file.c of mine gets\n    trashed.\n\n 2. Related, the UI \"git add <ignored>\" presents is just \"Use -f if you\n    really want to add them\". Users who aren't careful will just think\n    \"oh, I just need -f in this case\" and not alter .gitignore, leaving\n    a timebomb for future users.\n\n    Those new users will have no way of knowing that they've cloned a\n    repo with a broken overzealous .gitignore, e.g. there's nothing on\n    clone that says \"you've just cloned a repo with N files, all of\n    which are ignored, so git clean etc. will likely wipe out anything\n    you have in the checkout\".\n\n 3. Since we implictly expose this \"you need a one-off action to\n    override .gitignore\" noted in #2 users can and *do* use this for\n    \"soft\" ignores.\n\n    E.g. in a big work repo there's an ignore for *.png, even though the\n    repo has thousands of such files, because it's not considered good\n    practice to add them anymore (there's another static repo), and\n    someone thought to use .gitignore to enforce that suggestion.\n\n    I have a personal repo where I only want *.gpg files, and due to the\n    inability to re-include files recursively (noted in #1) I just\n    ignore '*' and use git veeery carefully. I was only worried about\n    'git clean' so far, but now I see I need to worry about \"checkout\"\n    as well.\n\nBut maybe the use-cases I'm mentioning are highly unusual and the repos\nat work have ended up in some bizarre state and nobody else cares about\nthis.\n\nIt would be interesting if someone at a big git hosting providers (hint:\nJeff :) could provide some numbers about how common it is to have a\nrepository containing tracked files ignored by a .gitignore the\nrepository itself carries. This wouldn't cover all of #1-3 above, but is\nprobably a pretty good proxy metric.\n\nI thought this could be done by:\n\n    git ls-tree -r --name-only HEAD  | git check-ignore --no-index --stdin\n\nBut I see that e.g. on git.git this goes wrong due to\nt/helper/.gitignore. So I don't know how one would answer \"does this\nrepo have .gitignored files tracked?\" in a one-liner.\n"},{"id":"360630","messageId":"CACsJy8BzjrvCMzhLKmktywfpa-8_OSvmQ8A_uRv2jfMa8_MbLA@mail.gmail.com","threadId":"49573","inReplyTo":"871s8qdzph.fsf@evledraar.gmail.com","subject":"Re: Ignored files being silently overwritten when switching branches","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2018-10-16T15:05:55Z","receivedAt":"2018-10-16T15:06:25Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Oct 16, 2018 at 11:12 AM Ævar Arnfjörð Bjarmason\n<avarab@gmail.com> wrote:\n>\n>\n> On Tue, Oct 16 2018, Jeff King wrote:\n>\n> > On Mon, Oct 15, 2018 at 01:01:50PM +0000, Per Lundberg wrote:\n> >\n> >> Sorry if this question has been asked before; I skimmed through the list\n> >> archives and the FAQ but couldn't immediately find it - please point me\n> >> in the right direction if it has indeed been discussed before.\n> >\n> > It is a frequently asked question, but it doesn't seem to be in any FAQ\n> > that I could find. The behavior you're seeing is intended. See this\n> > message (and the rest of the thread) for discussion:\n> >\n> >   https://public-inbox.org/git/7viq39avay.fsf@alter.siamese.dyndns.org/\n> >\n> >> So my question is: is this by design or should this be considered a bug\n> >> in git? Of course, it depends largely on what .gitignore is being used\n> >> for - if we are talking about files which can easily be regenerated\n> >> (build artifacts, node_modules folders etc.) I can totally understand\n> >> the current behavior, but when dealing with more sensitive & important\n> >> content it's a bit inconvenient.\n> >\n> > Basically: yes. It would be nice to have that \"do not track this, but do\n> > not trash it either\" state for a file, but Git does not currently\n> > support that.\n>\n> There's some patches in that thread that could be picked up by someone\n> interested. I think the approach mentioned by Matthieu Moy here makes\n> the most sense:\n> https://public-inbox.org/git/vpqd3t9656k.fsf@bauges.imag.fr/\n>\n> I don't think the rationale mentioned by Junio in\n> https://public-inbox.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/ is\n> very convincing.\n\nJust fyi I also have some wip changes that add the forth \"precious\"\nclass in addition to tracked, untracked and ignored [1]. If someone\nhas time it could be another option to pick up.\n\n[1] https://gitlab.com/pclouds/git/commit/0e7f7afa1879b055369ebd3f1224311c43c8a32b\n-- \nDuy\n"},{"id":"360802","messageId":"xmqqwoqgvx0j.fsf@gitster-ct.c.googlers.com","threadId":"49573","inReplyTo":"CACsJy8BzjrvCMzhLKmktywfpa-8_OSvmQ8A_uRv2jfMa8_MbLA@mail.gmail.com","subject":"Re: Ignored files being silently overwritten when switching branches","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-10-18T01:55:40Z","receivedAt":"2018-10-18T01:55:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Duy Nguyen <pclouds@gmail.com> writes:\n\n> Just fyi I also have some wip changes that add the forth \"precious\"\n> class in addition to tracked, untracked and ignored [1]. If someone\n> has time it could be another option to pick up.\n\nIt is much more sensible than gaining the ability to express\nprecious by trading away the ability to express expendable, I would\nthink.\n\n"}]}