{"thread":{"id":"63815","subject":"Precious files and the .jj directory","startedAt":"2025-07-22T07:31:24Z","lastAt":"2025-07-24T23:11:06Z","messageCount":4,"participants":["Jade Lovelace","Elijah Newren","Junio C Hamano","Martin von Zweigbergk"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"522394","messageId":"CAFA9we_yDs9SPL2pJU_WiUz8CqvQ8ZPHwXBfTzHHbt-QGV34qA@mail.gmail.com","threadId":"63815","inReplyTo":null,"subject":"Precious files and the .jj directory","fromName":"Jade Lovelace","fromEmail":"lists@jade.fyi","sentAt":"2025-07-22T07:31:11Z","receivedAt":"2025-07-22T07:31:24Z","isPatch":false,"sender":{"key":"lists@jade.fyi","avatar":null},"body":"I'm aware of the many discussions about precious files [1] [2] [3],\nbut I wanted to highlight a particularly pernicious category of\nprecious files that are really hard to do the right thing about with\nGit: namely, other version control systems. In particular, as I\nlearned in [4], `git clean -ixd` will of course list `.jj` to delete\nand delete it if you have it in gitignore. But yet having it untracked\nresults in it possibly accidentally getting added and also clutters up\n`git status`.\n\nIt's my understanding that git has more file deletion edge cases of\ngitignored files than of untracked ones so the latter is theoretically\nsafer. Is that correct?\n\n[1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/\n[2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/\n[3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/\n[4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs\n\nThe part about these that is especially pernicious is that git does\nthe right thing to `.git`, there are not that many of *these*\nparticular file patterns, and they generally merit the same treatment\nas .git as deleting them results in somewhere between frustrating and\ncatastrophic loss of work. The one other example of one I've seen\nother than .jj is .sl, though that's only colocated with git if you\nare up to serious shimming shenanigans with broken tools (nix flakes\netc) as AFAIK it is not supposed to be used colocated normally.\n\nShould these be special cased somehow? Should they be simply caught by\nthe precious-files work when it eventually gets done?\n\nRegards,\nJade\n"},{"id":"522447","messageId":"CABPp-BH-bVrX+w7ZtQie_kJ4vaD9subtqsAG5+WHYCmG408xNw@mail.gmail.com","threadId":"63815","inReplyTo":"CAFA9we_yDs9SPL2pJU_WiUz8CqvQ8ZPHwXBfTzHHbt-QGV34qA@mail.gmail.com","subject":"Re: Precious files and the .jj directory","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-22T15:19:32Z","receivedAt":"2025-07-22T15:19:44Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Tue, Jul 22, 2025 at 12:31 AM Jade Lovelace <lists@jade.fyi> wrote:\n>\n> I'm aware of the many discussions about precious files [1] [2] [3],\n> but I wanted to highlight a particularly pernicious category of\n> precious files that are really hard to do the right thing about with\n> Git: namely, other version control systems. In particular, as I\n> learned in [4], `git clean -ixd` will of course list `.jj` to delete\n> and delete it if you have it in gitignore. But yet having it untracked\n> results in it possibly accidentally getting added and also clutters up\n> `git status`.\n>\n> It's my understanding that git has more file deletion edge cases of\n> gitignored files than of untracked ones so the latter is theoretically\n> safer. Is that correct?\n\nI'm not sure what you mean here.  What I can say is that ignored files\nare treated as expendable; thus, for example, if another branch has a\nfile with the same name as an ignored file and you try to switch to\nthat branch, git will silently remove the ignored file in the way and\nreplace it with the file from the other branch.  However, I wouldn't\nconsider that an edge case.  I'd consider edge cases to be e.g. `git\nstash` is implemented via forking a number of other git commands, and\none of those wasn't careful to avoid deleting files when our intention\nwas to avoid deleting them.  But, in the case of ignored files,\nremoving files in the way of the operation is not accidental; it's\ndocumented as intended whenever that ignored file is in the way.\n\nUntracked files are generally not treated as expendable, meaning we do\nnot intend to delete them.  The main caveat is that you can request\nthey be removed as needed by specifying a forcing flag (e.g. git\ncheckout --force, or git reset --hard) for git to delete those.  There\nare a few edge cases, where commands invoke subcommands that might not\nhave been careful about flags they specify, resulting in the files\nbeing deleted when they shouldn't be.  Most of these cases were fixed\na few years ago, though I documented a few extra cases.  Link [6] from\nyour Link [1] will lead you to those.\n\nThe precious file proposal is about splitting ignored files into two\ncategories -- trashable (what all ignored files currently fall under),\nand precious (ignored but not expendable).  Until someone pushes that\neffort, you have to decide whether it's more important to you that the\nfiles aren't deleted (in which case I'd leave them as untracked) or\nthat they don't show up in `git status` and that you don't\naccidentally add them when you're not careful about which files you\nare adding (in which case you can mark them as ignored).\n\n>\n> [1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/\n> [2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/\n> [3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/\n> [4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs\n>\n> The part about these that is especially pernicious is that git does\n> the right thing to `.git`, there are not that many of *these*\n> particular file patterns, and they generally merit the same treatment\n> as .git as deleting them results in somewhere between frustrating and\n> catastrophic loss of work.\n\nI can see that'd be pretty bad.  This probably arises in practice when\nfolks collectively put the files in all three categories, right?  In\nother words, they start off as untracked, but someone accidentally\ncommits them (making them be tracked in some commits or branches), and\nsomeone else decides to ignore them, and then when the person who\nignored these files tries to switch branches or bisect or rebase to or\non top of the other developer's accidental commit, then their files\nare nuked.\n\nIf the files had only been in the combination of {untracked, ignored}\nor {untracked, tracked} then you'd likely be fine.  You'd also be fine\nif they were always ignored from the beginning, since that'd cause\neveryone to be unlikely to commit them and make them be tracked.  It's\nonly when you end up with files that are both tracked in some commits\nand ignored by some developers that you significantly risk running\ninto problems.\n\nOr am I missing some case where this comes up?  Is one of the edge\ncases for the handling of untracked files biting you?\n\n> The one other example of one I've seen\n> other than .jj is .sl, though that's only colocated with git if you\n> are up to serious shimming shenanigans with broken tools (nix flakes\n> etc) as AFAIK it is not supposed to be used colocated normally.\n>\n> Should these be special cased somehow? Should they be simply caught by\n> the precious-files work when it eventually gets done?\n\nI'd rather avoid the special-casing and instead have it be solved by\nimplementing precious files.  It is an interesting case where it gives\nmore motivation to the need for precious files; thanks for passing it\nalong.\n"},{"id":"522473","messageId":"xmqqzfcwrw3k.fsf@gitster.g","threadId":"63815","inReplyTo":"CAFA9we_yDs9SPL2pJU_WiUz8CqvQ8ZPHwXBfTzHHbt-QGV34qA@mail.gmail.com","subject":"Re: Precious files and the .jj directory","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-22T17:09:19Z","receivedAt":"2025-07-22T17:09:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jade Lovelace <lists@jade.fyi> writes:\n\n> I'm aware of the many discussions about precious files [1] [2] [3],\n> but I wanted to highlight a particularly pernicious category of\n> precious files that are really hard to do the right thing about with\n> Git: namely, other version control systems. In particular, as I\n> learned in [4], `git clean -ixd` will of course list `.jj` to delete\n> and delete it if you have it in gitignore. But yet having it untracked\n> results in it possibly accidentally getting added and also clutters up\n> `git status`.\n\nYes, that is a very concise summary of what happens if you do not\nsupport \"ignored but precious\" as a class of files distinct from\n\"ignored and expendable\".  Git only supports the latter and we wish\nwe had also the former is where we are, after \"many discussions\" you\nhave read.\n\n> It's my understanding that git has more file deletion edge cases of\n> gitignored files than of untracked ones so the latter is theoretically\n> safer. Is that correct?\n\nSorry, but I am not sure what apples and oranges you are comparing.\n\nYou list a thing in the .gitignore file or the .git/info/exclude\nfile (collectively known as \"exclude mechanism\") and the thing\nbecomes \"ignored an dexpendable\".  Your \"add\" will warn when you try\nto add it, because it is ignored.  Your \"checkout\", \"merge\", etc.,\nwill happily overwrite such a path when it needs to be removed to\nmake room, because it is expendable.\n\nYou do not tell about a thing to the exclude mechanism, and the\nthing is \"untracked, not ignored\".  Your \"add\" will happily add such\na path, because it is not ignored.  Your \"checkout\" and others will\nplay safer and more careful, avoid removing it, stop operation that\nneeds to remove the path in order to continue, because it is not\nexpendable.\n\nThe only solution would be to add proper support for ignored but\nprecious, I would think.\n"},{"id":"522711","messageId":"CANiSa6gDgEQN31J6TGgYHHxi+-0O4U4YC4wyWeY0=GvTJw2swQ@mail.gmail.com","threadId":"63815","inReplyTo":"CABPp-BH-bVrX+w7ZtQie_kJ4vaD9subtqsAG5+WHYCmG408xNw@mail.gmail.com","subject":"Re: Precious files and the .jj directory","fromName":"Martin von Zweigbergk","fromEmail":"martinvonz@gmail.com","sentAt":"2025-07-24T23:10:55Z","receivedAt":"2025-07-24T23:11:06Z","isPatch":false,"sender":{"key":"martinvonz@gmail.com","avatar":"https://avatars.githubusercontent.com/u/891642?v=4"},"body":"On Tue, Jul 22, 2025 at 8:22 AM Elijah Newren <newren@gmail.com> wrote:\n> Or am I missing some case where this comes up?  Is one of the edge\n> cases for the handling of untracked files biting you?\n\nIf I understood the discussion correctly, it typically happens in what\nwe call \"colocated repos\" [1], where `.git/` and `.jj/` are siblings.\nI just sent https://github.com/jj-vcs/jj/pull/7071 to at least warn\nagainst running `git clean -fdx` when creating such setups.\n\n[1] https://jj-vcs.github.io/jj/latest/git-compatibility/#co-located-jujutsugit-repos\n"}]}