threads / discuss / 63815

Precious files and the .jj directory

Subject: Precious files and the .jj directory

## tl;dr

4 messages between Jul 22, 2025 and Jul 24, 2025.

replies: 3people: 4as markdown or json

Jade Lovelace· Jul 22, 2025, 07:31 UTC · lore
I'm aware of the many discussions about precious files [1] [2] [3],
but I wanted to highlight a particularly pernicious category of
precious files that are really hard to do the right thing about with
Git: namely, other version control systems. In particular, as I
learned in [4], `git clean -ixd` will of course list `.jj` to delete
and delete it if you have it in gitignore. But yet having it untracked
results in it possibly accidentally getting added and also clutters up
`git status`.

It's my understanding that git has more file deletion edge cases of gitignored files than of untracked ones so the latter is theoretically safer. Is that correct?

[1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/ [2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/ [3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/ [4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs

The part about these that is especially pernicious is that git does the right thing to `.git`, there are not that many of *these* particular file patterns, and they generally merit the same treatment as .git as deleting them results in somewhere between frustrating and catastrophic loss of work. The one other example of one I've seen other than .jj is .sl, though that's only colocated with git if you are up to serious shimming shenanigans with broken tools (nix flakes etc) as AFAIK it is not supposed to be used colocated normally.

Should these be special cased somehow? Should they be simply caught by the precious-files work when it eventually gets done?

Regards, Jade

Elijah Newren· Jul 22, 2025, 15:19 UTC · re: Jade Lovelace · lore

Re: Precious files and the .jj directory

On Tue, Jul 22, 2025 at 12:31 AM Jade Lovelace <lists@jade.fyi> wrote:
Show 13 quoted lines
>
> I'm aware of the many discussions about precious files [1] [2] [3],
> but I wanted to highlight a particularly pernicious category of
> precious files that are really hard to do the right thing about with
> Git: namely, other version control systems. In particular, as I
> learned in [4], `git clean -ixd` will of course list `.jj` to delete
> and delete it if you have it in gitignore. But yet having it untracked
> results in it possibly accidentally getting added and also clutters up
> `git status`.
>
> It's my understanding that git has more file deletion edge cases of
> gitignored files than of untracked ones so the latter is theoretically
> safer. Is that correct?

I'm not sure what you mean here. What I can say is that ignored files are treated as expendable; thus, for example, if another branch has a file with the same name as an ignored file and you try to switch to that branch, git will silently remove the ignored file in the way and replace it with the file from the other branch. However, I wouldn't consider that an edge case. I'd consider edge cases to be e.g. `git stash` is implemented via forking a number of other git commands, and one of those wasn't careful to avoid deleting files when our intention was to avoid deleting them. But, in the case of ignored files, removing files in the way of the operation is not accidental; it's documented as intended whenever that ignored file is in the way.

Untracked files are generally not treated as expendable, meaning we do not intend to delete them. The main caveat is that you can request they be removed as needed by specifying a forcing flag (e.g. git checkout --force, or git reset --hard) for git to delete those. There are a few edge cases, where commands invoke subcommands that might not have been careful about flags they specify, resulting in the files being deleted when they shouldn't be. Most of these cases were fixed a few years ago, though I documented a few extra cases. Link [6] from your Link [1] will lead you to those.

The precious file proposal is about splitting ignored files into two categories -- trashable (what all ignored files currently fall under), and precious (ignored but not expendable). Until someone pushes that effort, you have to decide whether it's more important to you that the files aren't deleted (in which case I'd leave them as untracked) or that they don't show up in `git status` and that you don't accidentally add them when you're not careful about which files you are adding (in which case you can mark them as ignored).

Show 11 quoted lines
>
> [1]: https://lore.kernel.org/git/pull.1627.git.1703643931314.gitgitgadget@gmail.com/
> [2]: https://lore.kernel.org/git/871s7r4wuv.fsf@evledraar.gmail.com/
> [3]: https://lore.kernel.org/git/7v4oepaup7.fsf@alter.siamese.dyndns.org/
> [4]: https://maddie.wtf/posts/2025-07-21-jujutsu-for-busy-devs
>
> The part about these that is especially pernicious is that git does
> the right thing to `.git`, there are not that many of *these*
> particular file patterns, and they generally merit the same treatment
> as .git as deleting them results in somewhere between frustrating and
> catastrophic loss of work.

I can see that'd be pretty bad. This probably arises in practice when folks collectively put the files in all three categories, right? In other words, they start off as untracked, but someone accidentally commits them (making them be tracked in some commits or branches), and someone else decides to ignore them, and then when the person who ignored these files tries to switch branches or bisect or rebase to or on top of the other developer's accidental commit, then their files are nuked.

If the files had only been in the combination of {untracked, ignored} or {untracked, tracked} then you'd likely be fine. You'd also be fine if they were always ignored from the beginning, since that'd cause everyone to be unlikely to commit them and make them be tracked. It's only when you end up with files that are both tracked in some commits and ignored by some developers that you significantly risk running into problems.

Or am I missing some case where this comes up? Is one of the edge cases for the handling of untracked files biting you?

Show 7 quoted lines
> The one other example of one I've seen
> other than .jj is .sl, though that's only colocated with git if you
> are up to serious shimming shenanigans with broken tools (nix flakes
> etc) as AFAIK it is not supposed to be used colocated normally.
>
> Should these be special cased somehow? Should they be simply caught by
> the precious-files work when it eventually gets done?

I'd rather avoid the special-casing and instead have it be solved by implementing precious files. It is an interesting case where it gives more motivation to the need for precious files; thanks for passing it along.

Martin von Zweigbergk· Jul 24, 2025, 23:10 UTC · re: Elijah Newren · lore

Re: Precious files and the .jj directory

On Tue, Jul 22, 2025 at 8:22 AM Elijah Newren <newren@gmail.com> wrote:
> Or am I missing some case where this comes up?  Is one of the edge
> cases for the handling of untracked files biting you?

If I understood the discussion correctly, it typically happens in what we call "colocated repos" [1], where `.git/` and `.jj/` are siblings. I just sent https://github.com/jj-vcs/jj/pull/7071 to at least warn against running `git clean -fdx` when creating such setups.

[1] https://jj-vcs.github.io/jj/latest/git-compatibility/#co-located-jujutsugit-repos
Junio C Hamano· Jul 22, 2025, 17:09 UTC · re: Jade Lovelace · lore

Re: Precious files and the .jj directory

Jade Lovelace <lists@jade.fyi> writes:
Show 8 quoted lines
> I'm aware of the many discussions about precious files [1] [2] [3],
> but I wanted to highlight a particularly pernicious category of
> precious files that are really hard to do the right thing about with
> Git: namely, other version control systems. In particular, as I
> learned in [4], `git clean -ixd` will of course list `.jj` to delete
> and delete it if you have it in gitignore. But yet having it untracked
> results in it possibly accidentally getting added and also clutters up
> `git status`.

Yes, that is a very concise summary of what happens if you do not support "ignored but precious" as a class of files distinct from "ignored and expendable". Git only supports the latter and we wish we had also the former is where we are, after "many discussions" you have read.

> It's my understanding that git has more file deletion edge cases of
> gitignored files than of untracked ones so the latter is theoretically
> safer. Is that correct?
Sorry, but I am not sure what apples and oranges you are comparing.

You list a thing in the .gitignore file or the .git/info/exclude file (collectively known as "exclude mechanism") and the thing becomes "ignored an dexpendable". Your "add" will warn when you try to add it, because it is ignored. Your "checkout", "merge", etc., will happily overwrite such a path when it needs to be removed to make room, because it is expendable.

You do not tell about a thing to the exclude mechanism, and the thing is "untracked, not ignored". Your "add" will happily add such a path, because it is not ignored. Your "checkout" and others will play safer and more careful, avoid removing it, stop operation that needs to remove the path in order to continue, because it is not expendable.

The only solution would be to add proper support for ignored but precious, I would think.

← back to recent threads