patch, 3 partsclean: add `config.exclude` and `--remove-excluded`

This series extends the concept of "excluded files" in `git clean` to make it useful to protect "precious files" that might be present in a specific developer's working tree (see below).

Specifically, this series adds a `config.exclude` knob to configure "always excluded" files (same as `-e` on the command line), and a `--remove-excluded` flag (intentionally without a short form) to "REALLY remove everything, dammit!"

This might seem like euphemism treadmill, but there is a specific use-case for all of the exclusion methods and options:

.gitignore:     files that _the project_ does not want to track or touch
                (build artifacts)
clean.exclude:  files that _the user_ does not want to track or touch
                (IDE configuration)
git clean -x:   remove build artifacts, but keep precious files
                (when a pristine build is desired)
git clean -x --remove-excluded:
                remove everything, including precious files
                (e.g. for redistribution)

For instance, if I use Sublime Text or JetBrains IDEs to work on projects, I might want to add this to my ~/.gitconfig:

[clean]
  exclude = /*.sublime-*
  exclude = /.idea

Or, if I make use of the Bear compiler wrapper to generate the compilation database in those projects that do not use any of the modern build-systems to automate such generation, I might write:

[clean]
  exclude = /compile_commands.json

This way, even if I run `git clean -fxd` to test a clean build, I do not need to worry about accidentally removing the compilation database that would take a bunch of CPU-time to regenerate.

Ivan Shapovalov (3):
  clean, dir: add and use new helper `add_patterns_from_string_list()`
  clean: rename `ignored` -> `remove_ignored`
  clean: add `config.exclude` and `--remove-excluded`

 Documentation/config/clean.txt | 11 +++++++++++
 Documentation/git-clean.txt    | 22 +++++++++++++++-------
 builtin/clean.c                | 32 +++++++++++++++++++++-----------
 dir.c                          | 15 +++++++++++++++
 dir.h                          |  4 ++++
 5 files changed, 66 insertions(+), 18 deletions(-)

Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
---
 builtin/clean.c |  5 +----
 dir.c           | 15 +++++++++++++++
 dir.h           |  4 ++++
 3 files changed, 20 insertions(+), 4 deletions(-)

No behavior change.

Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
---
 builtin/clean.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

Add `config.exclude` to configure "always excluded" files (same as `-e` on the command line), and `--remove-excluded` (intentionally without a short form) to "REALLY remove everything, dammit!"

This might seem like euphemism treadmill, but there is a specific use-case for all of the exclusion methods and options:

.gitignore:     files that _the project_ does not want to track or touch
                (build artifacts)
clean.exclude:  files that _the user_ does not want to track or touch
                (IDE configuration)
git clean -x:   remove build artifacts, but keep precious files
                (when a pristine build is desired)
git clean -x --remove-excluded:
                remove everything, including precious files
                (e.g. for redistribution)

Signed-off-by: Ivan Shapovalov <intelfx@intelfx.name>
---
 Documentation/config/clean.txt | 11 +++++++++++
 Documentation/git-clean.txt    | 22 +++++++++++++++-------
 builtin/clean.c                | 19 ++++++++++++++++---
 3 files changed, 42 insertions(+), 10 deletions(-)

Ivan Shapovalov <intelfx@intelfx.name> writes:

A few comments on the proposed semantics.

 - It is questionable to throw paths that match command line "-e"
   patterns into the 'precious' class.  It breaks backward
   compatibility and established end-user expectations, doesn't it?

 - It is nice to see that an effort is made to sift "excluded" into
   two classes.  The traditional "ignored/excluded are expendable,
   so "git clean" will remove them, "git switch", when path F is
   excluded and there is a file F, would remove it when it needs to
   check out a tree that has paths under directory F.  "git add F"
   and "git add ." would not add it unless forced.  We would want
   another class of files `precious` that are ignored in the same
   sense by "git add", but yet excempt from removal by "git clean"
   and things like "git switch".

 - On the other hand, it is a good idea to use a new source of
   patterns that the command never paid attention to, like a new
   configuration variable.  Since nobody has ever used it for
   "excluded and expendable", there is no risk of breaking end-user
   expectations.

 - This particular implementation falls far short of the ideal of
   "precious files" class, though.  Since "git clean" is the only
   thing that pays attention to clean.exclude, "git add ." would
   happily add those paths that match the pattern, and "git switch"
   to check out a directory whose pathname matches the pattern would
   happily nuke a precious file that is in the working tree at that
   path.

Earlier discussions proposed extended syntax added to .gitignore mechanism and relevant codepaths, not just "git clean", all pay attention to the new "precious" paths, but one idea proposed by this series that is much better than the previous designs is the use of separate sources of patterns---we do not have to worry about backward compatibility issues at all with that design.

In my earlier review, I said it was clever to recognize that precious would be of personal nature, but I now realize that there are valid reasons why projects may _know_ what paths are precious and would want to distribute that knowledge to its participants. For example, our project would benefit from having config.mak marked as precious for everybody, so that nobody commits such a file by mistake and then ask us to pull from them.

As a configuration variable does not propagate across repositories by design, it would not work as a way for the project to share its idea of what paths are in the "precious" class, so in addition to the clean.exclude (which probably is a bad name, as (1) we want not just clean but things like add and switch also pay attention to it, and (2) the class it defines is distinct from "exclude", and would want to have the word "precious" in it) variable, we'd probably need to have a way to record them in tracked files, either in .gitignore files using some extended syntax, or separate .gitprecious files.

Thanks.

Ivan Shapovalov <intelfx@intelfx.name> writes:

How does it interact with "git status"?

I am not sure if this uses the adjective `precious` to mean the same thing as we historically talked about `precious`, in the context of "Git does not have `precious files`. What we call `ignored` are synoymous to `expendables`, and we'd eventually want to add the `precious` class of files that are separate from `ignored` files".

If the feature is about _turning_ the existing `ignored/excluded` into precious and require a new option to clean those files that have always been treated as expendables, then that is a grave usability regression. I am hoping that it is not the case.

Let's read on.

The above two share the same "does not want to track or touch" explanation and readers do not know if you want them to have distinct meaning, or just two different places the user has to store the same information, one project-wide, given by and shared with others, the other personal.

You need to say something like "`clean.exclude` introduces a new `precious` class, the user does nto want to track or touch but unlike those that match the patterns in .gitignore, they are not expendables" here, if that is what you are trying to say (I am just guessing).

Without that ...

... this would merely be a wishful thinking, but once the reader understands that you are introducing a new class, yes, it does make sense. And it is backward compatible enhancement, which is very good.

Ditto.

Another common theme around `precious` is not IDE configuration but things like config.mak file we have. Or perhaps deploy key files?

It is a clever UI hack to notice that the `precious` things are not something you'd share with the project, and to take advantage of the distinction between the project-wide vs personal preference in the configuration system to introduce the `precious` class. For that, it might even make sense to call the variable "clean.precious", as its semantics is VASTLY different from what we called `exclude` or `ignore` (they are synonyms---and they mean expendable files that are not to be tracked).

And when people want non-project-wide but personal paths that are excluded and expendable, they can use $GIT_DIR/info/exclude file. So a possible alternative is to have the dir.[ch] infrastructure to start paying attention to a new file $GIT_DIR/info/precious instead of the configuration variables. I am not making an assessment on the relative merit between clean.precious vs $GIT_DIR/info/precious yet---just throwing an alternative for others to discuss.

By the way, I notice Ævar is CC'ed, but I haven't seen him for quite a while around here, and am wondering how you decided to do so. Did you have private conversations with and got suggestions from him or something? Just being curious, but at the same time, if somebody's influence in the resulting design is big enough, crediting them with "Helped-by:" or some other trailer might be worth considering.

On 2025-02-11 at 10:37 -0800, Junio C Hamano wrote:

In the same way as `git clean -e`, i.e., there is no interaction.

There were no implications behind my usage of the word "precious".

I don't think I'm trying to introduce any new fundamental concepts to Git. This patch is merely extending an existing command line option into a configuration knob, because I noticed myself passing the same arguments over and over and eventually creating an alias that does nothing but `git clean -e ...`, with the `-e` flag repeated a good 20 or so times.

The above descriptions are just that, free-form descriptions to help understand the intended use-case. I'm not sure I understand the reasons behind the "wishful thinking" label applied here.

config.mak is precisely one of such files that I now have in my own `clean.exclude`.

This email was part of the `perl contrib/contacts/git-contacts` output for this patchset, as documented in Documentation/SubmittingPatches and Documentation/MyFirstContribution.txt. Should I have not done that?

Ivan Shapovalov <intelfx@intelfx.name> writes:

That is dissapointing. I was hoping that "git status -u" would list precious and ignored ones in two separate sections.

Then you should ;-) We'd like to see us use the same language to refer to the same concept within this same project (and more importantly, avoid misleading people by calling two different things with the same phrase).

No, as I said, I was curious if he is getting involved with the project back again behind the curtain.

Thanks.

On 2025-02-11 at 13:24 -0800, Junio C Hamano wrote:

Do I need to implement those interactions in order for this patch set to be considered viable?

I did not intend to mislead anyone (as evident by the fact that I was simply not aware of any preexisting connotations). I'd appreciate suggestions for a replacement term.