Slow git add . performance in large repo

We have a relatively large git repo and have noticed that `git add .` operations are slow (~1.5-2s). We have core.fsmonitor and core.untrackedCache set to true and `git status` executes in ~300ms. When I turn on trace2 I can see that almost all the time is spent in read_directo and it's visiting 26960 directories and 77989 paths.

I can use `git add <foo>` or `git add -u .` to speed things up but `git add .` is the most convenient for us. I created a small script to pipe the results of `git status` to `git add` and that runs in <500ms. This leaves me confused as to why the built-in performance is so slow.

git version is 2.49.0, mac os x 15.3

On 2025-03-17 at 18:53:10, Yissachar Radcliffe wrote:

What you're asking for with those commands is different. `git add -u .` says, "Please enumerate only those files that are in the index, and if they are modified or removed, update the index." `git add .` says, "Please enumerate every file in the working tree recursively and determine if there are any non-ignored changes, and then update the index." (Note that a file that matches an ignore pattern but is already tracked is not ignored, which affects the performance here.)

Notably, the former does not add new files that are untracked, but the latter does. That means that the code needs to know if there are any new untracked files. The untracked cache is not used when you specify a pathspec on the command line because in the general case, it doesn't have to be just `.` and it could be something like a match on an attribute or a glob pattern, which would make the code very complex in dealing with that case. It is, however, used when you _don't_ specify a pathspec (such as `git add -u`), as well as for `git status`, since those operate on the whole tree without any pathspecs.

When you pipe the results of `git status` to `git add`, you are effectively using the `-u` option, since that will only ever list files that are tracked.

I realize `git add .` is very convenient, but it does ask to do substantially more work than `git add -u` (which I use quite frequently), and so it can definitely perform worse, especially in large repositories. You can, of course, continue to use it, but you can't expect them to perform identically. My recommendation would be to use `git add -u` unless you need to add new files, since that's going to perform better. Once you get used to it, it's pretty easy to use.

I'm not sure what you mean by this; `git status` lists untracked files. For instance, if I `touch foo.txt` and `git add -u .` then foo.txt will not be staged. But if I pipe the changes from `git status` into `git add` then it will be added.

Is there a reason `git add .` couldn't use the untracked cache even if other pathspecs didn't? I have to imagine that `.` is by far the most common pathspec used and there would be value in speeding that up.

I wouldn't expect them to perform identically, but given how much faster it runs when piping in the data from `git status` I think it's reasonable to expect it to run much faster than it does today.

On 2025-03-18 at 02:04:43, Yissachar Radcliffe wrote:

Ah, I thought you meant piping the entries without `??`, which are already in the index. Yes, this is faster because it uses the untracked cache in many cases.

I don't see why it's impossible, but nobody has sent a patch. Most Git developers don't use `git add .` because there are better options and typically it isn't recommend to just add everything, so it hasn't been implemented.

As I said above, `git add .` isn't something I expect most Git developers use on a daily basis. It's very easy to accidentally add something you didn't intend, such as a build product that was formerly ignored but now is not (because it's no longer generated and someone removed the pattern), so it's not an approach that we typically recommend for that reason. The possibility of files that have accidentally not been ignored properly is not at all uncommon, and I run into it probably a couple times a month between work and home, even though I work with people who usually range from moderately to intimately familiar with Git.

If you feel strongly that this should exist, then the code is in `dir.c` (search for `pathspec`), and you could add a special case for this and send a patch. That doesn't guarantee that it will be accepted, but it certainly is more likely if you send a patch.