{"thread":{"id":"64243","subject":"Untracked files cache not used when --untracked-files is used","startedAt":"2025-10-03T11:05:31Z","lastAt":"2025-10-07T09:14:21Z","messageCount":5,"participants":["Devste Devste","Matthew Hughes"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"527873","messageId":"CANM0SV3+4k5tsgRw0g8-HoVSFGb1AwZr+2hCUXLriFGSd33WQQ@mail.gmail.com","threadId":"64243","inReplyTo":null,"subject":"Untracked files cache not used when --untracked-files is used","fromName":"Devste Devste","fromEmail":"devstemail@gmail.com","sentAt":"2025-10-03T11:05:17Z","receivedAt":"2025-10-03T11:05:31Z","isPatch":false,"sender":{"key":"devstemail@gmail.com","avatar":null},"body":"I am using:\ngit version 2.51.0.windows.1\n\nRun:\ntime git status --untracked-files\n\nI get:\nIt took 2.75 seconds to enumerate untracked files,\nbut the results were cached, and subsequent runs may be faster.\nSee 'git help status' for information on how to improve this.\n\nreal    0m2.959s\nuser    0m0.000s\nsys     0m0.000s\n\n---\n\nwhen I run it again, I get the same thing again\n\nHowever, when I run it again WITHOUT --untracked-files its much faster now:\nreal    0m0.753s\nuser    0m0.000s\nsys     0m0.015s\n\nwhich means that the untracked cache does work (bc deleting the\nuntracked cache, it will be slow and also show the message)\nper the docs:\n>When -u option is not used, untracked files and directories are shown (i.e. the same as specifying normal)\n\nWhich I can confirm also with:\ngit update-index --test-untracked-cache\n>Testing mtime in 'C:/Foo/bar' ...... OK\n\nIt seems that using --untracked-files(=all) causes it to either not\nuse the untracked files cache (or untracked files are not stored in\nthe untracked files cache if they are in an untracked directory?)\nSince various tools and IDEs use that hardcoded, fixing this would be\na massive performance boost for many users\n"},{"id":"527921","messageId":"lrahpvp3nj4rtewf3zbf67qsvwhc4mg6irbmrhklbizxbhqwom@cc25oeqex4il","threadId":"64243","inReplyTo":"CANM0SV3+4k5tsgRw0g8-HoVSFGb1AwZr+2hCUXLriFGSd33WQQ@mail.gmail.com","subject":"Re: Untracked files cache not used when --untracked-files is used","fromName":"Matthew Hughes","fromEmail":"matthewhughes934@gmail.com","sentAt":"2025-10-05T00:20:38Z","receivedAt":"2025-10-05T00:20:42Z","isPatch":false,"sender":{"key":"matthewhughes934@gmail.com","avatar":"https://avatars.githubusercontent.com/u/34972397?v=4"},"body":"Devste Devste wrote:\n> It seems that using --untracked-files(=all) causes it to either not\n> use the untracked files cache (or untracked files are not stored in\n> the untracked files cache if they are in an untracked directory?)\n> Since various tools and IDEs use that hardcoded, fixing this would be\n> a massive performance boost for many users\n\nWhat's the value of your `status.showUntrackedFiles` config var? I ask because\nI looked around a bit and found commit e6a653554bb49c26d105f3b478cbdbb1c0648f65\n(untracked-cache: support '--untracked-files=all' if configured), which\nincludes:\n\n> For most users there will be no change in behavior. Users who need\n> '--untracked-files=all' to perform well will now have the option of\n> setting \"status.showuntrackedfiles\" to \"all\" for better / more\n> consistent performance.\n\nTesting this out on a big repo (on my Linux machine):\n\n    $ git init .\n    # create ~100_000 files with plenty of directories\n    $ for i in {1..10000}; do echo dir_$i/{1,2,3,4}/nested_{1,2}; done | xargs mkdir -p\n    $ for i in {1..10000}; do echo dir_$i/{foo,bar,baz}/file.txt; done | xargs touch\n    $ git add .\n\nAs expected, status with untracked files and no untracked cache is rather slow:\n\n    $ time GIT_CONFIG_GLOBAL=/dev/null git status --untracked-files=all >/dev/null\n\n    real\t0m1.237s\n    user\t0m0.484s\n    sys\t0m1.150s\n\nStatus with untracked files and `core.untrackedCache=true` is just as slow:\n\n    $ time GIT_CONFIG_GLOBAL=/dev/null git -c 'core.untrackedCache=true' status --untracked-files=all >/dev/null\n\n    real\t0m1.250s\n    user\t0m0.435s\n    sys\t0m1.216s\n\nHowever, with `status.untrackedFiles=all` (i.e. matching the `--untracked-files` flag) it's much quicker:\n\n    $ time GIT_CONFIG_GLOBAL=/dev/null git -c 'core.untrackedCache=true' -c 'status.showUntrackedFiles=all' status --untracked-files=all >/dev/null\n\n    real\t0m0.382s\n    user\t0m0.214s\n    sys\t0m0.568s\n"},{"id":"527933","messageId":"CANM0SV38YcQ+THbAyTR5cnaX3iDfb92qG8gyAzUaJ5jMK7FfYQ@mail.gmail.com","threadId":"64243","inReplyTo":"lrahpvp3nj4rtewf3zbf67qsvwhc4mg6irbmrhklbizxbhqwom@cc25oeqex4il","subject":"Re: Untracked files cache not used when --untracked-files is used","fromName":"Devste Devste","fromEmail":"devstemail@gmail.com","sentAt":"2025-10-05T11:14:52Z","receivedAt":"2025-10-05T11:15:06Z","isPatch":false,"sender":{"key":"devstemail@gmail.com","avatar":null},"body":"Thank you, this solved the problem indeed, status.showUntrackedFiles\nwas not set at all in my case.\n\nHowever: it did not work/update the cache initially with the command\nthe IDE/GUI usually runs:\n\ngit config status.showUntrackedFiles all\n\ntime git -c diff.mnemonicprefix=false -c core.quotepath=false\n--no-optional-locks status --porcelain --ignore-submodules=dirty\n--untracked-files=all --no-ahead-behind\nreal    0m3.463s\nuser    0m0.000s\nsys     0m0.000s\n\ntime git -c diff.mnemonicprefix=false -c core.quotepath=false\n--no-optional-locks status --porcelain --ignore-submodules=dirty\n--untracked-files=all --no-ahead-behind\nreal    0m3.423s\nuser    0m0.031s\nsys     0m0.015s\n\ntime git status --untracked-files\n...\n\nIt took 3.41 seconds to enumerate untracked files,\nbut the results were cached, and subsequent runs may be faster.\nSee 'git help status' for information on how to improve this.\n...\nreal    0m3.691s\nuser    0m0.000s\nsys     0m0.015s\n\ntime git status --untracked-files\n...\nreal    0m0.773s\nuser    0m0.000s\nsys     0m0.000s\n\ntime git -c diff.mnemonicprefix=false -c core.quotepath=false\n--no-optional-locks status --porcelain --ignore-submodules=dirty\n--untracked-files=all --no-ahead-behind\n...\nreal    0m0.818s\nuser    0m0.000s\nsys     0m0.015s\n\n---\n\nOne of --no-optional-locks --porcelain --ignore-submodules=dirty\n--no-ahead-behind causes it to not update the cache it seems.\nUnfortunately, I cannot tell which exactly, because now, even when\nunsetting status.showUntrackedFiles it uses the cache for\n--untracked-files=all\nThis means, that if the untracked cache was created with\nstatus.showUntrackedFiles all, it will always use the untracked-files\ncache for --untracked-files\n\nOn Sun, 5 Oct 2025 at 02:20, Matthew Hughes <matthewhughes934@gmail.com> wrote:\n>\n> Devste Devste wrote:\n> > It seems that using --untracked-files(=all) causes it to either not\n> > use the untracked files cache (or untracked files are not stored in\n> > the untracked files cache if they are in an untracked directory?)\n> > Since various tools and IDEs use that hardcoded, fixing this would be\n> > a massive performance boost for many users\n>\n> What's the value of your `status.showUntrackedFiles` config var? I ask because\n> I looked around a bit and found commit e6a653554bb49c26d105f3b478cbdbb1c0648f65\n> (untracked-cache: support '--untracked-files=all' if configured), which\n> includes:\n>\n> > For most users there will be no change in behavior. Users who need\n> > '--untracked-files=all' to perform well will now have the option of\n> > setting \"status.showuntrackedfiles\" to \"all\" for better / more\n> > consistent performance.\n>\n> Testing this out on a big repo (on my Linux machine):\n>\n>     $ git init .\n>     # create ~100_000 files with plenty of directories\n>     $ for i in {1..10000}; do echo dir_$i/{1,2,3,4}/nested_{1,2}; done | xargs mkdir -p\n>     $ for i in {1..10000}; do echo dir_$i/{foo,bar,baz}/file.txt; done | xargs touch\n>     $ git add .\n>\n> As expected, status with untracked files and no untracked cache is rather slow:\n>\n>     $ time GIT_CONFIG_GLOBAL=/dev/null git status --untracked-files=all >/dev/null\n>\n>     real        0m1.237s\n>     user        0m0.484s\n>     sys 0m1.150s\n>\n> Status with untracked files and `core.untrackedCache=true` is just as slow:\n>\n>     $ time GIT_CONFIG_GLOBAL=/dev/null git -c 'core.untrackedCache=true' status --untracked-files=all >/dev/null\n>\n>     real        0m1.250s\n>     user        0m0.435s\n>     sys 0m1.216s\n>\n> However, with `status.untrackedFiles=all` (i.e. matching the `--untracked-files` flag) it's much quicker:\n>\n>     $ time GIT_CONFIG_GLOBAL=/dev/null git -c 'core.untrackedCache=true' -c 'status.showUntrackedFiles=all' status --untracked-files=all >/dev/null\n>\n>     real        0m0.382s\n>     user        0m0.214s\n>     sys 0m0.568s\n"},{"id":"528007","messageId":"l62blkvi5ouzhkbfmimumhzdkv6tmszsxuxhlfkmu2lkspyihy@75466fqmo6ez","threadId":"64243","inReplyTo":"CANM0SV38YcQ+THbAyTR5cnaX3iDfb92qG8gyAzUaJ5jMK7FfYQ@mail.gmail.com","subject":"Re: Untracked files cache not used when --untracked-files is used","fromName":"Matthew Hughes","fromEmail":"matthewhughes934@gmail.com","sentAt":"2025-10-06T17:25:51Z","receivedAt":"2025-10-06T17:25:55Z","isPatch":false,"sender":{"key":"matthewhughes934@gmail.com","avatar":"https://avatars.githubusercontent.com/u/34972397?v=4"},"body":"> One of --no-optional-locks --porcelain --ignore-submodules=dirty\n> --no-ahead-behind causes it to not update the cache it seems.\n> Unfortunately, I cannot tell which exactly, because now, even when\n> unsetting status.showUntrackedFiles it uses the cache for\n> --untracked-files=all\n> This means, that if the untracked cache was created with\n> status.showUntrackedFiles all, it will always use the untracked-files\n> cache for --untracked-files\n\nYou can disable the untracked cache with `git update-index\n--no-untracked-cache`. Experimenting with that, on my machine the culprit looks\nto be `--no-optional-locks`:\n\n    time GIT_CONFIG_GLOBAL=/dev/null git \\\n        --no-optional-locks \\\n        -c 'diff.mnemonicprefix=false' \\\n        -c 'core.quotepath=false' \\\n        -c 'core.untrackedCache=true' \\\n        -c 'status.showUntrackedFiles=all' \\\n        status \\\n        --porcelain \\\n        --ignore-submodules=dirty \\\n        --no-ahead-behind \\\n        --untracked-files=all  >/dev/null\n\nWill consistently, on repeated runs, take >1s. After removing\n`--no-optional-locks` one more run is still slow, but after that it drops to\n~300ms.\n\nGlancing at the code: the likely cause is the `repo_update_index_if_able` call\nin `cmd_status` is only called when `use_optional_locks` returns a truthy\nvalue.\n"},{"id":"528057","messageId":"CANM0SV1p3auirRoHTfh2r=JuP-QU7uyK4o1+PZNVPT1mCVSqXw@mail.gmail.com","threadId":"64243","inReplyTo":"l62blkvi5ouzhkbfmimumhzdkv6tmszsxuxhlfkmu2lkspyihy@75466fqmo6ez","subject":"Re: Untracked files cache not used when --untracked-files is used","fromName":"Devste Devste","fromEmail":"devstemail@gmail.com","sentAt":"2025-10-07T09:14:07Z","receivedAt":"2025-10-07T09:14:21Z","isPatch":false,"sender":{"key":"devstemail@gmail.com","avatar":null},"body":"Which means essentially, I have to run git status --untracked-files\nonce a day to keep the untracked cache up to date, since I can't\nchange the git commands SourceTree uses\n\nOn Mon, 6 Oct 2025 at 19:25, Matthew Hughes <matthewhughes934@gmail.com> wrote:\n>\n> > One of --no-optional-locks --porcelain --ignore-submodules=dirty\n> > --no-ahead-behind causes it to not update the cache it seems.\n> > Unfortunately, I cannot tell which exactly, because now, even when\n> > unsetting status.showUntrackedFiles it uses the cache for\n> > --untracked-files=all\n> > This means, that if the untracked cache was created with\n> > status.showUntrackedFiles all, it will always use the untracked-files\n> > cache for --untracked-files\n>\n> You can disable the untracked cache with `git update-index\n> --no-untracked-cache`. Experimenting with that, on my machine the culprit looks\n> to be `--no-optional-locks`:\n>\n>     time GIT_CONFIG_GLOBAL=/dev/null git \\\n>         --no-optional-locks \\\n>         -c 'diff.mnemonicprefix=false' \\\n>         -c 'core.quotepath=false' \\\n>         -c 'core.untrackedCache=true' \\\n>         -c 'status.showUntrackedFiles=all' \\\n>         status \\\n>         --porcelain \\\n>         --ignore-submodules=dirty \\\n>         --no-ahead-behind \\\n>         --untracked-files=all  >/dev/null\n>\n> Will consistently, on repeated runs, take >1s. After removing\n> `--no-optional-locks` one more run is still slow, but after that it drops to\n> ~300ms.\n>\n> Glancing at the code: the likely cause is the `repo_update_index_if_able` call\n> in `cmd_status` is only called when `use_optional_locks` returns a truthy\n> value.\n"}]}