{"thread":{"id":"58628","subject":"[PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","startedAt":"2022-10-15T13:04:18Z","lastAt":"2023-05-11T05:18:14Z","messageCount":58,"participants":["Rudy Rigot via GitGitGadget","Rudy Rigot","Jeff Hostetler","Taylor Blau","Ævar Arnfjörð Bjarmason","Derrick Stolee","Eric Sunshine","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"465017","messageId":"pull.1384.git.1665839050813.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":null,"subject":"[PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-15T13:04:10Z","receivedAt":"2022-10-15T13:04:18Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nCurrently, if git-status takes more than 2 seconds for enumerating\nuntracked files, a piece of advice is given to the user to consider\nignoring untracked files. This is somewhat at odds with the UX\nupsides from having fsmonitor enabled, since fsmonitor will be\nhere to take care of mitigating the performance downsides from\nthose untracked files.\n\nI considered just suppressing that piece of advice entirely for\nrepositories with fsmonitor disabled, but I decided to replace\nit with another piece of advice instead, letting the user know\nthat this run may have been slow, but the next ones should be faster.\nOf course, please let me know if the phrasing can be improved. To\nkeep consistent with other pieces of advice, this new one can be\nhidden with a new advice.statusFsmonitor config.\n\nIf the repository does not have fsmonitor enabled, or if the new\npiece of advice is hidden by config, the behavior falls back to\ntoday's behavior: show the message advising to ignore untracked\nfiles, as long as it wasn't disabled with the existing advice.statusUoption\nconfig.\n\nTest-wise, I tried to figure out ways to mock the behavior of a\nslow git-status, but I couldn't figure it out, so I could use some\nadvice. I tracked down Commit 6a38ef2ced (status: advise to consider\nuse of -u when read_directory takes too long, 2013-03-13), and it\nalso didn't have tests, so I'm questioning whether it can in fact\nbe reasonably done. Thanks in advance for any guidance.\n\nSigned-off-by: Rudy Rigot \n---\n fsmonitor: long status advice adapted to the fsmonitor use case\n \n Currently, if git-status takes more than 2 seconds for enumerating\n untracked files, a piece of advice is given to the user to consider\n ignoring untracked files. This is somewhat at odds with the UX upsides\n from having fsmonitor enabled, since fsmonitor will be here to take care\n of mitigating the performance downsides from those untracked files.\n \n I considered just suppressing that piece of advice entirely for\n repositories with fsmonitor disabled, but I decided to replace it with\n another piece of advice instead, letting the user know that this run may\n have been slow, but the next ones should be faster. Of course, please\n let me know if the phrasing can be improved. To keep consistent with\n other pieces of advice, this new one can be hidden with a new\n advice.statusFsmonitor config.\n \n If the repository does not have fsmonitor enabled, or if the new piece\n of advice is hidden by config, the behavior falls back to today's\n behavior: show the message advising to ignore untracked files, as long\n as it wasn't disabled with the existing advice.statusUoption config.\n \n Test-wise, I tried to figure out ways to mock the behavior of a slow\n git-status, but I couldn't figure it out, so I could use some advice. I\n tracked down Commit 6a38ef2 (status: advise to consider use of -u when\n read_directory takes too long, 2013-03-13), and it also didn't have\n tests, so I'm questioning whether it can in fact be reasonably done.\n Thanks in advance for any guidance.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\n Documentation/config/advice.txt | 3 +++\n advice.c | 1 +\n advice.h | 1 +\n wt-status.c | 23 ++++++++++++++++-------\n 4 files changed, 21 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt\nindex a00d0100a82..e8ebcf1b023 100644\n--- a/Documentation/config/advice.txt\n+++ b/Documentation/config/advice.txt\n@@ -57,6 +57,9 @@ advice.*::\n \t\tand that calculation takes longer than expected. Will not\n \t\tappear if `status.aheadBehind` is false or the option\n \t\t`--no-ahead-behind` is given.\n+\tstatusFsmonitor::\n+\t\tShown when the linkgit:git-status[1] command takes more than\n+\t\t2 seconds to enumerate untracked files, and fsmonitor is enabled.\n \tstatusHints::\n \t\tShow directions on how to proceed from the current\n \t\tstate in the output of linkgit:git-status[1], in\ndiff --git a/advice.c b/advice.c\nindex fd189689437..448b11ef273 100644\n--- a/advice.c\n+++ b/advice.c\n@@ -69,6 +69,7 @@ static struct {\n \t[ADVICE_SET_UPSTREAM_FAILURE]\t\t\t= { \"setUpstreamFailure\", 1 },\n \t[ADVICE_SKIPPED_CHERRY_PICKS]\t\t\t= { \"skippedCherryPicks\", 1 },\n \t[ADVICE_STATUS_AHEAD_BEHIND_WARNING]\t\t= { \"statusAheadBehindWarning\", 1 },\n+\t[ADVICE_STATUS_FSMONITOR]\t\t\t= { \"statusFsmonitor\", 1 },\n \t[ADVICE_STATUS_HINTS]\t\t\t\t= { \"statusHints\", 1 },\n \t[ADVICE_STATUS_U_OPTION]\t\t\t= { \"statusUoption\", 1 },\n \t[ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE] = { \"submoduleAlternateErrorStrategyDie\", 1 },\ndiff --git a/advice.h b/advice.h\nindex 07e0f76833e..a458619a160 100644\n--- a/advice.h\n+++ b/advice.h\n@@ -43,6 +43,7 @@ struct string_list;\n \tADVICE_SEQUENCER_IN_USE,\n \tADVICE_SET_UPSTREAM_FAILURE,\n \tADVICE_STATUS_AHEAD_BEHIND_WARNING,\n+\tADVICE_STATUS_FSMONITOR,\n \tADVICE_STATUS_HINTS,\n \tADVICE_STATUS_U_OPTION,\n \tADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE,\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..d6c7f0fa21a 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,6 +18,7 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n \n@@ -1814,6 +1815,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1872,20 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\tif (2000 < s->untracked_in_ms) {\n+\t\t\tif (advice_enabled(ADVICE_STATUS_FSMONITOR) && fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took a while to check your git status this time, but the results\\n\"\n+\t\t\t\t\t\t\"were cached, and your next runs should be faster.\"));\n+\t\t\t} else if (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n+\t\t\t\t\t\t\"may speed it up, but you have to be careful not to forget to add\\n\"\n+\t\t\t\t\t\t\"new files yourself (see 'git help status').\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: d420dda0576340909c3faff364cfbd1485f70376\n-- \ngitgitgadget\n"},{"id":"465018","messageId":"CANaDLW+xgfEFHWX9T45MT-WaPOXHeij80K17PCnuzJU636rMkQ@mail.gmail.com","threadId":"58628","inReplyTo":"pull.1384.git.1665839050813.gitgitgadget@gmail.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-10-15T13:08:45Z","receivedAt":"2022-10-15T13:09:04Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> I considered just suppressing that piece of advice entirely for\nrepositories with fsmonitor disabled\n\nSorry, I just noticed that I misspoke here. I meant repositories with\nfsmonitor *enabled*, of course.\n"},{"id":"465109","messageId":"087d3fca-d01e-f36a-85f5-7e861e4d11ca@jeffhostetler.com","threadId":"58628","inReplyTo":"pull.1384.git.1665839050813.gitgitgadget@gmail.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-10-17T15:39:46Z","receivedAt":"2022-10-17T15:39:54Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 10/15/22 9:04 AM, Rudy Rigot via GitGitGadget wrote:\n> From: Rudy Rigot \n> \n> Currently, if git-status takes more than 2 seconds for enumerating\n> untracked files, a piece of advice is given to the user to consider\n> ignoring untracked files. This is somewhat at odds with the UX\n> upsides from having fsmonitor enabled, since fsmonitor will be\n> here to take care of mitigating the performance downsides from\n> those untracked files.\n\nYes, the original advice needs some updating. Thanks!\n\n\nWe should be careful here, FSMonitor only helps with untracked\nfiles if the untracked cache (UC) is also turned on. They do work\nwell together and they greatly speed up things, but if either is\nturned off, `git status` will still need to scan. (Small nerd\ndetail: the UC by itself does speed things up -- it only needs\nto lstat known directories most of the time, rather than actually\nreaddir them. When both are on, we don't even need to do that.)\n\nThe original message suggested turning off the untracked file (UF)\nscan completely. Perhaps, we could have advice say to turn off\nUF scan -or- turn on FSM & UC ?\n\n\n> \n> I considered just suppressing that piece of advice entirely for\n> repositories with fsmonitor disabled, but I decided to replace\n> it with another piece of advice instead, letting the user know\n> that this run may have been slow, but the next ones should be faster.\n> Of course, please let me know if the phrasing can be improved. To\n> keep consistent with other pieces of advice, this new one can be\n> hidden with a new advice.statusFsmonitor config.\n> \n> If the repository does not have fsmonitor enabled, or if the new\n> piece of advice is hidden by config, the behavior falls back to\n> today's behavior: show the message advising to ignore untracked\n> files, as long as it wasn't disabled with the existing advice.statusUoption\n> config.\n\n\nWe also need to be careful here. With FSMonitor the \"first one is\nslow, the second one is fast\" happens because `git status` is secretly\nupdating the index (despite looking like a read-only command). That\ncauses the index to have an updated FSM token (so that subsequent\nFSM responses are relative to a more recent checkpoint). See [1].\nSo it isn't always correct that the second status will be faster.\nIt usually is, but it just depends on the threshold -- and more\nimportantly, that the UC is turned on. (So I guess what I'm saying\nis that again, we should advise to turn UF or turn on both FSM & UC.)\n\n\n[1] 26b9f34ab3 (fsmonitor: force update index after large responses, \n2022-03-25)\n\n> \n> Test-wise, I tried to figure out ways to mock the behavior of a\n> slow git-status, but I couldn't figure it out, so I could use some\n> advice. I tracked down Commit 6a38ef2ced (status: advise to consider\n> use of -u when read_directory takes too long, 2013-03-13), and it\n> also didn't have tests, so I'm questioning whether it can in fact\n> be reasonably done. Thanks in advance for any guidance.\n[...]\n\nWRT testing, you might do something like:\n\n\t#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n\n\tstatic inline int uf_was_slow(uint32_t untracked_in_ms)\n\t{\n\t#ifdef GIT_TEST_UF_DELAY_WARNING // or maybe use getenv() here\n\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n\t#endif\n\n\t\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n\t}\n\nAnd then you can set the GIT_TEST_ symbol (or env var) during the\ntest scripts and confirm that we get the messages that you expect.\n\nHope this helps,\nJeff\n"},{"id":"465113","messageId":"CANaDLWKcF07=FQgT7ZTKmcgworH45YdNy8hy2faMBg3CGYEf+w@mail.gmail.com","threadId":"58628","inReplyTo":"087d3fca-d01e-f36a-85f5-7e861e4d11ca@jeffhostetler.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-10-17T16:59:02Z","receivedAt":"2022-10-17T16:59:17Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> We should be careful here, FSMonitor only helps with untracked\nfiles if the untracked cache (UC) is also turned on. They do work\nwell together and they greatly speed up things, but if either is\nturned off, `git status` will still need to scan.\n\nOh, thanks, I didn't realize! With that, I agree that the messaging I'm\nproposing is not technically correct, and needs to be fixed.\n\nI agree with your advice about the message when FSMonitor and\nuntracked cache are both turned off. I'm also trying to think about\nwhat advice to give when they are turned on, and git status was\nslow because the update-index was building the cache on that call.\n\nImportantly, I'm trying to think of ways to keep the messaging\naccessible even when the user is not familiar with those concepts.\nI'm thinking a user may not even know what is currently enabled or\nnot in their environment, so there's probably value in detecting their\nsituation, and best adapting the messaging to it.\n\nFor context, in our case, we set core.fsmonitor and core.untrackedCache\nas part of our dev environment setup script, because we don't expect\nour least advanced developers to ramp up on what they are. And yet,\nit is useful to all of them to have them enabled, our git status is about\n30 seconds long without FSMonitor and UC.\n\nAs a result, we have been receiving negative feedback that git status is\nslow, but when we inform the user that it is cached, they run it again and\nconfirm that it is fine like this. The problem being about educating\nusers and not a technical issue, of course we're adding the info to our\nsetup doc, but I figured other large repos may hit this usability issue, so\nhere I am.\n\nWhat do you think of those phrasings?\n\n- If neither FSMonitor nor the untracked cache are turned on, changing\nthe current advice to: \"It took %.2f seconds to enumerate untracked files.\nYou may want to skip that part with 'status -uno' but you have to be\ncareful not to forget to add new files yourself (see 'git help status').\nOtherwise, you can enable the core.untrackedCache config to have\nit be cached, and potentially the core.fsmonitor config to further improve\nthe cache's performance.\"\n\n- If only the untracked cache is turned on, since you said it could already\nimprove some: \"It took %.2f seconds to enumerate untracked files.\nYour untracked cache is enabled, but you may want to enable the\ncore.fsmonitor config to further improve the cache's performance.\nOtherwise, you may want to skip that part with 'status -uno' but you have\nto be careful not to forget to add new files yourself (see 'git help status').\"\n\n- If they're both turned on: \"It took %.2f seconds to enumerate untracked\nfiles. Your untracked cache is enabled and fsmonitor is on,\nso your next calls may be faster. Otherwise, you may want to skip that part\nwith 'status -uno' but you have to be careful not to forget to add new\nfiles yourself (see 'git help status').\"\n\nI intentionally phrased it all in a way that manages expectations, \"may be\nfaster\", since you said the state of the FSM token could be impacted by a\nvariety of things, so we can't exactly commit to anything for sure about the\nnext call.\n\nLet me know what you think, I definitely want to do this right.\n\n\n> WRT testing\n\nOh, I didn't realize I could do this! Alright, I need time to look into it.\nWith that, it means I can test the existing advice message use case,\nwhich I don't believe is currently tested, so I'm double-glad!\n\n\nThanks a lot for all the insight.\n"},{"id":"465292","messageId":"d696b07f-cfa9-be45-b6d2-adb72811a205@jeffhostetler.com","threadId":"58628","inReplyTo":"CANaDLWKcF07=FQgT7ZTKmcgworH45YdNy8hy2faMBg3CGYEf+w@mail.gmail.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-10-20T12:56:48Z","receivedAt":"2022-10-20T12:57:03Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 10/17/22 12:59 PM, Rudy Rigot wrote:\n>> We should be careful here, FSMonitor only helps with untracked\n> files if the untracked cache (UC) is also turned on. They do work\n> well together and they greatly speed up things, but if either is\n> turned off, `git status` will still need to scan.\n> \n> Oh, thanks, I didn't realize! With that, I agree that the messaging I'm\n> proposing is not technically correct, and needs to be fixed.\n> \n> I agree with your advice about the message when FSMonitor and\n> untracked cache are both turned off. I'm also trying to think about\n> what advice to give when they are turned on, and git status was\n> slow because the update-index was building the cache on that call.\n> \n> Importantly, I'm trying to think of ways to keep the messaging\n> accessible even when the user is not familiar with those concepts.\n\nAgreed. We sometimes forget that not everyone is an expert in the\nsubtleties and terms of Git. Even the original message \"...enumerate\nuntracked files...\" can be a little obscure to the casual user.\n\n> I'm thinking a user may not even know what is currently enabled or\n> not in their environment, so there's probably value in detecting their\n> situation, and best adapting the messaging to it.\n> \n> For context, in our case, we set core.fsmonitor and core.untrackedCache\n> as part of our dev environment setup script, because we don't expect\n> our least advanced developers to ramp up on what they are. And yet,\n> it is useful to all of them to have them enabled, our git status is about\n> 30 seconds long without FSMonitor and UC.\n> \n> As a result, we have been receiving negative feedback that git status is\n> slow, but when we inform the user that it is cached, they run it again and\n> confirm that it is fine like this. The problem being about educating\n> users and not a technical issue, of course we're adding the info to our\n> setup doc, but I figured other large repos may hit this usability issue, so\n> here I am.\n> \n> What do you think of those phrasings?\n> \n> - If neither FSMonitor nor the untracked cache are turned on, changing\n> the current advice to: \"It took %.2f seconds to enumerate untracked files.\n> You may want to skip that part with 'status -uno' but you have to be\n> careful not to forget to add new files yourself (see 'git help status').\n> Otherwise, you can enable the core.untrackedCache config to have\n> it be cached, and potentially the core.fsmonitor config to further improve\n> the cache's performance.\"\n> \n> - If only the untracked cache is turned on, since you said it could already\n> improve some: \"It took %.2f seconds to enumerate untracked files.\n> Your untracked cache is enabled, but you may want to enable the\n> core.fsmonitor config to further improve the cache's performance.\n> Otherwise, you may want to skip that part with 'status -uno' but you have\n> to be careful not to forget to add new files yourself (see 'git help status').\"\n> \n> - If they're both turned on: \"It took %.2f seconds to enumerate untracked\n> files. Your untracked cache is enabled and fsmonitor is on,\n> so your next calls may be faster. Otherwise, you may want to skip that part\n> with 'status -uno' but you have to be careful not to forget to add new\n> files yourself (see 'git help status').\"\n\nAll three of these are a bit wordy and/or awkward -- but I'm not sure\nhow to say it either, so don't feel bad. I think it is just the quality\nof the corner that we've been backed into -- there are just too many\nknobs and not enough space to do them justice. And I've always found\nthe \"turn this on, but be careful...\" advice/warning a bit odd.\n\n\nAs an alternative, and definitely just thinking out loud here.\n\nWould it be better to add a \"Untracked Files and Status Speed\"\nsection to the bottom of https://git-scm.com/docs/git-status\n(aka `Documentation/git-status.txt`) near\nhttps://git-scm.com/docs/git-status#_background_refresh\n\nDescribe the various combination of config options in more detail\nand discuss the pros/cons of each, what the defaults are, and how to\nset them, how to tell what they are currently set to, and so on.\n\nThen have the advice message reference that new section.\n\nWe might also reword the large paragraph (\"When `-u` option is not...\")\nin the `--untracked-files` argument to reference the new section. It is\na little stale and just as awkward.\n\nJust a thought...\n\n\nJeff\n"},{"id":"465331","messageId":"CANaDLWJrBHAD4gu-h1sRgHXPbhf6YneJDF_+rhcRVDueKZfpLA@mail.gmail.com","threadId":"58628","inReplyTo":"d696b07f-cfa9-be45-b6d2-adb72811a205@jeffhostetler.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-10-20T20:17:06Z","receivedAt":"2022-10-20T20:17:24Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Oooh, I definitely like where your mind is with this. I think it makes a\nlot of sense, there used to be one way to act upon a slow status, now there\nare a few, so I can see how any in-depth explanation here would add to the\nconfusion for a user in a terminal who is just trying to get things done.\nAnd I see how the current messaging already kinda infringes on that.\n\nAlright, I can write the first draft of the documentation changes you were\nmentioning. Heads up though: I'm going to need your tight review of it,\nbecause I'm not as completely comfortable with what each option does as I\nwish I was, so I worry I may write something accidentally inaccurate. I'll\ntake the time to read up on it though, and then I'll try my hand at it and\nupdate it here, and let's take it from there if that sounds good.\n\nWith that, I'm thinking the \"slow status\" advice could be turned into\nsomething as simple as:\n\n> Your git status run was slow, here are some ways to optimize it.\n> https://git-scm.com/docs/git-status#_performance_optimizations\n\n(To be clear, I'm very un-opinionated about phrasing.)\n\nThere's one thing I'm still worried about though, which is what you\nmentioned earlier, and which brought me here: the fact that git-status\nfeels like a read-only command, but secretly isn't. I'm thinking the\nconfusing use case is when the repository was in fact set in a way that\nthings are cached, and yet git-status is still slow because it was\ngenerating that cache, and the user doesn't have a way to know that.\n\nAdding to the murkiness, it might not have been the reason, so I understand\nwe can't say things in such a confident manner as \"it will be faster now\",\nbecause of course it depends.\n\nSo I'm thinking, after the message above when git-status was slow, in the\nspecific case where the untracked cache is on (whether FSMonitor is on or\nnot, since that sounds more like under-the-hood detail), we could display\nsomething like the additional line here:\n\n> Your git status run was slow, here are some ways to optimize it.\n> https://git-scm.com/docs/git-status#_performance_optimizations\n>\n> Your git status run was cached.\n\nIf the untracked cache is on, I'm assuming that would be always accurate\ninformation, is that correct?\n\nIf you're concerned that users may not understand what it means for them,\nwe could also make it more obvious without over-committing about it:\n\n> Your git status run was slow, here are some ways to optimize it.\n> https://git-scm.com/docs/git-status#_performance_optimizations\n>\n> Your git status run was cached, it may be faster on your next runs.\n\nWhat do you think?\n\nTo summarize, next steps for me:\n- Make the first draft for the doc updates.\n- Change the advice messaging based on our discussion above and what you\nthink of it.\n- I still need to look into your test-related advice from last time, I\nhaven't yet. I really would like to give tests to all this.\n\nThanks a lot for all this, it helps tremendously!\n"},{"id":"465617","messageId":"09b566b7-bdaa-af52-eb4f-63b2dbedcd89@jeffhostetler.com","threadId":"58628","inReplyTo":"CANaDLWJrBHAD4gu-h1sRgHXPbhf6YneJDF_+rhcRVDueKZfpLA@mail.gmail.com","subject":"Re: [PATCH] fsmonitor: long status advice adapted to the fsmonitor use case","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-10-24T14:55:25Z","receivedAt":"2022-10-24T16:05:34Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 10/20/22 4:17 PM, Rudy Rigot wrote:\n> Oooh, I definitely like where your mind is with this. I think it makes a\n> lot of sense, there used to be one way to act upon a slow status, now there\n> are a few, so I can see how any in-depth explanation here would add to the\n> confusion for a user in a terminal who is just trying to get things done.\n> And I see how the current messaging already kinda infringes on that.\n> \n> Alright, I can write the first draft of the documentation changes you were\n> mentioning. Heads up though: I'm going to need your tight review of it,\n> because I'm not as completely comfortable with what each option does as I\n> wish I was, so I worry I may write something accidentally inaccurate. I'll\n> take the time to read up on it though, and then I'll try my hand at it and\n> update it here, and let's take it from there if that sounds good.\n > [...]\n\nyeah, i'm happy to help. just let me know when you have a first draft.\n\nJeff\n"},{"id":"465996","messageId":"pull.1384.v2.git.1667002005494.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.git.1665839050813.gitgitgadget@gmail.com","subject":"[PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-10-29T00:06:45Z","receivedAt":"2022-10-29T00:06:55Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nCurrently, if git-status takes more than 2 seconds for enumerating untracked\nfiles, a piece of advice is given to the user to consider ignoring untracked\nfiles. But Git now offers more possibilities to resolve that situation\n(untracked cache, fsmonitor) with different downsides.\n\nThis change is about refreshing that advice message. A new section in the\ndocumentation is introduced to present the possibilities, and the advice\nmessage links to it. I'm also introducing tests for this advice message,\nwhich was untested so far.\n\nOne of the downsides of untracked cache / fsmonitor, is that the first call\nmay be long in order to generate the cache, but the user may not know what\ntheir current configuration is. When collecting feedback from users of our\nvery large repo, that's the most common point of confusion that keeps coming\nback: people complain about git status being slow, but are satisfied when\nwe inform them that it's being cached and they should run it again to check.\nAs a result, the advice message tries to keep them informed of their current\nconfiguration.\n\nSigned-off-by: Rudy Rigot \n---\n status: long status advice adapted to recent capabilities\n \n Here is version 2 for this patch to change the long status advice to\n adapt to recent capabilities (untracked cache, FSMonitor).\n \n Clerical note: I /preview'd this with GitGitGadget, and this seems to be\n expressed as a patch on top of my first patch (\"Range-diff vs v1\")\n first, before showing the actual patch as refreshed. I'm assuming that\n is what is expected, please let me know if I'm doing this wrong.\n \n Changes since v1:\n \n * Introduced a new section in git status's docs to explain the various\n options, that the new advice can link to. I realized there's already\n solid details about untracked cache and fsmonitor in the\n update-index's docs, so I kept the new section very high-level and\n strategic, linking to update-index's docs for more details. I don't\n think anything's inaccurate, but I really could use some eyes on this\n anyway to be sure.\n * Changed the advice message depending on context (see test files to\n see every possible phrasing the end user can meet). I am very open to\n feedback, of course. One of my priorities was to keep the\n already-optimized user in the loop about what optimizations they may\n already have, in case they're only seeing this message because it's\n the git status run that did the first caching, since that's what's\n been confusing our users when using fsmonitor.\n * Introduced tests for all this. I chose to simulate the slowness\n thanks to a an environment variable rather than a symbol, just\n because it was also helpful to set it from the outside at dev time,\n but I'm not strongly opinionated about whether it should be one or\n the other.\n * Removed the advice.statusFsmonitor config I had introduced in v1,\n since it's all one short consolidated advice message now.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v1:\n\n 1: cc372e7c9d0 ! 1: 9ef7f1834b7 fsmonitor: long status advice adapted to the fsmonitor use case\n @@ Metadata\n Author: Rudy Rigot \n \n ## Commit message ##\n - fsmonitor: long status advice adapted to the fsmonitor use case\n + status: long status advice adapted to recent capabilities\n \n - Currently, if git-status takes more than 2 seconds for enumerating\n - untracked files, a piece of advice is given to the user to consider\n - ignoring untracked files. This is somewhat at odds with the UX\n - upsides from having fsmonitor enabled, since fsmonitor will be\n - here to take care of mitigating the performance downsides from\n - those untracked files.\n + Currently, if git-status takes more than 2 seconds for enumerating untracked\n + files, a piece of advice is given to the user to consider ignoring untracked\n + files. But Git now offers more possibilities to resolve that situation\n + (untracked cache, fsmonitor) with different downsides.\n \n - I considered just suppressing that piece of advice entirely for\n - repositories with fsmonitor disabled, but I decided to replace\n - it with another piece of advice instead, letting the user know\n - that this run may have been slow, but the next ones should be faster.\n - Of course, please let me know if the phrasing can be improved. To\n - keep consistent with other pieces of advice, this new one can be\n - hidden with a new advice.statusFsmonitor config.\n + This change is about refreshing that advice message. A new section in the\n + documentation is introduced to present the possibilities, and the advice\n + message links to it. I'm also introducing tests for this advice message,\n + which was untested so far.\n \n - If the repository does not have fsmonitor enabled, or if the new\n - piece of advice is hidden by config, the behavior falls back to\n - today's behavior: show the message advising to ignore untracked\n - files, as long as it wasn't disabled with the existing advice.statusUoption\n - config.\n -\n - Test-wise, I tried to figure out ways to mock the behavior of a\n - slow git-status, but I couldn't figure it out, so I could use some\n - advice. I tracked down Commit 6a38ef2ced (status: advise to consider\n - use of -u when read_directory takes too long, 2013-03-13), and it\n - also didn't have tests, so I'm questioning whether it can in fact\n - be reasonably done. Thanks in advance for any guidance.\n + One of the downsides of untracked cache / fsmonitor, is that the first call\n + may be long in order to generate the cache, but the user may not know what\n + their current configuration is. When collecting feedback from users of our\n + very large repo, that's the most common point of confusion that keeps coming\n + back: people complain about git status being slow, but are satisfied when\n + we inform them that it's being cached and they should run it again to check.\n + As a result, the advice message tries to keep them informed of their current\n + configuration.\n \n Signed-off-by: Rudy Rigot \n \n - ## Documentation/config/advice.txt ##\n -@@ Documentation/config/advice.txt: advice.*::\n - \t\tand that calculation takes longer than expected. Will not\n - \t\tappear if `status.aheadBehind` is false or the option\n - \t\t`--no-ahead-behind` is given.\n -+\tstatusFsmonitor::\n -+\t\tShown when the linkgit:git-status[1] command takes more than\n -+\t\t2 seconds to enumerate untracked files, and fsmonitor is enabled.\n - \tstatusHints::\n - \t\tShow directions on how to proceed from the current\n - \t\tstate in the output of linkgit:git-status[1], in\n + ## Documentation/git-status.txt ##\n +@@ Documentation/git-status.txt: during the write may conflict with other simultaneous processes, causing\n + them to fail. Scripts running `status` in the background should consider\n + using `git --no-optional-locks status` (see linkgit:git[1] for details).\n + \n ++UNTRACKED FILES AND STATUS SPEED\n ++--------------------------------\n ++\n ++If your untracked files take an unusual amount of time to enumerate, your\n ++repository certainly has a lot of them, and an advice message will display\n ++about it. Here are some configurations to consider in order to improve the\n ++situation:\n ++\n ++* Setting the `core.untrackedCache` configuration as `true` will allow for\n ++`git status` to keep track of the mtime of folders, in order to cache past\n ++`status` results and be sure to only browse folders that changed on subsequent\n ++runs, for filesystems that can support it (see linkgit:git-update-index[1]\n ++for details).\n ++* Used in conjonction with `core.untrackedCache`, setting the `core.fsmonitor`\n ++configuration as `true` will allow for `git status` to keep track of what\n ++files recently changed, in order to cache past `status` results and be sure\n ++to only focus on those files on subsequent runs (see linkgit:git-update-index[1]\n ++for details).\n ++* If none of the above options are satisfactory, setting the\n ++`status.showUntrackedFiles` configuration as `no` will cause `git status`\n ++to not attempt to list untracked files anymore, in which case you have to be\n ++careful not to forget to add new files yourself.\n ++\n ++If none of the above solutions are satisfactory, and you are bothered with\n ++the advice message, you can disable it by setting the `advice.statusUoption`\n ++configuration to `false`.\n ++\n + SEE ALSO\n + --------\n + linkgit:gitignore[5]\n +\n + ## t/t7065-wtstatus-slow.sh (new) ##\n +@@\n ++#!/bin/sh\n ++\n ++test_description='test status when slow untracked files'\n ++\n ++. ./test-lib.sh\n ++\n ++DATA=\"$TEST_DIRECTORY/t7065\"\n ++\n ++GIT_TEST_UF_DELAY_WARNING=1\n ++export GIT_TEST_UF_DELAY_WARNING\n ++\n ++test_expect_success setup '\n ++\tgit checkout -b test\n ++'\n ++\n ++test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n ++\ttest_must_fail git config --get core.untrackedCache &&\n ++\ttest_must_fail git config --get core.fsmonitor &&\n ++ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++ test_cmp \"$DATA/no_untrackedcache_no_fsmonitor\" ../actual &&\n ++ rm -fr ../actual\n ++'\n ++\n ++test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n ++ git config core.untrackedCache true &&\n ++\ttest_must_fail git config --get core.fsmonitor &&\n ++ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++ test_cmp \"$DATA/with_untrackedcache_no_fsmonitor\" ../actual &&\n ++ rm -fr ../actual\n ++'\n ++\n ++test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n ++ git config core.untrackedCache true &&\n ++\tgit config core.fsmonitor true &&\n ++ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++ test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n ++ rm -fr ../actual\n ++'\n ++\n ++test_done\n + \\ No newline at end of file\n \n - ## advice.c ##\n -@@ advice.c: static struct {\n - \t[ADVICE_SET_UPSTREAM_FAILURE]\t\t\t= { \"setUpstreamFailure\", 1 },\n - \t[ADVICE_SKIPPED_CHERRY_PICKS]\t\t\t= { \"skippedCherryPicks\", 1 },\n - \t[ADVICE_STATUS_AHEAD_BEHIND_WARNING]\t\t= { \"statusAheadBehindWarning\", 1 },\n -+\t[ADVICE_STATUS_FSMONITOR]\t\t\t= { \"statusFsmonitor\", 1 },\n - \t[ADVICE_STATUS_HINTS]\t\t\t\t= { \"statusHints\", 1 },\n - \t[ADVICE_STATUS_U_OPTION]\t\t\t= { \"statusUoption\", 1 },\n - \t[ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE] = { \"submoduleAlternateErrorStrategyDie\", 1 },\n + ## t/t7065/no_untrackedcache_no_fsmonitor (new) ##\n +@@\n ++On branch test\n ++\n ++No commits yet\n ++\n ++\n ++It took X seconds to enumerate untracked files.\n ++See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n ++for configuration options that may improve that time.\n ++\n ++nothing to commit (create/copy files and use \"git add\" to track)\n \n - ## advice.h ##\n -@@ advice.h: struct string_list;\n - \tADVICE_SEQUENCER_IN_USE,\n - \tADVICE_SET_UPSTREAM_FAILURE,\n - \tADVICE_STATUS_AHEAD_BEHIND_WARNING,\n -+\tADVICE_STATUS_FSMONITOR,\n - \tADVICE_STATUS_HINTS,\n - \tADVICE_STATUS_U_OPTION,\n - \tADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE,\n + ## t/t7065/with_untrackedcache_no_fsmonitor (new) ##\n +@@\n ++On branch test\n ++\n ++No commits yet\n ++\n ++\n ++It took X seconds to enumerate untracked files,\n ++but this is currently being cached, with fsmonitor OFF.\n ++See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n ++for configuration options that may improve that time.\n ++\n ++nothing to commit (create/copy files and use \"git add\" to track)\n +\n + ## t/t7065/with_untrackedcache_with_fsmonitor (new) ##\n +@@\n ++On branch test\n ++\n ++No commits yet\n ++\n ++\n ++It took X seconds to enumerate untracked files,\n ++but this is currently being cached, with fsmonitor ON.\n ++See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n ++for configuration options that may improve that time.\n ++\n ++nothing to commit (create/copy files and use \"git add\" to track)\n \n ## wt-status.c ##\n @@\n @@ wt-status.c\n +#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n ++#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n + \n + static const char cut_line[] =\n + \"------------------------ >8 ------------------------\\n\";\n +@@ wt-status.c: static void wt_longstatus_print_tracking(struct wt_status *s)\n + \tstrbuf_release(&sb);\n + }\n \n ++static inline int uf_was_slow(uint32_t untracked_in_ms)\n ++{\n ++\tconst char *x;\n ++\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n ++\tif (x) {\n ++\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n ++\t}\n ++\n ++\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n ++}\n ++\n + static void show_merge_in_progress(struct wt_status *s,\n + \t\t\t\t const char *color)\n + {\n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n -\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n -\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n -\t\t\t\t\t s->untracked_in_ms / 1000.0);\n -+\t\tif (2000 < s->untracked_in_ms) {\n -+\t\t\tif (advice_enabled(ADVICE_STATUS_FSMONITOR) && fsm_mode > FSMONITOR_MODE_DISABLED) {\n ++\t\tif (uf_was_slow(s->untracked_in_ms)) {\n ++\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n ++\t\t\t\tif (s->repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) {\n ++\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n ++\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n ++\t\t\t\t\t\t\t\"but this is currently being cached, with fsmonitor %s.\"),\n ++\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0,\n ++\t\t\t\t\t\t\t(fsm_mode > FSMONITOR_MODE_DISABLED) ? \"ON\" : \"OFF\");\n ++\t\t\t\t} else {\n ++\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n ++\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n ++\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n ++\t\t\t\t}\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t_(\"It took a while to check your git status this time, but the results\\n\"\n -+\t\t\t\t\t\t\"were cached, and your next runs should be faster.\"));\n -+\t\t\t} else if (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n ++\t\t\t\t\t\t_(\"See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\\n\"\n ++\t\t\t\t\t\t\"for configuration options that may improve that time.\"));\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n -+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n -+\t\t\t\t\t\t\"may speed it up, but you have to be careful not to forget to add\\n\"\n -+\t\t\t\t\t\t\"new files yourself (see 'git help status').\"),\n -+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n +\t\t\t}\n \t\t}\n \t} else if (s->committable)\n\n\n Documentation/git-status.txt | 27 +++++++++++++++\n t/t7065-wtstatus-slow.sh | 40 ++++++++++++++++++++++\n t/t7065/no_untrackedcache_no_fsmonitor | 10 ++++++\n t/t7065/with_untrackedcache_no_fsmonitor | 11 ++++++\n t/t7065/with_untrackedcache_with_fsmonitor | 11 ++++++\n wt-status.c | 40 ++++++++++++++++++----\n 6 files changed, 132 insertions(+), 7 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n create mode 100644 t/t7065/no_untrackedcache_no_fsmonitor\n create mode 100644 t/t7065/with_untrackedcache_no_fsmonitor\n create mode 100644 t/t7065/with_untrackedcache_with_fsmonitor\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 54a4b29b473..3d92e5fd018 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,33 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+If your untracked files take an unusual amount of time to enumerate, your\n+repository certainly has a lot of them, and an advice message will display\n+about it. Here are some configurations to consider in order to improve the\n+situation:\n+\n+* Setting the `core.untrackedCache` configuration as `true` will allow for\n+`git status` to keep track of the mtime of folders, in order to cache past\n+`status` results and be sure to only browse folders that changed on subsequent\n+runs, for filesystems that can support it (see linkgit:git-update-index[1]\n+for details).\n+* Used in conjonction with `core.untrackedCache`, setting the `core.fsmonitor`\n+configuration as `true` will allow for `git status` to keep track of what\n+files recently changed, in order to cache past `status` results and be sure\n+to only focus on those files on subsequent runs (see linkgit:git-update-index[1]\n+for details).\n+* If none of the above options are satisfactory, setting the\n+`status.showUntrackedFiles` configuration as `no` will cause `git status`\n+to not attempt to list untracked files anymore, in which case you have to be\n+careful not to forget to add new files yourself.\n+\n+If none of the above solutions are satisfactory, and you are bothered with\n+the advice message, you can disable it by setting the `advice.statusUoption`\n+configuration to `false`.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..92c053eaa64\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,40 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+. ./test-lib.sh\n+\n+DATA=\"$TEST_DIRECTORY/t7065\"\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+test_expect_success setup '\n+\tgit checkout -b test\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_must_fail git config --get core.untrackedCache &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/no_untrackedcache_no_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+ git config core.untrackedCache true &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/with_untrackedcache_no_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+ git config core.untrackedCache true &&\n+\tgit config core.fsmonitor true &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_done\n\\ No newline at end of file\ndiff --git a/t/t7065/no_untrackedcache_no_fsmonitor b/t/t7065/no_untrackedcache_no_fsmonitor\nnew file mode 100644\nindex 00000000000..e346deaa1db\n--- /dev/null\n+++ b/t/t7065/no_untrackedcache_no_fsmonitor\n@@ -0,0 +1,10 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files.\n+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n+for configuration options that may improve that time.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/t/t7065/with_untrackedcache_no_fsmonitor b/t/t7065/with_untrackedcache_no_fsmonitor\nnew file mode 100644\nindex 00000000000..a649d367493\n--- /dev/null\n+++ b/t/t7065/with_untrackedcache_no_fsmonitor\n@@ -0,0 +1,11 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files,\n+but this is currently being cached, with fsmonitor OFF.\n+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n+for configuration options that may improve that time.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/t/t7065/with_untrackedcache_with_fsmonitor b/t/t7065/with_untrackedcache_with_fsmonitor\nnew file mode 100644\nindex 00000000000..d5e95d984f8\n--- /dev/null\n+++ b/t/t7065/with_untrackedcache_with_fsmonitor\n@@ -0,0 +1,11 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files,\n+but this is currently being cached, with fsmonitor ON.\n+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n+for configuration options that may improve that time.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..be903c6e294 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,17 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static inline int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tconst char *x;\n+\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n+\tif (x) {\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\t}\n+\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1884,25 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\tif (uf_was_slow(s->untracked_in_ms)) {\n+\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t\tif (s->repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) {\n+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\t\"but this is currently being cached, with fsmonitor %s.\"),\n+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0,\n+\t\t\t\t\t\t\t(fsm_mode > FSMONITOR_MODE_DISABLED) ? \"ON\" : \"OFF\");\n+\t\t\t\t} else {\n+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t\t}\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\\n\"\n+\t\t\t\t\t\t\"for configuration options that may improve that time.\"));\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t}\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: bbe21b64a08f89475d8a3818e20c111378daa621\n-- \ngitgitgadget\n"},{"id":"466329","messageId":"8abc5272-4e01-e793-5155-ea116e9ad4fd@jeffhostetler.com","threadId":"58628","inReplyTo":"pull.1384.v2.git.1667002005494.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-11-02T19:45:18Z","receivedAt":"2022-11-02T19:45:25Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 10/28/22 8:06 PM, Rudy Rigot via GitGitGadget wrote:\n> From: Rudy Rigot \n> \n> Currently, if git-status takes more than 2 seconds for enumerating untracked\n> files, a piece of advice is given to the user to consider ignoring untracked\n> files. But Git now offers more possibilities to resolve that situation\n> (untracked cache, fsmonitor) with different downsides.\n> \n> This change is about refreshing that advice message. A new section in the\n> documentation is introduced to present the possibilities, and the advice\n> message links to it. I'm also introducing tests for this advice message,\n> which was untested so far.\n> \n> One of the downsides of untracked cache / fsmonitor, is that the first call\n> may be long in order to generate the cache, but the user may not know what\n> their current configuration is. When collecting feedback from users of our\n> very large repo, that's the most common point of confusion that keeps coming\n> back: people complain about git status being slow, but are satisfied when\n> we inform them that it's being cached and they should run it again to check.\n> As a result, the advice message tries to keep them informed of their current\n> configuration.\n\nLet me suggest an alternative commit message. We want to lead with a\n\"command\" -- as in: \"make Git do this\" or \"teach Git to do this\". Then\nexplain why. Maybe something like:\n\n Improve the advice displayed when `git status` is slow because\n of excessive numbers of untracked files. Update the `git status`\n man page to explain the various configuration options.\n\n `git status` can be slow when there are a large number of untracked\n files and directories, because Git must search the entire worktree\n to enumerate them. Previously, Git would print an advice message\n with the elapsed search time and a suggestion to disable the search\n using the `-uno` option. This suggestion also carried a warning\n that might scare off some users.\n\n Git can reduce the size and time of the untracked file search when\n the `core.untrackedCache` and `core.fsmonitor` features are enabled\n by caching results from previous `git status` invocations.\n\n Update the advice to explain the various combinations of additional\n configuration options and refer to (new) documentation in the man\n page that explains it in more detail than what can be printed in an\n advice message.\n\n Finally, add new tests to verify the new functionality.\n\nOr something like that. :-)\n\n\n[...]\n> diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\n> index 54a4b29b473..3d92e5fd018 100644\n> --- a/Documentation/git-status.txt\n> +++ b/Documentation/git-status.txt\n> @@ -457,6 +457,33 @@ during the write may conflict with other simultaneous processes, causing\n> them to fail. Scripts running `status` in the background should consider\n> using `git --no-optional-locks status` (see linkgit:git[1] for details).\n> \n> +UNTRACKED FILES AND STATUS SPEED\n> +--------------------------------\n> +\n> +If your untracked files take an unusual amount of time to enumerate, your\n> +repository certainly has a lot of them, and an advice message will display\n> +about it. Here are some configurations to consider in order to improve the\n> +situation:\n> +\n> +* Setting the `core.untrackedCache` configuration as `true` will allow for\n> +`git status` to keep track of the mtime of folders, in order to cache past\n> +`status` results and be sure to only browse folders that changed on subsequent\n> +runs, for filesystems that can support it (see linkgit:git-update-index[1]\n> +for details).\n> +* Used in conjonction with `core.untrackedCache`, setting the `core.fsmonitor`\n> +configuration as `true` will allow for `git status` to keep track of what\n> +files recently changed, in order to cache past `status` results and be sure\n> +to only focus on those files on subsequent runs (see linkgit:git-update-index[1]\n> +for details).\n> +* If none of the above options are satisfactory, setting the\n> +`status.showUntrackedFiles` configuration as `no` will cause `git status`\n> +to not attempt to list untracked files anymore, in which case you have to be\n> +careful not to forget to add new files yourself.\n> +\n> +If none of the above solutions are satisfactory, and you are bothered with\n> +the advice message, you can disable it by setting the `advice.statusUoption`\n> +configuration to `false`.\n> +\n\nI hate to suggest a complete rewrite as I myself struggle with\nhow to phrase things all toooooo often, but let me offer another\nstarting point and see what you think:\n\n `git status` can be very slow in large worktrees if/when it\n needs to search for untracked files and directories. There are\n many configuration options available to speed this up by either\n avoiding the work or making use of cached results from previous\n Git commands. Since we all work in different ways, there is no\n single optimum set of settings right for everyone. Here is a\n brief summary of the relevant options to help you choose which\n is right for you. Each of these settings is independently\n documented elsewhere in more detail, so please refer to them\n for complete details.\n\n * `-uno` or `status.showUntrackedFiles=false` : just don't search\n and don't report on untracked files. This is the fastest.\n `git status` will not list the untracked files, so you need\n to be careful to remember if you create any new files and\n manually `git add` them.\n\n * `advice.statusUoption=false` : search, but don't complain if it\n takes too long.\n\n * `core.untrackedCache=true` : enable the untracked cache feature\n and only search directories that have been modified since the\n previous `git status` command. Git remembers the set of\n untracked files within each directory and assumes that if a\n directory has not been modified, then the set of untracked\n file within has not changed. This is much faster than\n enumerating the contents of every directory, but still not\n without cost, because Git still has to search for the set of\n modified directories.\n\n * `core.untrackedCache=true` and `core.fsmonitor=true` or\n `core.fsmonitor=` : enable both the\n untracked cache and FSMonitor features and only search\n directories that have been modified since the previous\n `git status` command. This is faster than using just the\n untracked cache alone because Git can also avoid searching\n for modified directories. Git only has to enumerate the\n exact set of directories that have changed recently.\n\n Note that after you turn on the untracked cache and/or FSMonitor\n features it may take a few `git status` commands for the various\n caches to warm up before you see improved command times. This is\n normal.\n\nSomething like that. Hope this helps. (And again, sorry for the\nrewrite.)\n\n\n[...]\n> diff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\n> new file mode 100755\n> index 00000000000..92c053eaa64\n> --- /dev/null\n> +++ b/t/t7065-wtstatus-slow.sh\n> @@ -0,0 +1,40 @@\n> +#!/bin/sh\n[...]\n> +\n> +test_done\n> \\ No newline at end of file\n\nsmall nit: we should have a final LF at the end of the file.\n\nI'm going to skip over the test cases because I'm running\nshort on time this afternoon.\n\n\n[...]\n> diff --git a/wt-status.c b/wt-status.c\n[...]\n> static void show_merge_in_progress(struct wt_status *s,\n> \t\t\t\t const char *color)\n> {\n> @@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)\n> {\n> \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n> \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n> +\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n> \n> \tif (s->branch) {\n> \t\tconst char *on_what = _(\"On branch \");\n> @@ -1870,13 +1884,25 @@ static void wt_longstatus_print(struct wt_status *s)\n> \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n> \t\tif (s->show_ignored_mode)\n> \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n> -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n> -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> -\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n> -\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n> -\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n> -\t\t\t\t\t s->untracked_in_ms / 1000.0);\n> +\t\tif (uf_was_slow(s->untracked_in_ms)) {\n> +\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> +\t\t\t\tif (s->repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) {\n> +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n> +\t\t\t\t\t\t\t\"but this is currently being cached, with fsmonitor %s.\"),\n> +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0,\n> +\t\t\t\t\t\t\t(fsm_mode > FSMONITOR_MODE_DISABLED) ? \"ON\" : \"OFF\");\n> +\t\t\t\t} else {\n> +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n> +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n> +\t\t\t\t}\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t_(\"See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\\n\"\n> +\t\t\t\t\t\t\"for configuration options that may improve that time.\"));\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> +\t\t\t}\n\nI'm not sure I like the various mixture of messages here. Maybe\nit would be better with a single simple message:\n\n _(\"It took %.2f seconds to enumerate untracked files.\\n\"\n \"See 'git help status' for information on how to improve this.\")\n\nThis keeps all of the information in the documentation rather\nthan having part of it here in the code.\n\nAlso, we should refer to the documentation via `git help` rather\nthan as a link to the website.\n\n\nThanks,\nJeff\n\n\n\n"},{"id":"466330","messageId":"CANaDLWJmWiA2HwXSA5z-uaz+wg3f0WNTePkaG3omrcQ-Jri4VQ@mail.gmail.com","threadId":"58628","inReplyTo":"8abc5272-4e01-e793-5155-ea116e9ad4fd@jeffhostetler.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-02T20:34:00Z","receivedAt":"2022-11-02T20:34:17Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Thanks a lot for all that feedback. Honestly you're way more familiar\nthan I am with both how the underlying features work, and how to best\ncontribute this change, so I'm comfortable taking your advice pretty\nmuch blindly.\n\n\n> Let me suggest an alternative commit message. We want to lead with a\n> \"command\" -- as in: \"make Git do this\" or \"teach Git to do this\". Then\n> explain why.\n\nOops, sorry for missing this. Well, your commit message suggestion\nis flawless, I will reuse this as is. Thanks a lot for spending\ntime polishing it.\n\n\n> I hate to suggest a complete rewrite\n> [...]\n> Something like that. Hope this helps. (And again, sorry for the\nrewrite.)\n\nThere's absolutely nothing to apologize about. This is great! I\nwill also reuse as is. Here too, thanks a lot for the time spent\non this!\n\n\n> small nit: we should have a final LF at the end of the file.\n\nSounds good, will fix.\n\n\n> I'm going to skip over the test cases because I'm running\n> short on time this afternoon.\n\nThat's all good, I need to align my submission to all this and\nresubmit anyway, so you got time! I'm pretty confident about them\ntoo, a lot more than in the phrasing of the user-facing content\nI had.\n\n\n> Also, we should refer to the documentation via `git help` rather\n> than as a link to the website.\n\nOops, I didn't realize people were getting the same content from\neither. I will fix.\n\n\n> I'm not sure I like the various mixture of messages here. Maybe\n> it would be better with a single simple message:\n\nThat's the one thing I'm a bit concerned about, that I would like\nto discuss more if that's ok.\n\nThe current confusion we're seeing with users of our very large repo,\nis that they run git status the first time and notice it being slow\n(~30s), and then they see the current advice message advising that\nthey're supposed to do something about it. What they don't know, is\nthat untrackedcache and fsmonitor were already set for their\nenvironment, by the script setting their entire environment up.\n\nI don't think it is unusual for users to not necessarily know how\ntheir environment was configured (either because someone/something\nelse did it for them, or because they forgot what they did for\nthis specific repo, for instance).\n\nSo with that, I worry about the phrasing \"See 'git help status' for\ninformation on how to improve this.\" in that use case, because\nit implies that there is something they are expected to go improve,\nwhile that was already done.\n\nHere are some solution ideas:\n\n* Changing the wording for all use cases to not convey that they\nmust do anything about it. For instance just \"See 'git help status'\".\n(I don't love this because I could imagine users being puzzled about\nwhy Git is telling them this, then.)\n* Informing the user of their current caching situation in ways\nthat they can deduce whether or not they should be doing something\nabout it. That's what I was attempting to do here, but reading your\nhelp content, I think I got something wrong: I didn't realize the\ncache would only need to warm up with untrackedcache + fsmonitor,\nand not with untrackedcache alone. So with that an improvement could\nbe to only display \"but this is currently being cached.\" when both\nuntrackedcache + fsmonitor are on, and not display anything different\nwhen only untrackedcache is on then when it's not.\n* Not displaying this advice message when fsmonitor is on, since\nthe best possible optimization was already applied. Not loving it,\nbecause some could also still want untracked files off on top of it.\nAlso, it doesn't resolve the frustration of noticing git status being\nslow the first time.\n\nFor now if you don't mind, I'll change things to the 2nd proposal up\nthere, but this is not because I'm rejecting your guidance and insisting\nthat adding a line here is the right solution to the situation of\nenvironments that were already optimized, and I'm not sure what\nis. But I do worry that telling those users that they should optimize\nthings while they/someone already did will surely be confusing.\n\nI'll work on the other fixes, and I am deeply interested in your\nthoughts about this one. Thanks a lot for working through this with\nme.\n"},{"id":"466332","messageId":"pull.1384.v3.git.1667424467505.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v2.git.1667002005494.gitgitgadget@gmail.com","subject":"[PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-02T21:27:47Z","receivedAt":"2022-11-02T21:28:16Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nImprove the advice displayed when `git status` is slow because\nof excessive numbers of untracked files. Update the `git status`\nman page to explain the various configuration options.\n\n`git status` can be slow when there are a large number of untracked\nfiles and directories, because Git must search the entire worktree\nto enumerate them. Previously, Git would print an advice message\nwith the elapsed search time and a suggestion to disable the search\nusing the `-uno` option. This suggestion also carried a warning\nthat might scare off some users.\n\nGit can reduce the size and time of the untracked file search when\nthe `core.untrackedCache` and `core.fsmonitor` features are enabled\nby caching results from previous `git status` invocations.\n\nUpdate the advice to explain the various combinations of additional\nconfiguration options and refer to (new) documentation in the man\npage that explains it in more detail than what can be printed in an\nadvice message.\n\nFinally, add new tests to verify the new functionality.\n\nSigned-off-by: Rudy Rigot \n---\n status: long status advice adapted to recent capabilities\n \n Here is version 3 for this patch.\n \n Changes since v2:\n \n * Replaced copy of the commit message with the better one suggested by\n Jeff.\n * Replaced copy of the doc and the default advice message with the\n better ones suggested by Jeff.\n * Fixed EOF.\n * Changed the approach for users who are already optimized, pending\n more conversation to see what makes most sense.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v2:\n\n 1: 9ef7f1834b7 ! 1: 3c98492cb82 status: long status advice adapted to recent capabilities\n @@ Metadata\n ## Commit message ##\n status: long status advice adapted to recent capabilities\n \n - Currently, if git-status takes more than 2 seconds for enumerating untracked\n - files, a piece of advice is given to the user to consider ignoring untracked\n - files. But Git now offers more possibilities to resolve that situation\n - (untracked cache, fsmonitor) with different downsides.\n + Improve the advice displayed when `git status` is slow because\n + of excessive numbers of untracked files. Update the `git status`\n + man page to explain the various configuration options.\n \n - This change is about refreshing that advice message. A new section in the\n - documentation is introduced to present the possibilities, and the advice\n - message links to it. I'm also introducing tests for this advice message,\n - which was untested so far.\n + `git status` can be slow when there are a large number of untracked\n + files and directories, because Git must search the entire worktree\n + to enumerate them. Previously, Git would print an advice message\n + with the elapsed search time and a suggestion to disable the search\n + using the `-uno` option. This suggestion also carried a warning\n + that might scare off some users.\n \n - One of the downsides of untracked cache / fsmonitor, is that the first call\n - may be long in order to generate the cache, but the user may not know what\n - their current configuration is. When collecting feedback from users of our\n - very large repo, that's the most common point of confusion that keeps coming\n - back: people complain about git status being slow, but are satisfied when\n - we inform them that it's being cached and they should run it again to check.\n - As a result, the advice message tries to keep them informed of their current\n - configuration.\n + Git can reduce the size and time of the untracked file search when\n + the `core.untrackedCache` and `core.fsmonitor` features are enabled\n + by caching results from previous `git status` invocations.\n +\n + Update the advice to explain the various combinations of additional\n + configuration options and refer to (new) documentation in the man\n + page that explains it in more detail than what can be printed in an\n + advice message.\n +\n + Finally, add new tests to verify the new functionality.\n \n Signed-off-by: Rudy Rigot \n \n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n +UNTRACKED FILES AND STATUS SPEED\n +--------------------------------\n +\n -+If your untracked files take an unusual amount of time to enumerate, your\n -+repository certainly has a lot of them, and an advice message will display\n -+about it. Here are some configurations to consider in order to improve the\n -+situation:\n -+\n -+* Setting the `core.untrackedCache` configuration as `true` will allow for\n -+`git status` to keep track of the mtime of folders, in order to cache past\n -+`status` results and be sure to only browse folders that changed on subsequent\n -+runs, for filesystems that can support it (see linkgit:git-update-index[1]\n -+for details).\n -+* Used in conjonction with `core.untrackedCache`, setting the `core.fsmonitor`\n -+configuration as `true` will allow for `git status` to keep track of what\n -+files recently changed, in order to cache past `status` results and be sure\n -+to only focus on those files on subsequent runs (see linkgit:git-update-index[1]\n -+for details).\n -+* If none of the above options are satisfactory, setting the\n -+`status.showUntrackedFiles` configuration as `no` will cause `git status`\n -+to not attempt to list untracked files anymore, in which case you have to be\n -+careful not to forget to add new files yourself.\n -+\n -+If none of the above solutions are satisfactory, and you are bothered with\n -+the advice message, you can disable it by setting the `advice.statusUoption`\n -+configuration to `false`.\n ++`git status` can be very slow in large worktrees if/when it\n ++needs to search for untracked files and directories. There are\n ++many configuration options available to speed this up by either\n ++avoiding the work or making use of cached results from previous\n ++Git commands. Since we all work in different ways, there is no\n ++single optimum set of settings right for everyone. Here is a\n ++brief summary of the relevant options to help you choose which\n ++is right for you. Each of these settings is independently\n ++documented elsewhere in more detail, so please refer to them\n ++for complete details.\n ++\n ++* `-uno` or `status.showUntrackedFiles=false` : just don't search\n ++ and don't report on untracked files. This is the fastest.\n ++ `git status` will not list the untracked files, so you need\n ++ to be careful to remember if you create any new files and\n ++ manually `git add` them.\n ++\n ++* `advice.statusUoption=false` : search, but don't complain if it\n ++ takes too long.\n ++\n ++* `core.untrackedCache=true` : enable the untracked cache feature\n ++ and only search directories that have been modified since the\n ++ previous `git status` command. Git remembers the set of\n ++ untracked files within each directory and assumes that if a\n ++ directory has not been modified, then the set of untracked\n ++ file within has not changed. This is much faster than\n ++ enumerating the contents of every directory, but still not\n ++ without cost, because Git still has to search for the set of\n ++ modified directories.\n ++\n ++* `core.untrackedCache=true` and `core.fsmonitor=true` or\n ++ `core.fsmonitor=` : enable both the\n ++ untracked cache and FSMonitor features and only search\n ++ directories that have been modified since the previous\n ++ `git status` command. This is faster than using just the\n ++ untracked cache alone because Git can also avoid searching\n ++ for modified directories. Git only has to enumerate the\n ++ exact set of directories that have changed recently.\n ++\n ++Note that after you turn on the untracked cache and/or FSMonitor\n ++features it may take a few `git status` commands for the various\n ++caches to warm up before you see improved command times. This is\n ++normal.\n +\n SEE ALSO\n --------\n @@ t/t7065-wtstatus-slow.sh (new)\n +'\n +\n +test_done\n - \\ No newline at end of file\n \n ## t/t7065/no_untrackedcache_no_fsmonitor (new) ##\n @@\n @@ t/t7065/no_untrackedcache_no_fsmonitor (new)\n +\n +\n +It took X seconds to enumerate untracked files.\n -+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n -+for configuration options that may improve that time.\n ++See 'git help status' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n \n @@ t/t7065/with_untrackedcache_no_fsmonitor (new)\n +No commits yet\n +\n +\n -+It took X seconds to enumerate untracked files,\n -+but this is currently being cached, with fsmonitor OFF.\n -+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n -+for configuration options that may improve that time.\n ++It took X seconds to enumerate untracked files.\n ++See 'git help status' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n \n @@ t/t7065/with_untrackedcache_with_fsmonitor (new)\n +\n +\n +It took X seconds to enumerate untracked files,\n -+but this is currently being cached, with fsmonitor ON.\n -+See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\n -+for configuration options that may improve that time.\n ++but this is currently being cached.\n ++See 'git help status' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n \n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n +\t\tif (uf_was_slow(s->untracked_in_ms)) {\n +\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n -+\t\t\t\tif (s->repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) {\n ++\t\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n -+\t\t\t\t\t\t\t\"but this is currently being cached, with fsmonitor %s.\"),\n -+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0,\n -+\t\t\t\t\t\t\t(fsm_mode > FSMONITOR_MODE_DISABLED) ? \"ON\" : \"OFF\");\n ++\t\t\t\t\t\t\t\"but this is currently being cached.\"),\n ++\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n +\t\t\t\t} else {\n +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n +\t\t\t\t}\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t_(\"See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\\n\"\n -+\t\t\t\t\t\t\"for configuration options that may improve that time.\"));\n ++\t\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n +\t\t\t}\n \t\t}\n\n\n Documentation/git-status.txt | 47 ++++++++++++++++++++++\n t/t7065-wtstatus-slow.sh | 40 ++++++++++++++++++\n t/t7065/no_untrackedcache_no_fsmonitor | 9 +++++\n t/t7065/with_untrackedcache_no_fsmonitor | 9 +++++\n t/t7065/with_untrackedcache_with_fsmonitor | 10 +++++\n wt-status.c | 38 +++++++++++++----\n 6 files changed, 146 insertions(+), 7 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n create mode 100644 t/t7065/no_untrackedcache_no_fsmonitor\n create mode 100644 t/t7065/with_untrackedcache_no_fsmonitor\n create mode 100644 t/t7065/with_untrackedcache_with_fsmonitor\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 54a4b29b473..95f4ed95e96 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,53 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. Since we all work in different ways, there is no\n+single optimum set of settings right for everyone. Here is a\n+brief summary of the relevant options to help you choose which\n+is right for you. Each of these settings is independently\n+documented elsewhere in more detail, so please refer to them\n+for complete details.\n+\n+* `-uno` or `status.showUntrackedFiles=false` : just don't search\n+ and don't report on untracked files. This is the fastest.\n+ `git status` will not list the untracked files, so you need\n+ to be careful to remember if you create any new files and\n+ manually `git add` them.\n+\n+* `advice.statusUoption=false` : search, but don't complain if it\n+ takes too long.\n+\n+* `core.untrackedCache=true` : enable the untracked cache feature\n+ and only search directories that have been modified since the\n+ previous `git status` command. Git remembers the set of\n+ untracked files within each directory and assumes that if a\n+ directory has not been modified, then the set of untracked\n+ file within has not changed. This is much faster than\n+ enumerating the contents of every directory, but still not\n+ without cost, because Git still has to search for the set of\n+ modified directories.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+ `core.fsmonitor=` : enable both the\n+ untracked cache and FSMonitor features and only search\n+ directories that have been modified since the previous\n+ `git status` command. This is faster than using just the\n+ untracked cache alone because Git can also avoid searching\n+ for modified directories. Git only has to enumerate the\n+ exact set of directories that have changed recently.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..23c37ea71e7\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,40 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+. ./test-lib.sh\n+\n+DATA=\"$TEST_DIRECTORY/t7065\"\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+test_expect_success setup '\n+\tgit checkout -b test\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_must_fail git config --get core.untrackedCache &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/no_untrackedcache_no_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+ git config core.untrackedCache true &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/with_untrackedcache_no_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+ git config core.untrackedCache true &&\n+\tgit config core.fsmonitor true &&\n+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+ test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n+ rm -fr ../actual\n+'\n+\n+test_done\ndiff --git a/t/t7065/no_untrackedcache_no_fsmonitor b/t/t7065/no_untrackedcache_no_fsmonitor\nnew file mode 100644\nindex 00000000000..91dc3719cda\n--- /dev/null\n+++ b/t/t7065/no_untrackedcache_no_fsmonitor\n@@ -0,0 +1,9 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files.\n+See 'git help status' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/t/t7065/with_untrackedcache_no_fsmonitor b/t/t7065/with_untrackedcache_no_fsmonitor\nnew file mode 100644\nindex 00000000000..91dc3719cda\n--- /dev/null\n+++ b/t/t7065/with_untrackedcache_no_fsmonitor\n@@ -0,0 +1,9 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files.\n+See 'git help status' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/t/t7065/with_untrackedcache_with_fsmonitor b/t/t7065/with_untrackedcache_with_fsmonitor\nnew file mode 100644\nindex 00000000000..89d2dd5c2e7\n--- /dev/null\n+++ b/t/t7065/with_untrackedcache_with_fsmonitor\n@@ -0,0 +1,10 @@\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files,\n+but this is currently being cached.\n+See 'git help status' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..4dfc8a8969b 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,17 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static inline int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tconst char *x;\n+\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n+\tif (x) {\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\t}\n+\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1884,23 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n-\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\tif (uf_was_slow(s->untracked_in_ms)) {\n+\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\t\"but this is currently being cached.\"),\n+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t\t} else {\n+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t\t}\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\t}\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: bbe21b64a08f89475d8a3818e20c111378daa621\n-- \ngitgitgadget\n"},{"id":"466357","messageId":"Y2MEXyhh2cJ14ba9@nand.local","threadId":"58628","inReplyTo":"8abc5272-4e01-e793-5155-ea116e9ad4fd@jeffhostetler.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-02T23:59:27Z","receivedAt":"2022-11-02T23:59:35Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Nov 02, 2022 at 03:45:18PM -0400, Jeff Hostetler wrote:\n> Let me suggest an alternative commit message. We want to lead with a\n> \"command\" -- as in: \"make Git do this\" or \"teach Git to do this\". Then\n> explain why. Maybe something like:\n>\n> [...]\n\nExcellent suggestions, thank you.\n\n> > @@ -1870,13 +1884,25 @@ static void wt_longstatus_print(struct wt_status *s)\n> > \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n> > \t\tif (s->show_ignored_mode)\n> > \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n> > -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n> > -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> > -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> > -\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n> > -\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n> > -\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n> > -\t\t\t\t\t s->untracked_in_ms / 1000.0);\n> > +\t\tif (uf_was_slow(s->untracked_in_ms)) {\n> > +\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n> > +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> > +\t\t\t\tif (s->repo->settings.core_untracked_cache == UNTRACKED_CACHE_WRITE) {\n> > +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> > +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n> > +\t\t\t\t\t\t\t\"but this is currently being cached, with fsmonitor %s.\"),\n> > +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0,\n> > +\t\t\t\t\t\t\t(fsm_mode > FSMONITOR_MODE_DISABLED) ? \"ON\" : \"OFF\");\n> > +\t\t\t\t} else {\n> > +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> > +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n> > +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n> > +\t\t\t\t}\n> > +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> > +\t\t\t\t\t\t_(\"See https://git-scm.com/docs/git-status#_untracked_files_and_status_speed\\n\"\n> > +\t\t\t\t\t\t\"for configuration options that may improve that time.\"));\n> > +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> > +\t\t\t}\n>\n> I'm not sure I like the various mixture of messages here. Maybe\n> it would be better with a single simple message:\n>\n> _(\"It took %.2f seconds to enumerate untracked files.\\n\"\n> \"See 'git help status' for information on how to improve this.\")\n>\n> This keeps all of the information in the documentation rather\n> than having part of it here in the code.\n>\n> Also, we should refer to the documentation via `git help` rather\n> than as a link to the website.\n\nI agree with your suggestion of not linking out to git-scm.com here, but\nI wonder if we could get by without mentioning 'git help' here, either.\nPresumably looking up an unknown configuration variable with 'man\ngit-config' is easy enough.\n\nThanks,\nTaylor\n"},{"id":"466403","messageId":"CANaDLWK6-KkfKP0mipuWccfQFacDWsLHFNjS7ogL_xWvvmrCfQ@mail.gmail.com","threadId":"58628","inReplyTo":"Y2MEXyhh2cJ14ba9@nand.local","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-03T14:28:56Z","receivedAt":"2022-11-03T14:29:27Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> looking up an unknown configuration variable with 'man\n> git-config' is easy enough.\n\nI'm not strongly opinionated, but I believe the initial idea behind\nredirecting them to the doc was because Git now comes with more\nconfiguration abilities to improve performance of git status, that may\nbe more or less relevant depending on use cases, so there\nisn't really a single git-config key for them to look up any more. Their\nideal solution could be core.untrackedCache=true, core.fsmonitor=true,\nadvice.statusUoption=false, status.showUntrackedFiles=false, or even\nsome combinations of those can be relevant.\n\nFrom there, the goal I believe we were going for with this new doc\nsection is to let users know what configs exist for their git status\nslowness pains and why, so they can then go look those configs up for\nmore details, which I agree would indeed be easy from there.\n\nAgain, I'm not strongly opinionated, and I hope I accurately represented\nthe inital thinking on this idea.\n\nOne slightly stronger opinion I have, is that if the advice message\nwas just\n\n> It took %.2f seconds to enumerate untracked files.\n\nand nothing else, I can definitely see a strong UX downside of not\ngiving a hint of next steps for users. Basically, \"you have a problem,\nand we're not helping you resolve it\". Were you thinking more of\nsomething like this?\n\n> It took %.2f seconds to enumerate untracked files.\n> Please look up the core.untrackedCache, core.fsmonitor\n> advice.statusUoption, and status.showUntrackedFiles configs\n> for potential solutions.\n\nI'd say that's probably somewhat cryptic and a bit verbose (which is\nwhat we were trying to avoid by telling them to go see the doc), but\nwe wouldn't be leaving the user stranded, so I can see how that would\nwork out ok.\n\nI'm very interested in what you think.\n\nThanks,\n"},{"id":"466493","messageId":"221104.867d0byu5e.gmgdl@evledraar.gmail.com","threadId":"58628","inReplyTo":"CANaDLWK6-KkfKP0mipuWccfQFacDWsLHFNjS7ogL_xWvvmrCfQ@mail.gmail.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-04T08:52:59Z","receivedAt":"2022-11-04T08:57:08Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, Nov 03 2022, Rudy Rigot wrote:\n\n>> looking up an unknown configuration variable with 'man\n>> git-config' is easy enough.\n>\n> I'm not strongly opinionated, but I believe the initial idea behind\n> redirecting them to the doc was because Git now comes with more\n> configuration abilities to improve performance of git status, that may\n> be more or less relevant depending on use cases, so there\n> isn't really a single git-config key for them to look up any more. Their\n> ideal solution could be core.untrackedCache=true, core.fsmonitor=true,\n> advice.statusUoption=false, status.showUntrackedFiles=false, or even\n> some combinations of those can be relevant.\n>\n> From there, the goal I believe we were going for with this new doc\n> section is to let users know what configs exist for their git status\n> slowness pains and why, so they can then go look those configs up for\n> more details, which I agree would indeed be easy from there.\n>\n> Again, I'm not strongly opinionated, and I hope I accurately represented\n> the inital thinking on this idea.\n>\n> One slightly stronger opinion I have, is that if the advice message\n> was just\n>\n>> It took %.2f seconds to enumerate untracked files.\n>\n> and nothing else, I can definitely see a strong UX downside of not\n> giving a hint of next steps for users. Basically, \"you have a problem,\n> and we're not helping you resolve it\". Were you thinking more of\n> something like this?\n>\n>> It took %.2f seconds to enumerate untracked files.\n>> Please look up the core.untrackedCache, core.fsmonitor\n>> advice.statusUoption, and status.showUntrackedFiles configs\n>> for potential solutions.\n>\n> I'd say that's probably somewhat cryptic and a bit verbose (which is\n> what we were trying to avoid by telling them to go see the doc), but\n> we wouldn't be leaving the user stranded, so I can see how that would\n> work out ok.\n>\n> I'm very interested in what you think.\n\nOn the topic in general: I think it's probably a good thing to show the\nadvice, but I just want to point out that it's not without cost.\n\nRight now we're showing users a pretty basic command they can try, but\nnow we're showing them other stuff that needs more complex setup.\n\nFor some they're probably way better off, e.g. the untracked cache is\npretty much an unambiguous win (we should probably turn it on on\ndefault, but we'd need to check on-the-fly if the FS supports it\nproperly).\n\nBut for e.g. fsmonitor the user may spend a lot of time fiddling with\nit, only to find it doesn't help their use-case much, if it all.\n\nShould we still point out these possibilities? Probably, but just say'n.\n\nOne thing that I find glaringly omitted, which since you're working on\nthis you might consider adding: Suggest to just try running the exact\nsame command again, maybe it was just the FS cache.\n\nI.e. we're suggesting all this advanced stuff, but by far the biggest\ndifference is made on e.g. a modern *nix box (particularly Linux) by\njust having all the repo's assets in the FS cache.\n\n\n\n\n\n"},{"id":"466541","messageId":"CANaDLWLNMjUsetvsc9_b9GfM0qEQnETJ7TQhRL8Y3K6JSqQEzQ@mail.gmail.com","threadId":"58628","inReplyTo":"221104.867d0byu5e.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-04T15:33:07Z","receivedAt":"2022-11-04T15:33:49Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> One thing that I find glaringly omitted, which since you're working on\n> this you might consider adding: Suggest to just try running the exact\n> same command again, maybe it was just the FS cache.\n\nI have to admit that would be by far my personal preference.\n\n\nWe've been having a number of very great points made by several people\non this thread, but a number of them contradicting each other across\npeople, and yet clearly nobody's wrong, everybody makes very real\npoints. I'm trying to turn this into actionable changes I should make,\nbut I think I need guidance on that. This is my first ever contribution\nto the project, so I'm lacking the organizational awareness of the\nproject to be able to drive this to a consensus on what we should do.\n\nHere's a proposal that tries to make opinionated moves towards what\nI understand to be the priorities that were expressed:\n\n\n1- We keep the new paragraph doc, because I'd say why not, it's\nwell-written (thanks Jeff!) and useful. When people are looking for\nways to make git status faster, it's good that there's a reference\nabout it, and I'd expect it to be a common need across all kinds\nof user situations.\n\n\n2- When untracked cache is not on, if I understand Ævar's suggestion,\nit would say something like:\n\n> It took %.2f seconds to enumerate untracked files.\n> Try to enable untracked cache to see if it helps make it faster\n> for you:\n> git config core.untrackedCache true\n\nIt would satisfy that the message gives concise advice with actionable\nnext steps, without making assumptions about whether it will or won't\nwork. (Untracked cache alone did not make much of a difference in our\nvery large repo's case.) And it doesn't point to the help anymore, in\norder not to saturate the user with too much detail.\n\n\n3- When untrackedcache is on but fsmonitor is off, and git status is\nstill slow (that's the situation we had on our very large repo), it\ncould say something like:\n\n> It took %.2f seconds to enumerate untracked files.\n> Try to enable FSMonitor to see if it helps make it faster for you:\n> git config core.fsmonitor true\n\nSame as before, concise, no assumptions.\n\nThis setup is more advanced, but we are in a case where untracked cache\nis not helping, so I'm thining that should be very few repos.\nIf the user feels a need to better understand what's up, the feature\nis mentioned by name, so they can look it up and dig in if they wish to.\n\n\n4- When fsmonitor is on:\n\n> It took %.2f seconds to enumerate untracked files.\n> Your runs are being cached, try running git status again to see if\n> it's faster.\n\nSame as before, concise, no assumptions, and matches Ævar's suggestion\nabove that was also my preference, as it would apply perfectly to our\nvery large repo's use case and the grievances we've received.\n\n\nPlease let me know what your thoughts are about it all. A downside with\nall that is the option to disable untracked files is not mentioned at\nall, but if we keep the doc as it is, and it gets painful enough that\nthey search for other ways, I'm hopefuly the user would find it there.\n\n\nI want to say it again: I'm not very opinionated about any of this,\njust trying to collate feedback into an actionable plan. If I understood\nfeedback wrong, or my plan is not the best based on the feedback, that\nis very fine, but I will need guidance to know what makes more sense.\n\n\n> the untracked cache is\n> pretty much an unambiguous win (we should probably turn it on on\n> default, but we'd need to check on-the-fly if the FS supports it\n> properly).\n\nI could take on the work to make untracked cache on by default after\nthis, as another patch, if it sounds relevant to try. I feel I\nlack the technical understanding of what we need to check that you're\nmentioning here, so I'll have questions, but I'd be on board with\ntrying.\n"},{"id":"466555","messageId":"Y2WGYIEAZwBOpvxe@nand.local","threadId":"58628","inReplyTo":"CANaDLWK6-KkfKP0mipuWccfQFacDWsLHFNjS7ogL_xWvvmrCfQ@mail.gmail.com","subject":"Re: [PATCH v2] status: long status advice adapted to recent capabilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-04T21:38:40Z","receivedAt":"2022-11-04T21:38:49Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Nov 03, 2022 at 09:28:56AM -0500, Rudy Rigot wrote:\n> One slightly stronger opinion I have, is that if the advice message\n> was just\n>\n> > It took %.2f seconds to enumerate untracked files.\n>\n> and nothing else, I can definitely see a strong UX downside of not\n> giving a hint of next steps for users. Basically, \"you have a problem,\n> and we're not helping you resolve it\". Were you thinking more of\n> something like this?\n>\n> > It took %.2f seconds to enumerate untracked files.\n> > Please look up the core.untrackedCache, core.fsmonitor\n> > advice.statusUoption, and status.showUntrackedFiles configs\n> > for potential solutions.\n>\n> I'd say that's probably somewhat cryptic and a bit verbose (which is\n> what we were trying to avoid by telling them to go see the doc), but\n> we wouldn't be leaving the user stranded, so I can see how that would\n> work out ok.\n>\n> I'm very interested in what you think.\n\nI see what you're saying. On the one hand, it feels redundant to say,\n\"we noticed 'git status' is running slowly for you, try running 'git\nhelp status' to figure out why\".\n\nBut on the other hand, enumerating all possible configuration values\nthat you may or may not want to set given your particular circumstance\nisn't practical either.\n\nThinking on it more, I think what you wrote originally is reasonable.\n\nThanks,\nTaylor\n"},{"id":"466556","messageId":"Y2WG6ursW7qT29lc@nand.local","threadId":"58628","inReplyTo":"pull.1384.v3.git.1667424467505.gitgitgadget@gmail.com","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-04T21:40:58Z","receivedAt":"2022-11-04T21:41:03Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Nov 02, 2022 at 09:27:47PM +0000, Rudy Rigot via GitGitGadget wrote:\n> From: Rudy Rigot \n>\n> Improve the advice displayed when `git status` is slow because\n> of excessive numbers of untracked files. Update the `git status`\n> man page to explain the various configuration options.\n\nThis one is looking good to me. Jeff: do you agree? If so, I'm ready to\nstart merging this one down.\n\nThanks,\nTaylor\n"},{"id":"466729","messageId":"0e2de99a-da7e-f65f-aefe-117fb468ce55@github.com","threadId":"58628","inReplyTo":"pull.1384.v3.git.1667424467505.gitgitgadget@gmail.com","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-11-07T20:01:28Z","receivedAt":"2022-11-07T20:01:35Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/2/22 5:27 PM, Rudy Rigot via GitGitGadget wrote:> +UNTRACKED FILES AND STATUS SPEED\n> +--------------------------------\n> +\n> +`git status` can be very slow in large worktrees if/when it\n> +needs to search for untracked files and directories. There are\n> +many configuration options available to speed this up by either\n> +avoiding the work or making use of cached results from previous\n> +Git commands. Since we all work in different ways, there is no\n> +single optimum set of settings right for everyone. Here is a\n> +brief summary of the relevant options to help you choose which\n> +is right for you. Each of these settings is independently\n> +documented elsewhere in more detail, so please refer to them\n> +for complete details.\n\nSorry I'm late to this series. This new section is a great idea.\n\n> +* `-uno` or `status.showUntrackedFiles=false` : just don't search\n\nTwo nits:\n\n1. Is it clear that `-uno` is an option to `git status`? Should\n we say \"The `-uno` flag or the `status.showUntrackedfiles=false`\n config\"?\n\n2. Drop the \"just\" as it implies simplicity but is unnecessary. In\n fact, I'd replace the sentence with:\n\n Indicate that `git status` should not report untracked files.\n\n> + and don't report on untracked files. This is the fastest.\n\n \"This is the fastest option.\" ?\n\n> + `git status` will not list the untracked files, so you need\n> + to be careful to remember if you create any new files and\n> + manually `git add` them.\n> +\n> +* `advice.statusUoption=false` : search, but don't complain if it\n> + takes too long.\n\nPerhaps:\n\n\tThis config option disables a warning message when the\n\tsearch for untracked files takes longer than desired.\n\nand maybe even describe why:\n\n\tIn some large repositories, this message may appear\n\tfrequently and not be a helpful signal.\n\n> +* `core.untrackedCache=true` : enable the untracked cache feature\n> + and only search directories that have been modified since the\n> + previous `git status` command. Git remembers the set of\n> + untracked files within each directory and assumes that if a\n> + directory has not been modified, then the set of untracked\n> + file within has not changed. This is much faster than\n> + enumerating the contents of every directory, but still not\n> + without cost, because Git still has to search for the set of\n> + modified directories.\n\nIt might be helpful to mention that the untracked cache is stored\nin the .git/index file. The reduced cost searching for untracked\nfiles is offset slightly by the increased size of the index and\nthe cost of keeping it up-to-date. That reduced search time is\nusually worth the additional size.\n\n> +* `core.untrackedCache=true` and `core.fsmonitor=true` or\n> + `core.fsmonitor=` : enable both the\n> + untracked cache and FSMonitor features and only search\n> + directories that have been modified since the previous\n> + `git status` command. This is faster than using just the\n> + untracked cache alone because Git can also avoid searching\n> + for modified directories. Git only has to enumerate the\n> + exact set of directories that have changed recently.\n\nIt might be worth explicitly mentioning that while the FSMonitor\nfeature can be enabled without the untracked cache, the benefits\nare greatly reduced in that case.\n\n> +Note that after you turn on the untracked cache and/or FSMonitor\n> +features it may take a few `git status` commands for the various\n> +caches to warm up before you see improved command times. This is\n> +normal.\n> +\n> SEE ALSO\n> --------\n> linkgit:gitignore[5]\n> diff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\n> new file mode 100755\n> index 00000000000..23c37ea71e7\n> --- /dev/null\n> +++ b/t/t7065-wtstatus-slow.sh\n> @@ -0,0 +1,40 @@\n> +#!/bin/sh\n> +\n> +test_description='test status when slow untracked files'\n> +\n> +. ./test-lib.sh\n> +\n> +DATA=\"$TEST_DIRECTORY/t7065\"\n> +\n> +GIT_TEST_UF_DELAY_WARNING=1\n> +export GIT_TEST_UF_DELAY_WARNING\n> +\n> +test_expect_success setup '\n> +\tgit checkout -b test\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n> +\ttest_must_fail git config --get core.untrackedCache &&\n> +\ttest_must_fail git config --get core.fsmonitor &&\n> + git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n> + test_cmp \"$DATA/no_untrackedcache_no_fsmonitor\" ../actual &&\n> + rm -fr ../actual\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n> + git config core.untrackedCache true &&\n> +\ttest_must_fail git config --get core.fsmonitor &&\n> + git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n> + test_cmp \"$DATA/with_untrackedcache_no_fsmonitor\" ../actual &&\n> + rm -fr ../actual\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n> + git config core.untrackedCache true &&\n> +\tgit config core.fsmonitor true &&\n> + git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n> + test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n> + rm -fr ../actual\n> +'\n> +\n> +test_done\n> diff --git a/t/t7065/no_untrackedcache_no_fsmonitor b/t/t7065/no_untrackedcache_no_fsmonitor\n> diff --git a/t/t7065/with_untrackedcache_no_fsmonitor b/t/t7065/with_untrackedcache_no_fsmonitor\n> diff --git a/t/t7065/with_untrackedcache_with_fsmonitor b/t/t7065/with_untrackedcache_with_fsmonitor\n\nThese files are small enough that I think I'd rather see them\nbe created in the test script. You can use this kind of syntax:\n\n\tcat >expect <<-\\EOF\n\t\n\tEOF &&\n\nand then the test is self-contained. This is particularly\nhelpful when a test fails due to some future change in this\nmessage.\n\n> +static inline int uf_was_slow(uint32_t untracked_in_ms)\n> +{\n> +\tconst char *x;\n> +\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n> +\tif (x) {\n> +\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n> +\t}\n> +\n> +\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n> +}\n> +\n> static void show_merge_in_progress(struct wt_status *s,\n> \t\t\t\t const char *color)\n> {\n> @@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)\n> {\n> \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n> \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n> +\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n> \n> \tif (s->branch) {\n> \t\tconst char *on_what = _(\"On branch \");\n> @@ -1870,13 +1884,23 @@ static void wt_longstatus_print(struct wt_status *s)\n> \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n> \t\tif (s->show_ignored_mode)\n> \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n> -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n> -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> -\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> -\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n> -\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n> -\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n> -\t\t\t\t\t s->untracked_in_ms / 1000.0);\n> +\t\tif (uf_was_slow(s->untracked_in_ms)) {\n> +\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n\nSince there isn't an \"else\" for either of these cases, they can\nbe combined with \"&&\" in the top-level if, reducing the tab\ndepth for the body.\n\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> +\t\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n> +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n> +\t\t\t\t\t\t\t\"but this is currently being cached.\"),\n> +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n> +\t\t\t\t} else {\n> +\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n> +\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n> +\t\t\t\t}\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n> +\t\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n> +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> +\t\t\t}\n> \t\t}\n\nother than that nit, the code looks good to me.\n\nThanks for working on this!\n-Stolee\n"},{"id":"466730","messageId":"32e86ad3-2576-b90e-444b-636131bc168d@github.com","threadId":"58628","inReplyTo":"Y2WG6ursW7qT29lc@nand.local","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Derrick Stolee","fromEmail":"derrickstolee@github.com","sentAt":"2022-11-07T20:02:09Z","receivedAt":"2022-11-07T20:02:16Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 11/4/22 5:40 PM, Taylor Blau wrote:\n> On Wed, Nov 02, 2022 at 09:27:47PM +0000, Rudy Rigot via GitGitGadget wrote:\n>> From: Rudy Rigot \n>>\n>> Improve the advice displayed when `git status` is slow because\n>> of excessive numbers of untracked files. Update the `git status`\n>> man page to explain the various configuration options.\n> \n> This one is looking good to me. Jeff: do you agree? If so, I'm ready to\n> start merging this one down.\n\nSorry, I came in with some late review. I think one more round\nwould be helpful.\n\nThanks,\n-Stolee\n"},{"id":"466733","messageId":"CAPig+cTHbM6sXCkMHaDVNs6JJG-5pN-4Nf5tpBrOtYC82=8VvQ@mail.gmail.com","threadId":"58628","inReplyTo":"0e2de99a-da7e-f65f-aefe-117fb468ce55@github.com","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-07T20:20:12Z","receivedAt":"2022-11-07T20:20:36Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Nov 7, 2022 at 3:07 PM Derrick Stolee wrote:\n> On 11/2/22 5:27 PM, Rudy Rigot via GitGitGadget wrote:> +UNTRACKED FILES AND STATUS SPEED\n> > +test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n> > + git config core.untrackedCache true &&\n> > + git config core.fsmonitor true &&\n> > + git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n> > + test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n> > + rm -fr ../actual\n> > +'\n> > +\n> > diff --git a/t/t7065/no_untrackedcache_no_fsmonitor b/t/t7065/no_untrackedcache_no_fsmonitor\n> > diff --git a/t/t7065/with_untrackedcache_no_fsmonitor b/t/t7065/with_untrackedcache_no_fsmonitor\n> > diff --git a/t/t7065/with_untrackedcache_with_fsmonitor b/t/t7065/with_untrackedcache_with_fsmonitor\n>\n> These files are small enough that I think I'd rather see them\n> be created in the test script. You can use this kind of syntax:\n>\n> cat >expect <<-\\EOF\n> \n> EOF &&\n>\n> and then the test is self-contained. This is particularly\n> helpful when a test fails due to some future change in this\n> message.\n\nI've been meaning to respond to say the exact same thing about using\nhere-doc instead.\n\nAlso, the new tests seem to have an oddball mixture of indentation\nusing spaces and TAB. They should be using TAB only.\n"},{"id":"466734","messageId":"CANaDLWLVjWkSmSteNT8bi2ys3Ocg0uBUF=JouvdEB88+zE07KA@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTHbM6sXCkMHaDVNs6JJG-5pN-4Nf5tpBrOtYC82=8VvQ@mail.gmail.com","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-07T20:31:23Z","receivedAt":"2022-11-07T20:31:39Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Thanks to both of you Derrick and Eric, this is all great feedback.\nI've been through all of it, it's all crystal clear, and I'm planning\nto just integrate all of it as is. I'll have a new iteration later this\nweek (it's small stuff, I know, but I'm volunteering as a poll worker\ntoday and tomorrow).\n\nThanks again!\n"},{"id":"466767","messageId":"Y2mSabWsOfgBqNcR@nand.local","threadId":"58628","inReplyTo":"32e86ad3-2576-b90e-444b-636131bc168d@github.com","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-11-07T23:19:05Z","receivedAt":"2022-11-07T23:19:12Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Nov 07, 2022 at 03:02:09PM -0500, Derrick Stolee wrote:\n> On 11/4/22 5:40 PM, Taylor Blau wrote:\n> > On Wed, Nov 02, 2022 at 09:27:47PM +0000, Rudy Rigot via GitGitGadget wrote:\n> >> From: Rudy Rigot \n> >>\n> >> Improve the advice displayed when `git status` is slow because\n> >> of excessive numbers of untracked files. Update the `git status`\n> >> man page to explain the various configuration options.\n> >\n> > This one is looking good to me. Jeff: do you agree? If so, I'm ready to\n> > start merging this one down.\n>\n> Sorry, I came in with some late review. I think one more round\n> would be helpful.\n\nThanks, all. Will wait for one more round before we start merging this\ndown.\n\n\nThanks,\nTaylor\n"},{"id":"467033","messageId":"pull.1384.v4.git.1668055574050.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v3.git.1667424467505.gitgitgadget@gmail.com","subject":"[PATCH v4] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-10T04:46:13Z","receivedAt":"2022-11-10T04:46:24Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nImprove the advice displayed when `git status` is slow because\nof excessive numbers of untracked files. Update the `git status`\nman page to explain the various configuration options.\n\n`git status` can be slow when there are a large number of untracked\nfiles and directories, because Git must search the entire worktree\nto enumerate them. Previously, Git would print an advice message\nwith the elapsed search time and a suggestion to disable the search\nusing the `-uno` option. This suggestion also carried a warning\nthat might scare off some users.\n\nGit can reduce the size and time of the untracked file search when\nthe `core.untrackedCache` and `core.fsmonitor` features are enabled\nby caching results from previous `git status` invocations.\n\nUpdate the advice to explain the various combinations of additional\nconfiguration options and refer to (new) documentation in the man\npage that explains it in more detail than what can be printed in an\nadvice message.\n\nFinally, add new tests to verify the new functionality.\n\nSigned-off-by: Rudy Rigot \n---\n status: long status advice adapted to recent capabilities\n \n Here is version 4 for this patch.\n \n Changes since v3:\n \n * Integrated all feedback on the doc content itself, as is.\n * Moved the small test files into here docs in the test code.\n * Fix some awkward indentations.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v3:\n\n 1: 3c98492cb82 ! 1: 85b35882c02 status: long status advice adapted to recent capabilities\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n +documented elsewhere in more detail, so please refer to them\n +for complete details.\n +\n -+* `-uno` or `status.showUntrackedFiles=false` : just don't search\n -+ and don't report on untracked files. This is the fastest.\n -+ `git status` will not list the untracked files, so you need\n -+ to be careful to remember if you create any new files and\n -+ manually `git add` them.\n ++* The `-uno` flag or the `status.showUntrackedfiles=false`\n ++ config : indicate that `git status` should not report untracked\n ++\tfiles. This is the fastest option. `git status` will not list\n ++\tthe untracked files, so you need to be careful to remember if\n ++\tyou create any new files and manually `git add` them.\n +\n -+* `advice.statusUoption=false` : search, but don't complain if it\n -+ takes too long.\n ++* `advice.statusUoption=false` : this config option disables a\n ++\twarning message when the search for untracked files takes longer\n ++\tthan desired. In some large repositories, this message may appear\n ++\tfrequently and not be a helpful signal.\n +\n +* `core.untrackedCache=true` : enable the untracked cache feature\n + and only search directories that have been modified since the\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n + file within has not changed. This is much faster than\n + enumerating the contents of every directory, but still not\n + without cost, because Git still has to search for the set of\n -+ modified directories.\n ++ modified directories. The untracked cache is stored in the\n ++\t.git/index file. The reduced cost searching for untracked\n ++\tfiles is offset slightly by the increased size of the index and\n ++\tthe cost of keeping it up-to-date. That reduced search time is\n ++\tusually worth the additional size.\n +\n +* `core.untrackedCache=true` and `core.fsmonitor=true` or\n + `core.fsmonitor=` : enable both the\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n + `git status` command. This is faster than using just the\n + untracked cache alone because Git can also avoid searching\n + for modified directories. Git only has to enumerate the\n -+ exact set of directories that have changed recently.\n ++ exact set of directories that have changed recently. While\n ++\tthe FSMonitor feature can be enabled without the untracked\n ++\tcache, the benefits are greatly reduced in that case.\n +\n +Note that after you turn on the untracked cache and/or FSMonitor\n +features it may take a few `git status` commands for the various\n @@ t/t7065-wtstatus-slow.sh (new)\n +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n +\ttest_must_fail git config --get core.untrackedCache &&\n +\ttest_must_fail git config --get core.fsmonitor &&\n -+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+ test_cmp \"$DATA/no_untrackedcache_no_fsmonitor\" ../actual &&\n -+ rm -fr ../actual\n -+'\n -+\n -+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n -+ git config core.untrackedCache true &&\n -+\ttest_must_fail git config --get core.fsmonitor &&\n -+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+ test_cmp \"$DATA/with_untrackedcache_no_fsmonitor\" ../actual &&\n -+ rm -fr ../actual\n -+'\n -+\n -+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n -+ git config core.untrackedCache true &&\n -+\tgit config core.fsmonitor true &&\n -+ git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+ test_cmp \"$DATA/with_untrackedcache_with_fsmonitor\" ../actual &&\n -+ rm -fr ../actual\n -+'\n -+\n -+test_done\n -\n - ## t/t7065/no_untrackedcache_no_fsmonitor (new) ##\n -@@\n ++\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++\tcat >../expected <<-EOF &&\n +On branch test\n +\n +No commits yet\n +\n +\n +It took X seconds to enumerate untracked files.\n -+See 'git help status' for information on how to improve this.\n ++See '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n -\n - ## t/t7065/with_untrackedcache_no_fsmonitor (new) ##\n -@@\n ++\tEOF\n ++\ttest_cmp ../expected ../actual &&\n ++\trm -fr ../actual ../expected\n ++'\n ++\n ++test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n ++\tgit config core.untrackedCache true &&\n ++\ttest_must_fail git config --get core.fsmonitor &&\n ++\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++\tcat >../expected <<-EOF &&\n +On branch test\n +\n +No commits yet\n +\n +\n +It took X seconds to enumerate untracked files.\n -+See 'git help status' for information on how to improve this.\n ++See '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n -\n - ## t/t7065/with_untrackedcache_with_fsmonitor (new) ##\n -@@\n ++\tEOF\n ++\ttest_cmp ../expected ../actual &&\n ++\trm -fr ../actual ../expected\n ++'\n ++\n ++test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n ++\tgit config core.untrackedCache true &&\n ++\tgit config core.fsmonitor true &&\n ++\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n ++\tcat >../expected <<-EOF &&\n +On branch test\n +\n +No commits yet\n @@ t/t7065/with_untrackedcache_with_fsmonitor (new)\n +\n +It took X seconds to enumerate untracked files,\n +but this is currently being cached.\n -+See 'git help status' for information on how to improve this.\n ++See '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n +nothing to commit (create/copy files and use \"git add\" to track)\n ++\tEOF\n ++\ttest_cmp ../expected ../actual &&\n ++\trm -fr ../actual ../expected\n ++'\n ++\n ++test_done\n \n ## wt-status.c ##\n @@\n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n --\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n --\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n ++\t\tif (uf_was_slow(s->untracked_in_ms) && advice_enabled(ADVICE_STATUS_U_OPTION)) {\n + \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n ++\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n ++\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n ++\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n ++\t\t\t\t\t\t\"but this is currently being cached.\"),\n ++\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n ++\t\t\t} else {\n ++\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n ++\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n ++\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n ++\t\t\t}\n + \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n -\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n -\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n -\t\t\t\t\t s->untracked_in_ms / 1000.0);\n -+\t\tif (uf_was_slow(s->untracked_in_ms)) {\n -+\t\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION)) {\n -+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n -+\t\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n -+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n -+\t\t\t\t\t\t\t\"but this is currently being cached.\"),\n -+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n -+\t\t\t\t} else {\n -+\t\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n -+\t\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n -+\t\t\t\t}\n -+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n -+\t\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n -+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n -+\t\t\t}\n ++\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n ++\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\n\n Documentation/git-status.txt | 55 +++++++++++++++++++++++++++\n t/t7065-wtstatus-slow.sh | 74 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 32 +++++++++++++---\n 3 files changed, 156 insertions(+), 5 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 54a4b29b473..c51ba5e79e1 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,61 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. Since we all work in different ways, there is no\n+single optimum set of settings right for everyone. Here is a\n+brief summary of the relevant options to help you choose which\n+is right for you. Each of these settings is independently\n+documented elsewhere in more detail, so please refer to them\n+for complete details.\n+\n+* The `-uno` flag or the `status.showUntrackedfiles=false`\n+ config : indicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` : this config option disables a\n+\twarning message when the search for untracked files takes longer\n+\tthan desired. In some large repositories, this message may appear\n+\tfrequently and not be a helpful signal.\n+\n+* `core.untrackedCache=true` : enable the untracked cache feature\n+ and only search directories that have been modified since the\n+ previous `git status` command. Git remembers the set of\n+ untracked files within each directory and assumes that if a\n+ directory has not been modified, then the set of untracked\n+ file within has not changed. This is much faster than\n+ enumerating the contents of every directory, but still not\n+ without cost, because Git still has to search for the set of\n+ modified directories. The untracked cache is stored in the\n+\t.git/index file. The reduced cost searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+ `core.fsmonitor=` : enable both the\n+ untracked cache and FSMonitor features and only search\n+ directories that have been modified since the previous\n+ `git status` command. This is faster than using just the\n+ untracked cache alone because Git can also avoid searching\n+ for modified directories. Git only has to enumerate the\n+ exact set of directories that have changed recently. While\n+\tthe FSMonitor feature can be enabled without the untracked\n+\tcache, the benefits are greatly reduced in that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..468b8934836\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,74 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+. ./test-lib.sh\n+\n+DATA=\"$TEST_DIRECTORY/t7065\"\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+test_expect_success setup '\n+\tgit checkout -b test\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_must_fail git config --get core.untrackedCache &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+\tcat >../expected <<-EOF &&\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files.\n+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\n+\tEOF\n+\ttest_cmp ../expected ../actual &&\n+\trm -fr ../actual ../expected\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+\tgit config core.untrackedCache true &&\n+\ttest_must_fail git config --get core.fsmonitor &&\n+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+\tcat >../expected <<-EOF &&\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files.\n+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\n+\tEOF\n+\ttest_cmp ../expected ../actual &&\n+\trm -fr ../actual ../expected\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+\tgit config core.untrackedCache true &&\n+\tgit config core.fsmonitor true &&\n+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n+\tcat >../expected <<-EOF &&\n+On branch test\n+\n+No commits yet\n+\n+\n+It took X seconds to enumerate untracked files,\n+but this is currently being cached.\n+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+nothing to commit (create/copy files and use \"git add\" to track)\n+\tEOF\n+\ttest_cmp ../expected ../actual &&\n+\trm -fr ../actual ../expected\n+'\n+\n+test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..336f41e6d9f 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,17 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static inline int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tconst char *x;\n+\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n+\tif (x) {\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\t}\n+\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1884,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (uf_was_slow(s->untracked_in_ms) && advice_enabled(ADVICE_STATUS_U_OPTION)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but this is currently being cached.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: bbe21b64a08f89475d8a3818e20c111378daa621\n-- \ngitgitgadget\n"},{"id":"467034","messageId":"CAPig+cTGG-y6myEYOVeF8W9QBdCjhqeghsepi-2R9V-v7=YwZA@mail.gmail.com","threadId":"58628","inReplyTo":"pull.1384.v4.git.1668055574050.gitgitgadget@gmail.com","subject":"Re: [PATCH v4] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-10T05:42:43Z","receivedAt":"2022-11-10T05:43:00Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 9, 2022 at 11:46 PM Rudy Rigot via GitGitGadget\n wrote:\n> Improve the advice displayed when `git status` is slow because\n> of excessive numbers of untracked files. Update the `git status`\n> man page to explain the various configuration options.\n>\n> `git status` can be slow when there are a large number of untracked\n> files and directories, because Git must search the entire worktree\n> to enumerate them. Previously, Git would print an advice message\n> with the elapsed search time and a suggestion to disable the search\n> using the `-uno` option. This suggestion also carried a warning\n> that might scare off some users.\n>\n> Git can reduce the size and time of the untracked file search when\n> the `core.untrackedCache` and `core.fsmonitor` features are enabled\n> by caching results from previous `git status` invocations.\n>\n> Update the advice to explain the various combinations of additional\n> configuration options and refer to (new) documentation in the man\n> page that explains it in more detail than what can be printed in an\n> advice message.\n>\n> Finally, add new tests to verify the new functionality.\n>\n> Signed-off-by: Rudy Rigot \n> ---\n> diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\n> @@ -457,6 +457,61 @@ during the write may conflict with other simultaneous processes, causing\n> +UNTRACKED FILES AND STATUS SPEED\n> +--------------------------------\n> +\n> +`git status` can be very slow in large worktrees if/when it\n> +needs to search for untracked files and directories. There are\n> +many configuration options available to speed this up by either\n> +avoiding the work or making use of cached results from previous\n> +Git commands. Since we all work in different ways, there is no\n> +single optimum set of settings right for everyone. Here is a\n\nNot necessarily worth a reroll, but the \"since we all work...\"\nfragment could be dropped without any loss of clarity:\n\n There is no single optimum configuration suitable\n to every situation.\n\nwould probably be sufficient.\n\n> +brief summary of the relevant options to help you choose which\n> +is right for you. Each of these settings is independently\n> +documented elsewhere in more detail, so please refer to them\n> +for complete details.\n\nThis leaves the reader hanging somewhat by not saying where\n\"elsewhere\" actually is. You can use gitlink: to insert links to the\nappropriate documentation.\n\n> +* The `-uno` flag or the `status.showUntrackedfiles=false`\n> + config : indicate that `git status` should not report untracked\n> + files. This is the fastest option. `git status` will not list\n> + the untracked files, so you need to be careful to remember if\n> + you create any new files and manually `git add` them.\n\nThis and all the other bullet points use an odd mix of spaces and TAB\nfor indentation. They should consistently use one or the other.\n\nEarlier in this thread, Ævar suggested that it might be worthwhile to\nmention an additional possibility: that simply rerunning `git status`\nmight be sufficient to achieve reasonable speed since the second run\nof `git status` may benefit from the filesystem cache having been\nprimed by the first invocation of `git status`. As such, it may be\noverkill for a user to dive into the various options described here.\nHence, to mitigate the possibility of a user doing a lot of research\nand extra unnecessary configuration, it might make sense for the very\nfirst bullet point (before this one) to say merely:\n\n * Do nothing. Subsequent invocations of `git status` may be faster\n simply because the first `git status` primed the filesystem cache.\n\nor something like that (probably more formal, though).\n\n> diff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\n> @@ -0,0 +1,74 @@\n> +#!/bin/sh\n> +\n> +test_description='test status when slow untracked files'\n> +\n> +. ./test-lib.sh\n> +\n> +DATA=\"$TEST_DIRECTORY/t7065\"\n\nThis variable is unused, isn't it, now that you're inlining everything\nwith here-docs?\n\n> +GIT_TEST_UF_DELAY_WARNING=1\n> +export GIT_TEST_UF_DELAY_WARNING\n> +\n> +test_expect_success setup '\n> + git checkout -b test\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n> + test_must_fail git config --get core.untrackedCache &&\n> + test_must_fail git config --get core.fsmonitor &&\n\nRather than asserting that some preceding code has left the\nconfiguration in a state this test wants, it would be more robust for\nthis test to forcibly set up the state it wants. Something like this\nshould work:\n\n test_might_fail git config ---unset-all core.untrackedCache &&\n test_might_fail git config ---unset-all core.fsmonitor &&\n\n> + git status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n\nAt all costs, avoid escaping the \"trash\" directory in which this test\nis running. All files created by this test should be within the\n\"trash\" directory hierarchy. Therefore, drop \"../\" from all the\nfilenames, and instead create these files directly in the \"trash\"\ndirectory or in some subdirectory of the \"trash\" directory.\n\nI presume the reason you're escaping the \"trash\" directory is because\nyou don't want these untracked \"actual\" and \"expected\" files to\npollute the `git status` output you're testing? If so, then you should\nprobably instead set up a .gitignore file in the \"trash\" directory to\nmake `git status` ignore them.\n\nWe usually want to avoid having a Git command upstream of a pipe since\nthe exit code of the Git command will be lost. So, we'd typically do\nthis instead:\n\n git status >out &&\n sed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n\n> + cat >../expected <<-EOF &&\n> +On branch test\n> +\n> +No commits yet\n> +\n> +\n> +It took X seconds to enumerate untracked files.\n> +See '\"'\"'git help status'\"'\"' for information on how to improve this.\n> +\n> +nothing to commit (create/copy files and use \"git add\" to track)\n> + EOF\n\nTwo comments. First, use <<-\\EOF rather than <<-EOF to make it clear\nto readers that you're not interpolating any variables in the here-doc\nbody. Second, the <<- operator allows you to indent the here-doc body\n(with TABs, not spaces), so you can align the body with the rest of\nthe code:\n\n cat >expected <<-\\EOF &&\n On branch test\n ...\n EOF\n\n> + test_cmp ../expected ../actual &&\n> + rm -fr ../actual ../expected\n> +'\n\nWe usually don't bother cleaning up these files if they don't impact\nsubsequent tests negatively. However, note that if any code above the\n`rm -fr` command fails, then the `rm -fr` command itself won't be run,\nhence the files won't get cleaned up upon test failure. To ensure\ncleanup (if that's what you desire), use test_when_finished() _before_\ncode which may fail. So, something like this is typical:\n\n test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n test_when_finished \"rm -fr actual expected\" &&\n test_might_fail git config ---unset-all core.untrackedCache &&\n test_might_fail git config ---unset-all core.fsmonitor &&\n ...\n '\n\nSame comments apply to the other new tests added by this patch.\n\n> diff --git a/wt-status.c b/wt-status.c\n> @@ -1205,6 +1207,17 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n> +static inline int uf_was_slow(uint32_t untracked_in_ms)\n> +{\n\nDoes this need to be inline? Is this a hot piece of code, or is this\nmerely a premature optimization?\n\n> + const char *x;\n> + x = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n> + if (x) {\n> + untracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n> + }\n\nStyle is to avoid { } braces for these one-liner bodies.\n\nI think we don't even need the variable \"x\" in this case, so:\n\n if (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n untracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n\nwould be cleaner.\n\n> + return UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n> +}\n"},{"id":"467070","messageId":"CANaDLWK9ZhtqdpJJCNvOJ24x0jtUzZjZE5WKdzBPnePA4eGqTg@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTGG-y6myEYOVeF8W9QBdCjhqeghsepi-2R9V-v7=YwZA@mail.gmail.com","subject":"Re: [PATCH v4] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-10T17:01:49Z","receivedAt":"2022-11-10T17:02:08Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> the <<- operator allows you to indent the here-doc body\n> (with TABs, not spaces), so you can align the body with the rest of\n> the code\n\nUnfortunately, that's how I had done it first, but since some of those\nlines are blank, the test code had lines just made of \"\" and\nnothing else, which made the check-whitespace check fail. I considered\nreplacing empty line with something on the fly with sed (like just an\n\"x\" character for instance), but this felt hacky and brittle (in the\nunlikely case where an actual \"x\" would find itself genuinely lost in\nthe middle of that output, the test would mistakenly pass). I went\nwith the solution I'm presenting here because the readability\ndownsides of missing that indentation felt less bad. Definitely\nwilling to be convinced though.\n\nI have no issues with anything else from your review, and I'm planning\nto integrate it all. More specifically:\n\n> I presume the reason you're escaping the \"trash\" directory is because\n> you don't want these untracked \"actual\" and \"expected\" files to\n> pollute the `git status` output you're testing?\n\nYou are presuming right! The test was being flappy in CI runs before I\nchanged this, which I found used as a solution in other\ngit-status-related tests currently in the codebase. I'm not familiar\nwith the trash directory approach, but I'll figure it out.\n\n> Does this need to be inline? Is this a hot piece of code, or is this\n> merely a premature optimization?\n\nI'll admit my limits, I'm not familiar enough to know. If you feel the\ninline is unnecessary here, I'm glad to trust you on it, I'll remove\nit.\n\nThanks a lot for your in-depth review, I am planning to integrate all\nthe other feedback, and bring another iteration forward, possibly\ntoday.\n"},{"id":"467072","messageId":"CAPig+cTFRV=Np2oV5QJDpmwOwBaTVnjmAqcz-Ny7hCi6PexQUA@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLWK9ZhtqdpJJCNvOJ24x0jtUzZjZE5WKdzBPnePA4eGqTg@mail.gmail.com","subject":"Re: [PATCH v4] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-10T17:30:00Z","receivedAt":"2022-11-10T17:30:16Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Nov 10, 2022 at 12:02 PM Rudy Rigot wrote:\n> > the <<- operator allows you to indent the here-doc body\n> > (with TABs, not spaces), so you can align the body with the rest of\n> > the code\n>\n> Unfortunately, that's how I had done it first, but since some of those\n> lines are blank, the test code had lines just made of \"\" and\n> nothing else, which made the check-whitespace check fail. I considered\n> replacing empty line with something on the fly with sed (like just an\n> \"x\" character for instance), but this felt hacky and brittle (in the\n> unlikely case where an actual \"x\" would find itself genuinely lost in\n> the middle of that output, the test would mistakenly pass). I went\n> with the solution I'm presenting here because the readability\n> downsides of missing that indentation felt less bad. Definitely\n> willing to be convinced though.\n\nOkay, I see what you're getting at. Fortunately, there is a simple\nsolution as long as those lines are truly blank as emitted by `git\nstatus`: just leave the blank lines completely blank in the here-doc\nbody (don't bother inserting a TAB on the blank line). This should\nproduct the exact output you want:\n\n cat >../expected <<-\\EOF &&\n On branch test\n\n No commits yet\n ...\n EOF\n\nAlthough it should not be needed here, the `sed` approach is generally\nfine, and we use it often enough in tests, though usually with a more\nuncommon letter such as \"Q\". See, for instance, the q_to_nul(),\nq_to_tab(), etc. functions in t/test-lib-functions.sh.\n\n> > I presume the reason you're escaping the \"trash\" directory is because\n> > you don't want these untracked \"actual\" and \"expected\" files to\n> > pollute the `git status` output you're testing?\n>\n> You are presuming right! The test was being flappy in CI runs before I\n> changed this, which I found used as a solution in other\n> git-status-related tests currently in the codebase. I'm not familiar\n> with the trash directory approach, but I'll figure it out.\n\nEach test script is run in a temporary \"trash\" directory which gets\nthrown away when the script finishes. We want tests to constrain\nthemselves to the trash directory so they don't inadvertently destroy\na user's files outside the directory.\n\nI see what you mean about some existing status-related tests using\nfiles such as \"../actual\" and \"../expect\". It's not at all obvious in\na lot of those cases but they are safe[*] because those tests have\nalready cd'd into a subdirectory of the \"trash\" directory, thus\n\"../actual\" is referring to the \"trash\" directory itself, hence the\ntests do constrain themselves to \"trash\".\n\nAnyhow, I suspect that crafting a custom .gitignore file in the test\nsetup should satisfy this particular case and allow \"actual\" and\n\"expected\" to reside in the \"trash\" directory itself without mucking\nup `git status` output.\n\n[*] Unfortunately, some of those scripts are poorly structured because\nthey `cd` around between tests, which can leave CWD in an unexpected\nstate if some test fails and subsequent tests expect CWD to be\nsomewhere other than where it was left by the failed test. These days,\nwe only allow tests to `cd` within a subshell so that CWD is restored\nautomatically whether the test itself succeeds or fails. So, this is\nsafe:\n\n test_expect_success 'title' '\n do something &&\n mkdir foo &&\n (\n cd foo &&\n do something else >../actual &&\n )\n grep foo actual\n '\n"},{"id":"467073","messageId":"CANaDLWKR93_qfOkT6_u_qvUYZqjAZ_z_YXtzDFHXB+RY8nZhTg@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTFRV=Np2oV5QJDpmwOwBaTVnjmAqcz-Ny7hCi6PexQUA@mail.gmail.com","subject":"Re: [PATCH v4] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-10T17:47:29Z","receivedAt":"2022-11-10T17:47:47Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Oh, thanks, all of this helps a ton. I know exactly what to do about\nthose two things now.\n\n\nOn Thu, Nov 10, 2022 at 11:30 AM Eric Sunshine wrote:\n>\n> On Thu, Nov 10, 2022 at 12:02 PM Rudy Rigot wrote:\n> > > the <<- operator allows you to indent the here-doc body\n> > > (with TABs, not spaces), so you can align the body with the rest of\n> > > the code\n> >\n> > Unfortunately, that's how I had done it first, but since some of those\n> > lines are blank, the test code had lines just made of \"\" and\n> > nothing else, which made the check-whitespace check fail. I considered\n> > replacing empty line with something on the fly with sed (like just an\n> > \"x\" character for instance), but this felt hacky and brittle (in the\n> > unlikely case where an actual \"x\" would find itself genuinely lost in\n> > the middle of that output, the test would mistakenly pass). I went\n> > with the solution I'm presenting here because the readability\n> > downsides of missing that indentation felt less bad. Definitely\n> > willing to be convinced though.\n>\n> Okay, I see what you're getting at. Fortunately, there is a simple\n> solution as long as those lines are truly blank as emitted by `git\n> status`: just leave the blank lines completely blank in the here-doc\n> body (don't bother inserting a TAB on the blank line). This should\n> product the exact output you want:\n>\n> cat >../expected <<-\\EOF &&\n> On branch test\n>\n> No commits yet\n> ...\n> EOF\n>\n> Although it should not be needed here, the `sed` approach is generally\n> fine, and we use it often enough in tests, though usually with a more\n> uncommon letter such as \"Q\". See, for instance, the q_to_nul(),\n> q_to_tab(), etc. functions in t/test-lib-functions.sh.\n>\n> > > I presume the reason you're escaping the \"trash\" directory is because\n> > > you don't want these untracked \"actual\" and \"expected\" files to\n> > > pollute the `git status` output you're testing?\n> >\n> > You are presuming right! The test was being flappy in CI runs before I\n> > changed this, which I found used as a solution in other\n> > git-status-related tests currently in the codebase. I'm not familiar\n> > with the trash directory approach, but I'll figure it out.\n>\n> Each test script is run in a temporary \"trash\" directory which gets\n> thrown away when the script finishes. We want tests to constrain\n> themselves to the trash directory so they don't inadvertently destroy\n> a user's files outside the directory.\n>\n> I see what you mean about some existing status-related tests using\n> files such as \"../actual\" and \"../expect\". It's not at all obvious in\n> a lot of those cases but they are safe[*] because those tests have\n> already cd'd into a subdirectory of the \"trash\" directory, thus\n> \"../actual\" is referring to the \"trash\" directory itself, hence the\n> tests do constrain themselves to \"trash\".\n>\n> Anyhow, I suspect that crafting a custom .gitignore file in the test\n> setup should satisfy this particular case and allow \"actual\" and\n> \"expected\" to reside in the \"trash\" directory itself without mucking\n> up `git status` output.\n>\n> [*] Unfortunately, some of those scripts are poorly structured because\n> they `cd` around between tests, which can leave CWD in an unexpected\n> state if some test fails and subsequent tests expect CWD to be\n> somewhere other than where it was left by the failed test. These days,\n> we only allow tests to `cd` within a subshell so that CWD is restored\n> automatically whether the test itself succeeds or fails. So, this is\n> safe:\n>\n> test_expect_success 'title' '\n> do something &&\n> mkdir foo &&\n> (\n> cd foo &&\n> do something else >../actual &&\n> )\n> grep foo actual\n> '\n"},{"id":"467083","messageId":"pull.1384.v5.git.1668110679098.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v4.git.1668055574050.gitgitgadget@gmail.com","subject":"[PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-10T20:04:38Z","receivedAt":"2022-11-10T20:04:51Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nImprove the advice displayed when `git status` is slow because\nof excessive numbers of untracked files. Update the `git status`\nman page to explain the various configuration options.\n\n`git status` can be slow when there are a large number of untracked\nfiles and directories, because Git must search the entire worktree\nto enumerate them. Previously, Git would print an advice message\nwith the elapsed search time and a suggestion to disable the search\nusing the `-uno` option. This suggestion also carried a warning\nthat might scare off some users.\n\nGit can reduce the size and time of the untracked file search when\nthe `core.untrackedCache` and `core.fsmonitor` features are enabled\nby caching results from previous `git status` invocations.\n\nUpdate the advice to explain the various combinations of additional\nconfiguration options and refer to (new) documentation in the man\npage that explains it in more detail than what can be printed in an\nadvice message.\n\nFinally, add new tests to verify the new functionality.\n\nSigned-off-by: Rudy Rigot \n---\n status: long status advice adapted to recent capabilities\n \n Here is version 5 for this patch.\n \n Changes since v4:\n \n * Test improvements (readability, isolation. ...)\n * Doc improvements (referencing other docs, adding advice to try again,\n ...)\n * Minor logic simplifications\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v4:\n\n 1: 85b35882c02 ! 1: 8b1b9ee094f status: long status advice adapted to recent capabilities\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n +--------------------------------\n +\n +`git status` can be very slow in large worktrees if/when it\n -+needs to search for untracked files and directories. There are\n ++needs to search for untracked files and directories. There are\n +many configuration options available to speed this up by either\n +avoiding the work or making use of cached results from previous\n -+Git commands. Since we all work in different ways, there is no\n -+single optimum set of settings right for everyone. Here is a\n -+brief summary of the relevant options to help you choose which\n -+is right for you. Each of these settings is independently\n -+documented elsewhere in more detail, so please refer to them\n -+for complete details.\n -+\n -+* The `-uno` flag or the `status.showUntrackedfiles=false`\n -+ config : indicate that `git status` should not report untracked\n ++Git commands. There is no single optimum set of settings right\n ++for everyone. Here is a brief summary of the relevant options\n ++to help you choose which is right for you.\n ++\n ++* First, you may want to run `git status` again. Your current\n ++\tconfiguration may already be caching `git status` results,\n ++\tso it could be faster on subsequent runs.\n ++\n ++* The `--untracked-files=no` flag or the\n ++\t`status.showUntrackedfiles=false` config (see above for both) :\n ++\tindicate that `git status` should not report untracked\n +\tfiles. This is the fastest option. `git status` will not list\n +\tthe untracked files, so you need to be careful to remember if\n +\tyou create any new files and manually `git add` them.\n +\n -+* `advice.statusUoption=false` : this config option disables a\n -+\twarning message when the search for untracked files takes longer\n -+\tthan desired. In some large repositories, this message may appear\n -+\tfrequently and not be a helpful signal.\n -+\n -+* `core.untrackedCache=true` : enable the untracked cache feature\n -+ and only search directories that have been modified since the\n -+ previous `git status` command. Git remembers the set of\n -+ untracked files within each directory and assumes that if a\n -+ directory has not been modified, then the set of untracked\n -+ file within has not changed. This is much faster than\n -+ enumerating the contents of every directory, but still not\n -+ without cost, because Git still has to search for the set of\n -+ modified directories. The untracked cache is stored in the\n ++* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n ++\tthis config option disables a warning message when the search\n ++\tfor untracked files takes longer than desired. In some large\n ++\trepositories, this message may appear frequently and not be a\n ++\thelpful signal.\n ++\n ++* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n ++\tenable the untracked cache feature and only search directories\n ++\tthat have been modified since the previous `git status` command.\n ++\tGit remembers the set of untracked files within each directory\n ++\tand assumes that if a directory has not been modified, then\n ++\tthe set of untracked file within has not changed. This is much\n ++\tfaster than enumerating the contents of every directory, but still\n ++\tnot without cost, because Git still has to search for the set of\n ++\tmodified directories. The untracked cache is stored in the\n +\t.git/index file. The reduced cost searching for untracked\n +\tfiles is offset slightly by the increased size of the index and\n +\tthe cost of keeping it up-to-date. That reduced search time is\n +\tusually worth the additional size.\n +\n +* `core.untrackedCache=true` and `core.fsmonitor=true` or\n -+ `core.fsmonitor=` : enable both the\n -+ untracked cache and FSMonitor features and only search\n -+ directories that have been modified since the previous\n -+ `git status` command. This is faster than using just the\n -+ untracked cache alone because Git can also avoid searching\n -+ for modified directories. Git only has to enumerate the\n -+ exact set of directories that have changed recently. While\n -+\tthe FSMonitor feature can be enabled without the untracked\n -+\tcache, the benefits are greatly reduced in that case.\n ++\t`core.fsmonitor=` (see\n ++\tlinkgit:git-update-index[1]) : enable both the untracked cache\n ++\tand FSMonitor features and only search directories that have\n ++\tbeen modified since the previous `git status` command. This\n ++\tis faster than using just the untracked cache alone because\n ++\tGit can also avoid searching for modified directories. Git\n ++\tonly has to enumerate the exact set of directories that have\n ++\tchanged recently. While the FSMonitor feature can be enabled\n ++\twithout the untracked cache, the benefits are greatly reduced\n ++\tin that case.\n +\n +Note that after you turn on the untracked cache and/or FSMonitor\n +features it may take a few `git status` commands for the various\n @@ t/t7065-wtstatus-slow.sh (new)\n +\n +. ./test-lib.sh\n +\n -+DATA=\"$TEST_DIRECTORY/t7065\"\n -+\n +GIT_TEST_UF_DELAY_WARNING=1\n +export GIT_TEST_UF_DELAY_WARNING\n +\n +test_expect_success setup '\n -+\tgit checkout -b test\n ++\tgit checkout -b test &&\n ++\techo \"actual\" >> .gitignore &&\n ++\techo \"expected\" >> .gitignore &&\n ++\techo \"out\" >> .gitignore &&\n ++\tgit add .gitignore &&\n ++\tgit commit -m \"Add .gitignore\"\n +'\n +\n +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n -+\ttest_must_fail git config --get core.untrackedCache &&\n -+\ttest_must_fail git config --get core.fsmonitor &&\n -+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+\tcat >../expected <<-EOF &&\n -+On branch test\n -+\n -+No commits yet\n -+\n ++\ttest_might_fail git config --unset-all core.untrackedCache &&\n ++\ttest_might_fail git config --unset-all core.fsmonitor &&\n ++\tgit status >out &&\n ++\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\tcat >expected <<-\\EOF &&\n ++\t\tOn branch test\n +\n -+It took X seconds to enumerate untracked files.\n -+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\t\tIt took X seconds to enumerate untracked files.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+nothing to commit (create/copy files and use \"git add\" to track)\n ++\t\tnothing to commit, working tree clean\n +\tEOF\n -+\ttest_cmp ../expected ../actual &&\n -+\trm -fr ../actual ../expected\n ++\ttest_cmp expected actual\n +'\n +\n +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n +\tgit config core.untrackedCache true &&\n -+\ttest_must_fail git config --get core.fsmonitor &&\n -+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+\tcat >../expected <<-EOF &&\n -+On branch test\n ++\ttest_might_fail git config --unset-all core.fsmonitor &&\n ++\tgit status >out &&\n ++\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\tcat >expected <<-\\EOF &&\n ++\t\tOn branch test\n +\n -+No commits yet\n ++\t\tIt took X seconds to enumerate untracked files.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+\n -+It took X seconds to enumerate untracked files.\n -+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n -+\n -+nothing to commit (create/copy files and use \"git add\" to track)\n ++\t\tnothing to commit, working tree clean\n +\tEOF\n -+\ttest_cmp ../expected ../actual &&\n -+\trm -fr ../actual ../expected\n ++\ttest_cmp expected actual\n +'\n +\n +test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n +\tgit config core.untrackedCache true &&\n +\tgit config core.fsmonitor true &&\n -+\tgit status | sed \"s/[0-9]\\.[0-9][0-9]/X/g\" >../actual &&\n -+\tcat >../expected <<-EOF &&\n -+On branch test\n -+\n -+No commits yet\n -+\n ++\tgit status >out &&\n ++\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\tcat >expected <<-\\EOF &&\n ++\t\tOn branch test\n +\n -+It took X seconds to enumerate untracked files,\n -+but this is currently being cached.\n -+See '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\t\tIt took X seconds to enumerate untracked files,\n ++\t\tbut this is currently being cached.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+nothing to commit (create/copy files and use \"git add\" to track)\n ++\t\tnothing to commit, working tree clean\n +\tEOF\n -+\ttest_cmp ../expected ../actual &&\n -+\trm -fr ../actual ../expected\n ++\ttest_cmp expected actual\n +'\n +\n +test_done\n @@ wt-status.c: static void wt_longstatus_print_tracking(struct wt_status *s)\n \n +static inline int uf_was_slow(uint32_t untracked_in_ms)\n +{\n -+\tconst char *x;\n -+\tx = getenv(\"GIT_TEST_UF_DELAY_WARNING\");\n -+\tif (x) {\n ++\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\")) {\n +\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n +\t}\n +\n\n\n Documentation/git-status.txt | 59 +++++++++++++++++++++++++++++++\n t/t7065-wtstatus-slow.sh | 68 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 30 +++++++++++++---\n 3 files changed, 152 insertions(+), 5 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 5e438a7fdc1..ed1ae3bd35c 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,65 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. There is no single optimum set of settings right\n+for everyone. Here is a brief summary of the relevant options\n+to help you choose which is right for you.\n+\n+* First, you may want to run `git status` again. Your current\n+\tconfiguration may already be caching `git status` results,\n+\tso it could be faster on subsequent runs.\n+\n+* The `--untracked-files=no` flag or the\n+\t`status.showUntrackedfiles=false` config (see above for both) :\n+\tindicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n+\tthis config option disables a warning message when the search\n+\tfor untracked files takes longer than desired. In some large\n+\trepositories, this message may appear frequently and not be a\n+\thelpful signal.\n+\n+* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n+\tenable the untracked cache feature and only search directories\n+\tthat have been modified since the previous `git status` command.\n+\tGit remembers the set of untracked files within each directory\n+\tand assumes that if a directory has not been modified, then\n+\tthe set of untracked file within has not changed. This is much\n+\tfaster than enumerating the contents of every directory, but still\n+\tnot without cost, because Git still has to search for the set of\n+\tmodified directories. The untracked cache is stored in the\n+\t.git/index file. The reduced cost searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+\t`core.fsmonitor=` (see\n+\tlinkgit:git-update-index[1]) : enable both the untracked cache\n+\tand FSMonitor features and only search directories that have\n+\tbeen modified since the previous `git status` command. This\n+\tis faster than using just the untracked cache alone because\n+\tGit can also avoid searching for modified directories. Git\n+\tonly has to enumerate the exact set of directories that have\n+\tchanged recently. While the FSMonitor feature can be enabled\n+\twithout the untracked cache, the benefits are greatly reduced\n+\tin that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..bc624756622\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,68 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+. ./test-lib.sh\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+test_expect_success setup '\n+\tgit checkout -b test &&\n+\techo \"actual\" >> .gitignore &&\n+\techo \"expected\" >> .gitignore &&\n+\techo \"out\" >> .gitignore &&\n+\tgit add .gitignore &&\n+\tgit commit -m \"Add .gitignore\"\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_might_fail git config --unset-all core.untrackedCache &&\n+\ttest_might_fail git config --unset-all core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\t\tOn branch test\n+\n+\t\tIt took X seconds to enumerate untracked files.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+\tgit config core.untrackedCache true &&\n+\ttest_might_fail git config --unset-all core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\t\tOn branch test\n+\n+\t\tIt took X seconds to enumerate untracked files.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+\tgit config core.untrackedCache true &&\n+\tgit config core.fsmonitor true &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\t\tOn branch test\n+\n+\t\tIt took X seconds to enumerate untracked files,\n+\t\tbut this is currently being cached.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..5093bf8c894 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,15 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static inline int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\")) {\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\t}\n+\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1825,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1882,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (uf_was_slow(s->untracked_in_ms) && advice_enabled(ADVICE_STATUS_U_OPTION)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but this is currently being cached.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: 319605f8f00e402f3ea758a02c63534ff800a711\n-- \ngitgitgadget\n"},{"id":"467312","messageId":"4cba2921-1497-ccc0-adfa-742d5ce75e3a@jeffhostetler.com","threadId":"58628","inReplyTo":"Y2WG6ursW7qT29lc@nand.local","subject":"Re: [PATCH v3] status: long status advice adapted to recent capabilities","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-11-15T16:38:53Z","receivedAt":"2022-11-15T16:38:57Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 11/4/22 5:40 PM, Taylor Blau wrote:\n> On Wed, Nov 02, 2022 at 09:27:47PM +0000, Rudy Rigot via GitGitGadget wrote:\n>> From: Rudy Rigot \n>>\n>> Improve the advice displayed when `git status` is slow because\n>> of excessive numbers of untracked files. Update the `git status`\n>> man page to explain the various configuration options.\n> \n> This one is looking good to me. Jeff: do you agree? If so, I'm ready to\n> start merging this one down.\n> \n> Thanks,\n> Taylor\n\nThis series is looking good to me too. I think V5 has addressed\nall of the concerns.\n\nJeff\n"},{"id":"467313","messageId":"454089ac-3d0d-9820-26a4-e5650c2604d2@jeffhostetler.com","threadId":"58628","inReplyTo":"pull.1384.v5.git.1668110679098.gitgitgadget@gmail.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-11-15T16:39:47Z","receivedAt":"2022-11-15T16:39:52Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 11/10/22 3:04 PM, Rudy Rigot via GitGitGadget wrote:\n> From: Rudy Rigot \n> \n> Improve the advice displayed when `git status` is slow because\n> of excessive numbers of untracked files. Update the `git status`\n> man page to explain the various configuration options.\n> \n> `git status` can be slow when there are a large number of untracked\n> files and directories, because Git must search the entire worktree\n> to enumerate them. Previously, Git would print an advice message\n> with the elapsed search time and a suggestion to disable the search\n> using the `-uno` option. This suggestion also carried a warning\n> that might scare off some users.\n> \n> Git can reduce the size and time of the untracked file search when\n> the `core.untrackedCache` and `core.fsmonitor` features are enabled\n> by caching results from previous `git status` invocations.\n> \n> Update the advice to explain the various combinations of additional\n> configuration options and refer to (new) documentation in the man\n> page that explains it in more detail than what can be printed in an\n> advice message.\n> \n> Finally, add new tests to verify the new functionality.\n> \n> Signed-off-by: Rudy Rigot \n\nI think V5 looks good.\nThanks for your persistence!\n\nJeff\n\n"},{"id":"467314","messageId":"CANaDLWJVQV_RRTmjv2C1+WRJFmZN+LPNCDdawBs_-pLHcYamKQ@mail.gmail.com","threadId":"58628","inReplyTo":"454089ac-3d0d-9820-26a4-e5650c2604d2@jeffhostetler.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-15T16:42:26Z","receivedAt":"2022-11-15T16:43:03Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> I think V5 has addressed all of the concerns.\n\nI confirm I don't think there was any concern expressed so far that I\ndidn't find a way to address in some way.\n\n> Thanks for your persistence!\n\nMy absolute pleasure!\n"},{"id":"467317","messageId":"CAPig+cTO3NPg_Kx3dZhFMEtbMe9hRvaumZYxMnSJRyXqUA=p0g@mail.gmail.com","threadId":"58628","inReplyTo":"pull.1384.v5.git.1668110679098.gitgitgadget@gmail.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-15T17:26:33Z","receivedAt":"2022-11-15T17:27:20Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Nov 10, 2022 at 3:04 PM Rudy Rigot via GitGitGadget\n wrote:\n> Improve the advice displayed when `git status` is slow because\n> of excessive numbers of untracked files. Update the `git status`\n> man page to explain the various configuration options.\n>\n> `git status` can be slow when there are a large number of untracked\n> files and directories, because Git must search the entire worktree\n> to enumerate them. Previously, Git would print an advice message\n> with the elapsed search time and a suggestion to disable the search\n> using the `-uno` option. This suggestion also carried a warning\n> that might scare off some users.\n>\n> Git can reduce the size and time of the untracked file search when\n> the `core.untrackedCache` and `core.fsmonitor` features are enabled\n> by caching results from previous `git status` invocations.\n>\n> Update the advice to explain the various combinations of additional\n> configuration options and refer to (new) documentation in the man\n> page that explains it in more detail than what can be printed in an\n> advice message.\n>\n> Finally, add new tests to verify the new functionality.\n>\n> Signed-off-by: Rudy Rigot \n\nMostly just some minor style-related comments below, but also a couple\ngrammos in the new documentation, and a question or two...\n\n> diff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\n> @@ -457,6 +457,65 @@ during the write may conflict with other simultaneous processes, causing\n> +* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n> + enable the untracked cache feature and only search directories\n> + that have been modified since the previous `git status` command.\n> + Git remembers the set of untracked files within each directory\n> + and assumes that if a directory has not been modified, then\n> + the set of untracked file within has not changed. This is much\n\ns/file/files/\n\n> + faster than enumerating the contents of every directory, but still\n> + not without cost, because Git still has to search for the set of\n> + modified directories. The untracked cache is stored in the\n> + .git/index file. The reduced cost searching for untracked\n\nMight want backticks around the literal filename: `.git/index`\n\nAlso: s/cost searching/cost of searching/\n\n> + files is offset slightly by the increased size of the index and\n> + the cost of keeping it up-to-date. That reduced search time is\n> + usually worth the additional size.\n> diff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\n> @@ -0,0 +1,68 @@\n> +test_expect_success setup '\n> + git checkout -b test &&\n> + echo \"actual\" >> .gitignore &&\n> + echo \"expected\" >> .gitignore &&\n> + echo \"out\" >> .gitignore &&\n\nStyle: drop space after redirection operator:\n\n echo \"actual\" >>.gitignore &&\n echo \"expected\" >>.gitignore &&\n echo \"out\" >>.gitignore &&\n\nBy the way, this is an excellent opportunity to use a here-doc as\nyou're now doing elsewhere in the script:\n\n cat >.gitignore <<-\\EOF &&\n actual\n expected\n out\n EOF\n\nDo we want to anchor these .gitignore patterns to make it clear to\nfuture readers the precise expectations of these tests?\n\n cat >.gitignore <<-\\EOF &&\n /actual\n /expected\n /out\n EOF\n\n> + git add .gitignore &&\n> + git commit -m \"Add .gitignore\"\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n> + test_might_fail git config --unset-all core.untrackedCache &&\n> + test_might_fail git config --unset-all core.fsmonitor &&\n> + git status >out &&\n> + sed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n> + cat >expected <<-\\EOF &&\n> + On branch test\n> +\n> + It took X seconds to enumerate untracked files.\n> + See '\"'\"'git help status'\"'\"' for information on how to improve this.\n> +\n> + nothing to commit, working tree clean\n> + EOF\n\nStyle: We normally make the here-doc body indentation match the\nindentation of the command itself:\n\n cat >expected <<-\\EOF &&\n On branch test\n ...\n EOF\n\n> + test_cmp expected actual\n> +'\n> +\n> +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n> + git config core.untrackedCache true &&\n\nIt's perhaps not super important in this case since each subsequent\ntest is setting up its configuration exactly as it wants it, but it is\ncommon elsewhere in the test scripts to use test_config() which\nautomatically unsets the configuration when the test finishes:\n\n test_config core.untrackedCache true &&\n\nI'm on the fence as to whether or not to use test_config() in this\ncase, but it shouldn't hurt to do so.\n\n> + test_might_fail git config --unset-all core.fsmonitor &&\n> + git status >out &&\n> + sed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n> + cat >expected <<-\\EOF &&\n> + On branch test\n> +\n> + It took X seconds to enumerate untracked files.\n> + See '\"'\"'git help status'\"'\"' for information on how to improve this.\n> +\n> + nothing to commit, working tree clean\n> + EOF\n> + test_cmp expected actual\n> diff --git a/wt-status.c b/wt-status.c\n> @@ -1205,6 +1207,15 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n> +static inline int uf_was_slow(uint32_t untracked_in_ms)\n\nDoes this need to be \"inline\"?\n\n> +{\n> + if (getenv(\"GIT_TEST_UF_DELAY_WARNING\")) {\n> + untracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n> + }\n> +\n> + return UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n> +}\n\nStyle:\n\n if (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n untracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n return UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n\n> @@ -1870,13 +1882,21 @@ static void wt_longstatus_print(struct wt_status *s)\n> - if (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n> + if (uf_was_slow(s->untracked_in_ms) && advice_enabled(ADVICE_STATUS_U_OPTION)) {\n\nWas there a specific reason you switched around the condition so it\nchecks advice_enabled() _after_ checking for slowness? If so, the\nreason may not be obvious to future readers and might deserve mention\nin the commit message. If it was just a whim, then future readers\nmight end up wondering if there was some reason which they are unable\nto figure out, in which case it would be better to retain the original\nordering of conditions.\n\n> status_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n> + if (fsm_mode > FSMONITOR_MODE_DISABLED) {\n> + status_printf_ln(s, GIT_COLOR_NORMAL,\n> + _(\"It took %.2f seconds to enumerate untracked files,\\n\"\n> + \"but this is currently being cached.\"),\n> + s->untracked_in_ms / 1000.0);\n\nTo what does \"this\" refer? Is it this repository? Or something else?\n\n> + } else {\n> + status_printf_ln(s, GIT_COLOR_NORMAL,\n> + _(\"It took %.2f seconds to enumerate untracked files.\"),\n> + s->untracked_in_ms / 1000.0);\n> + }\n> status_printf_ln(s, GIT_COLOR_NORMAL,\n> + _(\"See 'git help status' for information on how to improve this.\"));\n\nOkay.\n\n> + status_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n"},{"id":"467321","messageId":"CANaDLW+Ec0kY4AW5dGvnCaHgcvFOZQZO5EAi595KbVKj7KDg3g@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTO3NPg_Kx3dZhFMEtbMe9hRvaumZYxMnSJRyXqUA=p0g@mail.gmail.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-15T17:45:49Z","receivedAt":"2022-11-15T17:46:04Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Thanks for the feedback, and I'll integrate it into a new patch, most\nlikely today.\n\n> Does this need to be \"inline\"?\n\nOops. Just as I said I don't think I left any feedback unaddressed.\nYou did mention it last time, and it fell through the cracks. I'll fix\nit this time, sorry about that.\n\n> Was there a specific reason you switched around the condition\n\nTotally a whim. After several iterations, the code had changed enough\nthat the original ordering was lost. I'll switch it back.\n\n> To what does \"this\" refer? Is it this repository? Or something else?\n\nHah, good point. The accurate answer is \"the status of currently\nexisting files is being cached, and we'll watch what files changed\nafter now so we only run things on those next time\". Obviously that\nwould be too verbose for the inexperienced user hitting this, really\nthis line is here to convey \"if you run it again, it's probably going\nto be faster\".\n\nHere are some ideas:\n- \"but this result is currently being cached.\"\n- \"but git status results are currently being cached.\" (true but not\nperfectly accurate since index updates don't only happen on git\nstatus)\n- \"but untracked files are currently being cached.\" (not completely\naccurate, I believe the index is updated for all files; the untracked\nfiles are only the interesting ones for this specific performance\nconsideration)\n- \"but the results were cached, and your next runs may be faster.\"\n\nI could use some guidance on what would make most sense here. I\nstrongly feel like the user should know of it since that's been what's\nbeen confusing the users of our very large repo specifically when\ntheir git status is temporarily slow; but I don't have any opinions at\nall about the right phrasing. For now, I'm planning to use the latter\nbullet point in my next patch because it's the most explicit, but I'd\nbe glad to apply someone else's take on this instead.\n"},{"id":"467323","messageId":"CAPig+cT2+nitwD64FzG=6FvO7eUn3q-cq_CmmYeOXxOMyzvUnw@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLW+Ec0kY4AW5dGvnCaHgcvFOZQZO5EAi595KbVKj7KDg3g@mail.gmail.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-15T18:06:34Z","receivedAt":"2022-11-15T18:07:13Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 15, 2022 at 12:46 PM Rudy Rigot wrote:\n> Thanks for the feedback, and I'll integrate it into a new patch, most\n> likely today.\n\nThanks. I think this is _almost_ there; much more polished than\nearlier iterations. Hopefully, one final reroll will make it complete.\n\n> > To what does \"this\" refer? Is it this repository? Or something else?\n>\n> Hah, good point. The accurate answer is \"the status of currently\n> existing files is being cached, and we'll watch what files changed\n> after now so we only run things on those next time\". Obviously that\n> would be too verbose for the inexperienced user hitting this, really\n> this line is here to convey \"if you run it again, it's probably going\n> to be faster\".\n>\n> Here are some ideas:\n> - \"but this result is currently being cached.\"\n> - \"but git status results are currently being cached.\" (true but not\n> perfectly accurate since index updates don't only happen on git\n> status)\n> - \"but untracked files are currently being cached.\" (not completely\n> accurate, I believe the index is updated for all files; the untracked\n> files are only the interesting ones for this specific performance\n> consideration)\n> - \"but the results were cached, and your next runs may be faster.\"\n>\n> I could use some guidance on what would make most sense here. I\n> strongly feel like the user should know of it since that's been what's\n> been confusing the users of our very large repo specifically when\n> their git status is temporarily slow; but I don't have any opinions at\n> all about the right phrasing. For now, I'm planning to use the latter\n> bullet point in my next patch because it's the most explicit, but I'd\n> be glad to apply someone else's take on this instead.\n\nReading the proposals while wearing the hat of someone who has never\nhad to deal with speeding up untracked-file bookkeeping and who may\nnot even know that remedies are available (as discussed in the\ndocumentation), I find that the first three bullet points convey no\nmeaning at all; they leave the reader hanging. The final bullet point,\non the other hand, tells the user something conclusive. I _very_ much\nprefer the final proposal.\n\n(In \"but the results were cached, and your next runs may be faster.\",\nI might suggest dropping \"your\" -- i.e. \"... and subsequent runs may\nbe faster\".)\n"},{"id":"467324","messageId":"CANaDLWLT45GWgMLwj6Ec3nZf-A+Wx5YAkx7pFbmG-AAFf-oxaA@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cT2+nitwD64FzG=6FvO7eUn3q-cq_CmmYeOXxOMyzvUnw@mail.gmail.com","subject":"Re: [PATCH v5] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-15T18:08:54Z","receivedAt":"2022-11-15T18:09:11Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"That makes a ton of sense, I agree with your thinking. Thanks for\nthat, I'll do exactly that.\n"},{"id":"467342","messageId":"pull.1384.v6.git.1668547188070.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v5.git.1668110679098.gitgitgadget@gmail.com","subject":"[PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-15T21:19:47Z","receivedAt":"2022-11-15T21:20:19Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\nImprove the advice displayed when `git status` is slow because\nof excessive numbers of untracked files. Update the `git status`\nman page to explain the various configuration options.\n\n`git status` can be slow when there are a large number of untracked\nfiles and directories, because Git must search the entire worktree\nto enumerate them. Previously, Git would print an advice message\nwith the elapsed search time and a suggestion to disable the search\nusing the `-uno` option. This suggestion also carried a warning\nthat might scare off some users.\n\nGit can reduce the size and time of the untracked file search when\nthe `core.untrackedCache` and `core.fsmonitor` features are enabled\nby caching results from previous `git status` invocations.\n\nUpdate the advice to explain the various combinations of additional\nconfiguration options and refer to (new) documentation in the man\npage that explains it in more detail than what can be printed in an\nadvice message.\n\nFinally, add new tests to verify the new functionality.\n\nSigned-off-by: Rudy Rigot \n---\n status: long status advice adapted to recent capabilities\n \n Here is version 6 for this patch.\n \n Changes since v5:\n \n * End of sentence for fsmonitor case was changed to \"but the results\n were cached, and subsequent runs may be faster.\"\n * Except for that, mostly style and doc fixes.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v6\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v6\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v5:\n\n 1: 8b1b9ee094f ! 1: ff3aa0e01c0 status: long status advice adapted to recent capabilities\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n +\tthat have been modified since the previous `git status` command.\n +\tGit remembers the set of untracked files within each directory\n +\tand assumes that if a directory has not been modified, then\n -+\tthe set of untracked file within has not changed. This is much\n ++\tthe set of untracked files within has not changed. This is much\n +\tfaster than enumerating the contents of every directory, but still\n +\tnot without cost, because Git still has to search for the set of\n +\tmodified directories. The untracked cache is stored in the\n -+\t.git/index file. The reduced cost searching for untracked\n ++\t`.git/index` file. The reduced cost of searching for untracked\n +\tfiles is offset slightly by the increased size of the index and\n +\tthe cost of keeping it up-to-date. That reduced search time is\n +\tusually worth the additional size.\n @@ t/t7065-wtstatus-slow.sh (new)\n +\n +test_expect_success setup '\n +\tgit checkout -b test &&\n -+\techo \"actual\" >> .gitignore &&\n -+\techo \"expected\" >> .gitignore &&\n -+\techo \"out\" >> .gitignore &&\n ++\tcat >.gitignore <<-\\EOF &&\n ++\t/actual\n ++\t/expected\n ++\t/out\n ++\tEOF\n +\tgit add .gitignore &&\n +\tgit commit -m \"Add .gitignore\"\n +'\n @@ t/t7065-wtstatus-slow.sh (new)\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\t\tOn branch test\n ++\tOn branch test\n +\n -+\t\tIt took X seconds to enumerate untracked files.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\tIt took X seconds to enumerate untracked files.\n ++\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+\t\tnothing to commit, working tree clean\n ++\tnothing to commit, working tree clean\n +\tEOF\n +\ttest_cmp expected actual\n +'\n +\n +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n -+\tgit config core.untrackedCache true &&\n ++\ttest_config core.untrackedCache true &&\n +\ttest_might_fail git config --unset-all core.fsmonitor &&\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\t\tOn branch test\n ++\tOn branch test\n +\n -+\t\tIt took X seconds to enumerate untracked files.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\tIt took X seconds to enumerate untracked files.\n ++\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+\t\tnothing to commit, working tree clean\n ++\tnothing to commit, working tree clean\n +\tEOF\n +\ttest_cmp expected actual\n +'\n +\n +test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n -+\tgit config core.untrackedCache true &&\n -+\tgit config core.fsmonitor true &&\n ++\ttest_config core.untrackedCache true &&\n ++\ttest_config core.fsmonitor true &&\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\t\tOn branch test\n ++\tOn branch test\n +\n -+\t\tIt took X seconds to enumerate untracked files,\n -+\t\tbut this is currently being cached.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\tIt took X seconds to enumerate untracked files,\n ++\tbut the results were cached, and subsequent runs may be faster.\n ++\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n +\n -+\t\tnothing to commit, working tree clean\n ++\tnothing to commit, working tree clean\n +\tEOF\n +\ttest_cmp expected actual\n +'\n @@ wt-status.c: static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n -+static inline int uf_was_slow(uint32_t untracked_in_ms)\n ++static int uf_was_slow(uint32_t untracked_in_ms)\n +{\n -+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\")) {\n ++\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n +\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n -+\t}\n -+\n +\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n +}\n +\n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n -+\t\tif (uf_was_slow(s->untracked_in_ms) && advice_enabled(ADVICE_STATUS_U_OPTION)) {\n ++\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s->untracked_in_ms)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n +\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n +\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n -+\t\t\t\t\t\t\"but this is currently being cached.\"),\n ++\t\t\t\t\t\t\"but the results were cached, and subsequent runs may be faster.\"),\n +\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n +\t\t\t} else {\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n\n\n Documentation/git-status.txt | 59 ++++++++++++++++++++++++++++++\n t/t7065-wtstatus-slow.sh | 70 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 28 ++++++++++++---\n 3 files changed, 152 insertions(+), 5 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 5e438a7fdc1..570c36e07c1 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,65 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. There is no single optimum set of settings right\n+for everyone. Here is a brief summary of the relevant options\n+to help you choose which is right for you.\n+\n+* First, you may want to run `git status` again. Your current\n+\tconfiguration may already be caching `git status` results,\n+\tso it could be faster on subsequent runs.\n+\n+* The `--untracked-files=no` flag or the\n+\t`status.showUntrackedfiles=false` config (see above for both) :\n+\tindicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n+\tthis config option disables a warning message when the search\n+\tfor untracked files takes longer than desired. In some large\n+\trepositories, this message may appear frequently and not be a\n+\thelpful signal.\n+\n+* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n+\tenable the untracked cache feature and only search directories\n+\tthat have been modified since the previous `git status` command.\n+\tGit remembers the set of untracked files within each directory\n+\tand assumes that if a directory has not been modified, then\n+\tthe set of untracked files within has not changed. This is much\n+\tfaster than enumerating the contents of every directory, but still\n+\tnot without cost, because Git still has to search for the set of\n+\tmodified directories. The untracked cache is stored in the\n+\t`.git/index` file. The reduced cost of searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+\t`core.fsmonitor=` (see\n+\tlinkgit:git-update-index[1]) : enable both the untracked cache\n+\tand FSMonitor features and only search directories that have\n+\tbeen modified since the previous `git status` command. This\n+\tis faster than using just the untracked cache alone because\n+\tGit can also avoid searching for modified directories. Git\n+\tonly has to enumerate the exact set of directories that have\n+\tchanged recently. While the FSMonitor feature can be enabled\n+\twithout the untracked cache, the benefits are greatly reduced\n+\tin that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..8d08a962f88\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,70 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+. ./test-lib.sh\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+test_expect_success setup '\n+\tgit checkout -b test &&\n+\tcat >.gitignore <<-\\EOF &&\n+\t/actual\n+\t/expected\n+\t/out\n+\tEOF\n+\tgit add .gitignore &&\n+\tgit commit -m \"Add .gitignore\"\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_might_fail git config --unset-all core.untrackedCache &&\n+\ttest_might_fail git config --unset-all core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch test\n+\n+\tIt took X seconds to enumerate untracked files.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+\ttest_config core.untrackedCache true &&\n+\ttest_might_fail git config --unset-all core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch test\n+\n+\tIt took X seconds to enumerate untracked files.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+\ttest_config core.untrackedCache true &&\n+\ttest_config core.fsmonitor true &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch test\n+\n+\tIt took X seconds to enumerate untracked files,\n+\tbut the results were cached, and subsequent runs may be faster.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..1f6d64e759f 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,13 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1823,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1880,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s->untracked_in_ms)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but the results were cached, and subsequent runs may be faster.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: 319605f8f00e402f3ea758a02c63534ff800a711\n-- \ngitgitgadget\n"},{"id":"467656","messageId":"CAPig+cRPQ7bmG6+U+oQGGUFiSiHoMMpMk8FDJ7GMJvwCXifa9g@mail.gmail.com","threadId":"58628","inReplyTo":"pull.1384.v6.git.1668547188070.gitgitgadget@gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-21T05:06:10Z","receivedAt":"2022-11-21T05:06:26Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 15, 2022 at 4:19 PM Rudy Rigot via GitGitGadget\n wrote:\n> Improve the advice displayed when `git status` is slow because\n> of excessive numbers of untracked files. Update the `git status`\n> man page to explain the various configuration options.\n>\n> Signed-off-by: Rudy Rigot \n> ---\n> Here is version 6 for this patch.\n>\n> Changes since v5:\n>\n> * End of sentence for fsmonitor case was changed to \"but the results\n> were cached, and subsequent runs may be faster.\"\n> * Except for that, mostly style and doc fixes.\n\nThanks, I think v6 addresses all my earlier review comments. Just one\nor two notes below...\n\n> +test_expect_success setup '\n> + git checkout -b test &&\n> + cat >.gitignore <<-\\EOF &&\n> + /actual\n> + /expected\n> + /out\n> + EOF\n> + git add .gitignore &&\n> + git commit -m \"Add .gitignore\"\n> +'\n\nI suppose I wasn't paying close enough attention earlier, but I see\nnow that this is creating a branch named \"test\". If I remove the `git\ncheckout -b test` line entirely and change \"test\" to \"master\" in the\n\"expected\" files, all the tests pass just fine. So, it's not apparent\nwhy you need to create a specially-named branch here rather than\nsimply accepting the default branch name. The reason I bring this up\nis that unnecessary code, whether in tests or elsewhere, can confuse\nfuture readers (and reviewers, such as myself) into wondering if\nsomething subtle is going on which the reader is overlooking. (This is\nvery much the same sort of concern as when I asked in an earlier\nreview why the conditions in an `if` got switched around from the way\nthey were in the original code; such unnecessary changes can confuse\nfuture readers.)\n\nIf the answer is that you wanted to avoid the term \"master\", then an\nalternative would have been to override the default branch name at the\ntop of the script:\n\n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n\nThat said, this is minor, and I'm not keen on eating up more of your\ntime or reviewer time, so I doubt this is worth a reroll.\n\n> +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n> + test_might_fail git config --unset-all core.untrackedCache &&\n> + test_might_fail git config --unset-all core.fsmonitor &&\n\nWhen I suggested the above code, it had slipped my mind that we have a\ntest_unconfig() function in t/test-lib-functions.sh which does this\nmore concisely:\n\n test_unconfig core.untrackedCache &&\n test_unconfig core.fsmonitor &&\n\nBut what you have here in v6 is good enough; it's not worth wasting\nyour time or reviewer time rerolling just to make this change. I\nmention it merely for future reference.\n"},{"id":"467695","messageId":"CANaDLWJM1VRivm8VLqxg+w8K-+49E0km6AgOzWzN9X=TgzaEiA@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cRPQ7bmG6+U+oQGGUFiSiHoMMpMk8FDJ7GMJvwCXifa9g@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-21T15:54:51Z","receivedAt":"2022-11-21T15:55:07Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> That said, this is minor, and I'm not keen on eating up more of your\n> time or reviewer time, so I doubt this is worth a reroll.\n\nEh, there's nothing wrong with striving for perfection. Lemme do one\nmore reroll...\n\n> So, it's not apparent\n> why you need to create a specially-named branch here rather than\n> simply accepting the default branch name.\n\nThe reason was that it failed some CI pipelines before I did this,\nwith some pipelines printing \"main\" instead of \"master\" into the git\nstatus output. I fixed it right away, so I don't know if it was a CI\nglitch that day or if it would still be the same running it now. I\ncould have redacted the branch name away from the output, but it\nseemed simpler and more readable to just set the branch name in stone\nfor all pipelines.\n\n> an alternative would have been to override the default branch name at the\n> top of the script:\n\nOh, this seems like a better way to do what I was trying to do. I'll\nchange it now.\n\n> we have a test_unconfig() function\n\nI'll use that.\n\nNew patch coming!\n"},{"id":"467696","messageId":"CAPig+cQgu=i6pZTzoNYGZ_6X=DGdmwa=dPhSQVqD+eLCZCGJSg@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLWJM1VRivm8VLqxg+w8K-+49E0km6AgOzWzN9X=TgzaEiA@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-21T16:17:36Z","receivedAt":"2022-11-21T16:18:08Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Nov 21, 2022 at 10:55 AM Rudy Rigot wrote:\n> > That said, this is minor, and I'm not keen on eating up more of your\n> > time or reviewer time, so I doubt this is worth a reroll.\n>\n> Eh, there's nothing wrong with striving for perfection. Lemme do one\n> more reroll...\n\nReviewer time is a scarce resource on the mailing list these days,\nwhich is why I'm hesitant to see rerolls for minor or subjective\nchanges. However, if you're going to reroll anyhow, I have a couple\nmore things to say... (below)\n\n> > So, it's not apparent\n> > why you need to create a specially-named branch here rather than\n> > simply accepting the default branch name.\n>\n> The reason was that it failed some CI pipelines before I did this,\n> with some pipelines printing \"main\" instead of \"master\" into the git\n> status output. I fixed it right away, so I don't know if it was a CI\n> glitch that day or if it would still be the same running it now. I\n> could have redacted the branch name away from the output, but it\n> seemed simpler and more readable to just set the branch name in stone\n> for all pipelines.\n\nMost likely it wasn't a glitch, but rather (I'd guess) that Windows CI\nuses \"main\" already, whereas Unix CI's still use \"master\".\n\n> > an alternative would have been to override the default branch name at the\n> > top of the script:\n>\n> Oh, this seems like a better way to do what I was trying to do. I'll\n> change it now.\n>\n> > we have a test_unconfig() function\n>\n> I'll use that.\n>\n> New patch coming!\n\nIf you're going to reroll, then I'll mention a couple more things\nwhich I held back before since I want to use reviewer time wisely.\nNevertheless...\n\nFirst, having the commit message explain the problem first and then\nthe solution is more reviewer-friendly, not the solution and then the\nproblem as this patch is doing. Additionally, the commit message\nshould be written in imperative mood. Documentation/SubmittingPatches\nhas a good discussion of these points. It's also typically unnecessary\nfor the commit message to say that the patch is adding new tests;\nreviewers assume that you will do so when appropriate, and the patch\nitself shows plainly enough that you did. Taking these points into\nconsideration, you might write the commit message like this:\n\n status: modernize git-status \"slow untracked files\" advice\n\n `git status` can be slow when there are a large number of\n untracked files and directories since Git must search the entire\n worktree to enumerate them. When it is too slow, Git prints\n advice with the elapsed search time and a suggestion to disable\n the search using the `-uno` option. This suggestion also carries\n a warning that might scare off some users.\n\n However, these days, `-uno` isn't the only option. Git can reduce\n the size and time of the untracked file search when the\n `core.untrackedCache` and `core.fsmonitor` features are enabled by\n caching results from previous `git status` invocations.\n\n Therefore, update the `git status` man page to explain the various\n configuration options, and update the advice to provide more\n detail about the current configuration and to refer to the updated\n documentation.\n\nSecond, we usually don't want to waste a test script number (such as\n\"t7065\") if we can avoid it, especially for so few tests and such\nminor functionality. So, if there is an existing test script in which\nthese new tests might fit, it's better to add them to that script\ninstead, usually at the end of the script. (I haven't checked, but\nmaybe you can find an existing script which would be a good fit; if\nnot, then placing them in a new standalone script, as the patch is\nalready doing, may be okay.)\n"},{"id":"467773","messageId":"CANaDLWJ+Suye98QKub9nfnknLEsyQ4PK1LxDkPmzGC_-hApkFw@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cQgu=i6pZTzoNYGZ_6X=DGdmwa=dPhSQVqD+eLCZCGJSg@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-22T16:52:21Z","receivedAt":"2022-11-22T16:52:49Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> Most likely it wasn't a glitch, but rather (I'd guess) that Windows CI\n> uses \"main\" already, whereas Unix CI's still use \"master\".\n\nMy intuition was that it was something like that indeed.\n\n> you might write the commit message like this\n\nThe current phrasing was initially copied as is from a past review\nfeedback; I have no issues at all replacing it with yours.\n\n> So, if there is an existing test script in which\n> these new tests might fit, it's better to add them to that script\n> instead\n\nOh, sorry about that, it hadn't occurred to me that there could be a\ndownside to using new test script numbers.\n\nI'm 100% on board with the thinking, but I'm struggling quite a bit to\nimplement it. There are several existing test scripts where these new\ntests would fit very well semantically (t7060-wtstatus.sh,c,\nt7512-status-help.sh, t7519-status-fsmonitor.sh, ...), and I spent\nquite some time yesterday trying to move the 3 news tests to those.\nFor some reason, test_cmp is not giving me a diff anymore when working\nin those script files, so I feel in the dark about what the tests are\nfailing about, and I'm stumped about what to try next.\n\nWhat I mean: for instance, if I introduce an intentional mistake in\nthe test and run './t7065-wtstatus-slow.sh -v', I get this section\nthat clarifies what the issue is:\n\n--- expected 2022-11-21 23:46:00.000000000 +0000\n+++ actual 2022-11-21 23:46:00.000000000 +0000\n@@ -1,4 +1,4 @@\n-On branch maine\n+On branch main\n\nHere is a gist https://gist.github.com/rudyrigot/b31fcb6384e829ca7586818758e48d0b,\nwith:\n\n- the patch as I currently have it on t7508-status.sh (it's a bit\nlonger than it was, without the isolation in a separate script I've\nhad to do a few things to mitigate the side effects from other tests\nin the script)\n- the end of what I get when running 'sh ./t7508-status.sh -v':\nhttps://gist.github.com/rudyrigot/ee80f3d59231f25698c9dd6c48d8ab85. It\nseems like 2 of my 3 tests are failing, but the output isn't very\nhelpful to figure out why.\n\nWould you (or someone else) have pointers to help me get through this one?\n\nI'm tempted to throw in the towel, since it sounded like it wasn't too\nhuge a deal if this lived in its separate script file, and that other\npeople's bandwidth (which I'm aware is what I'm requesting here) is an\neven more scarce resource. So I'll submit a new patch with everything\nelse but this, so there's the option to still proceed with it if\nthat's the most sensible path forward. But I have to admit I'm quite\nfrustrated that I couldn't figure this last one out by myself, so I'm\nmore than happy to dig more into it, if anyone has guidance.\n\nThanks a lot.\n"},{"id":"467774","messageId":"pull.1384.v7.git.1669136378754.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v6.git.1668547188070.gitgitgadget@gmail.com","subject":"[PATCH v7] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-22T16:59:38Z","receivedAt":"2022-11-22T16:59:55Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\n`git status` can be slow when there are a large number of\nuntracked files and directories since Git must search the entire\nworktree to enumerate them. When it is too slow, Git prints\nadvice with the elapsed search time and a suggestion to disable\nthe search using the `-uno` option. This suggestion also carries\na warning that might scare off some users.\n\nHowever, these days, `-uno` isn't the only option. Git can reduce\nthe size and time of the untracked file search when the\n`core.untrackedCache` and `core.fsmonitor` features are enabled by\ncaching results from previous `git status` invocations.\n\nTherefore, update the `git status` man page to explain the various\nconfiguration options, and update the advice to provide more\ndetail about the current configuration and to refer to the updated\ndocumentation.\n\nSigned-off-by: Rudy Rigot \n---\n status: modernize git-status \"slow untracked files\" advice\n \n Here is version 7 for this patch.\n \n Changes since v6:\n \n * Readability improvements in tests\n * Rewrite of the commit message\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v7\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v7\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v6:\n\n 1: ff3aa0e01c0 ! 1: 871a9becbdf status: long status advice adapted to recent capabilities\n @@ Metadata\n Author: Rudy Rigot \n \n ## Commit message ##\n - status: long status advice adapted to recent capabilities\n + status: modernize git-status \"slow untracked files\" advice\n \n - Improve the advice displayed when `git status` is slow because\n - of excessive numbers of untracked files. Update the `git status`\n - man page to explain the various configuration options.\n + `git status` can be slow when there are a large number of\n + untracked files and directories since Git must search the entire\n + worktree to enumerate them. When it is too slow, Git prints\n + advice with the elapsed search time and a suggestion to disable\n + the search using the `-uno` option. This suggestion also carries\n + a warning that might scare off some users.\n \n - `git status` can be slow when there are a large number of untracked\n - files and directories, because Git must search the entire worktree\n - to enumerate them. Previously, Git would print an advice message\n - with the elapsed search time and a suggestion to disable the search\n - using the `-uno` option. This suggestion also carried a warning\n - that might scare off some users.\n + However, these days, `-uno` isn't the only option. Git can reduce\n + the size and time of the untracked file search when the\n + `core.untrackedCache` and `core.fsmonitor` features are enabled by\n + caching results from previous `git status` invocations.\n \n - Git can reduce the size and time of the untracked file search when\n - the `core.untrackedCache` and `core.fsmonitor` features are enabled\n - by caching results from previous `git status` invocations.\n -\n - Update the advice to explain the various combinations of additional\n - configuration options and refer to (new) documentation in the man\n - page that explains it in more detail than what can be printed in an\n - advice message.\n -\n - Finally, add new tests to verify the new functionality.\n + Therefore, update the `git status` man page to explain the various\n + configuration options, and update the advice to provide more\n + detail about the current configuration and to refer to the updated\n + documentation.\n \n Signed-off-by: Rudy Rigot \n \n @@ t/t7065-wtstatus-slow.sh (new)\n +\n +test_description='test status when slow untracked files'\n +\n -+. ./test-lib.sh\n ++GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n ++export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n +\n +GIT_TEST_UF_DELAY_WARNING=1\n +export GIT_TEST_UF_DELAY_WARNING\n +\n ++. ./test-lib.sh\n ++\n +test_expect_success setup '\n -+\tgit checkout -b test &&\n +\tcat >.gitignore <<-\\EOF &&\n +\t/actual\n +\t/expected\n @@ t/t7065-wtstatus-slow.sh (new)\n +'\n +\n +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n -+\ttest_might_fail git config --unset-all core.untrackedCache &&\n -+\ttest_might_fail git config --unset-all core.fsmonitor &&\n ++\ttest_unconfig core.untrackedCache &&\n ++\ttest_unconfig core.fsmonitor &&\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\tOn branch test\n ++\tOn branch main\n +\n +\tIt took X seconds to enumerate untracked files.\n +\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n @@ t/t7065-wtstatus-slow.sh (new)\n +\n +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n +\ttest_config core.untrackedCache true &&\n -+\ttest_might_fail git config --unset-all core.fsmonitor &&\n ++\ttest_unconfig core.fsmonitor &&\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\tOn branch test\n ++\tOn branch main\n +\n +\tIt took X seconds to enumerate untracked files.\n +\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n @@ t/t7065-wtstatus-slow.sh (new)\n +\tgit status >out &&\n +\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n +\tcat >expected <<-\\EOF &&\n -+\tOn branch test\n ++\tOn branch main\n +\n +\tIt took X seconds to enumerate untracked files,\n +\tbut the results were cached, and subsequent runs may be faster.\n\n\n Documentation/git-status.txt | 59 +++++++++++++++++++++++++++++\n t/t7065-wtstatus-slow.sh | 72 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 28 +++++++++++---\n 3 files changed, 154 insertions(+), 5 deletions(-)\n create mode 100755 t/t7065-wtstatus-slow.sh\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 5e438a7fdc1..570c36e07c1 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,65 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. There is no single optimum set of settings right\n+for everyone. Here is a brief summary of the relevant options\n+to help you choose which is right for you.\n+\n+* First, you may want to run `git status` again. Your current\n+\tconfiguration may already be caching `git status` results,\n+\tso it could be faster on subsequent runs.\n+\n+* The `--untracked-files=no` flag or the\n+\t`status.showUntrackedfiles=false` config (see above for both) :\n+\tindicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n+\tthis config option disables a warning message when the search\n+\tfor untracked files takes longer than desired. In some large\n+\trepositories, this message may appear frequently and not be a\n+\thelpful signal.\n+\n+* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n+\tenable the untracked cache feature and only search directories\n+\tthat have been modified since the previous `git status` command.\n+\tGit remembers the set of untracked files within each directory\n+\tand assumes that if a directory has not been modified, then\n+\tthe set of untracked files within has not changed. This is much\n+\tfaster than enumerating the contents of every directory, but still\n+\tnot without cost, because Git still has to search for the set of\n+\tmodified directories. The untracked cache is stored in the\n+\t`.git/index` file. The reduced cost of searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+\t`core.fsmonitor=` (see\n+\tlinkgit:git-update-index[1]) : enable both the untracked cache\n+\tand FSMonitor features and only search directories that have\n+\tbeen modified since the previous `git status` command. This\n+\tis faster than using just the untracked cache alone because\n+\tGit can also avoid searching for modified directories. Git\n+\tonly has to enumerate the exact set of directories that have\n+\tchanged recently. While the FSMonitor feature can be enabled\n+\twithout the untracked cache, the benefits are greatly reduced\n+\tin that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh\nnew file mode 100755\nindex 00000000000..6733e8ba36b\n--- /dev/null\n+++ b/t/t7065-wtstatus-slow.sh\n@@ -0,0 +1,72 @@\n+#!/bin/sh\n+\n+test_description='test status when slow untracked files'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+GIT_TEST_UF_DELAY_WARNING=1\n+export GIT_TEST_UF_DELAY_WARNING\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\tcat >.gitignore <<-\\EOF &&\n+\t/actual\n+\t/expected\n+\t/out\n+\tEOF\n+\tgit add .gitignore &&\n+\tgit commit -m \"Add .gitignore\"\n+'\n+\n+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n+\ttest_unconfig core.untrackedCache &&\n+\ttest_unconfig core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch main\n+\n+\tIt took X seconds to enumerate untracked files.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n+\ttest_config core.untrackedCache true &&\n+\ttest_unconfig core.fsmonitor &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch main\n+\n+\tIt took X seconds to enumerate untracked files.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n+\ttest_config core.untrackedCache true &&\n+\ttest_config core.fsmonitor true &&\n+\tgit status >out &&\n+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\tcat >expected <<-\\EOF &&\n+\tOn branch main\n+\n+\tIt took X seconds to enumerate untracked files,\n+\tbut the results were cached, and subsequent runs may be faster.\n+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\tnothing to commit, working tree clean\n+\tEOF\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..1f6d64e759f 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,13 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1823,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1880,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s->untracked_in_ms)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but the results were cached, and subsequent runs may be faster.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: 319605f8f00e402f3ea758a02c63534ff800a711\n-- \ngitgitgadget\n"},{"id":"467776","messageId":"CAPig+cTrpnVOW0Y2m5xtPhLudY=rPCn3qPQA0RSso7ueFytZbQ@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLWJ+Suye98QKub9nfnknLEsyQ4PK1LxDkPmzGC_-hApkFw@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-22T17:18:07Z","receivedAt":"2022-11-22T17:18:23Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 11:52 AM Rudy Rigot wrote:\n> > you might write the commit message like this\n>\n> The current phrasing was initially copied as is from a past review\n> feedback; I have no issues at all replacing it with yours.\n\nApparently, I overlooked that when scanning backward through the thread.\n\n> > So, if there is an existing test script in which\n> > these new tests might fit, it's better to add them to that script\n> > instead\n>\n> Oh, sorry about that, it hadn't occurred to me that there could be a\n> downside to using new test script numbers.\n\nNo need to apologize. Reviewers understand that there is a lot to\nabsorb and figure out.\n\n> I'm 100% on board with the thinking, but I'm struggling quite a bit to\n> implement it. There are several existing test scripts where these new\n> tests would fit very well semantically (t7060-wtstatus.sh,c,\n> t7512-status-help.sh, t7519-status-fsmonitor.sh, ...), and I spent\n> quite some time yesterday trying to move the 3 news tests to those.\n> For some reason, test_cmp is not giving me a diff anymore when working\n> in those script files, so I feel in the dark about what the tests are\n> failing about, and I'm stumped about what to try next.\n>\n> Here is a gist https://gist.github.com/rudyrigot/b31fcb6384e829ca7586818758e48d0b,\n> with:\n>\n> - the patch as I currently have it on t7508-status.sh (it's a bit\n> longer than it was, without the isolation in a separate script I've\n> had to do a few things to mitigate the side effects from other tests\n> in the script)\n\nI probably should have thought to warn about possible interaction with\nearlier tests when I made the suggestion to place the tests in an\nexisting script. Nevertheless, we should be able to sidestep all that\n\"isolation goop\" you had to add to the tests... (more below).\n\n> - the end of what I get when running 'sh ./t7508-status.sh -v':\n> https://gist.github.com/rudyrigot/ee80f3d59231f25698c9dd6c48d8ab85. It\n> seems like 2 of my 3 tests are failing, but the output isn't very\n> helpful to figure out why.\n>\n> Would you (or someone else) have pointers to help me get through this one?\n\nThe second 'gist' URL seems broken, so I can't comment on the exact\noutput. Without having seen the actual problem, I can't really provide\ndirect feedback for fixing the precise issue, however...\n\nWe actually don't want you to pollute your new tests with goop to\nclean up after earlier tests in the script since that just introduces\na bunch of code in your tests which is not directly relevant to what\nyour tests are checking. So, perhaps the cleanest way to approach this\nis to have your \"setup\" test create a new repository in the trash\ndirectory and then just run your remaining new tests in that\nrepository. That way, your tests don't have to deal with any gunk from\nearlier tests. This basically means taking your tests pretty much as\nyou had them in v6, but with a little extra boilerplate. Something\nlike this:\n\n test_expect_success setup '\n git init slowstatus &&\n (\n cd slowstatus &&\n cat >.gitignore <<-\\EOF &&\n /actual\n /expected\n /out\n EOF\n git add .gitignore &&\n git commit -m \"Add .gitignore\"\n )\n '\n\n test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n (\n cd slowstatus &&\n test_unconfig core.untrackedCache &&\n test_unconfig core.fsmonitor &&\n ...\n )\n '\n\nand so on.\n\n> I'm tempted to throw in the towel, since it sounded like it wasn't too\n> huge a deal if this lived in its separate script file, and that other\n> people's bandwidth (which I'm aware is what I'm requesting here) is an\n> even more scarce resource.\n\nMy comment about possibly leaving them in a separate script if you\ncouldn't find a suitable existing script was just general advice. I\ncan't speak for Junio and thus can't say what he will or will not\naccept in his tree.\n\n> So I'll submit a new patch with everything\n> else but this, so there's the option to still proceed with it if\n> that's the most sensible path forward. But I have to admit I'm quite\n> frustrated that I couldn't figure this last one out by myself, so I'm\n> more than happy to dig more into it, if anyone has guidance.\n\nPerhaps the suggestion I outlined above will help.\n"},{"id":"467777","messageId":"CAPig+cR=kkWVuBDh2FS+869a_P_xaLj5NaCgW7q3M_utLrgSsg@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTrpnVOW0Y2m5xtPhLudY=rPCn3qPQA0RSso7ueFytZbQ@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-22T17:24:09Z","receivedAt":"2022-11-22T17:24:28Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 12:18 PM Eric Sunshine wrote:\n> On Tue, Nov 22, 2022 at 11:52 AM Rudy Rigot wrote:\n> > > you might write the commit message like this\n> > - the end of what I get when running 'sh ./t7508-status.sh -v':\n> > https://gist.github.com/rudyrigot/ee80f3d59231f25698c9dd6c48d8ab85. It\n> > seems like 2 of my 3 tests are failing, but the output isn't very\n> > helpful to figure out why.\n>\n> The second 'gist' URL seems broken, so I can't comment on the exact\n> output. Without having seen the actual problem, I can't really provide\n> direct feedback for fixing the precise issue, however...\n\nDespite the URL of the second git being broken, I notice that you have\nsome \"-v\" output tacked onto the first gist. Indeed, the \"-v\" output\nisn't very helpful here. What I normally do when debugging failing\ntests is run with \"-x\" and \"-i\". The \"-x\" shows all the output the\ntest produces, including any error messages. So \"./t7508-status.sh -x\n-i\" would likely make it easier to diagnose such failures. (But first\ntry the suggestion I made in my previous reply for isolating the new\ntests in their own repository.)\n"},{"id":"467778","messageId":"CANaDLWLLVqJ6_razCc1-On5dWBv5xC3QRn_Vw1M-yodCs-0XrA@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cR=kkWVuBDh2FS+869a_P_xaLj5NaCgW7q3M_utLrgSsg@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-22T17:29:45Z","receivedAt":"2022-11-22T17:30:02Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Oops, here is the only Gist URL I had meant to share, sorry about the\nconfusion, the other one was a copy-paste mistake:\nhttps://gist.github.com/rudyrigot/b31fcb6384e829ca7586818758e48d0b\n\nBut the suggestions you just gave me are wonderfully helpful! Thanks a\nlot for that, let me give those a try now...\n"},{"id":"467780","messageId":"CAPig+cQF8vjGNUux-ZMBRxbEd3V0p27oLWZ7k2=mf40kAkWVeg@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cTrpnVOW0Y2m5xtPhLudY=rPCn3qPQA0RSso7ueFytZbQ@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-22T17:40:33Z","receivedAt":"2022-11-22T17:40:50Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 12:18 PM Eric Sunshine wrote:\n> This basically means taking your tests pretty much as\n> you had them in v6, but with a little extra boilerplate. Something\n> like this:\n>\n> test_expect_success setup '\n> git init slowstatus &&\n> (\n> cd slowstatus &&\n> cat >.gitignore <<-\\EOF &&\n> /actual\n> /expected\n> /out\n> EOF\n> git add .gitignore &&\n> git commit -m \"Add .gitignore\"\n> )\n> '\n\nA minor additional comment if you do go this route and place the new\ntests in an existing script...\n\nAlthough \"setup\" was fine as a title in a standalone script, when\nadding the new tests to an existing script, you'd probably want to\nchoose a more meaningful name. Perhaps \"setup slow untracked-cache\nstatus\" or something.\n"},{"id":"467788","messageId":"CAPig+cSR0MAYRLtPS1YcegqMZn4FDbdRvbCbuDfXWR=wF_ofGw@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cQF8vjGNUux-ZMBRxbEd3V0p27oLWZ7k2=mf40kAkWVeg@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-22T18:07:36Z","receivedAt":"2022-11-22T18:08:13Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 12:40 PM Eric Sunshine wrote:\n> A minor additional comment if you do go this route and place the new\n> tests in an existing script...\n\nAnd one more comment...\n\nBy placing:\n\n GIT_TEST_UF_DELAY_WARNING=1\n export GIT_TEST_UF_DELAY_WARNING\n\nat the top of the existing script into which you add the new tests, we\nhave to worry about potential side-effects in other tests in the\nscripts. Better would be to place these lines just above the new\ntests, so that the effects are better isolated. However, even better\nthan that would be to isolate the environment variable to exactly the\npoint it is needed. For instance:\n\n test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n test_unconfig core.untrackedCache &&\n test_unconfig core.fsmonitor &&\n GIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n ...\n '\n"},{"id":"467802","messageId":"CANaDLW+3HfVDrXmqNwcmbdbfeYBvCAR2pjo4FSuiGn_S=sOL5g@mail.gmail.com","threadId":"58628","inReplyTo":"CAPig+cSR0MAYRLtPS1YcegqMZn4FDbdRvbCbuDfXWR=wF_ofGw@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-22T19:19:10Z","receivedAt":"2022-11-22T19:19:33Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Holy snap it worked! Patch coming as soon as CI confirms it's good\neverywhere. I believe I was able to integrate all of your advice.\nThanks a lot for your guidance!\n"},{"id":"467805","messageId":"CAPig+cQrKZZwo9suXmVFzNEPwCJZ6J0Sau4dpjkH=PSoQav2yA@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLW+3HfVDrXmqNwcmbdbfeYBvCAR2pjo4FSuiGn_S=sOL5g@mail.gmail.com","subject":"Re: [PATCH v6] status: long status advice adapted to recent capabilities","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-11-22T19:48:33Z","receivedAt":"2022-11-22T19:48:58Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 2:19 PM Rudy Rigot wrote:\n> Holy snap it worked! Patch coming as soon as CI confirms it's good\n> everywhere. I believe I was able to integrate all of your advice.\n> Thanks a lot for your guidance!\n\nGlad to hear that it all worked out.\n"},{"id":"467814","messageId":"pull.1384.v8.git.1669154823035.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v7.git.1669136378754.gitgitgadget@gmail.com","subject":"[PATCH v8] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-22T22:07:02Z","receivedAt":"2022-11-22T22:07:38Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\n`git status` can be slow when there are a large number of\nuntracked files and directories since Git must search the entire\nworktree to enumerate them. When it is too slow, Git prints\nadvice with the elapsed search time and a suggestion to disable\nthe search using the `-uno` option. This suggestion also carries\na warning that might scare off some users.\n\nHowever, these days, `-uno` isn't the only option. Git can reduce\nthe size and time of the untracked file search when the\n`core.untrackedCache` and `core.fsmonitor` features are enabled by\ncaching results from previous `git status` invocations.\n\nTherefore, update the `git status` man page to explain the various\nconfiguration options, and update the advice to provide more\ndetail about the current configuration and to refer to the updated\ndocumentation.\n\nSigned-off-by: Rudy Rigot \n---\n status: modernize git-status \"slow untracked files\" advice\n \n Here is version 8 for this patch.\n \n Changes since v7:\n \n * Moved tests from new test script to existing one, in order not to\n needlessly waste a test script number for such a small feature. Two\n caveats:\n * The use of test_config in a subshell result in: 'error: bug in the\n test script: test_when_finished does nothing in a subshell', so\n I've had to resort to using plain old git config instead.\n * A test higher in the script is doing git config --global\n advice.statusuoption false, so I now have to set it locally in my\n isolated test repo.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v8\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v8\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v7:\n\n 1: 871a9becbdf ! 1: 16e3721515b status: modernize git-status \"slow untracked files\" advice\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n --------\n linkgit:gitignore[5]\n \n - ## t/t7065-wtstatus-slow.sh (new) ##\n -@@\n -+#!/bin/sh\n -+\n -+test_description='test status when slow untracked files'\n -+\n -+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n -+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n -+\n -+GIT_TEST_UF_DELAY_WARNING=1\n -+export GIT_TEST_UF_DELAY_WARNING\n -+\n -+. ./test-lib.sh\n -+\n -+test_expect_success setup '\n -+\tcat >.gitignore <<-\\EOF &&\n -+\t/actual\n -+\t/expected\n -+\t/out\n -+\tEOF\n -+\tgit add .gitignore &&\n -+\tgit commit -m \"Add .gitignore\"\n + ## t/t7508-status.sh ##\n +@@ t/t7508-status.sh: test_expect_success 'racy timestamps will be fixed for dirty worktree' '\n + \t! test_is_magic_mtime .git/index\n + '\n + \n ++test_expect_success 'setup slow status advice' '\n ++\tGIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main git init slowstatus &&\n ++\t(\n ++\t\tcd slowstatus &&\n ++\t\tcat >.gitignore <<-\\EOF &&\n ++\t\t/actual\n ++\t\t/expected\n ++\t\t/out\n ++\t\tEOF\n ++\t\tgit add .gitignore &&\n ++\t\tgit commit -m \"Add .gitignore\" &&\n ++\t\tgit config advice.statusuoption true\n ++\t)\n +'\n +\n -+test_expect_success 'when core.untrackedCache and fsmonitor are unset' '\n -+\ttest_unconfig core.untrackedCache &&\n -+\ttest_unconfig core.fsmonitor &&\n -+\tgit status >out &&\n -+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n -+\tcat >expected <<-\\EOF &&\n -+\tOn branch main\n -+\n -+\tIt took X seconds to enumerate untracked files.\n -+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n -+\n -+\tnothing to commit, working tree clean\n -+\tEOF\n -+\ttest_cmp expected actual\n ++test_expect_success 'slow status advice when core.untrackedCache and fsmonitor are unset' '\n ++\t(\n ++\t\tcd slowstatus &&\n ++\t\tgit config core.untrackedCache false &&\n ++\t\tgit config core.fsmonitor false &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n ++\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tcat >expected <<-\\EOF &&\n ++\t\tOn branch main\n ++\n ++\t\tIt took X seconds to enumerate untracked files.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\n ++\t\tnothing to commit, working tree clean\n ++\t\tEOF\n ++\t\ttest_cmp expected actual\n ++\t)\n +'\n +\n -+test_expect_success 'when core.untrackedCache true, but not fsmonitor' '\n -+\ttest_config core.untrackedCache true &&\n -+\ttest_unconfig core.fsmonitor &&\n -+\tgit status >out &&\n -+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n -+\tcat >expected <<-\\EOF &&\n -+\tOn branch main\n -+\n -+\tIt took X seconds to enumerate untracked files.\n -+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n -+\n -+\tnothing to commit, working tree clean\n -+\tEOF\n -+\ttest_cmp expected actual\n ++test_expect_success 'slow status advice when core.untrackedCache true, but not fsmonitor' '\n ++\t(\n ++\t\tcd slowstatus &&\n ++\t\tgit config core.untrackedCache true &&\n ++\t\tgit config core.fsmonitor false &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n ++\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tcat >expected <<-\\EOF &&\n ++\t\tOn branch main\n ++\n ++\t\tIt took X seconds to enumerate untracked files.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\n ++\t\tnothing to commit, working tree clean\n ++\t\tEOF\n ++\t\ttest_cmp expected actual\n ++\t)\n +'\n +\n -+test_expect_success 'when core.untrackedCache true, and fsmonitor' '\n -+\ttest_config core.untrackedCache true &&\n -+\ttest_config core.fsmonitor true &&\n -+\tgit status >out &&\n -+\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n -+\tcat >expected <<-\\EOF &&\n -+\tOn branch main\n -+\n -+\tIt took X seconds to enumerate untracked files,\n -+\tbut the results were cached, and subsequent runs may be faster.\n -+\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n -+\n -+\tnothing to commit, working tree clean\n -+\tEOF\n -+\ttest_cmp expected actual\n ++test_expect_success 'slow status advice when core.untrackedCache true, and fsmonitor' '\n ++\t(\n ++\t\tcd slowstatus &&\n ++\t\tgit config core.untrackedCache true &&\n ++\t\tgit config core.fsmonitor true &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n ++\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tcat >expected <<-\\EOF &&\n ++\t\tOn branch main\n ++\n ++\t\tIt took X seconds to enumerate untracked files,\n ++\t\tbut the results were cached, and subsequent runs may be faster.\n ++\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\n ++\t\tnothing to commit, working tree clean\n ++\t\tEOF\n ++\t\ttest_cmp expected actual\n ++\t)\n +'\n +\n -+test_done\n + test_done\n \n ## wt-status.c ##\n @@\n\n\n Documentation/git-status.txt | 59 +++++++++++++++++++++++++++++\n t/t7508-status.sh | 73 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 28 +++++++++++---\n 3 files changed, 155 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 5e438a7fdc1..570c36e07c1 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,65 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND STATUS SPEED\n+--------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. There is no single optimum set of settings right\n+for everyone. Here is a brief summary of the relevant options\n+to help you choose which is right for you.\n+\n+* First, you may want to run `git status` again. Your current\n+\tconfiguration may already be caching `git status` results,\n+\tso it could be faster on subsequent runs.\n+\n+* The `--untracked-files=no` flag or the\n+\t`status.showUntrackedfiles=false` config (see above for both) :\n+\tindicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n+\tthis config option disables a warning message when the search\n+\tfor untracked files takes longer than desired. In some large\n+\trepositories, this message may appear frequently and not be a\n+\thelpful signal.\n+\n+* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n+\tenable the untracked cache feature and only search directories\n+\tthat have been modified since the previous `git status` command.\n+\tGit remembers the set of untracked files within each directory\n+\tand assumes that if a directory has not been modified, then\n+\tthe set of untracked files within has not changed. This is much\n+\tfaster than enumerating the contents of every directory, but still\n+\tnot without cost, because Git still has to search for the set of\n+\tmodified directories. The untracked cache is stored in the\n+\t`.git/index` file. The reduced cost of searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+\t`core.fsmonitor=` (see\n+\tlinkgit:git-update-index[1]) : enable both the untracked cache\n+\tand FSMonitor features and only search directories that have\n+\tbeen modified since the previous `git status` command. This\n+\tis faster than using just the untracked cache alone because\n+\tGit can also avoid searching for modified directories. Git\n+\tonly has to enumerate the exact set of directories that have\n+\tchanged recently. While the FSMonitor feature can be enabled\n+\twithout the untracked cache, the benefits are greatly reduced\n+\tin that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7508-status.sh b/t/t7508-status.sh\nindex 2b7ef6c41a4..02d641f0413 100755\n--- a/t/t7508-status.sh\n+++ b/t/t7508-status.sh\n@@ -1676,4 +1676,77 @@ test_expect_success 'racy timestamps will be fixed for dirty worktree' '\n \t! test_is_magic_mtime .git/index\n '\n \n+test_expect_success 'setup slow status advice' '\n+\tGIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main git init slowstatus &&\n+\t(\n+\t\tcd slowstatus &&\n+\t\tcat >.gitignore <<-\\EOF &&\n+\t\t/actual\n+\t\t/expected\n+\t\t/out\n+\t\tEOF\n+\t\tgit add .gitignore &&\n+\t\tgit commit -m \"Add .gitignore\" &&\n+\t\tgit config advice.statusuoption true\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache and fsmonitor are unset' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache false &&\n+\t\tgit config core.fsmonitor false &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took X seconds to enumerate untracked files.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache true, but not fsmonitor' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache true &&\n+\t\tgit config core.fsmonitor false &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took X seconds to enumerate untracked files.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache true, and fsmonitor' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache true &&\n+\t\tgit config core.fsmonitor true &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took X seconds to enumerate untracked files,\n+\t\tbut the results were cached, and subsequent runs may be faster.\n+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..1f6d64e759f 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,13 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static int uf_was_slow(uint32_t untracked_in_ms)\n+{\n+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1823,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1880,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s->untracked_in_ms)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but the results were cached, and subsequent runs may be faster.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: 319605f8f00e402f3ea758a02c63534ff800a711\n-- \ngitgitgadget\n"},{"id":"467967","messageId":"xmqq5yf3fx4s.fsf@gitster.g","threadId":"58628","inReplyTo":"pull.1384.v8.git.1669154823035.gitgitgadget@gmail.com","subject":"Re: [PATCH v8] status: modernize git-status \"slow untracked files\" advice","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-11-25T04:58:43Z","receivedAt":"2022-11-25T04:58:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Rudy Rigot via GitGitGadget\" writes:\n\n> From: Rudy Rigot \n>\n> `git status` can be slow when there are a large number of\n> untracked files and directories since Git must search the entire\n> worktree to enumerate them. When it is too slow, Git prints\n> advice with the elapsed search time and a suggestion to disable\n> the search using the `-uno` option. This suggestion also carries\n> a warning that might scare off some users.\n>\n> However, these days, `-uno` isn't the only option. Git can reduce\n> the size and time of the untracked file search when the\n\n\"time\" I can sort of understand (\"can reduce the time taken to\nenumerate untracked files\" is how I may phrase it, though), but\nwhat did you want to say with \"size\"?\n\n> `core.untrackedCache` and `core.fsmonitor` features are enabled by\n> caching results from previous `git status` invocations.\n>\n> Therefore, update the `git status` man page to explain the various\n> configuration options, and update the advice to provide more ...\n\nLose \"Therefore, \"; the resulting text would be much easier to\nfollow.\n\n> +UNTRACKED FILES AND STATUS SPEED\n\n\"STATUS SPEED\" somehow does not sound quite grammatical. Perhaps\n\"untracked files and performance\" or something instead?\n\n> +--------------------------------\n> +\n> +`git status` can be very slow in large worktrees if/when it\n> +needs to search for untracked files and directories. There are\n> +many configuration options available to speed this up by either\n> +avoiding the work or making use of cached results from previous\n> +Git commands. There is no single optimum set of settings right\n> +for everyone. Here is a brief summary of the relevant options\n> +to help you choose which is right for you.\n\nGood.\n\n> +* First, you may want to run `git status` again. Your current\n> +\tconfiguration may already be caching `git status` results,\n> +\tso it could be faster on subsequent runs.\n\nThe above may be a good advice, but it is misleading to make it as\nif it is another alternative of equal footing with everything else\nlisted. It may likely make the resulting text much easier to follow\nif you fold it into \"Here is a summary\", perhaps like...\n\n ... right for everyone. We'll list a summary of the relevant\n options to help you, but before going into the list, you may\n want to run `git status` again, because your configuration may\n already be ...\n\n\n> +* The `--untracked-files=no` flag or the\n> +\t`status.showUntrackedfiles=false` config (see above for both) :\n\nLose the SP before the \":\" (applies to all other entries, too).\n\n> +\tindicate that `git status` should not report untracked\n> +\tfiles. This is the fastest option. `git status` will not list\n> +\tthe untracked files, so you need to be careful to remember if\n> +\tyou create any new files and manually `git add` them.\n\nOK.\n\n> +* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n> +\tthis config option disables a warning message when the search\n> +\tfor untracked files takes longer than desired. In some large\n> +\trepositories, this message may appear frequently and not be a\n> +\thelpful signal.\n\nThis is not technically wrong per-se, except that \"desired\" in\n\"takes longer than desired\" may simply be wrong.\n\nThe reason why the message may not be a \"helpful signal\" is in such\na repository and project the user may have already accepted the\ncurrent trade-off as _desirable_, iow, the user is WILLING to wait\nfor 2 seconds. And in such a case, it indeed is the most sensible\noption to disable the advice.\n\nWe should also stress the fact that this has nothing to do with\nspeeding up, unlike other pieces of advice you are giving here.\nIt's not like disabling the advice will allow us to omit something\nwe need to do to compute the advice (in other words, if the overhead\nto measure the time taken to list untracked files is large, this may\nmatter, but that is hardly the case).\n\nPerhaps\n\n Setting this variable to `false` disables the warning message\n given when enumerating untracked files takes more than 2\n seconds. In a large project, it may take longer and the user\n may have already accepted the trade off (e.g. using \"-uno\" may\n not be an acceptable option for the user), in which case, there\n is no point issuing the warning message, and in such a case,\n disabling the warning may be the best.\n\nor something like that.\n\n> +test_expect_success 'setup slow status advice' '\n> +\tGIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main git init slowstatus &&\n> +\t(\n> +\t\tcd slowstatus &&\n> +\t\tcat >.gitignore <<-\\EOF &&\n> +\t\t/actual\n> +\t\t/expected\n> +\t\t/out\n> +\t\tEOF\n> +\t\tgit add .gitignore &&\n> +\t\tgit commit -m \"Add .gitignore\" &&\n> +\t\tgit config advice.statusuoption true\n> +\t)\n> +'\n> +\n> +test_expect_success 'slow status advice when core.untrackedCache and fsmonitor are unset' '\n> +\t(\n> +\t\tcd slowstatus &&\n> +\t\tgit config core.untrackedCache false &&\n> +\t\tgit config core.fsmonitor false &&\n> +\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n> +\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n\nWhat if it takes more than 10 seconds, e.g.\n\n\t\"It took 92.34 seconds to enumerate...\"\n\nWouldn't it be redacted into \"It took 9X seconds to enumerate\"?\n\nIt probably does not happen, only because you are forcing the code\nto pretend that it took 2.001 seconds or something, I suspect. But\nif you are forcing with GIT_TEST_UF_DELAY_WARNING to pretend that it\ntook some unacceptably long time, it may be more robust to\n\n * pass \"struct wt_status *s\" to uf_was_slow(), instead of passing\n s->untracked_in_ms\n\n * when GIT_TEST_UF_DETAIL_WARNING tells us we are pretending a long\n delay for the purpose of running tests, ASSIGN a known value to\n s->untracked_in_ms\n\n * get rid of \"out\" and use of \"sed\" in these test, and instead\n check for exact output.\n\ne.g.\n\n\tstatic int uf_was_slow(struct wt_status *s)\n\t{\n\t\tif (getenv(\"GIT_TEST_UF_DETAIL_WARNING\"))\n\t\t\ts->untracked_in_ms = 3.25;\n\t\treturn UF_DELAY_WARNING_IN_MS < s->untracked_in_ms;\n\t}\n\nplus\n\n\tGIT_TEST_UF_DETAIL_WARNING=1 git status >actual &&\n\tcat >expect <<-\\EOF &&\n\t...\n\tIt took 3.25 seconds to enumerate ...\n\tEOF\n\ttest_cmp expect actual\n\nAlso, what do you need /g modifier in \"sed\" script for? I do not\nthink we give more than one such number in the message we are\ntesting.\n\n> +\t\tcat >expected <<-\\EOF &&\n> +\t\tOn branch main\n> +\n> +\t\tIt took X seconds to enumerate untracked files.\n> +\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n\nThis is not wrong per-se, but it is more customary to do say:\n\n\t\tSee '\\''git help status'\\'' for information on ...\n\nAll of the comments for this test apply to other two new tests.\n\n> +\t\tnothing to commit, working tree clean\n> +\t\tEOF\n> +\t\ttest_cmp expected actual\n> +\t)\n> +'\n\nAdditionally (read: you do not _have_ to do this to make this topic\nacceptable, but it probably is worth thinking about), if we need to\nintroduce a new helper function uf_was_slow() anyway, a much better\nchange may be to make the 2 seconds cut-off configurable, than\ninventing GIT_TEST_UF_DETAIL_WARNING used only for tests. You can\nintroduce, say, \"status.enumerateUntrackedDelayMS\", a configuration\nvariable that can be set to override the hardcoded 2000 milliseconds\n(i.e. UF_DELAY_WARNING_IN_MS) to control what delay is acceptable\nfor the repository.\n\nThen you can run the tests with the configuration set to a negative\nvalue (i.e. no time is acceptably short, even 0 milliseconds). If\nyou go that route, then you do need to redirect to \"out\" and redact\nwith \"sed\" (make sure you are prepared to see a delay more than 10\nseconds in such a case).\n\nThanks.\n"},{"id":"468192","messageId":"CANaDLW+ukK2GU7NzkCvXVNc9DX3_93Pp+PHq-WcLpRJizPidVA@mail.gmail.com","threadId":"58628","inReplyTo":"xmqq5yf3fx4s.fsf@gitster.g","subject":"Re: [PATCH v8] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-29T15:21:51Z","receivedAt":"2022-11-29T15:22:41Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Thanks for the amazing feedback, and sorry for the delay.\n\n\n> \"time\" I can sort of understand (\"can reduce the time taken to\n> enumerate untracked files\" is how I may phrase it, though), but\n> what did you want to say with \"size\"?\n\nThis bit was provided by a past reviewer so I hope I don't\nmisrepresent it, but I think the idea was to convey that the reason\nit's faster is because the part of the codebase it will do active work\non is smaller. I don't think it provides compellingly more information\nfor end users, compared to just mentioning time, so I'll simplify with\nyour proposal here.\n\n\n> Additionally (read: you do not _have_ to do this to make this topic\n> acceptable, but it probably is worth thinking about), if we need to\n> introduce a new helper function uf_was_slow() anyway, a much better\n> change may be to make the 2 seconds cut-off configurable, than\n> inventing GIT_TEST_UF_DETAIL_WARNING used only for tests.\n\nI agree it would be an improvement, I'm going to try to do this. This\ndoesn't feel like a much more involved change compared to your\nalternative suggestion of setting a hardcoded test value for\n`s->untracked_in_ms`, so it feels like there's not much to lose from\ndoing it this way, while users would gain some nice configurability.\n\nFor transparency, my intuition is that I'm not sure there will be use\ncases where the config will be meaningfully leveraged by users. My gut\nis that the current cut-off time is an arbitrary UX perception\ncut-off, so I'm not sure it would need to be different depending on\ngiven repo situations. But I'm also very aware that I could be wrong,\nand this could open use cases that I'm not thinking about. And to your\npoint, it would be sensible to use it as a test input anyway, so we\nmight as well make it a user-facing tweak; so, I'm on board.\n\n\n> Wouldn't it be redacted into \"It took 9X seconds to enumerate\"?\n> It probably does not happen, only because you are forcing the code\n> to pretend that it took 2.001 seconds or something, I suspect.\n\nYup, you guessed right. I'll be sure to change the regular expression\nto be resilient to double-digit times.\n\n\n> Also, what do you need /g modifier in \"sed\" script for? I do not\n> think we give more than one such number in the message we are\n> testing.\n\nIndeed, it's not useful to anything, and shouldn't be there, I'll fix this too.\n\n\nAll of the other comments are crystal clear, and I intend to implement\nthem exactly as advised. You can expect a new patch this week, maybe\neven today.\n\n\nThanks a lot again!\n"},{"id":"468216","messageId":"CANaDLW+Zuwpk_7jTO5LmWTXDT8LRPPcGARkNtaV6ORioWyZ0tg@mail.gmail.com","threadId":"58628","inReplyTo":"CANaDLW+ukK2GU7NzkCvXVNc9DX3_93Pp+PHq-WcLpRJizPidVA@mail.gmail.com","subject":"Re: [PATCH v8] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-11-30T00:51:25Z","receivedAt":"2022-11-30T00:52:02Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Alright, I tried the \"status.enumerateUntrackedDelayMS\" approach, but\nI couldn't pull it off and now I am stumped.\n\n\nThis is somewhat frustrating, so I'd welcome guidance if anyone has\ntime and is interested. Since this doesn't actually work, I don't\nthink I should create an actual patch for it on the mailing list, so\nhere are two other ways to show what I've got, I hope they're\nacceptable:\n\n- in Gist form:\nhttps://gist.github.com/rudyrigot/aa3e8e5ddb4f71fdc7fc0e92d9b7a4b8\n- in GitHub compare form:\nhttps://github.com/git/git/compare/master...rudyrigot:git:status_enumerateUntrackedDelayMS\n\nThe issues I'm seeing:\n\n- No matter how I set the config from the test, it doesn't seem to\nhave any effect. I'm thinking I might be doing something wrong in how\nI set the value, which I've done in git_status_config in\nbuiltin/commit.c, which very well may be the wrong place.\n- Therefore, I've been testing things by changing the default value in\nwt_status_prepare in wt-status.c. Setting it at 0 and making the\noperator <= instead of < makes the advice display, which tells me that\nthe logic is sound. Setting at its intended value of 2000 doesn't\ndisplay the advice message, as expected. But setting it at -1 also\ndoesn't display it. I'm a bit puzzled about why that would be, and I'm\nwondering: maybe the int is unsigned? It doesn't look like it based on\nhow the structure field is declared in wt-status.h, but I know my own\nlimits in C so I could be wrong.\n\n\nNow, I'm also well aware that Junio raised that advice leaving the\ndoor wide open to not actually solve this as part of this patch; and I\ndid express in my previous reply that I am not intuitively convinced\nthere is much value to it for users, although I could be wrong of\ncourse. So with that, if it's better to let it go, that is fine by me\ntoo.\n\nWith that in mind, I implemented the alternative that Junio was\nproposing instead (assigning the value of `s->untracked_in_ms`), and\nit seems to work all good. It just passed CI, so I'm about to submit\nthat as a patch, with every other piece of feedback also addressed.\n\n\nUnrelated note: I noticed that the first 2 bits of feedback applied to\ndocs that were part of past patches, but were removed in the last\npatch. The rest of the doc feedback was current, so I was able to\nimplement them, but obviously I couldn't implement the first 2 ones,\nsince the issues they're about are gone.\n"},{"id":"468217","messageId":"pull.1384.v9.git.1669769536707.gitgitgadget@gmail.com","threadId":"58628","inReplyTo":"pull.1384.v8.git.1669154823035.gitgitgadget@gmail.com","subject":"[PATCH v9] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2022-11-30T00:52:16Z","receivedAt":"2022-11-30T00:52:49Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"From: Rudy Rigot \n\n`git status` can be slow when there are a large number of\nuntracked files and directories since Git must search the entire\nworktree to enumerate them. When it is too slow, Git prints\nadvice with the elapsed search time and a suggestion to disable\nthe search using the `-uno` option. This suggestion also carries\na warning that might scare off some users.\n\nHowever, these days, `-uno` isn't the only option. Git can reduce\nthe size and time of the untracked file search when the\n`core.untrackedCache` and `core.fsmonitor` features are enabled by\ncaching results from previous `git status` invocations.\n\nTherefore, update the `git status` man page to explain the various\nconfiguration options, and update the advice to provide more\ndetail about the current configuration and to refer to the updated\ndocumentation.\n\nSigned-off-by: Rudy Rigot \n---\n status: modernize git-status \"slow untracked files\" advice\n \n Here is version 9 for this patch.\n \n Changes since v8:\n \n * Improved tests.\n * The untracked files delay measured is now set to always the same\n value in test cases. That has allowed to remove all sed calls from\n tests.\n * Improved documentation.\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-1384%2Frudyrigot%2Fadvice_statusFsmonitor-v9\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1384/rudyrigot/advice_statusFsmonitor-v9\nPull-Request: https://github.com/gitgitgadget/git/pull/1384\n\nRange-diff vs v8:\n\n 1: 16e3721515b ! 1: fcb298e6e5a status: modernize git-status \"slow untracked files\" advice\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n -+UNTRACKED FILES AND STATUS SPEED\n -+--------------------------------\n ++UNTRACKED FILES AND PERFORMANCE\n ++-------------------------------\n +\n +`git status` can be very slow in large worktrees if/when it\n +needs to search for untracked files and directories. There are\n +many configuration options available to speed this up by either\n +avoiding the work or making use of cached results from previous\n +Git commands. There is no single optimum set of settings right\n -+for everyone. Here is a brief summary of the relevant options\n -+to help you choose which is right for you.\n -+\n -+* First, you may want to run `git status` again. Your current\n -+\tconfiguration may already be caching `git status` results,\n -+\tso it could be faster on subsequent runs.\n ++for everyone. We'll list a summary of the relevant options to help\n ++you, but before going into the list, you may want to run `git status`\n ++again, because your configuration may already be caching `git status`\n ++results, so it could be faster on subsequent runs.\n +\n +* The `--untracked-files=no` flag or the\n -+\t`status.showUntrackedfiles=false` config (see above for both) :\n ++\t`status.showUntrackedfiles=false` config (see above for both):\n +\tindicate that `git status` should not report untracked\n +\tfiles. This is the fastest option. `git status` will not list\n +\tthe untracked files, so you need to be careful to remember if\n +\tyou create any new files and manually `git add` them.\n +\n -+* `advice.statusUoption=false` (see linkgit:git-config[1]) :\n -+\tthis config option disables a warning message when the search\n -+\tfor untracked files takes longer than desired. In some large\n -+\trepositories, this message may appear frequently and not be a\n -+\thelpful signal.\n ++* `advice.statusUoption=false` (see linkgit:git-config[1]):\n ++\tsetting this variable to `false` disables the warning message\n ++\tgiven when enumerating untracked files takes more than 2\n ++\tseconds. In a large project, it may take longer and the user\n ++\tmay have already accepted the trade off (e.g. using \"-uno\" may\n ++\tnot be an acceptable option for the user), in which case, there\n ++\tis no point issuing the warning message, and in such a case,\n ++\tdisabling the warning may be the best.\n +\n -+* `core.untrackedCache=true` (see linkgit:git-update-index[1]) :\n ++* `core.untrackedCache=true` (see linkgit:git-update-index[1]):\n +\tenable the untracked cache feature and only search directories\n +\tthat have been modified since the previous `git status` command.\n +\tGit remembers the set of untracked files within each directory\n @@ Documentation/git-status.txt: during the write may conflict with other simultane\n +\n +* `core.untrackedCache=true` and `core.fsmonitor=true` or\n +\t`core.fsmonitor=` (see\n -+\tlinkgit:git-update-index[1]) : enable both the untracked cache\n ++\tlinkgit:git-update-index[1]): enable both the untracked cache\n +\tand FSMonitor features and only search directories that have\n +\tbeen modified since the previous `git status` command. This\n +\tis faster than using just the untracked cache alone because\n @@ t/t7508-status.sh: test_expect_success 'racy timestamps will be fixed for dirty\n +\t\tcd slowstatus &&\n +\t\tgit config core.untrackedCache false &&\n +\t\tgit config core.fsmonitor false &&\n -+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n -+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n +\t\tcat >expected <<-\\EOF &&\n +\t\tOn branch main\n +\n -+\t\tIt took X seconds to enumerate untracked files.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\t\tIt took 3.25 seconds to enumerate untracked files.\n ++\t\tSee '\\''git help status'\\'' for information on how to improve this.\n +\n +\t\tnothing to commit, working tree clean\n +\t\tEOF\n @@ t/t7508-status.sh: test_expect_success 'racy timestamps will be fixed for dirty\n +\t\tcd slowstatus &&\n +\t\tgit config core.untrackedCache true &&\n +\t\tgit config core.fsmonitor false &&\n -+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n -+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n +\t\tcat >expected <<-\\EOF &&\n +\t\tOn branch main\n +\n -+\t\tIt took X seconds to enumerate untracked files.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\t\tIt took 3.25 seconds to enumerate untracked files.\n ++\t\tSee '\\''git help status'\\'' for information on how to improve this.\n +\n +\t\tnothing to commit, working tree clean\n +\t\tEOF\n @@ t/t7508-status.sh: test_expect_success 'racy timestamps will be fixed for dirty\n +\t\tcd slowstatus &&\n +\t\tgit config core.untrackedCache true &&\n +\t\tgit config core.fsmonitor true &&\n -+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n -+\t\tsed \"s/[0-9]\\.[0-9][0-9]/X/g\" out >actual &&\n ++\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n +\t\tcat >expected <<-\\EOF &&\n +\t\tOn branch main\n +\n -+\t\tIt took X seconds to enumerate untracked files,\n ++\t\tIt took 3.25 seconds to enumerate untracked files,\n +\t\tbut the results were cached, and subsequent runs may be faster.\n -+\t\tSee '\"'\"'git help status'\"'\"' for information on how to improve this.\n ++\t\tSee '\\''git help status'\\'' for information on how to improve this.\n +\n +\t\tnothing to commit, working tree clean\n +\t\tEOF\n @@ wt-status.c: static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n -+static int uf_was_slow(uint32_t untracked_in_ms)\n ++static int uf_was_slow(struct wt_status *s)\n +{\n +\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n -+\t\tuntracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;\n -+\treturn UF_DELAY_WARNING_IN_MS < untracked_in_ms;\n ++\t\ts->untracked_in_ms = 3250;\n ++\treturn UF_DELAY_WARNING_IN_MS < s->untracked_in_ms;\n +}\n +\n static void show_merge_in_progress(struct wt_status *s,\n @@ wt-status.c: static void wt_longstatus_print(struct wt_status *s)\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n -\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n -+\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s->untracked_in_ms)) {\n ++\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n +\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n +\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n\n\n Documentation/git-status.txt | 60 +++++++++++++++++++++++++++++++\n t/t7508-status.sh | 70 ++++++++++++++++++++++++++++++++++++\n wt-status.c | 28 ++++++++++++---\n 3 files changed, 153 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-status.txt b/Documentation/git-status.txt\nindex 5e438a7fdc1..a051b1e8f38 100644\n--- a/Documentation/git-status.txt\n+++ b/Documentation/git-status.txt\n@@ -457,6 +457,66 @@ during the write may conflict with other simultaneous processes, causing\n them to fail. Scripts running `status` in the background should consider\n using `git --no-optional-locks status` (see linkgit:git[1] for details).\n \n+UNTRACKED FILES AND PERFORMANCE\n+-------------------------------\n+\n+`git status` can be very slow in large worktrees if/when it\n+needs to search for untracked files and directories. There are\n+many configuration options available to speed this up by either\n+avoiding the work or making use of cached results from previous\n+Git commands. There is no single optimum set of settings right\n+for everyone. We'll list a summary of the relevant options to help\n+you, but before going into the list, you may want to run `git status`\n+again, because your configuration may already be caching `git status`\n+results, so it could be faster on subsequent runs.\n+\n+* The `--untracked-files=no` flag or the\n+\t`status.showUntrackedfiles=false` config (see above for both):\n+\tindicate that `git status` should not report untracked\n+\tfiles. This is the fastest option. `git status` will not list\n+\tthe untracked files, so you need to be careful to remember if\n+\tyou create any new files and manually `git add` them.\n+\n+* `advice.statusUoption=false` (see linkgit:git-config[1]):\n+\tsetting this variable to `false` disables the warning message\n+\tgiven when enumerating untracked files takes more than 2\n+\tseconds. In a large project, it may take longer and the user\n+\tmay have already accepted the trade off (e.g. using \"-uno\" may\n+\tnot be an acceptable option for the user), in which case, there\n+\tis no point issuing the warning message, and in such a case,\n+\tdisabling the warning may be the best.\n+\n+* `core.untrackedCache=true` (see linkgit:git-update-index[1]):\n+\tenable the untracked cache feature and only search directories\n+\tthat have been modified since the previous `git status` command.\n+\tGit remembers the set of untracked files within each directory\n+\tand assumes that if a directory has not been modified, then\n+\tthe set of untracked files within has not changed. This is much\n+\tfaster than enumerating the contents of every directory, but still\n+\tnot without cost, because Git still has to search for the set of\n+\tmodified directories. The untracked cache is stored in the\n+\t`.git/index` file. The reduced cost of searching for untracked\n+\tfiles is offset slightly by the increased size of the index and\n+\tthe cost of keeping it up-to-date. That reduced search time is\n+\tusually worth the additional size.\n+\n+* `core.untrackedCache=true` and `core.fsmonitor=true` or\n+\t`core.fsmonitor=` (see\n+\tlinkgit:git-update-index[1]): enable both the untracked cache\n+\tand FSMonitor features and only search directories that have\n+\tbeen modified since the previous `git status` command. This\n+\tis faster than using just the untracked cache alone because\n+\tGit can also avoid searching for modified directories. Git\n+\tonly has to enumerate the exact set of directories that have\n+\tchanged recently. While the FSMonitor feature can be enabled\n+\twithout the untracked cache, the benefits are greatly reduced\n+\tin that case.\n+\n+Note that after you turn on the untracked cache and/or FSMonitor\n+features it may take a few `git status` commands for the various\n+caches to warm up before you see improved command times. This is\n+normal.\n+\n SEE ALSO\n --------\n linkgit:gitignore[5]\ndiff --git a/t/t7508-status.sh b/t/t7508-status.sh\nindex 2b7ef6c41a4..aed07c5b622 100755\n--- a/t/t7508-status.sh\n+++ b/t/t7508-status.sh\n@@ -1676,4 +1676,74 @@ test_expect_success 'racy timestamps will be fixed for dirty worktree' '\n \t! test_is_magic_mtime .git/index\n '\n \n+test_expect_success 'setup slow status advice' '\n+\tGIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main git init slowstatus &&\n+\t(\n+\t\tcd slowstatus &&\n+\t\tcat >.gitignore <<-\\EOF &&\n+\t\t/actual\n+\t\t/expected\n+\t\t/out\n+\t\tEOF\n+\t\tgit add .gitignore &&\n+\t\tgit commit -m \"Add .gitignore\" &&\n+\t\tgit config advice.statusuoption true\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache and fsmonitor are unset' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache false &&\n+\t\tgit config core.fsmonitor false &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took 3.25 seconds to enumerate untracked files.\n+\t\tSee '\\''git help status'\\'' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache true, but not fsmonitor' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache true &&\n+\t\tgit config core.fsmonitor false &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took 3.25 seconds to enumerate untracked files.\n+\t\tSee '\\''git help status'\\'' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n+test_expect_success 'slow status advice when core.untrackedCache true, and fsmonitor' '\n+\t(\n+\t\tcd slowstatus &&\n+\t\tgit config core.untrackedCache true &&\n+\t\tgit config core.fsmonitor true &&\n+\t\tGIT_TEST_UF_DELAY_WARNING=1 git status >actual &&\n+\t\tcat >expected <<-\\EOF &&\n+\t\tOn branch main\n+\n+\t\tIt took 3.25 seconds to enumerate untracked files,\n+\t\tbut the results were cached, and subsequent runs may be faster.\n+\t\tSee '\\''git help status'\\'' for information on how to improve this.\n+\n+\t\tnothing to commit, working tree clean\n+\t\tEOF\n+\t\ttest_cmp expected actual\n+\t)\n+'\n+\n test_done\ndiff --git a/wt-status.c b/wt-status.c\nindex 5813174896c..b430d25da43 100644\n--- a/wt-status.c\n+++ b/wt-status.c\n@@ -18,8 +18,10 @@\n #include \"worktree.h\"\n #include \"lockfile.h\"\n #include \"sequencer.h\"\n+#include \"fsmonitor-settings.h\"\n \n #define AB_DELAY_WARNING_IN_MS (2 * 1000)\n+#define UF_DELAY_WARNING_IN_MS (2 * 1000)\n \n static const char cut_line[] =\n \"------------------------ >8 ------------------------\\n\";\n@@ -1205,6 +1207,13 @@ static void wt_longstatus_print_tracking(struct wt_status *s)\n \tstrbuf_release(&sb);\n }\n \n+static int uf_was_slow(struct wt_status *s)\n+{\n+\tif (getenv(\"GIT_TEST_UF_DELAY_WARNING\"))\n+\t\ts->untracked_in_ms = 3250;\n+\treturn UF_DELAY_WARNING_IN_MS < s->untracked_in_ms;\n+}\n+\n static void show_merge_in_progress(struct wt_status *s,\n \t\t\t\t const char *color)\n {\n@@ -1814,6 +1823,7 @@ static void wt_longstatus_print(struct wt_status *s)\n {\n \tconst char *branch_color = color(WT_STATUS_ONBRANCH, s);\n \tconst char *branch_status_color = color(WT_STATUS_HEADER, s);\n+\tenum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);\n \n \tif (s->branch) {\n \t\tconst char *on_what = _(\"On branch \");\n@@ -1870,13 +1880,21 @@ static void wt_longstatus_print(struct wt_status *s)\n \t\twt_longstatus_print_other(s, &s->untracked, _(\"Untracked files\"), \"add\");\n \t\tif (s->show_ignored_mode)\n \t\t\twt_longstatus_print_other(s, &s->ignored, _(\"Ignored files\"), \"add -f\");\n-\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {\n+\t\tif (advice_enabled(ADVICE_STATUS_U_OPTION) && uf_was_slow(s)) {\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n+\t\t\tif (fsm_mode > FSMONITOR_MODE_DISABLED) {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files,\\n\"\n+\t\t\t\t\t\t\"but the results were cached, and subsequent runs may be faster.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t} else {\n+\t\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n+\t\t\t\t\t\t_(\"It took %.2f seconds to enumerate untracked files.\"),\n+\t\t\t\t\t\ts->untracked_in_ms / 1000.0);\n+\t\t\t}\n \t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL,\n-\t\t\t\t\t _(\"It took %.2f seconds to enumerate untracked files. 'status -uno'\\n\"\n-\t\t\t\t\t \"may speed it up, but you have to be careful not to forget to add\\n\"\n-\t\t\t\t\t \"new files yourself (see 'git help status').\"),\n-\t\t\t\t\t s->untracked_in_ms / 1000.0);\n+\t\t\t\t\t_(\"See 'git help status' for information on how to improve this.\"));\n+\t\t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, \"%s\", \"\");\n \t\t}\n \t} else if (s->committable)\n \t\tstatus_printf_ln(s, GIT_COLOR_NORMAL, _(\"Untracked files not listed%s\"),\n\nbase-commit: 319605f8f00e402f3ea758a02c63534ff800a711\n-- \ngitgitgadget\n"},{"id":"468302","messageId":"xmqq4juftyan.fsf@gitster.g","threadId":"58628","inReplyTo":"pull.1384.v9.git.1669769536707.gitgitgadget@gmail.com","subject":"Re: [PATCH v9] status: modernize git-status \"slow untracked files\" advice","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-12-01T06:48:00Z","receivedAt":"2022-12-01T06:48:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Rudy Rigot via GitGitGadget\" writes:\n\n> From: Rudy Rigot \n>\n> `git status` can be slow when there are a large number of\n> untracked files and directories since Git must search the entire\n> worktree to enumerate them. When it is too slow, Git prints\n> advice with the elapsed search time and a suggestion to disable\n> the search using the `-uno` option. This suggestion also carries\n> a warning that might scare off some users.\n>\n> However, these days, `-uno` isn't the only option. Git can reduce\n> the size and time of the untracked file search when the\n> `core.untrackedCache` and `core.fsmonitor` features are enabled by\n> caching results from previous `git status` invocations.\n>\n> Therefore, update the `git status` man page to explain the various\n> configuration options, and update the advice to provide more\n> detail about the current configuration and to refer to the updated\n> documentation.\n>\n> Signed-off-by: Rudy Rigot \n> ---\n> status: modernize git-status \"slow untracked files\" advice\n> \n> Here is version 9 for this patch.\n> \n> Changes since v8:\n> \n> * Improved tests.\n> * The untracked files delay measured is now set to always the same\n> value in test cases. That has allowed to remove all sed calls from\n> tests.\n> * Improved documentation.\n\nLooking pretty good (I thought you were going to update the proposed\nlog message, too, though). Let's replace and merge it down to 'next'\nto cook during the pre-release freeze period.\n\nThanks.\n"},{"id":"468325","messageId":"CANaDLWLdkaJs96KTBA2B-h+Ei+f7ayS-gvvN06Y4T7w=GgPGrg@mail.gmail.com","threadId":"58628","inReplyTo":"xmqq4juftyan.fsf@gitster.g","subject":"Re: [PATCH v9] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-12-01T15:16:46Z","receivedAt":"2022-12-01T15:17:12Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"> (I thought you were going to update the proposed\n> log message, too, though)\n\nOh, I'm sorry, what do you mean by updating the proposed log message?\nI'm looking back at past feedback, and I can't seem to match it to an\nunaddressed one. It was not intentional, so I would be very glad to\nfix it if you'd like, I'll gladly leave it up to you.\n\nThanks a lot!\n"},{"id":"468360","messageId":"xmqqedtispyc.fsf@gitster.g","threadId":"58628","inReplyTo":"CANaDLWLdkaJs96KTBA2B-h+Ei+f7ayS-gvvN06Y4T7w=GgPGrg@mail.gmail.com","subject":"Re: [PATCH v9] status: modernize git-status \"slow untracked files\" advice","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-12-01T22:45:47Z","receivedAt":"2022-12-01T22:45:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Rudy Rigot writes:\n\n>> (I thought you were going to update the proposed\n>> log message, too, though)\n>\n> Oh, I'm sorry, what do you mean by updating the proposed log message?\n> I'm looking back at past feedback, and I can't seem to match it to an\n> unaddressed one. It was not intentional, so I would be very glad to\n> fix it if you'd like, I'll gladly leave it up to you.\n>\n> Thanks a lot!\n\nI was referring to this part:\n\n>> ... I don't think it provides compellingly more information\n>> for end users, compared to just mentioning time, so I'll simplify with\n>> your proposal here.\n\nin\n\n https://lore.kernel.org/git/CANaDLW+ukK2GU7NzkCvXVNc9DX3_93Pp+PHq-WcLpRJizPidVA@mail.gmail.com/\n\nyou sent earlier.\n\nNo big deal, and thanks for working on this.\n\n\n"},{"id":"468365","messageId":"CANaDLW+aMk3ph25VohqOve6wC46+Zig1a=UtUYHTAwX1K6bDYw@mail.gmail.com","threadId":"58628","inReplyTo":"xmqqedtispyc.fsf@gitster.g","subject":"Re: [PATCH v9] status: modernize git-status \"slow untracked files\" advice","fromName":"Rudy Rigot","fromEmail":"rudy.rigot@gmail.com","sentAt":"2022-12-01T22:57:09Z","receivedAt":"2022-12-01T22:57:26Z","isPatch":true,"sender":{"key":"rudy.rigot@gmail.com","avatar":"https://avatars.githubusercontent.com/u/552279?v=4"},"body":"Oh, now I understand!\n\nBut I had skipped it because I couldn't find it in my patch, and even\nnow, I'm afraid I can't. It looks to me like part of an old patch, a\nparagraph that was rewritten 2 patches back. Is it possible that you\nfound it in the lines that were being removed on the patch that you\ncommented on, for instance?\n\nIf I'm correct, then yeay! No problem in the end, this won't reach users.\nIf I'm looking wrong though, then I'm very sorry about the miss. I\nalso do get how this may not be worth a new round just for this fix.\n\nThanks a lot for your (and everybody else's) guidance through this.\nThis was my first patch, and I'm delighted about the solid shape it's\nlanding with; I'm hoping it is the first of many.\n\nThanks a lot!\n"},{"id":"477022","messageId":"CAPig+cR+mNDT1NX1dSp92UAvvFwoHGU_D60MSnVkMmuwSx=fhw@mail.gmail.com","threadId":"58628","inReplyTo":"pull.1384.v8.git.1669154823035.gitgitgadget@gmail.com","subject":"Re: [PATCH v8] status: modernize git-status \"slow untracked files\" advice","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2023-05-11T05:17:56Z","receivedAt":"2023-05-11T05:18:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Nov 22, 2022 at 5:07 PM Rudy Rigot via GitGitGadget\n wrote:\n> `git status` can be slow when there are a large number of\n> untracked files and directories since Git must search the entire\n> worktree to enumerate them. When it is too slow, Git prints\n> advice with the elapsed search time and a suggestion to disable\n> the search using the `-uno` option. This suggestion also carries\n> a warning that might scare off some users.\n>\n> However, these days, `-uno` isn't the only option. Git can reduce\n> the size and time of the untracked file search when the\n> `core.untrackedCache` and `core.fsmonitor` features are enabled by\n> caching results from previous `git status` invocations.\n>\n> Therefore, update the `git status` man page to explain the various\n> configuration options, and update the advice to provide more\n> detail about the current configuration and to refer to the updated\n> documentation.\n>\n> Signed-off-by: Rudy Rigot \n> ---\n> Changes since v7:\n>\n> * Moved tests from new test script to existing one, in order not to\n> needlessly waste a test script number for such a small feature. Two\n> caveats:\n> * The use of test_config in a subshell result in: 'error: bug in the\n> test script: test_when_finished does nothing in a subshell', so\n> I've had to resort to using plain old git config instead.\n\nSorry, I should have thought of this when making suggestions in my\nearlier reviews. For completeness, in case you run across this sort of\nissue again if submitting patches in the future, it is possible to get\nthis working by using the -C option (change to directory) with\ntest_config() outside of the subshell. For instance:\n\n test_expect_sucess 'some title' '\n test_config -C slowstatus core.untrackedCache false &&\n test_config -C slowstatus core.fsmonitor false &&\n (\n cd slowstatus &&\n GIT_TEST_UF_DELAY_WARNING=1 git status >out &&\n ...\n )\n '\n"}]}