git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3] status: long status advice adapted to recent capabilities

From
Derrick Stolee <derrickstolee@github.com>
Date
Nov 7, 2022, 20:01 UTC
Message-ID
<0e2de99a-da7e-f65f-aefe-117fb468ce55@github.com>
In-Reply-To
<pull.1384.v3.git.1667424467505.gitgitgadget@gmail.com>
On 11/2/22 5:27 PM, Rudy Rigot via GitGitGadget wrote:> +UNTRACKED FILES AND STATUS SPEED
Show 12 quoted lines
> +--------------------------------
> +
> +`git status` can be very slow in large worktrees if/when it
> +needs to search for untracked files and directories.  There are
> +many configuration options available to speed this up by either
> +avoiding the work or making use of cached results from previous
> +Git commands.  Since we all work in different ways, there is no
> +single optimum set of settings right for everyone.  Here is a
> +brief summary of the relevant options to help you choose which
> +is right for you.  Each of these settings is independently
> +documented elsewhere in more detail, so please refer to them
> +for complete details.
Sorry I'm late to this series. This new section is a great idea.
> +* `-uno` or `status.showUntrackedFiles=false` : just don't search
Two nits:
1. Is it clear that `-uno` is an option to `git status`? Should
   we say "The `-uno` flag or the `status.showUntrackedfiles=false`
   config"?
2. Drop the "just" as it implies simplicity but is unnecessary. In
   fact, I'd replace the sentence with:
   Indicate that `git status` should not report untracked files.
> +    and don't report on untracked files.  This is the fastest.
  "This is the fastest option." ?
Show 6 quoted lines
> +    `git status` will not list the untracked files, so you need
> +    to be careful to remember if you create any new files and
> +    manually `git add` them.
> +
> +* `advice.statusUoption=false` : search, but don't complain if it
> +    takes too long.
Perhaps:
	This config option disables a warning message when the
	search for untracked files takes longer than desired.
and maybe even describe why:
	In some large repositories, this message may appear
	frequently and not be a helpful signal.
Show 9 quoted lines
> +* `core.untrackedCache=true` : enable the untracked cache feature
> +    and only search directories that have been modified since the
> +    previous `git status` command.  Git remembers the set of
> +    untracked files within each directory and assumes that if a
> +    directory has not been modified, then the set of untracked
> +    file within has not changed.  This is much faster than
> +    enumerating the contents of every directory, but still not
> +    without cost, because Git still has to search for the set of
> +    modified directories.

It might be helpful to mention that the untracked cache is stored in the .git/index file. The reduced cost searching for untracked files is offset slightly by the increased size of the index and the cost of keeping it up-to-date. That reduced search time is usually worth the additional size.

Show 8 quoted lines
> +* `core.untrackedCache=true` and `core.fsmonitor=true` or
> +    `core.fsmonitor=<hook_command_pathname>` : enable both the
> +    untracked cache and FSMonitor features and only search
> +    directories that have been modified since the previous
> +    `git status` command.  This is faster than using just the
> +    untracked cache alone because Git can also avoid searching
> +    for modified directories.  Git only has to enumerate the
> +    exact set of directories that have changed recently.

It might be worth explicitly mentioning that while the FSMonitor feature can be enabled without the untracked cache, the benefits are greatly reduced in that case.

Show 57 quoted lines
> +Note that after you turn on the untracked cache and/or FSMonitor
> +features it may take a few `git status` commands for the various
> +caches to warm up before you see improved command times.  This is
> +normal.
> +
>  SEE ALSO
>  --------
>  linkgit:gitignore[5]
> diff --git a/t/t7065-wtstatus-slow.sh b/t/t7065-wtstatus-slow.sh
> new file mode 100755
> index 00000000000..23c37ea71e7
> --- /dev/null
> +++ b/t/t7065-wtstatus-slow.sh
> @@ -0,0 +1,40 @@
> +#!/bin/sh
> +
> +test_description='test status when slow untracked files'
> +
> +. ./test-lib.sh
> +
> +DATA="$TEST_DIRECTORY/t7065"
> +
> +GIT_TEST_UF_DELAY_WARNING=1
> +export GIT_TEST_UF_DELAY_WARNING
> +
> +test_expect_success setup '
> +	git checkout -b test
> +'
> +
> +test_expect_success 'when core.untrackedCache and fsmonitor are unset' '
> +	test_must_fail git config --get core.untrackedCache &&
> +	test_must_fail git config --get core.fsmonitor &&
> +    git status | sed "s/[0-9]\.[0-9][0-9]/X/g" >../actual &&
> +    test_cmp "$DATA/no_untrackedcache_no_fsmonitor" ../actual &&
> +    rm -fr ../actual
> +'
> +
> +test_expect_success 'when core.untrackedCache true, but not fsmonitor' '
> +    git config core.untrackedCache true &&
> +	test_must_fail git config --get core.fsmonitor &&
> +    git status | sed "s/[0-9]\.[0-9][0-9]/X/g" >../actual &&
> +    test_cmp "$DATA/with_untrackedcache_no_fsmonitor" ../actual &&
> +    rm -fr ../actual
> +'
> +
> +test_expect_success 'when core.untrackedCache true, and fsmonitor' '
> +    git config core.untrackedCache true &&
> +	git config core.fsmonitor true &&
> +    git status | sed "s/[0-9]\.[0-9][0-9]/X/g" >../actual &&
> +    test_cmp "$DATA/with_untrackedcache_with_fsmonitor" ../actual &&
> +    rm -fr ../actual
> +'
> +
> +test_done
> diff --git a/t/t7065/no_untrackedcache_no_fsmonitor b/t/t7065/no_untrackedcache_no_fsmonitor
> diff --git a/t/t7065/with_untrackedcache_no_fsmonitor b/t/t7065/with_untrackedcache_no_fsmonitor
> diff --git a/t/t7065/with_untrackedcache_with_fsmonitor b/t/t7065/with_untrackedcache_with_fsmonitor

These files are small enough that I think I'd rather see them be created in the test script. You can use this kind of syntax:

	cat >expect <<-\EOF
	<file contents here>
	EOF &&

and then the test is self-contained. This is particularly helpful when a test fails due to some future change in this message.

Show 35 quoted lines
> +static inline int uf_was_slow(uint32_t untracked_in_ms)
> +{
> +	const char *x;
> +	x = getenv("GIT_TEST_UF_DELAY_WARNING");
> +	if (x) {
> +		untracked_in_ms += UF_DELAY_WARNING_IN_MS + 1;
> +	}
> +
> +	return UF_DELAY_WARNING_IN_MS < untracked_in_ms;
> +}
> +
>  static void show_merge_in_progress(struct wt_status *s,
>  				   const char *color)
>  {
> @@ -1814,6 +1827,7 @@ static void wt_longstatus_print(struct wt_status *s)
>  {
>  	const char *branch_color = color(WT_STATUS_ONBRANCH, s);
>  	const char *branch_status_color = color(WT_STATUS_HEADER, s);
> +	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(s->repo);
>  
>  	if (s->branch) {
>  		const char *on_what = _("On branch ");
> @@ -1870,13 +1884,23 @@ static void wt_longstatus_print(struct wt_status *s)
>  		wt_longstatus_print_other(s, &s->untracked, _("Untracked files"), "add");
>  		if (s->show_ignored_mode)
>  			wt_longstatus_print_other(s, &s->ignored, _("Ignored files"), "add -f");
> -		if (advice_enabled(ADVICE_STATUS_U_OPTION) && 2000 < s->untracked_in_ms) {
> -			status_printf_ln(s, GIT_COLOR_NORMAL, "%s", "");
> -			status_printf_ln(s, GIT_COLOR_NORMAL,
> -					 _("It took %.2f seconds to enumerate untracked files. 'status -uno'\n"
> -					   "may speed it up, but you have to be careful not to forget to add\n"
> -					   "new files yourself (see 'git help status')."),
> -					 s->untracked_in_ms / 1000.0);
> +		if (uf_was_slow(s->untracked_in_ms)) {
> +			if (advice_enabled(ADVICE_STATUS_U_OPTION)) {

Since there isn't an "else" for either of these cases, they can be combined with "&&" in the top-level if, reducing the tab depth for the body.

Show 16 quoted lines
> +				status_printf_ln(s, GIT_COLOR_NORMAL, "%s", "");
> +				if (fsm_mode > FSMONITOR_MODE_DISABLED) {
> +					status_printf_ln(s, GIT_COLOR_NORMAL,
> +							_("It took %.2f seconds to enumerate untracked files,\n"
> +							"but this is currently being cached."),
> +							s->untracked_in_ms / 1000.0);
> +				} else {
> +					status_printf_ln(s, GIT_COLOR_NORMAL,
> +							_("It took %.2f seconds to enumerate untracked files."),
> +							s->untracked_in_ms / 1000.0);
> +				}
> +				status_printf_ln(s, GIT_COLOR_NORMAL,
> +						_("See 'git help status' for information on how to improve this."));
> +				status_printf_ln(s, GIT_COLOR_NORMAL, "%s", "");
> +			}
>  		}
other than that nit, the code looks good to me.

Thanks for working on this! -Stolee

Previous: Jeff HostetlerNext: Eric Sunshine
Message 21 of 58 in “fsmonitor: long status advice adapted to the fsmonitor use case”
  1. fsmonitor: long status advice adapted to the fsmonitor use caseRudy Rigot via GitGitGadget, Oct 15, 2022
  2. Rudy RigotOct 15, 2022
  3. Jeff HostetlerOct 17, 2022
  4. Rudy RigotOct 17, 2022
  5. Jeff HostetlerOct 20, 2022
  6. Rudy RigotOct 20, 2022
  7. Jeff HostetlerOct 24, 2022
  8. status: long status advice adapted to recent capabilitiesRudy Rigot via GitGitGadget, Oct 29, 2022
  9. Jeff HostetlerNov 2, 2022
  10. Rudy RigotNov 2, 2022
  11. Taylor BlauNov 2, 2022
  12. Rudy RigotNov 3, 2022
  13. Ævar Arnfjörð BjarmasonNov 4, 2022
  14. Rudy RigotNov 4, 2022
  15. Taylor BlauNov 4, 2022
  16. status: long status advice adapted to recent capabilitiesRudy Rigot via GitGitGadget, Nov 2, 2022
  17. Taylor BlauNov 4, 2022
  18. Derrick StoleeNov 7, 2022
  19. Taylor BlauNov 7, 2022
  20. Jeff HostetlerNov 15, 2022
  21. Derrick StoleeNov 7, 2022
  22. Eric SunshineNov 7, 2022
  23. Rudy RigotNov 7, 2022
  24. status: long status advice adapted to recent capabilitiesRudy Rigot via GitGitGadget, Nov 10, 2022
  25. Eric SunshineNov 10, 2022
  26. Rudy RigotNov 10, 2022
  27. Eric SunshineNov 10, 2022
  28. Rudy RigotNov 10, 2022
  29. status: long status advice adapted to recent capabilitiesRudy Rigot via GitGitGadget, Nov 10, 2022
  30. Jeff HostetlerNov 15, 2022
  31. Rudy RigotNov 15, 2022
  32. Eric SunshineNov 15, 2022
  33. Rudy RigotNov 15, 2022
  34. Eric SunshineNov 15, 2022
  35. Rudy RigotNov 15, 2022
  36. status: long status advice adapted to recent capabilitiesRudy Rigot via GitGitGadget, Nov 15, 2022
  37. Eric SunshineNov 21, 2022
  38. Rudy RigotNov 21, 2022
  39. Eric SunshineNov 21, 2022
  40. Rudy RigotNov 22, 2022
  41. Eric SunshineNov 22, 2022
  42. Eric SunshineNov 22, 2022
  43. Rudy RigotNov 22, 2022
  44. Eric SunshineNov 22, 2022
  45. Eric SunshineNov 22, 2022
  46. Rudy RigotNov 22, 2022
  47. Eric SunshineNov 22, 2022
  48. status: modernize git-status "slow untracked files" adviceRudy Rigot via GitGitGadget, Nov 22, 2022
  49. status: modernize git-status "slow untracked files" adviceRudy Rigot via GitGitGadget, Nov 22, 2022
  50. Junio C HamanoNov 25, 2022
  51. Rudy RigotNov 29, 2022
  52. Rudy RigotNov 30, 2022
  53. status: modernize git-status "slow untracked files" adviceRudy Rigot via GitGitGadget, Nov 30, 2022
  54. Junio C HamanoDec 1, 2022
  55. Rudy RigotDec 1, 2022
  56. Junio C HamanoDec 1, 2022
  57. Rudy RigotDec 1, 2022
  58. Eric SunshineMay 11, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.