git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 06/11] read-cache: log the number of lstat calls to trace2

From
JGJeff Hostetler via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 3, 2021, 15:34 UTC
Message-ID
<d150a2d4576e1519e3a4b228a30ad3c3e63fa150.1612366490.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.860.v2.git.1612366490.gitgitgadget@gmail.com>
From: Jeff Hostetler <jeffhost@microsoft.com>
Report the total number of calls made to lstat() inside of refresh_index().

FSMonitor improves the performance of commands like `git status` by avoiding scanning the disk for changed files. This can be seen in `refresh_index()`. Let's measure this.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 read-cache.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index ecf6f689940..893cc41e1d9 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1364,7 +1364,8 @@ int add_index_entry(struct index_state *istate, struct cache_entry *ce, int opti
 static struct cache_entry *refresh_cache_ent(struct index_state *istate,
 					     struct cache_entry *ce,
 					     unsigned int options, int *err,
-					     int *changed_ret)
+					     int *changed_ret,
+					     int *t2_did_lstat)
 {
 	struct stat st;
 	struct cache_entry *updated;
@@ -1406,6 +1407,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,
 		return NULL;
 	}
 
+	if (t2_did_lstat)
+		*t2_did_lstat = 1;
 	if (lstat(ce->name, &st) < 0) {
 		if (ignore_missing && errno == ENOENT)
 			return ce;
@@ -1519,6 +1522,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 	const char *added_fmt;
 	const char *unmerged_fmt;
 	struct progress *progress = NULL;
+	int t2_sum_lstat = 0;
 
 	if (flags & REFRESH_PROGRESS && isatty(2))
 		progress = start_delayed_progress(_("Refresh index"),
@@ -1536,11 +1540,13 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 	 * we only have to do the special cases that are left.
 	 */
 	preload_index(istate, pathspec, 0);
+	trace2_region_enter("index", "refresh", NULL);
 	for (i = 0; i < istate->cache_nr; i++) {
 		struct cache_entry *ce, *new_entry;
 		int cache_errno = 0;
 		int changed = 0;
 		int filtered = 0;
+		int t2_did_lstat = 0;
 
 		ce = istate->cache[i];
 		if (ignore_submodules && S_ISGITLINK(ce->ce_mode))
@@ -1566,7 +1572,10 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 		if (filtered)
 			continue;
 
-		new_entry = refresh_cache_ent(istate, ce, options, &cache_errno, &changed);
+		new_entry = refresh_cache_ent(istate, ce, options,
+					      &cache_errno, &changed,
+					      &t2_did_lstat);
+		t2_sum_lstat += t2_did_lstat;
 		if (new_entry == ce)
 			continue;
 		if (progress)
@@ -1602,6 +1611,8 @@ int refresh_index(struct index_state *istate, unsigned int flags,
 
 		replace_index_entry(istate, i, new_entry);
 	}
+	trace2_data_intmax("index", NULL, "refresh/sum_lstat", t2_sum_lstat);
+	trace2_region_leave("index", "refresh", NULL);
 	if (progress) {
 		display_progress(progress, istate->cache_nr);
 		stop_progress(&progress);
@@ -1614,7 +1625,7 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate,
 					struct cache_entry *ce,
 					unsigned int options)
 {
-	return refresh_cache_ent(istate, ce, options, NULL, NULL);
+	return refresh_cache_ent(istate, ce, options, NULL, NULL, NULL);
 }
 
 
-- 
gitgitgadget
Previous: Jeff Hostetler via GitGitGadgetNext: Jeff Hostetler via GitGitGadget
Message 21 of 30 in “FSMonitor Preliminary Commits”
  1. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 1, 2021
  2. 01/11 p7519: use xargs -0 rather than -d in testJeff Hostetler via GitGitGadget, Feb 1, 2021
  3. Junio C HamanoFeb 1, 2021
  4. Jeff HostetlerFeb 2, 2021
  5. Junio C HamanoFeb 2, 2021
  6. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 1, 2021
  7. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 1, 2021
  8. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 1, 2021
  9. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 1, 2021
  10. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 1, 2021
  11. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  12. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  13. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 1, 2021
  14. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  15. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  16. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 3, 2021
  17. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 3, 2021
  18. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 3, 2021
  19. 01/11 p7519: do not rely on "xargs -d" in testJeff Hostetler via GitGitGadget, Feb 3, 2021
  20. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 3, 2021
  21. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  22. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  23. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 3, 2021
  24. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 3, 2021
  25. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 3, 2021
  26. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  27. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  28. Jeff HostetlerFeb 16, 2021
  29. Junio C HamanoFeb 17, 2021
  30. Taylor BlauFeb 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.