git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 05/11] preload-index: log the number of lstat calls to trace2

From
JGJeff Hostetler via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 1, 2021, 22:02 UTC
Message-ID
<cdd49f1fdb18b85430c39f65d2a85890abf927e7.1612216941.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.860.git.1612216941.gitgitgadget@gmail.com>
From: Jeff Hostetler <jeffhost@microsoft.com>
Report the total number of calls made to lstat() inside preload_index().

FSMonitor improves the performance of commands like `git status` by avoiding scanning the disk for changed files. This can be seen in `preload_index()`. Let's measure this.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 preload-index.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
diff --git a/preload-index.c b/preload-index.c
index ed6eaa47388..e5529a58636 100644
--- a/preload-index.c
+++ b/preload-index.c
@@ -31,6 +31,7 @@ struct thread_data {
 	struct pathspec pathspec;
 	struct progress_data *progress;
 	int offset, nr;
+	int t2_nr_lstat;
 };
 
 static void *preload_thread(void *_data)
@@ -73,6 +74,7 @@ static void *preload_thread(void *_data)
 			continue;
 		if (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))
 			continue;
+		p->t2_nr_lstat++;
 		if (lstat(ce->name, &st))
 			continue;
 		if (ie_match_stat(index, ce, &st, CE_MATCH_RACY_IS_DIRTY|CE_MATCH_IGNORE_FSMONITOR))
@@ -98,6 +100,7 @@ void preload_index(struct index_state *index,
 	int threads, i, work, offset;
 	struct thread_data data[MAX_PARALLEL];
 	struct progress_data pd;
+	int t2_sum_lstat = 0;
 
 	if (!HAVE_THREADS || !core_preload_index)
 		return;
@@ -107,6 +110,9 @@ void preload_index(struct index_state *index,
 		threads = 2;
 	if (threads < 2)
 		return;
+
+	trace2_region_enter("index", "preload", NULL);
+
 	trace_performance_enter();
 	if (threads > MAX_PARALLEL)
 		threads = MAX_PARALLEL;
@@ -141,10 +147,14 @@ void preload_index(struct index_state *index,
 		struct thread_data *p = data+i;
 		if (pthread_join(p->pthread, NULL))
 			die("unable to join threaded lstat");
+		t2_sum_lstat += p->t2_nr_lstat;
 	}
 	stop_progress(&pd.progress);
 
 	trace_performance_leave("preload index");
+
+	trace2_data_intmax("index", NULL, "preload/sum_lstat", t2_sum_lstat);
+	trace2_region_leave("index", "preload", NULL);
 }
 
 int repo_read_index_preload(struct repository *repo,
-- 
gitgitgadget
Previous: Jeff Hostetler via GitGitGadgetNext: Jeff Hostetler via GitGitGadget
Message 11 of 30 in “FSMonitor Preliminary Commits”
  1. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 1, 2021
  2. 01/11 p7519: use xargs -0 rather than -d in testJeff Hostetler via GitGitGadget, Feb 1, 2021
  3. Junio C HamanoFeb 1, 2021
  4. Jeff HostetlerFeb 2, 2021
  5. Junio C HamanoFeb 2, 2021
  6. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 1, 2021
  7. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 1, 2021
  8. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 1, 2021
  9. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 1, 2021
  10. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 1, 2021
  11. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  12. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  13. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 1, 2021
  14. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  15. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  16. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 3, 2021
  17. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 3, 2021
  18. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 3, 2021
  19. 01/11 p7519: do not rely on "xargs -d" in testJeff Hostetler via GitGitGadget, Feb 3, 2021
  20. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 3, 2021
  21. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  22. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  23. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 3, 2021
  24. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 3, 2021
  25. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 3, 2021
  26. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  27. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  28. Jeff HostetlerFeb 16, 2021
  29. Junio C HamanoFeb 17, 2021
  30. Taylor BlauFeb 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.