git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 04/11] p7519: add trace logging during perf test

From
JGJeff Hostetler via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 3, 2021, 15:34 UTC
Message-ID
<f6ea0a51f50e95d943a33156c3013af97ddda470.1612366490.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.860.v2.git.1612366490.gitgitgadget@gmail.com>
From: Jeff Hostetler <jeffhost@microsoft.com>

Add optional trace logging to allow us to better compare performance of various fsmonitor providers and compare results with non-fsmonitor runs.

Currently, this includes Trace2 logging, but may be extended to include other trace targets, such as GIT_TRACE_FSMONITOR if desired.

Using this logging helped me explain an odd behavior on MacOS where the kernel was dropping events and causing the hook to Watchman to timeout.

Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
---
 t/perf/.gitignore         |  1 +
 t/perf/Makefile           |  4 ++--
 t/perf/p7519-fsmonitor.sh | 32 ++++++++++++++++++++++++++++++++
 3 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/t/perf/.gitignore b/t/perf/.gitignore
index 982eb8e3a94..72f5d0d3148 100644
--- a/t/perf/.gitignore
+++ b/t/perf/.gitignore
@@ -1,3 +1,4 @@
 /build/
 /test-results/
+/test-trace/
 /trash directory*/
diff --git a/t/perf/Makefile b/t/perf/Makefile
index fcb0e8865e4..2465770a782 100644
--- a/t/perf/Makefile
+++ b/t/perf/Makefile
@@ -7,10 +7,10 @@ perf: pre-clean
 	./run
 
 pre-clean:
-	rm -rf test-results
+	rm -rf test-results test-trace
 
 clean:
-	rm -rf build "trash directory".* test-results
+	rm -rf build "trash directory".* test-results test-trace
 
 test-lint:
 	$(MAKE) -C .. test-lint
diff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh
index 78e7d2c03f5..aa0ea0e2ec8 100755
--- a/t/perf/p7519-fsmonitor.sh
+++ b/t/perf/p7519-fsmonitor.sh
@@ -32,6 +32,8 @@ test_description="Test core.fsmonitor"
 #
 # GIT_PERF_7519_DROP_CACHE: if set, the OS caches are dropped between tests
 #
+# GIT_PERF_7519_TRACE: if set, enable trace logging during the test.
+#   Trace logs will be grouped by fsmonitor provider.
 
 test_perf_large_repo
 test_checkout_worktree
@@ -70,6 +72,32 @@ then
 	fi
 fi
 
+trace_start() {
+	if test -n "$GIT_PERF_7519_TRACE"
+	then
+		name="$1"
+		TEST_TRACE_DIR="$TEST_OUTPUT_DIRECTORY/test-trace/p7519/"
+		echo "Writing trace logging to $TEST_TRACE_DIR"
+
+		mkdir -p "$TEST_TRACE_DIR"
+
+		# Start Trace2 logging and any other GIT_TRACE_* logs that you
+		# want for this named test case.
+
+		GIT_TRACE2_PERF="$TEST_TRACE_DIR/$name.trace2perf"
+		export GIT_TRACE2_PERF
+
+		>"$GIT_TRACE2_PERF"
+	fi
+}
+
+trace_stop() {
+	if test -n "$GIT_PERF_7519_TRACE"
+	then
+		unset GIT_TRACE2_PERF
+	fi
+}
+
 test_expect_success "one time repo setup" '
 	# set untrackedCache depending on the environment
 	if test -n "$GIT_PERF_7519_UNTRACKED_CACHE"
@@ -213,6 +241,7 @@ test_fsmonitor_suite() {
 # such as Watchman.
 #
 
+trace_start fsmonitor-watchman
 if test -n "$GIT_PERF_7519_FSMONITOR"; then
 	for INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do
 		test_expect_success "setup for fsmonitor $INTEGRATION_PATH" 'setup_for_fsmonitor'
@@ -231,11 +260,13 @@ then
 	# preventing the removal of the trash directory
 	watchman shutdown-server >/dev/null 2>&1
 fi
+trace_stop
 
 #
 # Run a full set of perf tests with the fsmonitor feature disabled.
 #
 
+trace_start fsmonitor-disabled
 test_expect_success "setup without fsmonitor" '
 	unset INTEGRATION_SCRIPT &&
 	git config --unset core.fsmonitor &&
@@ -243,5 +274,6 @@ test_expect_success "setup without fsmonitor" '
 '
 
 test_fsmonitor_suite
+trace_stop
 
 test_done
-- 
gitgitgadget
Previous: Jeff Hostetler via GitGitGadgetNext: Jeff Hostetler via GitGitGadget
Message 18 of 30 in “FSMonitor Preliminary Commits”
  1. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 1, 2021
  2. 01/11 p7519: use xargs -0 rather than -d in testJeff Hostetler via GitGitGadget, Feb 1, 2021
  3. Junio C HamanoFeb 1, 2021
  4. Jeff HostetlerFeb 2, 2021
  5. Junio C HamanoFeb 2, 2021
  6. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 1, 2021
  7. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 1, 2021
  8. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 1, 2021
  9. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 1, 2021
  10. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 1, 2021
  11. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  12. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  13. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 1, 2021
  14. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  15. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 1, 2021
  16. 00/11 FSMonitor Preliminary CommitsJeff Hostetler via GitGitGadget, Feb 3, 2021
  17. 03/11 p7519: move watchman cleanup earlier in the testJeff Hostetler via GitGitGadget, Feb 3, 2021
  18. 04/11 p7519: add trace logging during perf testJeff Hostetler via GitGitGadget, Feb 3, 2021
  19. 01/11 p7519: do not rely on "xargs -d" in testJeff Hostetler via GitGitGadget, Feb 3, 2021
  20. 02/11 p7519: fix watchman watch-list test on WindowsJeff Hostetler via GitGitGadget, Feb 3, 2021
  21. 06/11 read-cache: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  22. 07/11 read-cache: log the number of scanned files to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  23. 11/11 fsmonitor: refactor initialization of fsmonitor_last_update tokenJeff Hostetler via GitGitGadget, Feb 3, 2021
  24. 09/11 fsmonitor: log FSMN token when reading and writing the indexJeff Hostetler via GitGitGadget, Feb 3, 2021
  25. 10/11 fsmonitor: allow all entries for a folder to be invalidatedKevin Willford via GitGitGadget, Feb 3, 2021
  26. 08/11 fsmonitor: log invocation of FSMonitor hook to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  27. 05/11 preload-index: log the number of lstat calls to trace2Jeff Hostetler via GitGitGadget, Feb 3, 2021
  28. Jeff HostetlerFeb 16, 2021
  29. Junio C HamanoFeb 17, 2021
  30. Taylor BlauFeb 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.