git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] perf: amend the grep tests to test grep.threads

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Dec 29, 2017, 22:59 UTC
Message-ID
<20171229225903.19688-1-avarab@gmail.com>

Ever since 5b594f457a ("Threaded grep", 2010-01-25) the number of threads git-grep uses under PTHREADS has been hardcoded to 8, but there's no performance test to check whether this is an optimal setting.

Amend the existing tests for the grep engines to support a mode where this can be tested, e.g.:

    GIT_PERF_GREP_THREADS='1 8 16' GIT_PERF_LARGE_REPO=~/g/linux ./run p782*
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
This looks less scary under diff -w.
 t/perf/p7820-grep-engines.sh       | 52 ++++++++++++++++++++++++++++-------
 t/perf/p7821-grep-engines-fixed.sh | 55 ++++++++++++++++++++++++++++++--------
 2 files changed, 86 insertions(+), 21 deletions(-)
diff --git a/t/perf/p7820-grep-engines.sh b/t/perf/p7820-grep-engines.sh
index 62aba19e76..8b09c5bf32 100755
--- a/t/perf/p7820-grep-engines.sh
+++ b/t/perf/p7820-grep-engines.sh
@@ -12,6 +12,9 @@ e.g. GIT_PERF_7820_GREP_OPTS=' -i'. Some options to try:
 	-vi
 	-vw
 	-viw
+
+If GIT_PERF_GREP_THREADS is set to a list of threads (e.g. '1 4 8'
+etc.) we will test the patterns under those numbers of threads.
 "
 
 . ./perf-lib.sh
@@ -19,6 +22,11 @@ e.g. GIT_PERF_7820_GREP_OPTS=' -i'. Some options to try:
 test_perf_large_repo
 test_checkout_worktree
 
+if test -n "$GIT_PERF_GREP_THREADS"
+then
+	test_set_prereq PERF_GREP_ENGINES_THREADS
+fi
+
 for pattern in \
 	'how.to' \
 	'^how to' \
@@ -39,18 +47,42 @@ do
 		else
 			prereq=""
 		fi
-		test_perf $prereq "$engine grep$GIT_PERF_7820_GREP_OPTS '$pattern'" "
-			git -c grep.patternType=$engine grep$GIT_PERF_7820_GREP_OPTS -- '$pattern' >'out.$engine' || :
-		"
-	done
-
-	test_expect_success "assert that all engines found the same for$GIT_PERF_7820_GREP_OPTS '$pattern'" '
-		test_cmp out.basic out.extended &&
-		if test_have_prereq PCRE
+		if ! test_have_prereq PERF_GREP_ENGINES_THREADS
 		then
-			test_cmp out.basic out.perl
+			test_perf $prereq "$engine grep$GIT_PERF_7820_GREP_OPTS '$pattern'" "
+				git -c grep.patternType=$engine grep$GIT_PERF_7820_GREP_OPTS -- '$pattern' >'out.$engine' || :
+			"
+		else
+			for threads in $GIT_PERF_GREP_THREADS
+			do
+				test_perf PTHREADS,$prereq "$engine grep$GIT_PERF_7820_GREP_OPTS '$pattern' with $threads threads" "
+					git -c grep.patternType=$engine -c grep.threads=$threads grep$GIT_PERF_7820_GREP_OPTS -- '$pattern' >'out.$engine.$threads' || :
+				"
+			done
 		fi
-	'
+	done
+
+	if ! test_have_prereq PERF_GREP_ENGINES_THREADS
+	then
+		test_expect_success "assert that all engines found the same for$GIT_PERF_7820_GREP_OPTS '$pattern'" '
+			test_cmp out.basic out.extended &&
+			if test_have_prereq PCRE
+			then
+				test_cmp out.basic out.perl
+			fi
+		'
+	else
+		for threads in $GIT_PERF_GREP_THREADS
+		do
+			test_expect_success PTHREADS "assert that all engines found the same for$GIT_PERF_7820_GREP_OPTS '$pattern' under threading" "
+				test_cmp out.basic.$threads out.extended.$threads &&
+				if test_have_prereq PCRE
+				then
+					test_cmp out.basic.$threads out.perl.$threads
+				fi
+			"
+		done
+	fi
 done
 
 test_done
diff --git a/t/perf/p7821-grep-engines-fixed.sh b/t/perf/p7821-grep-engines-fixed.sh
index c7ef1e198f..61e41b82cf 100755
--- a/t/perf/p7821-grep-engines-fixed.sh
+++ b/t/perf/p7821-grep-engines-fixed.sh
@@ -6,6 +6,9 @@ Set GIT_PERF_7821_GREP_OPTS in the environment to pass options to
 git-grep. Make sure to include a leading space,
 e.g. GIT_PERF_7821_GREP_OPTS=' -w'. See p7820-grep-engines.sh for more
 options to try.
+
+If GIT_PERF_7821_THREADS is set to a list of threads (e.g. '1 4 8'
+etc.) we will test the patterns under those numbers of threads.
 "
 
 . ./perf-lib.sh
@@ -13,6 +16,11 @@ options to try.
 test_perf_large_repo
 test_checkout_worktree
 
+if test -n "$GIT_PERF_GREP_THREADS"
+then
+	test_set_prereq PERF_GREP_ENGINES_THREADS
+fi
+
 for pattern in 'int' 'uncommon' 'æ'
 do
 	for engine in fixed basic extended perl
@@ -23,19 +31,44 @@ do
 		else
 			prereq=""
 		fi
-		test_perf $prereq "$engine grep$GIT_PERF_7821_GREP_OPTS $pattern" "
-			git -c grep.patternType=$engine grep$GIT_PERF_7821_GREP_OPTS $pattern >'out.$engine' || :
-		"
-	done
-
-	test_expect_success "assert that all engines found the same for$GIT_PERF_7821_GREP_OPTS $pattern" '
-		test_cmp out.fixed out.basic &&
-		test_cmp out.fixed out.extended &&
-		if test_have_prereq PCRE
+		if ! test_have_prereq PERF_GREP_ENGINES_THREADS
 		then
-			test_cmp out.fixed out.perl
+			test_perf $prereq "$engine grep$GIT_PERF_7821_GREP_OPTS $pattern" "
+				git -c grep.patternType=$engine grep$GIT_PERF_7821_GREP_OPTS $pattern >'out.$engine' || :
+			"
+		else
+			for threads in $GIT_PERF_GREP_THREADS
+			do
+				test_perf PTHREADS,$prereq "$engine grep$GIT_PERF_7821_GREP_OPTS $pattern with $threads threads" "
+					git -c grep.patternType=$engine -c grep.threads=$threads grep$GIT_PERF_7821_GREP_OPTS $pattern >'out.$engine.$threads' || :
+				"
+			done
 		fi
-	'
+	done
+
+	if ! test_have_prereq PERF_GREP_ENGINES_THREADS
+	then
+		test_expect_success "assert that all engines found the same for$GIT_PERF_7821_GREP_OPTS $pattern" '
+			test_cmp out.fixed out.basic &&
+			test_cmp out.fixed out.extended &&
+			if test_have_prereq PCRE
+			then
+				test_cmp out.fixed out.perl
+			fi
+		'
+	else
+		for threads in $GIT_PERF_GREP_THREADS
+		do
+			test_expect_success PTHREADS "assert that all engines found the same for$GIT_PERF_7821_GREP_OPTS $pattern under threading" "
+				test_cmp out.fixed.$threads out.basic.$threads &&
+				test_cmp out.fixed.$threads out.extended.$threads &&
+				if test_have_prereq PCRE
+				then
+					test_cmp out.fixed.$threads out.perl.$threads
+				fi
+			"
+		done
+	fi
 done
 
 test_done

-- 
2.15.1.424.g9478a66081
Next: Junio C Hamano
Message 1 of 4 in “perf: amend the grep tests to test grep.threads”
  1. perf: amend the grep tests to test grep.threadsÆvar Arnfjörð Bjarmason, Dec 29, 2017
  2. Junio C HamanoJan 4, 2018
  3. Ævar Arnfjörð BjarmasonJan 5, 2018
  4. Junio C HamanoJan 5, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.