git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/3] t: add "test-tool genbytes" to avoid slow tr(1) on macOS

From
Kristofer Karlsson via GitGitGadget <gitgitgadget@gmail.com>
Date
Nov 7, 2026, 14:59 UTC
Message-ID
<4c02ada9b85028f26f7e7bcc62e2ac12443da9c5.1794063565.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2301.git.1794063565.gitgitgadget@gmail.com>
From: Kristofer Karlsson <krka@spotify.com>

Several tests create fixture data consisting of a single repeated character with "test-tool genzeros <count> | tr '\000' <char>". This works well on platforms where tr(1) is implemented efficiently.

The implementation of tr(1) on macOS supports multi-byte characters, and processes its input one character at a time through getwchar() and putwchar(), even in the C locale. For small inputs that does not matter, but t4205 generates more than 2GB of data this way, which makes that test much slower on macOS than on Linux: on the macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on it, compared to about 6 seconds on Linux.

Fix this by adding "test-tool genbytes <byte> [<count>]", which writes the requested byte directly, and turn "genzeros" into a thin wrapper around the same code. Use the new helper in all tests that piped "genzeros" through tr(1), not just t4205, so that it is the obvious choice for new tests.

This makes t4205 much faster on macOS in CI. The Linux jobs, shown for comparison, only save the few seconds that GNU tr(1) took, which is within the usual noise:

  job                before    after   change
  osx-gcc            821.1s   243.3s   -70.4%
  osx-reftable       911.0s   194.0s   -78.7%
  linux-asan-ubsan   226.7s   222.1s    -2.0%
  linux-leaks        320.2s   298.8s    -6.7%
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
---
 t/helper/test-genbytes.c          | 52 ++++++++++++++++++++++---------
 t/helper/test-tool.c              |  1 +
 t/helper/test-tool.h              |  1 +
 t/t0613-reftable-write-options.sh |  2 +-
 t/t4205-log-pretty-formats.sh     |  2 +-
 t/t5300-pack-object.sh            |  6 ++--
 t/t7006-pager.sh                  |  3 +-
 t/t9850-shell.sh                  |  2 +-
 8 files changed, 46 insertions(+), 23 deletions(-)
diff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c
index b895436a32..293af78d0d 100644
--- a/t/helper/test-genbytes.c
+++ b/t/helper/test-genbytes.c
@@ -3,29 +3,26 @@
 #include "test-tool.h"
 #include "git-compat-util.h"
 
-int cmd__genzeros(int argc, const char **argv)
+/*
+ * Write "count" copies of "byte" to stdout, or an endless stream of them if
+ * "count" is negative.
+ */
+static int generate_bytes(char byte, intmax_t count)
 {
-	/* static, so that it is NUL-initialized */
-	static const char zeros[256 * 1024];
-	intmax_t count;
+	char buf[256 * 1024];
 	ssize_t n;
 
-	if (argc > 2) {
-		fprintf(stderr, "usage: %s [<count>]\n", argv[0]);
-		return 1;
-	}
-
-	count = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;
+	memset(buf, byte, sizeof(buf));
 
-	/* Writing out individual NUL bytes is slow... */
+	/* Writing out individual bytes is slow... */
 	while (count < 0)
-		if (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)
+		if (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)
 			die_errno("write error");
 
 	while (count > 0) {
-		n = xwrite(1, zeros,
-			   count < ARRAY_SIZE(zeros)
-			   ? count : ARRAY_SIZE(zeros));
+		n = xwrite(1, buf,
+			   count < ARRAY_SIZE(buf)
+			   ? count : ARRAY_SIZE(buf));
 
 		if (n < 0)
 			die_errno("write error");
@@ -35,3 +32,28 @@ int cmd__genzeros(int argc, const char **argv)
 
 	return 0;
 }
+
+static intmax_t parse_count(int argc, const char **argv, int pos)
+{
+	return pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;
+}
+
+int cmd__genbytes(int argc, const char **argv)
+{
+	if (argc < 2 || argc > 3 || strlen(argv[1]) != 1) {
+		fprintf(stderr, "usage: %s <byte> [<count>]\n", argv[0]);
+		return 1;
+	}
+
+	return generate_bytes(argv[1][0], parse_count(argc, argv, 2));
+}
+
+int cmd__genzeros(int argc, const char **argv)
+{
+	if (argc > 2) {
+		fprintf(stderr, "usage: %s [<count>]\n", argv[0]);
+		return 1;
+	}
+
+	return generate_bytes('\0', parse_count(argc, argv, 1));
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index b71a22b43b..af70e2bec7 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -33,6 +33,7 @@ static struct test_cmd cmds[] = {
 	{ "example-tap", cmd__example_tap },
 	{ "find-pack", cmd__find_pack },
 	{ "fsmonitor-client", cmd__fsmonitor_client },
+	{ "genbytes", cmd__genbytes },
 	{ "genrandom", cmd__genrandom },
 	{ "genzeros", cmd__genzeros },
 	{ "getcwd", cmd__getcwd },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index f2885b33d5..c35e7412af 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -26,6 +26,7 @@ int cmd__env_helper(int argc, const char **argv);
 int cmd__example_tap(int argc, const char **argv);
 int cmd__find_pack(int argc, const char **argv);
 int cmd__fsmonitor_client(int argc, const char **argv);
+int cmd__genbytes(int argc, const char **argv);
 int cmd__genrandom(int argc, const char **argv);
 int cmd__genzeros(int argc, const char **argv);
 int cmd__getcwd(int argc, const char **argv);
diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh
index 0b27d75fe3..0b74ff6157 100755
--- a/t/t0613-reftable-write-options.sh
+++ b/t/t0613-reftable-write-options.sh
@@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '
 	(
 		cd repo &&
 		test_commit A &&
-		test-tool genzeros 500 | tr "\000" "a" >logmsg &&
+		test-tool genbytes a 500 >logmsg &&
 		cat >expect <<-EOF &&
 		fatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large
 		EOF
diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh
index 01b97c8888..894d88c39b 100755
--- a/t/t4205-log-pretty-formats.sh
+++ b/t/t4205-log-pretty-formats.sh
@@ -1182,7 +1182,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes
 '
 
 test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '
-	test-tool genzeros 2147483649 | tr "\000" "1" >expect &&
+	test-tool genbytes 1 2147483649 >expect &&
 	huge_commit=$(git commit-tree -F expect HEAD^{tree})
 '
 
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index aac139e6a0..38dc4021b7 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -9,9 +9,9 @@ test_description='git pack-object'
 
 test_expect_success 'setup' '
 	rm -f .git/index* &&
-	test-tool genzeros 4096 | tr "\000" "a" >a &&
-	test-tool genzeros 4096 | tr "\000" "b" >b &&
-	test-tool genzeros 4096 | tr "\000" "c" >c &&
+	test-tool genbytes a 4096 >a &&
+	test-tool genbytes b 4096 >b &&
+	test-tool genbytes c 4096 >c &&
 	test-tool genrandom "seed a" 2097152 >a_big &&
 	test-tool genrandom "seed b" 2097152 >b_big &&
 	git update-index --add a a_big b b_big c &&
diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh
index 75ca7a25da..8f1f71bf12 100755
--- a/t/t7006-pager.sh
+++ b/t/t7006-pager.sh
@@ -662,8 +662,7 @@ test_expect_success 'setup trace2' '
 '
 
 test_expect_success 'setup large log output' '
-	test-tool genzeros 50000 |
-	tr "\000" "a" |
+	test-tool genbytes a 50000 |
 	sed "s/a/this is a long commit message/g" >commit-msg &&
 	git commit --allow-empty -F commit-msg
 '
diff --git a/t/t9850-shell.sh b/t/t9850-shell.sh
index 27cc49bbd1..d56edebda7 100755
--- a/t/t9850-shell.sh
+++ b/t/t9850-shell.sh
@@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '
 '
 
 test_expect_success 'shell complains of overlong commands' '
-	test-tool genzeros | tr "\000" "a" |
+	test-tool genbytes a |
 	test_must_fail git shell 2>err &&
 	test_grep "too long" err
 '
-- 
gitgitgadget
Previous: Kristofer KarlssonNext: Johannes Sixt
Message 8 of 39 in “t: add "test-tool genbytes" to avoid slow tr(1) on macOS”
  1. 0/3 t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 7, 2026
  2. 1/3 t/helper: rename test-genzeros.c to test-genbytes.cKristofer Karlsson via GitGitGadget, Nov 7, 2026
  3. Junio C HamanoNov 7, 2026
  4. Junio C HamanoNov 8, 2026
  5. Kristofer KarlssonNov 8, 2026
  6. Junio C HamanoNov 8, 2026
  7. Kristofer KarlssonNov 8, 2026
  8. 2/3 t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 7, 2026
  9. Johannes SixtNov 8, 2026
  10. Kristofer KarlssonNov 8, 2026
  11. Johannes SixtNov 8, 2026
  12. Kristofer KarlssonNov 8, 2026
  13. Johannes SixtNov 8, 2026
  14. Johannes SixtNov 9, 2026
  15. Kristofer KarlssonNov 9, 2026
  16. Johannes SixtNov 9, 2026
  17. Collin FunkNov 10, 2026
  18. Kristofer KarlssonNov 10, 2026
  19. Junio C HamanoNov 8, 2026
  20. 3/3 t: replace "test-tool genzeros" with "genbytes NUL"Kristofer Karlsson via GitGitGadget, Nov 7, 2026
  21. Junio C HamanoNov 7, 2026
  22. Kristofer KarlssonNov 8, 2026
  23. Junio C HamanoNov 7, 2026
  24. Kristofer KarlssonNov 8, 2026
  25. 0/2 t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 8, 2026
  26. 1/2 t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 8, 2026
  27. Junio C HamanoNov 9, 2026
  28. Kristofer KarlssonNov 9, 2026
  29. Junio C HamanoNov 9, 2026
  30. Patrick SteinhardtNov 9, 2026
  31. Kristofer KarlssonNov 9, 2026
  32. 2/2 t: replace "test-tool genzeros" with "genbytes 00"Kristofer Karlsson via GitGitGadget, Nov 8, 2026
  33. t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 10, 2026
  34. Junio C HamanoNov 10, 2026
  35. Patrick SteinhardtNov 11, 2026
  36. Junio C HamanoNov 11, 2026
  37. Kristofer KarlssonNov 11, 2026
  38. Junio C HamanoNov 11, 2026
  39. t: add "test-tool genbytes" to avoid slow tr(1) on macOSKristofer Karlsson via GitGitGadget, Nov 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.