{"thread":{"id":"66800","subject":"[PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","startedAt":"2026-11-07T14:59:22Z","lastAt":"2026-11-11T17:57:54Z","messageCount":39,"participants":["Kristofer Karlsson via GitGitGadget","Junio C Hamano","Johannes Sixt","Kristofer Karlsson","Patrick Steinhardt","Collin Funk"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"557791","messageId":"pull.2301.git.1794063565.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":null,"subject":"[PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-07T14:59:22Z","receivedAt":"2026-11-07T14:59:22Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"While looking for slow tests in CI, t4205 stood out on macOS: its EXPENSIVE\ntests take several times longer there than on Linux. A CI experiment showed\nthat much of the time was spent in tr(1), which is very slow on macOS for\nthe 2GB of data that t4205 generates.\n\nPatch 1 renames test-genzeros.c so that the actual changes in patch 2 show\nup as a regular diff. Patch 2 adds \"genbytes\" and uses it instead of\n\"genzeros | tr\", which is where the speedup comes from.\n\nOpen question: is patch 3 wanted? It folds \"genzeros\" into \"genbytes\",\nspelling the NUL byte as \"NUL\", so that there is a single command.\nAlternatively, we could drop it and keep \"genzeros\" as a thin wrapper,\navoiding changes to its 10 call sites. I kept patch 3 separate so that it is\neasy to drop, but it could also be squashed into patch 2.\n\nSome other decisions that may be worth pointing out:\n\n * The buffer is no longer static: it used to be static so that it was\n   zero-initialized for free, but now that it is filled with the requested\n   byte on every call, a local buffer avoids shared state. It keeps its size\n   of 256KB. Smaller buffers made no measurable difference in my tests, but\n   there was no reason to change it.\n\n * Invalid counts are still silently treated as 0, as before. That could be\n   tightened, but it is unrelated to this series.\n\nKristofer Karlsson (3):\n  t/helper: rename test-genzeros.c to test-genbytes.c\n  t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n  t: replace \"test-tool genzeros\" with \"genbytes NUL\"\n\n Makefile                          |  2 +-\n t/helper/meson.build              |  2 +-\n t/helper/test-genbytes.c          | 64 +++++++++++++++++++++++++++++++\n t/helper/test-genzeros.c          | 37 ------------------\n t/helper/test-tool.c              |  2 +-\n t/helper/test-tool.h              |  2 +-\n t/t0613-reftable-write-options.sh |  2 +-\n t/t1007-hash-object.sh            |  8 ++--\n t/t1051-large-conversion.sh       |  4 +-\n t/t1450-fsck.sh                   |  4 +-\n t/t4141-apply-too-large.sh        |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 +--\n t/t5318-commit-graph.sh           |  2 +-\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 16 files changed, 85 insertions(+), 59 deletions(-)\n create mode 100644 t/helper/test-genbytes.c\n delete mode 100644 t/helper/test-genzeros.c\n\n\nbase-commit: fb255012acbcbe81189cb8014da0280b09c66ecc\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2301%2Fspkrka%2Ftest-tool-genbytes-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2301/spkrka/test-tool-genbytes-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2301\n-- \ngitgitgadget\n\n"},{"id":"557792","messageId":"b4378b352769cdcc2e9cfdf40efe173123ea9ab4.1794063565.git.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-07T14:59:23Z","receivedAt":"2026-11-07T14:59:23Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nThe next commit adds a \"genbytes\" command to this file and turns\n\"genzeros\" into a thin wrapper around it.  Rename the file in a\nseparate step first, so that git can detect the rename and the actual\nchanges show up as a regular diff, which is easier to review.\n\nNo functional changes.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Makefile                                      | 2 +-\n t/helper/meson.build                          | 2 +-\n t/helper/{test-genzeros.c => test-genbytes.c} | 0\n 3 files changed, 2 insertions(+), 2 deletions(-)\n rename t/helper/{test-genzeros.c => test-genbytes.c} (100%)\n\ndiff --git a/Makefile b/Makefile\nindex c2a6098484..8d214286e0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -825,8 +825,8 @@ TEST_BUILTINS_OBJS += test-env-helper.o\n TEST_BUILTINS_OBJS += test-example-tap.o\n TEST_BUILTINS_OBJS += test-find-pack.o\n TEST_BUILTINS_OBJS += test-fsmonitor-client.o\n+TEST_BUILTINS_OBJS += test-genbytes.o\n TEST_BUILTINS_OBJS += test-genrandom.o\n-TEST_BUILTINS_OBJS += test-genzeros.o\n TEST_BUILTINS_OBJS += test-getcwd.o\n TEST_BUILTINS_OBJS += test-hash-speed.o\n TEST_BUILTINS_OBJS += test-hash.o\ndiff --git a/t/helper/meson.build b/t/helper/meson.build\nindex ae513b4cdc..f99688c49b 100644\n--- a/t/helper/meson.build\n+++ b/t/helper/meson.build\n@@ -22,8 +22,8 @@ test_tool_sources = [\n   'test-example-tap.c',\n   'test-find-pack.c',\n   'test-fsmonitor-client.c',\n+  'test-genbytes.c',\n   'test-genrandom.c',\n-  'test-genzeros.c',\n   'test-getcwd.c',\n   'test-hash-speed.c',\n   'test-hash.c',\ndiff --git a/t/helper/test-genzeros.c b/t/helper/test-genbytes.c\nsimilarity index 100%\nrename from t/helper/test-genzeros.c\nrename to t/helper/test-genbytes.c\n-- \ngitgitgadget\n\n\n"},{"id":"557793","messageId":"4c02ada9b85028f26f7e7bcc62e2ac12443da9c5.1794063565.git.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-07T14:59:24Z","receivedAt":"2026-11-07T14:59:24Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nSeveral tests create fixture data consisting of a single repeated\ncharacter with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\nworks well on platforms where tr(1) is implemented efficiently.\n\nThe implementation of tr(1) on macOS supports multi-byte characters,\nand processes its input one character at a time through\ngetwchar() and putwchar(), even in the C locale.  For small inputs\nthat does not matter, but t4205 generates more than 2GB of data this\nway, which makes that test much slower on macOS than on Linux: on the\nmacOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\nit, compared to about 6 seconds on Linux.\n\nFix this by adding \"test-tool genbytes <byte> [<count>]\", which writes\nthe requested byte directly, and turn \"genzeros\" into a thin\nwrapper around the same code.  Use the new helper in all tests that\npiped \"genzeros\" through tr(1), not just t4205, so that it is the\nobvious choice for new tests.\n\nThis makes t4205 much faster on macOS in CI.  The Linux jobs, shown\nfor comparison, only save the few seconds that GNU tr(1) took, which\nis within the usual noise:\n\n  job                before    after   change\n  osx-gcc            821.1s   243.3s   -70.4%\n  osx-reftable       911.0s   194.0s   -78.7%\n  linux-asan-ubsan   226.7s   222.1s    -2.0%\n  linux-leaks        320.2s   298.8s    -6.7%\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n t/helper/test-genbytes.c          | 52 ++++++++++++++++++++++---------\n t/helper/test-tool.c              |  1 +\n t/helper/test-tool.h              |  1 +\n t/t0613-reftable-write-options.sh |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 ++--\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 8 files changed, 46 insertions(+), 23 deletions(-)\n\ndiff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\nindex b895436a32..293af78d0d 100644\n--- a/t/helper/test-genbytes.c\n+++ b/t/helper/test-genbytes.c\n@@ -3,29 +3,26 @@\n #include \"test-tool.h\"\n #include \"git-compat-util.h\"\n \n-int cmd__genzeros(int argc, const char **argv)\n+/*\n+ * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n+ * \"count\" is negative.\n+ */\n+static int generate_bytes(char byte, intmax_t count)\n {\n-\t/* static, so that it is NUL-initialized */\n-\tstatic const char zeros[256 * 1024];\n-\tintmax_t count;\n+\tchar buf[256 * 1024];\n \tssize_t n;\n \n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n-\n-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n+\tmemset(buf, byte, sizeof(buf));\n \n-\t/* Writing out individual NUL bytes is slow... */\n+\t/* Writing out individual bytes is slow... */\n \twhile (count < 0)\n-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n+\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n \t\t\tdie_errno(\"write error\");\n \n \twhile (count > 0) {\n-\t\tn = xwrite(1, zeros,\n-\t\t\t   count < ARRAY_SIZE(zeros)\n-\t\t\t   ? count : ARRAY_SIZE(zeros));\n+\t\tn = xwrite(1, buf,\n+\t\t\t   count < ARRAY_SIZE(buf)\n+\t\t\t   ? count : ARRAY_SIZE(buf));\n \n \t\tif (n < 0)\n \t\t\tdie_errno(\"write error\");\n@@ -35,3 +32,28 @@ int cmd__genzeros(int argc, const char **argv)\n \n \treturn 0;\n }\n+\n+static intmax_t parse_count(int argc, const char **argv, int pos)\n+{\n+\treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n+}\n+\n+int cmd__genbytes(int argc, const char **argv)\n+{\n+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1) {\n+\t\tfprintf(stderr, \"usage: %s <byte> [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes(argv[1][0], parse_count(argc, argv, 2));\n+}\n+\n+int cmd__genzeros(int argc, const char **argv)\n+{\n+\tif (argc > 2) {\n+\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex b71a22b43b..af70e2bec7 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -33,6 +33,7 @@ static struct test_cmd cmds[] = {\n \t{ \"example-tap\", cmd__example_tap },\n \t{ \"find-pack\", cmd__find_pack },\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n+\t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n \t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex f2885b33d5..c35e7412af 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -26,6 +26,7 @@ int cmd__env_helper(int argc, const char **argv);\n int cmd__example_tap(int argc, const char **argv);\n int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n+int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex 0b27d75fe3..0b74ff6157 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n \t(\n \t\tcd repo &&\n \t\ttest_commit A &&\n-\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n+\t\ttest-tool genbytes a 500 >logmsg &&\n \t\tcat >expect <<-EOF &&\n \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n \t\tEOF\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 01b97c8888..894d88c39b 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1182,7 +1182,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n-\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+\ttest-tool genbytes 1 2147483649 >expect &&\n \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n '\n \ndiff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\nindex aac139e6a0..38dc4021b7 100755\n--- a/t/t5300-pack-object.sh\n+++ b/t/t5300-pack-object.sh\n@@ -9,9 +9,9 @@ test_description='git pack-object'\n \n test_expect_success 'setup' '\n \trm -f .git/index* &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n+\ttest-tool genbytes a 4096 >a &&\n+\ttest-tool genbytes b 4096 >b &&\n+\ttest-tool genbytes c 4096 >c &&\n \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n \tgit update-index --add a a_big b b_big c &&\ndiff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\nindex 75ca7a25da..8f1f71bf12 100755\n--- a/t/t7006-pager.sh\n+++ b/t/t7006-pager.sh\n@@ -662,8 +662,7 @@ test_expect_success 'setup trace2' '\n '\n \n test_expect_success 'setup large log output' '\n-\ttest-tool genzeros 50000 |\n-\ttr \"\\000\" \"a\" |\n+\ttest-tool genbytes a 50000 |\n \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n \tgit commit --allow-empty -F commit-msg\n '\ndiff --git a/t/t9850-shell.sh b/t/t9850-shell.sh\nindex 27cc49bbd1..d56edebda7 100755\n--- a/t/t9850-shell.sh\n+++ b/t/t9850-shell.sh\n@@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '\n '\n \n test_expect_success 'shell complains of overlong commands' '\n-\ttest-tool genzeros | tr \"\\000\" \"a\" |\n+\ttest-tool genbytes a |\n \ttest_must_fail git shell 2>err &&\n \ttest_grep \"too long\" err\n '\n-- \ngitgitgadget\n\n\n"},{"id":"557794","messageId":"169ca5968ea2aa40c26076697f8d5c83270f9888.1794063565.git.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH 3/3] t: replace \"test-tool genzeros\" with \"genbytes NUL\"","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-07T14:59:25Z","receivedAt":"2026-11-07T14:59:25Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nNow that \"genbytes\" exists, \"genzeros\" is merely a special case of it\nfor NUL bytes.  It is only kept as a separate command because a NUL\nbyte cannot be passed as a command-line argument.\n\nTeach \"genbytes\" to accept \"NUL\" as the name of that byte instead,\nconvert the remaining callers of \"genzeros\", and remove it, so that\nthere is a single command for generating repeated bytes.\n\nOnly some of these callers depend on the data being NUL bytes, like\nt1007, which compares against the object ID of 5GB of zeros, and\nt5318, which zeroes out part of a commit-graph file.  The others only\ncare about the size, but keep generating NUL bytes there, too, so\nthat what they test does not change.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n t/helper/test-genbytes.c    | 29 +++++++++++++++++------------\n t/helper/test-tool.c        |  1 -\n t/helper/test-tool.h        |  1 -\n t/t1007-hash-object.sh      |  8 ++++----\n t/t1051-large-conversion.sh |  4 ++--\n t/t1450-fsck.sh             |  4 ++--\n t/t4141-apply-too-large.sh  |  2 +-\n t/t5318-commit-graph.sh     |  2 +-\n 8 files changed, 27 insertions(+), 24 deletions(-)\n\ndiff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\nindex 293af78d0d..ad0fb2d2b4 100644\n--- a/t/helper/test-genbytes.c\n+++ b/t/helper/test-genbytes.c\n@@ -33,6 +33,19 @@ static int generate_bytes(char byte, intmax_t count)\n \treturn 0;\n }\n \n+/*\n+ * A NUL byte cannot be passed as an argument, so it is spelled out as\n+ * \"NUL\".  Returns -1 for anything that is not a single byte.\n+ */\n+static int parse_byte(const char *arg)\n+{\n+\tif (!strcmp(arg, \"NUL\"))\n+\t\treturn '\\0';\n+\tif (strlen(arg) == 1)\n+\t\treturn (unsigned char)arg[0];\n+\treturn -1;\n+}\n+\n static intmax_t parse_count(int argc, const char **argv, int pos)\n {\n \treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n@@ -40,20 +53,12 @@ static intmax_t parse_count(int argc, const char **argv, int pos)\n \n int cmd__genbytes(int argc, const char **argv)\n {\n-\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1) {\n-\t\tfprintf(stderr, \"usage: %s <byte> [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n+\tint byte;\n \n-\treturn generate_bytes(argv[1][0], parse_count(argc, argv, 2));\n-}\n-\n-int cmd__genzeros(int argc, const char **argv)\n-{\n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n+\tif (argc < 2 || argc > 3 || (byte = parse_byte(argv[1])) < 0) {\n+\t\tfprintf(stderr, \"usage: %s (<byte> | NUL) [<count>]\\n\", argv[0]);\n \t\treturn 1;\n \t}\n \n-\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n+\treturn generate_bytes(byte, parse_count(argc, argv, 2));\n }\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex af70e2bec7..16571fd702 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -35,7 +35,6 @@ static struct test_cmd cmds[] = {\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n \t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n-\t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\n \t{ \"hashmap\", cmd__hashmap },\n \t{ \"hash-speed\", cmd__hash_speed },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex c35e7412af..1b45c8c37b 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -28,7 +28,6 @@ int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n-int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\n int cmd__hashmap(int argc, const char **argv);\n int cmd__hash_speed(int argc, const char **argv);\ndiff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh\nindex 463b38f990..ea1a1949a1 100755\n--- a/t/t1007-hash-object.sh\n+++ b/t/t1007-hash-object.sh\n@@ -263,7 +263,7 @@ test_expect_success '--stdin outside of repository (uses default hash)' '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash literally' '\n-\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n+\ttest-tool genbytes NUL $((5*1024*1024*1024)) >big &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object --stdin --literally <big >actual &&\n \ttest_cmp expect actual\n@@ -271,7 +271,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash correctly via --stdin' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object --stdin <big >actual &&\n \ttest_cmp expect actual\n@@ -279,7 +279,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash correctly' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object -- big >actual &&\n \ttest_cmp expect actual\n@@ -289,7 +289,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n # We ensure that cleaning doesn't mangle large files on 64-bit Windows.\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'hash filtered files over 4GB correctly' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \ttest_config filter.null-filter.clean \"cat\" &&\n \techo \"big filter=null-filter\" >.gitattributes &&\ndiff --git a/t/t1051-large-conversion.sh b/t/t1051-large-conversion.sh\nindex 361afb679b..6364c396fd 100755\n--- a/t/t1051-large-conversion.sh\n+++ b/t/t1051-large-conversion.sh\n@@ -91,7 +91,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n \ttest_commit test small \"a small file\" &&\n \tsmall_size=$(test_file_size small) &&\n \ttest_config filter.makelarge.smudge \\\n-\t\t\"test-tool genzeros $((5*1024*1024*1024)) && cat\" &&\n+\t\t\"test-tool genbytes NUL $((5*1024*1024*1024)) && cat\" &&\n \techo \"small filter=makelarge\" >.gitattributes &&\n \trm small &&\n \tgit checkout -- small &&\n@@ -103,7 +103,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n # the actual size, we ensure that cleaning doesn't mangle large files on 64-bit Windows.\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n \t\t'files over 4GB convert on input' '\n-\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n+\ttest-tool genbytes NUL $((5*1024*1024*1024)) >big &&\n \ttest_config filter.checklarge.clean \"wc -c >big.size\" &&\n \techo \"big filter=checklarge\" >.gitattributes &&\n \tgit add big &&\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex 3e9eb251cb..86b01d071c 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -1111,7 +1111,7 @@ test_expect_success 'fsck error on gitattributes with excessive line lengths' '\n '\n \n test_expect_success 'fsck error on gitattributes with excessive size' '\n-\tblob=$(test-tool genzeros $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n+\tblob=$(test-tool genbytes NUL $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n \ttest_when_finished \"remove_object $blob\" &&\n \ttree=$(printf \"100644 blob %s\\t%s\\n\" $blob .gitattributes | git mktree) &&\n \ttest_when_finished \"remove_object $tree\" &&\n@@ -1157,7 +1157,7 @@ test_expect_success 'fsck with blob size equal to core.bigFileThreshold' '\n \tgit init big-blob-fsck &&\n \t(\n \t\tcd big-blob-fsck &&\n-\t\ttest-tool genzeros 1024 >blob &&\n+\t\ttest-tool genbytes NUL 1024 >blob &&\n \t\tgit add blob &&\n \t\tgit commit -m \"blob equal to threshold\" &&\n \t\tgit repack -ad &&\ndiff --git a/t/t4141-apply-too-large.sh b/t/t4141-apply-too-large.sh\nindex 6bc1c7a663..309807859c 100755\n--- a/t/t4141-apply-too-large.sh\n+++ b/t/t4141-apply-too-large.sh\n@@ -13,7 +13,7 @@ test_expect_success EXPENSIVE 'git apply rejects patches that are too large' '\n \t\t+++ b/file\n \t\t@@ -0,0 +1 @@\n \t\tEOF\n-\t\ttest-tool genzeros $((1024 * 1024 * 1023))\n+\t\ttest-tool genbytes NUL $((1024 * 1024 * 1023))\n \t} | test_must_fail git apply 2>err &&\n \ttest_grep \"patch too large\" err\n '\ndiff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh\nindex 6cecfbeaa1..974bd21f1e 100755\n--- a/t/t5318-commit-graph.sh\n+++ b/t/t5318-commit-graph.sh\n@@ -522,7 +522,7 @@ corrupt_graph_and_verify() {\n \tzero_pos=${4:-${orig_size}} &&\n \tprintf \"$data\" | dd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$pos\" conv=notrunc &&\n \tdd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$zero_pos\" if=/dev/null &&\n-\ttest-tool genzeros $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n+\ttest-tool genbytes NUL $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n \tcorrupt_graph_verify \"$grepstr\"\n \n }\n-- \ngitgitgadget\n\n"},{"id":"557821","messageId":"xmqq1p8w9rbf.fsf@gitster.g","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-07T23:04:20Z","receivedAt":"2026-11-07T23:04:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> While looking for slow tests in CI, t4205 stood out on macOS: its EXPENSIVE\n> tests take several times longer there than on Linux. A CI experiment showed\n> that much of the time was spent in tr(1), which is very slow on macOS for\n> the 2GB of data that t4205 generates.\n>\n> Patch 1 renames test-genzeros.c so that the actual changes in patch 2 show\n> up as a regular diff. Patch 2 adds \"genbytes\" and uses it instead of\n> \"genzeros | tr\", which is where the speedup comes from.\n\nJust being curious.  Would forcing C locale on it help?  I.e.,\n\n\tgenzeros | LANG=C LC_ALL=C tr ...\n\n\n"},{"id":"557822","messageId":"xmqqwlqo8c8y.fsf@gitster.g","threadId":"66800","inReplyTo":"b4378b352769cdcc2e9cfdf40efe173123ea9ab4.1794063565.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-07T23:15:09Z","receivedAt":"2026-11-07T23:15:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> From: Kristofer Karlsson <krka@spotify.com>\n>\n> The next commit adds a \"genbytes\" command to this file and turns\n> \"genzeros\" into a thin wrapper around it.  Rename the file in a\n> separate step first, so that git can detect the rename and the actual\n> changes show up as a regular diff, which is easier to review.\n>\n> No functional changes.\n>\n> Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n> ---\n>  Makefile                                      | 2 +-\n>  t/helper/meson.build                          | 2 +-\n>  t/helper/{test-genzeros.c => test-genbytes.c} | 0\n>  3 files changed, 2 insertions(+), 2 deletions(-)\n>  rename t/helper/{test-genzeros.c => test-genbytes.c} (100%)\n\nNeedless to say, with this patch applied, the following tests would\nnow fail, unable to find \"test-tool genzeros\".\n\n    t0613-reftable-write-options.sh\n    t1007-hash-object.sh\n    t1051-large-conversion.sh\n    t1450-fsck.sh\n    t4141-apply-too-large.sh\n    t4205-log-pretty-formats.sh\n    t5300-pack-object.sh\n    t5318-commit-graph.sh\n    t7006-pager.sh\n    t9850-shell.sh\n\nFWIW, if you apply [1/3] and [2/3], squash them together, then give\na custom rename factor to the command, e.g.,\n\n    $ git format-patch -M3 -1\n\nto format the resulting single patch, you will see a modification of\ngenzeros while the file gets renamed just fine, and you would not\nhave to break these tests in the middle of a series.\n\n"},{"id":"557823","messageId":"xmqqjymo8bv6.fsf@gitster.g","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-07T23:23:25Z","receivedAt":"2026-11-07T23:23:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> Open question: is patch 3 wanted? It folds \"genzeros\" into \"genbytes\",\n> spelling the NUL byte as \"NUL\", so that there is a single command.\n\nI would suggest genbytes to take the value of the byte in decimal or\nhex, instead of av[1][0], so that you do not have to treat NUL any\nspecially.  I would actually think it is even better to limit the\ninput format only to [0-9a-f][0-9a-f], i.e., two hexadecimal digits\nthat does not allow uppercase to represent values 0-255 (inclusive),\nwithout support for any other input format like decimal.\n\nAfter all, this is just a test helper, so ergonomics or flexibility\ndo not matter as much as simplicity.\n\n> Some other decisions that may be worth pointing out:\n>\n>  * The buffer is no longer static: it used to be static so that it was\n>    zero-initialized for free, but now that it is filled with the requested\n>    byte on every call, a local buffer avoids shared state. It keeps its size\n>    of 256KB. Smaller buffers made no measurable difference in my tests, but\n>    there was no reason to change it.\n\nShouldn't matter.\n\n>  * Invalid counts are still silently treated as 0, as before. That could be\n>    tightened, but it is unrelated to this series.\n\nShouldn't matter.  That's ergonomics and flexibility that we do not\nvalue as much as in a normal program.\n\nThanks.\n\n"},{"id":"557832","messageId":"xmqqpkwg53us.fsf@gitster.g","threadId":"66800","inReplyTo":"xmqqwlqo8c8y.fsf@gitster.g","subject":"Re: [PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-08T04:44:43Z","receivedAt":"2026-11-08T04:44:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Needless to say, with this patch applied, the following tests would\n> now fail, unable to find \"test-tool genzeros\".\n\nNot really.  Even though the source file is renamed, the dispatcher\ninside still recognises genzeros and calls cmd__genzeros.  In fact,\nat this stage, \"test-tool genbytes\" would not even work (and it does\nnot have to work as nobody calls it).\n\nSo, sorry for the false alarm, but my point about not split the\nrename + modification into two separate steps still stands.\n\nThanks.\n\n>\n>     t0613-reftable-write-options.sh\n>     t1007-hash-object.sh\n>     t1051-large-conversion.sh\n>     t1450-fsck.sh\n>     t4141-apply-too-large.sh\n>     t4205-log-pretty-formats.sh\n>     t5300-pack-object.sh\n>     t5318-commit-graph.sh\n>     t7006-pager.sh\n>     t9850-shell.sh\n>\n> FWIW, if you apply [1/3] and [2/3], squash them together, then give\n> a custom rename factor to the command, e.g.,\n>\n>     $ git format-patch -M3 -1\n>\n> to format the resulting single patch, you will see a modification of\n> genzeros while the file gets renamed just fine, and you would not\n> have to break these tests in the middle of a series.\n\n"},{"id":"557834","messageId":"50b6c35f-e9c3-47ef-8fca-aebd2620623a@kdbg.org","threadId":"66800","inReplyTo":"4c02ada9b85028f26f7e7bcc62e2ac12443da9c5.1794063565.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-11-08T08:54:23Z","receivedAt":"2026-11-08T08:54:23Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 07.11.26 um 15:59 schrieb Kristofer Karlsson via GitGitGadget:\n> Several tests create fixture data consisting of a single repeated\n> character with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\n> works well on platforms where tr(1) is implemented efficiently.\n> \n> The implementation of tr(1) on macOS supports multi-byte characters,\n> and processes its input one character at a time through\n> getwchar() and putwchar(), even in the C locale.  For small inputs\n> that does not matter, but t4205 generates more than 2GB of data this\n> way, which makes that test much slower on macOS than on Linux: on the\n> macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\n> it, compared to about 6 seconds on Linux.\n> \n> Fix this by adding \"test-tool genbytes <byte> [<count>]\", which writes\n> the requested byte directly, and turn \"genzeros\" into a thin\n> wrapper around the same code.\n\nAll cases that do not need a particular character can use\n\n\tprintf %0.500d 0\n\nThough, on my Linux box,\n\n\tprintf %0.2147483649d 0\n\nproduces nothing, but\n\n\tprintf 000000000%0.2147483640d 0\n\nhas 2147483649 bytes of output.\n\nIf this isn't insanely slow on macOS, use this pattern instead of a\nhighly engineered helper program? ;-)\n\n-- Hannes\n\n\n"},{"id":"557835","messageId":"CAL71e4MiH62mpy0poRG5pSz=GPpoLWo-qPi_inaav9CgqDn5hg@mail.gmail.com","threadId":"66800","inReplyTo":"xmqqpkwg53us.fsf@gitster.g","subject":"Re: [PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T09:10:02Z","receivedAt":"2026-11-08T09:10:02Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 05:44, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n> > Needless to say, with this patch applied, the following tests would\n> > now fail, unable to find \"test-tool genzeros\".\n>\n> Not really.  Even though the source file is renamed, the dispatcher\n> inside still recognises genzeros and calls cmd__genzeros.  In fact,\n> at this stage, \"test-tool genbytes\" would not even work (and it does\n> not have to work as nobody calls it).\n>\n> So, sorry for the false alarm, but my point about not split the\n> rename + modification into two separate steps still stands.\n>\n[...]\n> > FWIW, if you apply [1/3] and [2/3], squash them together, then give\n> > a custom rename factor to the command, e.g.,\n> >\n> >     $ git format-patch -M3 -1\n> >\n> > to format the resulting single patch, you will see a modification of\n> > genzeros while the file gets renamed just fine, and you would not\n> > have to break these tests in the middle of a series.\n\nWhen using the GGG workflow I think that option is not available,\nthough I will see if I can try formatting and sending the patch\nmanually for this one instead.  (Or try to submit a patch to GGG\nto allow for configuring this too.)\n\nI could squash it all into a single patch too, which I think\nwould make sense if we don't need the intermediate state\nwhere both genzeros and genbytes exist at the same time.\n\nWhat's your preference?\n\nThanks,\nKristofer\n\n"},{"id":"557836","messageId":"CAL71e4PXCY6bDEW6LyXKkskurzRTjbfK_zf6DqPiRMVAt=Yixw@mail.gmail.com","threadId":"66800","inReplyTo":"xmqq1p8w9rbf.fsf@gitster.g","subject":"Re: [PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T09:11:50Z","receivedAt":"2026-11-08T09:11:50Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 00:04, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> Just being curious.  Would forcing C locale on it help?  I.e.,\n>\n>         genzeros | LANG=C LC_ALL=C tr ...\n>\n\nNo, test-lib-functions.sh already sets those env variables,\nbut the macos implementation of tr always executes the slow path\nunfortunately.\n\nThanks,\nKristofer\n\n"},{"id":"557837","messageId":"CAL71e4OGJWYPO5Wbcj16JwbhiXB0+EiAerH+ur19jWiM2hkyKg@mail.gmail.com","threadId":"66800","inReplyTo":"xmqqjymo8bv6.fsf@gitster.g","subject":"Re: [PATCH 0/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T09:13:59Z","receivedAt":"2026-11-08T09:13:59Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 00:23, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> > Open question: is patch 3 wanted? It folds \"genzeros\" into \"genbytes\",\n> > spelling the NUL byte as \"NUL\", so that there is a single command.\n>\n> I would suggest genbytes to take the value of the byte in decimal or\n> hex, instead of av[1][0], so that you do not have to treat NUL any\n> specially.  I would actually think it is even better to limit the\n> input format only to [0-9a-f][0-9a-f], i.e., two hexadecimal digits\n> that does not allow uppercase to represent values 0-255 (inclusive),\n> without support for any other input format like decimal.\n>\n> After all, this is just a test helper, so ergonomics or flexibility\n> do not matter as much as simplicity.\n\nYes, I actually had one local prototype that used this approach,\nbut I held off on it since it would mean a semantically larger\nchange.  But I agree it would improve the ergonomics, so I can do it.\nAnd it's not too many call sites to convert anyway.\n\nThanks,\nKristofer\n\n"},{"id":"557838","messageId":"CAL71e4NFyAShDZPW9JRB7Bpg3aLY5mY4wdn5m_ZYoLQjHZw_jA@mail.gmail.com","threadId":"66800","inReplyTo":"50b6c35f-e9c3-47ef-8fca-aebd2620623a@kdbg.org","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T09:18:27Z","receivedAt":"2026-11-08T09:18:27Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 09:54, Johannes Sixt <j6t@kdbg.org> wrote:\n>\n> Am 07.11.26 um 15:59 schrieb Kristofer Karlsson via GitGitGadget:\n> > Several tests create fixture data consisting of a single repeated\n> > character with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\n> > works well on platforms where tr(1) is implemented efficiently.\n> >\n> > The implementation of tr(1) on macOS supports multi-byte characters,\n> > and processes its input one character at a time through\n> > getwchar() and putwchar(), even in the C locale.  For small inputs\n> > that does not matter, but t4205 generates more than 2GB of data this\n> > way, which makes that test much slower on macOS than on Linux: on the\n> > macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\n> > it, compared to about 6 seconds on Linux.\n> >\n> > Fix this by adding \"test-tool genbytes <byte> [<count>]\", which writes\n> > the requested byte directly, and turn \"genzeros\" into a thin\n> > wrapper around the same code.\n>\n> All cases that do not need a particular character can use\n>\n>         printf %0.500d 0\n>\n> Though, on my Linux box,\n>\n>         printf %0.2147483649d 0\n>\n> produces nothing, but\n>\n>         printf 000000000%0.2147483640d 0\n>\n> has 2147483649 bytes of output.\n>\n> If this isn't insanely slow on macOS, use this pattern instead of a\n> highly engineered helper program? ;-)\n>\n> -- Hannes\n>\n\nNice trick, and it would work for t4205, which does not care about\nthe byte value.  I tried it for the 2147483649 bytes there on Linux:\n\n  printf           time   peak memory\n  dash             6.5s   2.1GB\n  bash             6.0s   3MB\n  coreutils        3.7s   2.1GB\n\nIt is definitely fast enough (on the variants I tested).\nThe annoying thing (but not a real blocker) is the extra memory\nusage required and I am not sure I fully trust that all\nimplementations will behave the same for absurdly large padding.\n\n(Particularly for paddings larger than 2^31, as you noted too.)\n\nIt feels a bit like a hack, and tweaking the existing test-tool\nfeels like a more sane option that is easier to reason about.\n\nA gotcha here is that the printf hack only works for producing\n\"0\", so for the cases we truly need NUL characters it does not\nhelp us, and I think that is also a point in favor of\nsimply generalizing test-tool genzeros instead.\n\nThanks,\nKristofer\n\n"},{"id":"557839","messageId":"6992b7e9-8721-4e1f-9c00-9fd0e0704e80@kdbg.org","threadId":"66800","inReplyTo":"CAL71e4NFyAShDZPW9JRB7Bpg3aLY5mY4wdn5m_ZYoLQjHZw_jA@mail.gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-11-08T09:35:15Z","receivedAt":"2026-11-08T09:35:15Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 08.11.26 um 10:18 schrieb Kristofer Karlsson:\n> On Sun, 8 Nov 2026 at 09:54, Johannes Sixt <j6t@kdbg.org> wrote:\n>>         printf 000000000%0.2147483640d 0\n> \n> It is definitely fast enough (on the variants I tested).\n> The annoying thing (but not a real blocker) is the extra memory\n> usage required and I am not sure I fully trust that all\n> implementations will behave the same for absurdly large padding.\n> \n> (Particularly for paddings larger than 2^31, as you noted too.)\n\nI don't know how to measure memory requirements, but if it helps,\n\n\tprintf 000000000%02147483640d 0\n\n(without the fullstop) works, too.\n\n> It feels a bit like a hack, and tweaking the existing test-tool\n> feels like a more sane option that is easier to reason about.\n> \n> A gotcha here is that the printf hack only works for producing\n> \"0\", so for the cases we truly need NUL characters it does not\n> help us, and I think that is also a point in favor of\n> simply generalizing test-tool genzeros instead.\nOn the other hand, the only case you really care about is the one that\ntakes an insane amount of time. If you convert only that case to printf,\nthen we do not need all this churn in the first place.\n\n-- Hannes\n\n\n"},{"id":"557840","messageId":"CAL71e4NKsTZNoLFOuSPfiJPf7oS57S01MpM5V+UOr+bgMGjOkQ@mail.gmail.com","threadId":"66800","inReplyTo":"6992b7e9-8721-4e1f-9c00-9fd0e0704e80@kdbg.org","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T12:24:16Z","receivedAt":"2026-11-08T12:24:16Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 10:35, Johannes Sixt <j6t@kdbg.org> wrote:\n>\n> On the other hand, the only case you really care about is the one that\n> takes an insane amount of time. If you convert only that case to printf,\n> then we do not need all this churn in the first place.\n\nThat's a good point -- if we want to make a minimal fix, we could\nsimply switch t4205 to use printf instead of \"genzeros | tr\".\n\n-       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+       printf 000000000%02147483640d 0 >expect &&\n\nI tried this out and it works on Linux but fails on macOS[1].  What\nhappens is that printf only outputs the nine literal zeros\n(\"000000000\") there, and silently ignores the very large padding.\n\n[1] https://github.com/gitgitgadget/git/actions/runs/42475956442/job/128717309586\n\nThe setup and the next test still pass, since the latter compares\nagainst the same file, and only the test expecting the exact size in\nthe error message notices.  So I do not think we can rely on printf\nfor this, and I still think that generalizing the existing helper is\na reasonable way forward.\n\nI think even if printf _would_ have worked, the end state of having\na uniform way of creating this fake data is nicer than having three\nseparate ways:\n\n  - genzeros when we only need NUL bytes\n  - genzeros | tr for small inputs that need some other byte\n  - printf for large inputs where \"0\" bytes are valid\n\n(And if we ever need to support large inputs with a specific byte\nvalue, we'd get stuck again, though anticipating such a need is\nspeculative.)\n\nThanks,\nKristofer\n\n"},{"id":"557844","messageId":"c8688e1a-17b6-4d0b-ba9a-fc1e0fa8938b@kdbg.org","threadId":"66800","inReplyTo":"CAL71e4NKsTZNoLFOuSPfiJPf7oS57S01MpM5V+UOr+bgMGjOkQ@mail.gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-11-08T16:34:02Z","receivedAt":"2026-11-08T16:34:02Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 08.11.26 um 13:24 schrieb Kristofer Karlsson:\n> -       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n> +       printf 000000000%02147483640d 0 >expect &&\n> \n> I tried this out and it works on Linux but fails on macOS[1].  What\n> happens is that printf only outputs the nine literal zeros\n> (\"000000000\") there, and silently ignores the very large padding.\n\nFair enough.\n\n> [1] https://github.com/gitgitgadget/git/actions/runs/42475956442/job/128717309586\n\n-- Hannes\n\n\n"},{"id":"557848","messageId":"xmqqcxsf41en.fsf@gitster.g","threadId":"66800","inReplyTo":"CAL71e4MiH62mpy0poRG5pSz=GPpoLWo-qPi_inaav9CgqDn5hg@mail.gmail.com","subject":"Re: [PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-08T18:35:12Z","receivedAt":"2026-11-08T18:35:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> When using the GGG workflow I think that option is not available,\n> though I will see if I can try formatting and sending the patch\n> manually for this one instead.  (Or try to submit a patch to GGG\n> to allow for configuring this too.)\n>\n> I could squash it all into a single patch too, which I think\n> would make sense if we don't need the intermediate state\n> where both genzeros and genbytes exist at the same time.\n>\n> What's your preference?\n\nI have no strong preference between keeping both genzeros() and\ngenbytes() or rewriting users of genzeros() to use genbytes(), and I\nprefer to leave that out of the decision.\n\nMy preference is to optimize for those who consume the resulting\nhistory the most over the longer term.\n\nYour split between a \"pure rename\" and a \"modification in place\"\nis optimizing for those who review the patches on the mailing list\n(arguably due to tool limitations).  While they are an important\naudience of yours (after all, without their support, your change\nwould not end up in the final history), those who will read the\nhistory via \"git log\" or \"git show\" to see what happened later are\nthe more important readers of your work in the long run.  To them,\na single rename-and-modify commit is much easier to understand, I\nwould think.\n\n\n"},{"id":"557849","messageId":"xmqq7bin41an.fsf@gitster.g","threadId":"66800","inReplyTo":"50b6c35f-e9c3-47ef-8fca-aebd2620623a@kdbg.org","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-08T18:37:36Z","receivedAt":"2026-11-08T18:37:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt <j6t@kdbg.org> writes:\n\n> All cases that do not need a particular character can use\n>\n> \tprintf %0.500d 0\n\nI read this \"printf is an easy and efficient way to create a long\nsequence of bytes if you do not care what value these bytes have\"\ndiscussion and found it very illuminating (including the mention of\nglitches that are platform specific).\n\nThanks.\n\n\n"},{"id":"557853","messageId":"CAL71e4NSAUQz=s8DVSK42=ZK-626qqVBs+YSJra9d+SLV9DPSg@mail.gmail.com","threadId":"66800","inReplyTo":"xmqqcxsf41en.fsf@gitster.g","subject":"Re: [PATCH 1/3] t/helper: rename test-genzeros.c to test-genbytes.c","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-08T19:10:59Z","receivedAt":"2026-11-08T19:10:59Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Sun, 8 Nov 2026 at 19:35, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> > I could squash it all into a single patch too, which I think\n> > would make sense if we don't need the intermediate state\n> > where both genzeros and genbytes exist at the same time.\n> >\n> > What's your preference?\n>\n> I have no strong preference between keeping both genzeros() and\n> genbytes() or rewriting users of genzeros() to use genbytes(), and I\n> prefer to leave that out of the decision.\n>\n> My preference is to optimize for those who consume the resulting\n> history the most over the longer term.\n>\n> Your split between a \"pure rename\" and a \"modification in place\"\n> is optimizing for those who review the patches on the mailing list\n> (arguably due to tool limitations).  While they are an important\n> audience of yours (after all, without their support, your change\n> would not end up in the final history), those who will read the\n> history via \"git log\" or \"git show\" to see what happened later are\n> the more important readers of your work in the long run.  To them,\n> a single rename-and-modify commit is much easier to understand, I\n> would think.\n\nThat's fair -- For v2 I will squash patches 1 and 2 and keep\npatch 3.  I'll also add the lowercase hex as input format.\nThe similarity seems to be ~29%, so running it with -M20% would\nonly mildly reduce the diff, though I think the diff is\nmanageable to consume anyway.\n\nThanks,\nKristofer\n\n"},{"id":"557855","messageId":"pull.2301.v2.git.1794165304.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH v2 0/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-08T19:15:02Z","receivedAt":"2026-11-08T19:15:02Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"While looking for slow tests in CI, t4205 stood out on macOS: its EXPENSIVE\ntests take several times longer there than on Linux. A CI experiment showed\nthat much of the time was spent in tr(1), which is very slow on macOS for\nthe 2GB of data that t4205 generates.\n\nPatch 1 adds \"test-tool genbytes\" and uses it instead of \"genzeros | tr\",\nwhich is where the speedup comes from. Patch 2 converts the remaining\n\"genzeros\" callers and removes it.\n\nChanges since v1:\n\n * Squashed the rename of test-genzeros.c into the patch that changes it, as\n   suggested by Junio. The file changes enough that the default rename\n   detection shows it as deleted and newly created; \"git format-patch -M30%\"\n   shows it as a rename.\n\n * \"genbytes\" now takes the byte as two lowercase hexadecimal digits instead\n   of a literal character, as suggested by Junio. This also works for NUL\n   bytes, so patch 2 now uses \"genbytes 00\" instead of special-casing \"NUL\".\n\nKristofer Karlsson (2):\n  t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n  t: replace \"test-tool genzeros\" with \"genbytes 00\"\n\n Makefile                          |  2 +-\n t/helper/meson.build              |  2 +-\n t/helper/test-genbytes.c          | 51 +++++++++++++++++++++++++++++++\n t/helper/test-genzeros.c          | 37 ----------------------\n t/helper/test-tool.c              |  2 +-\n t/helper/test-tool.h              |  2 +-\n t/t0613-reftable-write-options.sh |  2 +-\n t/t1007-hash-object.sh            |  8 ++---\n t/t1051-large-conversion.sh       |  4 +--\n t/t1450-fsck.sh                   |  4 +--\n t/t4141-apply-too-large.sh        |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 ++--\n t/t5318-commit-graph.sh           |  2 +-\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 16 files changed, 72 insertions(+), 59 deletions(-)\n create mode 100644 t/helper/test-genbytes.c\n delete mode 100644 t/helper/test-genzeros.c\n\n\nbase-commit: fb255012acbcbe81189cb8014da0280b09c66ecc\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2301%2Fspkrka%2Ftest-tool-genbytes-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2301/spkrka/test-tool-genbytes-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/2301\n\nRange-diff vs v1:\n\n 1:  b4378b3527 < -:  ---------- t/helper: rename test-genzeros.c to test-genbytes.c\n 2:  4c02ada9b8 ! 1:  54139a3a97 t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n     @@ Commit message\n          macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\n          it, compared to about 6 seconds on Linux.\n      \n     -    Fix this by adding \"test-tool genbytes <byte> [<count>]\", which writes\n     -    the requested byte directly, and turn \"genzeros\" into a thin\n     -    wrapper around the same code.  Use the new helper in all tests that\n     -    piped \"genzeros\" through tr(1), not just t4205, so that it is the\n     -    obvious choice for new tests.\n     +    Fix this by adding \"test-tool genbytes <hex-byte> [<count>]\", which\n     +    writes the requested byte, given as two lowercase hexadecimal\n     +    digits, directly.  Turn \"genzeros\" into a thin wrapper around the\n     +    same code, and rename the file to match.  Use the new helper in all\n     +    tests that piped \"genzeros\" through tr(1), not just t4205, so that\n     +    it is the obvious choice for new tests.\n      \n          This makes t4205 much faster on macOS in CI.  The Linux jobs, shown\n          for comparison, only save the few seconds that GNU tr(1) took, which\n     @@ Commit message\n      \n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n     - ## t/helper/test-genbytes.c ##\n     + ## Makefile ##\n     +@@ Makefile: TEST_BUILTINS_OBJS += test-env-helper.o\n     + TEST_BUILTINS_OBJS += test-example-tap.o\n     + TEST_BUILTINS_OBJS += test-find-pack.o\n     + TEST_BUILTINS_OBJS += test-fsmonitor-client.o\n     ++TEST_BUILTINS_OBJS += test-genbytes.o\n     + TEST_BUILTINS_OBJS += test-genrandom.o\n     +-TEST_BUILTINS_OBJS += test-genzeros.o\n     + TEST_BUILTINS_OBJS += test-getcwd.o\n     + TEST_BUILTINS_OBJS += test-hash-speed.o\n     + TEST_BUILTINS_OBJS += test-hash.o\n     +\n     + ## t/helper/meson.build ##\n     +@@ t/helper/meson.build: test_tool_sources = [\n     +   'test-example-tap.c',\n     +   'test-find-pack.c',\n     +   'test-fsmonitor-client.c',\n     ++  'test-genbytes.c',\n     +   'test-genrandom.c',\n     +-  'test-genzeros.c',\n     +   'test-getcwd.c',\n     +   'test-hash-speed.c',\n     +   'test-hash.c',\n     +\n     + ## t/helper/test-genbytes.c (new) ##\n      @@\n     - #include \"test-tool.h\"\n     - #include \"git-compat-util.h\"\n     - \n     --int cmd__genzeros(int argc, const char **argv)\n     ++#define DISABLE_SIGN_COMPARE_WARNINGS\n     ++\n     ++#include \"test-tool.h\"\n     ++#include \"git-compat-util.h\"\n     ++#include \"hex-ll.h\"\n     ++\n      +/*\n      + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n      + * \"count\" is negative.\n      + */\n      +static int generate_bytes(char byte, intmax_t count)\n     - {\n     --\t/* static, so that it is NUL-initialized */\n     --\tstatic const char zeros[256 * 1024];\n     --\tintmax_t count;\n     ++{\n      +\tchar buf[256 * 1024];\n     - \tssize_t n;\n     - \n     --\tif (argc > 2) {\n     --\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n     --\t\treturn 1;\n     --\t}\n     --\n     --\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n     ++\tssize_t n;\n     ++\n      +\tmemset(buf, byte, sizeof(buf));\n     - \n     --\t/* Writing out individual NUL bytes is slow... */\n     ++\n      +\t/* Writing out individual bytes is slow... */\n     - \twhile (count < 0)\n     --\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n     ++\twhile (count < 0)\n      +\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n     - \t\t\tdie_errno(\"write error\");\n     - \n     - \twhile (count > 0) {\n     --\t\tn = xwrite(1, zeros,\n     --\t\t\t   count < ARRAY_SIZE(zeros)\n     --\t\t\t   ? count : ARRAY_SIZE(zeros));\n     ++\t\t\tdie_errno(\"write error\");\n     ++\n     ++\twhile (count > 0) {\n      +\t\tn = xwrite(1, buf,\n      +\t\t\t   count < ARRAY_SIZE(buf)\n      +\t\t\t   ? count : ARRAY_SIZE(buf));\n     - \n     - \t\tif (n < 0)\n     - \t\t\tdie_errno(\"write error\");\n     -@@ t/helper/test-genbytes.c: int cmd__genzeros(int argc, const char **argv)\n     - \n     - \treturn 0;\n     - }\n     ++\n     ++\t\tif (n < 0)\n     ++\t\t\tdie_errno(\"write error\");\n     ++\n     ++\t\tcount -= n;\n     ++\t}\n     ++\n     ++\treturn 0;\n     ++}\n      +\n      +static intmax_t parse_count(int argc, const char **argv, int pos)\n      +{\n     @@ t/helper/test-genbytes.c: int cmd__genzeros(int argc, const char **argv)\n      +\n      +int cmd__genbytes(int argc, const char **argv)\n      +{\n     -+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1) {\n     -+\t\tfprintf(stderr, \"usage: %s <byte> [<count>]\\n\", argv[0]);\n     ++\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 2 ||\n     ++\t    strspn(argv[1], \"0123456789abcdef\") != 2) {\n     ++\t\tfprintf(stderr, \"usage: %s <hex-byte> [<count>]\\n\", argv[0]);\n      +\t\treturn 1;\n      +\t}\n      +\n     -+\treturn generate_bytes(argv[1][0], parse_count(argc, argv, 2));\n     ++\treturn generate_bytes(hex2chr(argv[1]), parse_count(argc, argv, 2));\n      +}\n      +\n      +int cmd__genzeros(int argc, const char **argv)\n     @@ t/helper/test-genbytes.c: int cmd__genzeros(int argc, const char **argv)\n      +\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n      +}\n      \n     + ## t/helper/test-genzeros.c (deleted) ##\n     +@@\n     +-#define DISABLE_SIGN_COMPARE_WARNINGS\n     +-\n     +-#include \"test-tool.h\"\n     +-#include \"git-compat-util.h\"\n     +-\n     +-int cmd__genzeros(int argc, const char **argv)\n     +-{\n     +-\t/* static, so that it is NUL-initialized */\n     +-\tstatic const char zeros[256 * 1024];\n     +-\tintmax_t count;\n     +-\tssize_t n;\n     +-\n     +-\tif (argc > 2) {\n     +-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n     +-\t\treturn 1;\n     +-\t}\n     +-\n     +-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n     +-\n     +-\t/* Writing out individual NUL bytes is slow... */\n     +-\twhile (count < 0)\n     +-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n     +-\t\t\tdie_errno(\"write error\");\n     +-\n     +-\twhile (count > 0) {\n     +-\t\tn = xwrite(1, zeros,\n     +-\t\t\t   count < ARRAY_SIZE(zeros)\n     +-\t\t\t   ? count : ARRAY_SIZE(zeros));\n     +-\n     +-\t\tif (n < 0)\n     +-\t\t\tdie_errno(\"write error\");\n     +-\n     +-\t\tcount -= n;\n     +-\t}\n     +-\n     +-\treturn 0;\n     +-}\n     +\n       ## t/helper/test-tool.c ##\n      @@ t/helper/test-tool.c: static struct test_cmd cmds[] = {\n       \t{ \"example-tap\", cmd__example_tap },\n     @@ t/t0613-reftable-write-options.sh: test_expect_success 'small block size fails w\n       \t\tcd repo &&\n       \t\ttest_commit A &&\n      -\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n     -+\t\ttest-tool genbytes a 500 >logmsg &&\n     ++\t\ttest-tool genbytes 61 500 >logmsg &&\n       \t\tcat >expect <<-EOF &&\n       \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n       \t\tEOF\n     @@ t/t4205-log-pretty-formats.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'lo\n       \n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n      -\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n     -+\ttest-tool genbytes 1 2147483649 >expect &&\n     ++\ttest-tool genbytes 31 2147483649 >expect &&\n       \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n       '\n       \n     @@ t/t5300-pack-object.sh: test_description='git pack-object'\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n     -+\ttest-tool genbytes a 4096 >a &&\n     -+\ttest-tool genbytes b 4096 >b &&\n     -+\ttest-tool genbytes c 4096 >c &&\n     ++\ttest-tool genbytes 61 4096 >a &&\n     ++\ttest-tool genbytes 62 4096 >b &&\n     ++\ttest-tool genbytes 63 4096 >c &&\n       \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n       \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n       \tgit update-index --add a a_big b b_big c &&\n     @@ t/t7006-pager.sh: test_expect_success 'setup trace2' '\n       test_expect_success 'setup large log output' '\n      -\ttest-tool genzeros 50000 |\n      -\ttr \"\\000\" \"a\" |\n     -+\ttest-tool genbytes a 50000 |\n     ++\ttest-tool genbytes 61 50000 |\n       \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n       \tgit commit --allow-empty -F commit-msg\n       '\n     @@ t/t9850-shell.sh: test_expect_success 'shell allows interactive command' '\n       \n       test_expect_success 'shell complains of overlong commands' '\n      -\ttest-tool genzeros | tr \"\\000\" \"a\" |\n     -+\ttest-tool genbytes a |\n     ++\ttest-tool genbytes 61 |\n       \ttest_must_fail git shell 2>err &&\n       \ttest_grep \"too long\" err\n       '\n 3:  169ca5968e ! 2:  f49af0e33f t: replace \"test-tool genzeros\" with \"genbytes NUL\"\n     @@ Metadata\n      Author: Kristofer Karlsson <krka@spotify.com>\n      \n       ## Commit message ##\n     -    t: replace \"test-tool genzeros\" with \"genbytes NUL\"\n     +    t: replace \"test-tool genzeros\" with \"genbytes 00\"\n      \n     -    Now that \"genbytes\" exists, \"genzeros\" is merely a special case of it\n     -    for NUL bytes.  It is only kept as a separate command because a NUL\n     -    byte cannot be passed as a command-line argument.\n     -\n     -    Teach \"genbytes\" to accept \"NUL\" as the name of that byte instead,\n     -    convert the remaining callers of \"genzeros\", and remove it, so that\n     -    there is a single command for generating repeated bytes.\n     +    Now that \"genbytes\" exists, \"genzeros\" is merely the special case of\n     +    it that writes NUL bytes.  Convert its remaining callers to\n     +    \"genbytes 00\" and remove it, so that there is a single command for\n     +    generating repeated bytes.\n      \n          Only some of these callers depend on the data being NUL bytes, like\n          t1007, which compares against the object ID of 5GB of zeros, and\n     @@ Commit message\n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n       ## t/helper/test-genbytes.c ##\n     -@@ t/helper/test-genbytes.c: static int generate_bytes(char byte, intmax_t count)\n     - \treturn 0;\n     - }\n     +@@ t/helper/test-genbytes.c: int cmd__genbytes(int argc, const char **argv)\n       \n     -+/*\n     -+ * A NUL byte cannot be passed as an argument, so it is spelled out as\n     -+ * \"NUL\".  Returns -1 for anything that is not a single byte.\n     -+ */\n     -+static int parse_byte(const char *arg)\n     -+{\n     -+\tif (!strcmp(arg, \"NUL\"))\n     -+\t\treturn '\\0';\n     -+\tif (strlen(arg) == 1)\n     -+\t\treturn (unsigned char)arg[0];\n     -+\treturn -1;\n     -+}\n     -+\n     - static intmax_t parse_count(int argc, const char **argv, int pos)\n     - {\n     - \treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n     -@@ t/helper/test-genbytes.c: static intmax_t parse_count(int argc, const char **argv, int pos)\n     - \n     - int cmd__genbytes(int argc, const char **argv)\n     - {\n     --\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1) {\n     --\t\tfprintf(stderr, \"usage: %s <byte> [<count>]\\n\", argv[0]);\n     --\t\treturn 1;\n     --\t}\n     -+\tint byte;\n     - \n     --\treturn generate_bytes(argv[1][0], parse_count(argc, argv, 2));\n     --}\n     + \treturn generate_bytes(hex2chr(argv[1]), parse_count(argc, argv, 2));\n     + }\n      -\n      -int cmd__genzeros(int argc, const char **argv)\n      -{\n      -\tif (argc > 2) {\n      -\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n     -+\tif (argc < 2 || argc > 3 || (byte = parse_byte(argv[1])) < 0) {\n     -+\t\tfprintf(stderr, \"usage: %s (<byte> | NUL) [<count>]\\n\", argv[0]);\n     - \t\treturn 1;\n     - \t}\n     - \n     +-\t\treturn 1;\n     +-\t}\n     +-\n      -\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n     -+\treturn generate_bytes(byte, parse_count(argc, argv, 2));\n     - }\n     +-}\n      \n       ## t/helper/test-tool.c ##\n      @@ t/helper/test-tool.c: static struct test_cmd cmds[] = {\n     @@ t/t1007-hash-object.sh: test_expect_success '--stdin outside of repository (uses\n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       \t\t'files over 4GB hash literally' '\n      -\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n     -+\ttest-tool genbytes NUL $((5*1024*1024*1024)) >big &&\n     ++\ttest-tool genbytes 00 $((5*1024*1024*1024)) >big &&\n       \ttest_oid large5GB >expect &&\n       \tgit hash-object --stdin --literally <big >actual &&\n       \ttest_cmp expect actual\n     @@ t/t1007-hash-object.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       \t\t'files over 4GB hash correctly via --stdin' '\n      -\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n     -+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n     ++\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n       \ttest_oid large5GB >expect &&\n       \tgit hash-object --stdin <big >actual &&\n       \ttest_cmp expect actual\n     @@ t/t1007-hash-object.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       \t\t'files over 4GB hash correctly' '\n      -\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n     -+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n     ++\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n       \ttest_oid large5GB >expect &&\n       \tgit hash-object -- big >actual &&\n       \ttest_cmp expect actual\n     @@ t/t1007-hash-object.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n       \t\t'hash filtered files over 4GB correctly' '\n      -\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n     -+\t{ test -f big || test-tool genbytes NUL $((5*1024*1024*1024)) >big; } &&\n     ++\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n       \ttest_oid large5GB >expect &&\n       \ttest_config filter.null-filter.clean \"cat\" &&\n       \techo \"big filter=null-filter\" >.gitattributes &&\n     @@ t/t1051-large-conversion.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG\n       \tsmall_size=$(test_file_size small) &&\n       \ttest_config filter.makelarge.smudge \\\n      -\t\t\"test-tool genzeros $((5*1024*1024*1024)) && cat\" &&\n     -+\t\t\"test-tool genbytes NUL $((5*1024*1024*1024)) && cat\" &&\n     ++\t\t\"test-tool genbytes 00 $((5*1024*1024*1024)) && cat\" &&\n       \techo \"small filter=makelarge\" >.gitattributes &&\n       \trm small &&\n       \tgit checkout -- small &&\n     @@ t/t1051-large-conversion.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG\n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n       \t\t'files over 4GB convert on input' '\n      -\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n     -+\ttest-tool genbytes NUL $((5*1024*1024*1024)) >big &&\n     ++\ttest-tool genbytes 00 $((5*1024*1024*1024)) >big &&\n       \ttest_config filter.checklarge.clean \"wc -c >big.size\" &&\n       \techo \"big filter=checklarge\" >.gitattributes &&\n       \tgit add big &&\n     @@ t/t1450-fsck.sh: test_expect_success 'fsck error on gitattributes with excessive\n       \n       test_expect_success 'fsck error on gitattributes with excessive size' '\n      -\tblob=$(test-tool genzeros $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n     -+\tblob=$(test-tool genbytes NUL $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n     ++\tblob=$(test-tool genbytes 00 $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n       \ttest_when_finished \"remove_object $blob\" &&\n       \ttree=$(printf \"100644 blob %s\\t%s\\n\" $blob .gitattributes | git mktree) &&\n       \ttest_when_finished \"remove_object $tree\" &&\n     @@ t/t1450-fsck.sh: test_expect_success 'fsck with blob size equal to core.bigFileT\n       \t(\n       \t\tcd big-blob-fsck &&\n      -\t\ttest-tool genzeros 1024 >blob &&\n     -+\t\ttest-tool genbytes NUL 1024 >blob &&\n     ++\t\ttest-tool genbytes 00 1024 >blob &&\n       \t\tgit add blob &&\n       \t\tgit commit -m \"blob equal to threshold\" &&\n       \t\tgit repack -ad &&\n     @@ t/t4141-apply-too-large.sh: test_expect_success EXPENSIVE 'git apply rejects pat\n       \t\t@@ -0,0 +1 @@\n       \t\tEOF\n      -\t\ttest-tool genzeros $((1024 * 1024 * 1023))\n     -+\t\ttest-tool genbytes NUL $((1024 * 1024 * 1023))\n     ++\t\ttest-tool genbytes 00 $((1024 * 1024 * 1023))\n       \t} | test_must_fail git apply 2>err &&\n       \ttest_grep \"patch too large\" err\n       '\n     @@ t/t5318-commit-graph.sh: corrupt_graph_and_verify() {\n       \tprintf \"$data\" | dd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$pos\" conv=notrunc &&\n       \tdd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$zero_pos\" if=/dev/null &&\n      -\ttest-tool genzeros $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n     -+\ttest-tool genbytes NUL $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n     ++\ttest-tool genbytes 00 $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n       \tcorrupt_graph_verify \"$grepstr\"\n       \n       }\n\n-- \ngitgitgadget\n\n"},{"id":"557856","messageId":"54139a3a97127732e11fc5abbc9a3687542669f6.1794165304.git.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.v2.git.1794165304.gitgitgadget@gmail.com","subject":"[PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-08T19:15:03Z","receivedAt":"2026-11-08T19:15:03Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nSeveral tests create fixture data consisting of a single repeated\ncharacter with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\nworks well on platforms where tr(1) is implemented efficiently.\n\nThe implementation of tr(1) on macOS supports multi-byte characters,\nand processes its input one character at a time through\ngetwchar() and putwchar(), even in the C locale.  For small inputs\nthat does not matter, but t4205 generates more than 2GB of data this\nway, which makes that test much slower on macOS than on Linux: on the\nmacOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\nit, compared to about 6 seconds on Linux.\n\nFix this by adding \"test-tool genbytes <hex-byte> [<count>]\", which\nwrites the requested byte, given as two lowercase hexadecimal\ndigits, directly.  Turn \"genzeros\" into a thin wrapper around the\nsame code, and rename the file to match.  Use the new helper in all\ntests that piped \"genzeros\" through tr(1), not just t4205, so that\nit is the obvious choice for new tests.\n\nThis makes t4205 much faster on macOS in CI.  The Linux jobs, shown\nfor comparison, only save the few seconds that GNU tr(1) took, which\nis within the usual noise:\n\n  job                before    after   change\n  osx-gcc            821.1s   243.3s   -70.4%\n  osx-reftable       911.0s   194.0s   -78.7%\n  linux-asan-ubsan   226.7s   222.1s    -2.0%\n  linux-leaks        320.2s   298.8s    -6.7%\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n Makefile                          |  2 +-\n t/helper/meson.build              |  2 +-\n t/helper/test-genbytes.c          | 61 +++++++++++++++++++++++++++++++\n t/helper/test-genzeros.c          | 37 -------------------\n t/helper/test-tool.c              |  1 +\n t/helper/test-tool.h              |  1 +\n t/t0613-reftable-write-options.sh |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 +--\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 11 files changed, 72 insertions(+), 47 deletions(-)\n create mode 100644 t/helper/test-genbytes.c\n delete mode 100644 t/helper/test-genzeros.c\n\ndiff --git a/Makefile b/Makefile\nindex c2a6098484..8d214286e0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -825,8 +825,8 @@ TEST_BUILTINS_OBJS += test-env-helper.o\n TEST_BUILTINS_OBJS += test-example-tap.o\n TEST_BUILTINS_OBJS += test-find-pack.o\n TEST_BUILTINS_OBJS += test-fsmonitor-client.o\n+TEST_BUILTINS_OBJS += test-genbytes.o\n TEST_BUILTINS_OBJS += test-genrandom.o\n-TEST_BUILTINS_OBJS += test-genzeros.o\n TEST_BUILTINS_OBJS += test-getcwd.o\n TEST_BUILTINS_OBJS += test-hash-speed.o\n TEST_BUILTINS_OBJS += test-hash.o\ndiff --git a/t/helper/meson.build b/t/helper/meson.build\nindex ae513b4cdc..f99688c49b 100644\n--- a/t/helper/meson.build\n+++ b/t/helper/meson.build\n@@ -22,8 +22,8 @@ test_tool_sources = [\n   'test-example-tap.c',\n   'test-find-pack.c',\n   'test-fsmonitor-client.c',\n+  'test-genbytes.c',\n   'test-genrandom.c',\n-  'test-genzeros.c',\n   'test-getcwd.c',\n   'test-hash-speed.c',\n   'test-hash.c',\ndiff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\nnew file mode 100644\nindex 0000000000..b760326b24\n--- /dev/null\n+++ b/t/helper/test-genbytes.c\n@@ -0,0 +1,61 @@\n+#define DISABLE_SIGN_COMPARE_WARNINGS\n+\n+#include \"test-tool.h\"\n+#include \"git-compat-util.h\"\n+#include \"hex-ll.h\"\n+\n+/*\n+ * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n+ * \"count\" is negative.\n+ */\n+static int generate_bytes(char byte, intmax_t count)\n+{\n+\tchar buf[256 * 1024];\n+\tssize_t n;\n+\n+\tmemset(buf, byte, sizeof(buf));\n+\n+\t/* Writing out individual bytes is slow... */\n+\twhile (count < 0)\n+\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n+\t\t\tdie_errno(\"write error\");\n+\n+\twhile (count > 0) {\n+\t\tn = xwrite(1, buf,\n+\t\t\t   count < ARRAY_SIZE(buf)\n+\t\t\t   ? count : ARRAY_SIZE(buf));\n+\n+\t\tif (n < 0)\n+\t\t\tdie_errno(\"write error\");\n+\n+\t\tcount -= n;\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static intmax_t parse_count(int argc, const char **argv, int pos)\n+{\n+\treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n+}\n+\n+int cmd__genbytes(int argc, const char **argv)\n+{\n+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 2 ||\n+\t    strspn(argv[1], \"0123456789abcdef\") != 2) {\n+\t\tfprintf(stderr, \"usage: %s <hex-byte> [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes(hex2chr(argv[1]), parse_count(argc, argv, 2));\n+}\n+\n+int cmd__genzeros(int argc, const char **argv)\n+{\n+\tif (argc > 2) {\n+\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n+}\ndiff --git a/t/helper/test-genzeros.c b/t/helper/test-genzeros.c\ndeleted file mode 100644\nindex b895436a32..0000000000\n--- a/t/helper/test-genzeros.c\n+++ /dev/null\n@@ -1,37 +0,0 @@\n-#define DISABLE_SIGN_COMPARE_WARNINGS\n-\n-#include \"test-tool.h\"\n-#include \"git-compat-util.h\"\n-\n-int cmd__genzeros(int argc, const char **argv)\n-{\n-\t/* static, so that it is NUL-initialized */\n-\tstatic const char zeros[256 * 1024];\n-\tintmax_t count;\n-\tssize_t n;\n-\n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n-\n-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n-\n-\t/* Writing out individual NUL bytes is slow... */\n-\twhile (count < 0)\n-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n-\t\t\tdie_errno(\"write error\");\n-\n-\twhile (count > 0) {\n-\t\tn = xwrite(1, zeros,\n-\t\t\t   count < ARRAY_SIZE(zeros)\n-\t\t\t   ? count : ARRAY_SIZE(zeros));\n-\n-\t\tif (n < 0)\n-\t\t\tdie_errno(\"write error\");\n-\n-\t\tcount -= n;\n-\t}\n-\n-\treturn 0;\n-}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex b71a22b43b..af70e2bec7 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -33,6 +33,7 @@ static struct test_cmd cmds[] = {\n \t{ \"example-tap\", cmd__example_tap },\n \t{ \"find-pack\", cmd__find_pack },\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n+\t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n \t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex f2885b33d5..c35e7412af 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -26,6 +26,7 @@ int cmd__env_helper(int argc, const char **argv);\n int cmd__example_tap(int argc, const char **argv);\n int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n+int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex 0b27d75fe3..b2032a799d 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n \t(\n \t\tcd repo &&\n \t\ttest_commit A &&\n-\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n+\t\ttest-tool genbytes 61 500 >logmsg &&\n \t\tcat >expect <<-EOF &&\n \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n \t\tEOF\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 01b97c8888..aa51a70a34 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1182,7 +1182,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n-\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+\ttest-tool genbytes 31 2147483649 >expect &&\n \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n '\n \ndiff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\nindex aac139e6a0..3c2f83091c 100755\n--- a/t/t5300-pack-object.sh\n+++ b/t/t5300-pack-object.sh\n@@ -9,9 +9,9 @@ test_description='git pack-object'\n \n test_expect_success 'setup' '\n \trm -f .git/index* &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n+\ttest-tool genbytes 61 4096 >a &&\n+\ttest-tool genbytes 62 4096 >b &&\n+\ttest-tool genbytes 63 4096 >c &&\n \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n \tgit update-index --add a a_big b b_big c &&\ndiff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\nindex 75ca7a25da..f59b3a00a8 100755\n--- a/t/t7006-pager.sh\n+++ b/t/t7006-pager.sh\n@@ -662,8 +662,7 @@ test_expect_success 'setup trace2' '\n '\n \n test_expect_success 'setup large log output' '\n-\ttest-tool genzeros 50000 |\n-\ttr \"\\000\" \"a\" |\n+\ttest-tool genbytes 61 50000 |\n \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n \tgit commit --allow-empty -F commit-msg\n '\ndiff --git a/t/t9850-shell.sh b/t/t9850-shell.sh\nindex 27cc49bbd1..cf05944682 100755\n--- a/t/t9850-shell.sh\n+++ b/t/t9850-shell.sh\n@@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '\n '\n \n test_expect_success 'shell complains of overlong commands' '\n-\ttest-tool genzeros | tr \"\\000\" \"a\" |\n+\ttest-tool genbytes 61 |\n \ttest_must_fail git shell 2>err &&\n \ttest_grep \"too long\" err\n '\n-- \ngitgitgadget\n\n\n"},{"id":"557857","messageId":"f49af0e33fcbc298e8ab66924fcea81bc47ded97.1794165304.git.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.v2.git.1794165304.gitgitgadget@gmail.com","subject":"[PATCH v2 2/2] t: replace \"test-tool genzeros\" with \"genbytes 00\"","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-08T19:15:04Z","receivedAt":"2026-11-08T19:15:04Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nNow that \"genbytes\" exists, \"genzeros\" is merely the special case of\nit that writes NUL bytes.  Convert its remaining callers to\n\"genbytes 00\" and remove it, so that there is a single command for\ngenerating repeated bytes.\n\nOnly some of these callers depend on the data being NUL bytes, like\nt1007, which compares against the object ID of 5GB of zeros, and\nt5318, which zeroes out part of a commit-graph file.  The others only\ncare about the size, but keep generating NUL bytes there, too, so\nthat what they test does not change.\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n t/helper/test-genbytes.c    | 10 ----------\n t/helper/test-tool.c        |  1 -\n t/helper/test-tool.h        |  1 -\n t/t1007-hash-object.sh      |  8 ++++----\n t/t1051-large-conversion.sh |  4 ++--\n t/t1450-fsck.sh             |  4 ++--\n t/t4141-apply-too-large.sh  |  2 +-\n t/t5318-commit-graph.sh     |  2 +-\n 8 files changed, 10 insertions(+), 22 deletions(-)\n\ndiff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\nindex b760326b24..898d73c516 100644\n--- a/t/helper/test-genbytes.c\n+++ b/t/helper/test-genbytes.c\n@@ -49,13 +49,3 @@ int cmd__genbytes(int argc, const char **argv)\n \n \treturn generate_bytes(hex2chr(argv[1]), parse_count(argc, argv, 2));\n }\n-\n-int cmd__genzeros(int argc, const char **argv)\n-{\n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n-\n-\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n-}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex af70e2bec7..16571fd702 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -35,7 +35,6 @@ static struct test_cmd cmds[] = {\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n \t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n-\t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\n \t{ \"hashmap\", cmd__hashmap },\n \t{ \"hash-speed\", cmd__hash_speed },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex c35e7412af..1b45c8c37b 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -28,7 +28,6 @@ int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n-int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\n int cmd__hashmap(int argc, const char **argv);\n int cmd__hash_speed(int argc, const char **argv);\ndiff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh\nindex 463b38f990..9e28bcc4f0 100755\n--- a/t/t1007-hash-object.sh\n+++ b/t/t1007-hash-object.sh\n@@ -263,7 +263,7 @@ test_expect_success '--stdin outside of repository (uses default hash)' '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash literally' '\n-\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n+\ttest-tool genbytes 00 $((5*1024*1024*1024)) >big &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object --stdin --literally <big >actual &&\n \ttest_cmp expect actual\n@@ -271,7 +271,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash correctly via --stdin' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object --stdin <big >actual &&\n \ttest_cmp expect actual\n@@ -279,7 +279,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'files over 4GB hash correctly' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \tgit hash-object -- big >actual &&\n \ttest_cmp expect actual\n@@ -289,7 +289,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n # We ensure that cleaning doesn't mangle large files on 64-bit Windows.\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \\\n \t\t'hash filtered files over 4GB correctly' '\n-\t{ test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } &&\n+\t{ test -f big || test-tool genbytes 00 $((5*1024*1024*1024)) >big; } &&\n \ttest_oid large5GB >expect &&\n \ttest_config filter.null-filter.clean \"cat\" &&\n \techo \"big filter=null-filter\" >.gitattributes &&\ndiff --git a/t/t1051-large-conversion.sh b/t/t1051-large-conversion.sh\nindex 361afb679b..dd5902b3b3 100755\n--- a/t/t1051-large-conversion.sh\n+++ b/t/t1051-large-conversion.sh\n@@ -91,7 +91,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n \ttest_commit test small \"a small file\" &&\n \tsmall_size=$(test_file_size small) &&\n \ttest_config filter.makelarge.smudge \\\n-\t\t\"test-tool genzeros $((5*1024*1024*1024)) && cat\" &&\n+\t\t\"test-tool genbytes 00 $((5*1024*1024*1024)) && cat\" &&\n \techo \"small filter=makelarge\" >.gitattributes &&\n \trm small &&\n \tgit checkout -- small &&\n@@ -103,7 +103,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n # the actual size, we ensure that cleaning doesn't mangle large files on 64-bit Windows.\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \\\n \t\t'files over 4GB convert on input' '\n-\ttest-tool genzeros $((5*1024*1024*1024)) >big &&\n+\ttest-tool genbytes 00 $((5*1024*1024*1024)) >big &&\n \ttest_config filter.checklarge.clean \"wc -c >big.size\" &&\n \techo \"big filter=checklarge\" >.gitattributes &&\n \tgit add big &&\ndiff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh\nindex 3e9eb251cb..ae21cb897f 100755\n--- a/t/t1450-fsck.sh\n+++ b/t/t1450-fsck.sh\n@@ -1111,7 +1111,7 @@ test_expect_success 'fsck error on gitattributes with excessive line lengths' '\n '\n \n test_expect_success 'fsck error on gitattributes with excessive size' '\n-\tblob=$(test-tool genzeros $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n+\tblob=$(test-tool genbytes 00 $((100 * 1024 * 1024 + 1)) | git hash-object -w --stdin) &&\n \ttest_when_finished \"remove_object $blob\" &&\n \ttree=$(printf \"100644 blob %s\\t%s\\n\" $blob .gitattributes | git mktree) &&\n \ttest_when_finished \"remove_object $tree\" &&\n@@ -1157,7 +1157,7 @@ test_expect_success 'fsck with blob size equal to core.bigFileThreshold' '\n \tgit init big-blob-fsck &&\n \t(\n \t\tcd big-blob-fsck &&\n-\t\ttest-tool genzeros 1024 >blob &&\n+\t\ttest-tool genbytes 00 1024 >blob &&\n \t\tgit add blob &&\n \t\tgit commit -m \"blob equal to threshold\" &&\n \t\tgit repack -ad &&\ndiff --git a/t/t4141-apply-too-large.sh b/t/t4141-apply-too-large.sh\nindex 6bc1c7a663..52639a62b0 100755\n--- a/t/t4141-apply-too-large.sh\n+++ b/t/t4141-apply-too-large.sh\n@@ -13,7 +13,7 @@ test_expect_success EXPENSIVE 'git apply rejects patches that are too large' '\n \t\t+++ b/file\n \t\t@@ -0,0 +1 @@\n \t\tEOF\n-\t\ttest-tool genzeros $((1024 * 1024 * 1023))\n+\t\ttest-tool genbytes 00 $((1024 * 1024 * 1023))\n \t} | test_must_fail git apply 2>err &&\n \ttest_grep \"patch too large\" err\n '\ndiff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh\nindex 6cecfbeaa1..37b903359c 100755\n--- a/t/t5318-commit-graph.sh\n+++ b/t/t5318-commit-graph.sh\n@@ -522,7 +522,7 @@ corrupt_graph_and_verify() {\n \tzero_pos=${4:-${orig_size}} &&\n \tprintf \"$data\" | dd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$pos\" conv=notrunc &&\n \tdd of=\"full/$objdir/info/commit-graph\" bs=1 seek=\"$zero_pos\" if=/dev/null &&\n-\ttest-tool genzeros $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n+\ttest-tool genbytes 00 $(($orig_size - $zero_pos)) >>\"full/$objdir/info/commit-graph\" &&\n \tcorrupt_graph_verify \"$grepstr\"\n \n }\n-- \ngitgitgadget\n\n"},{"id":"557868","messageId":"xmqqeccu3hqq.fsf@gitster.g","threadId":"66800","inReplyTo":"54139a3a97127732e11fc5abbc9a3687542669f6.1794165304.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-09T01:39:57Z","receivedAt":"2026-11-09T01:39:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> diff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\n> new file mode 100644\n> index 0000000000..b760326b24\n> --- /dev/null\n> +++ b/t/helper/test-genbytes.c\n> @@ -0,0 +1,61 @@\n> +#define DISABLE_SIGN_COMPARE_WARNINGS\n> +\n> +#include \"test-tool.h\"\n> +#include \"git-compat-util.h\"\n> +#include \"hex-ll.h\"\n> +\n> +/*\n> + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n> + * \"count\" is negative.\n> + */\n> +static int generate_bytes(char byte, intmax_t count)\n> +{\n> +\tchar buf[256 * 1024];\n\nAllocating 256kB on stack may not be a great practice, but this may\nsufficiently work on most reasonable systems, as this is an internal\nof a \"run once and exit\" test helper function.\n\nStill, the stack memory for the primary thread of a process is\nlimited to 8MB or so on modern Linux systems, and I hear macOS also\nuses 8MB.  I am not familiar with Windows, but some documentation\nfound on the Web seems to say that their linker gives 1MB to their\nproduct by default, and among 1MB, 256kB is not an insignificant\nfraction.\n\nAgain, this is an internal of a \"run once and exit\" test helper, so\ndeclaring this as \"static char buf[]\" should not pose an issue; it is\nnot like you would need to have multiple simultaneous invocations of\nthis function, each of which needs to fill this buffer with\ndifferent values.\n\n"},{"id":"557877","messageId":"avFvYD5K4SKlveiI@pks.im","threadId":"66800","inReplyTo":"54139a3a97127732e11fc5abbc9a3687542669f6.1794165304.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-09T06:59:44Z","receivedAt":"2026-11-09T06:59:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Nov 08, 2026 at 07:15:03PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n[snip]\n> This makes t4205 much faster on macOS in CI.  The Linux jobs, shown\n> for comparison, only save the few seconds that GNU tr(1) took, which\n> is within the usual noise:\n> \n>   job                before    after   change\n>   osx-gcc            821.1s   243.3s   -70.4%\n>   osx-reftable       911.0s   194.0s   -78.7%\n>   linux-asan-ubsan   226.7s   222.1s    -2.0%\n>   linux-leaks        320.2s   298.8s    -6.7%\n\nNice improvement.\n\n> diff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\n> new file mode 100644\n> index 0000000000..b760326b24\n> --- /dev/null\n> +++ b/t/helper/test-genbytes.c\n> @@ -0,0 +1,61 @@\n\nOkay. The diff is unfortunately quite useless as we do the file rename\nplus all the changes in a single step. It might've been helpful to split\nthis up into:\n\n  1. A rename of the file.\n\n  2. A refactoring of `cmd_genzeros()` to use the write buffer.\n\n  3. A change to introduce `cmd_genbytes()`.\n\nBut maybe I'm also overdoing this.\n\n> +#define DISABLE_SIGN_COMPARE_WARNINGS\n> +\n> +#include \"test-tool.h\"\n> +#include \"git-compat-util.h\"\n> +#include \"hex-ll.h\"\n> +\n> +/*\n> + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n> + * \"count\" is negative.\n> + */\n> +static int generate_bytes(char byte, intmax_t count)\n> +{\n> +\tchar buf[256 * 1024];\n> +\tssize_t n;\n> +\n> +\tmemset(buf, byte, sizeof(buf));\n> +\n> +\t/* Writing out individual bytes is slow... */\n> +\twhile (count < 0)\n> +\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n> +\t\t\tdie_errno(\"write error\");\n> +\n> +\twhile (count > 0) {\n> +\t\tn = xwrite(1, buf,\n> +\t\t\t   count < ARRAY_SIZE(buf)\n> +\t\t\t   ? count : ARRAY_SIZE(buf));\n> +\n> +\t\tif (n < 0)\n> +\t\t\tdie_errno(\"write error\");\n> +\n> +\t\tcount -= n;\n> +\t}\n\nOkay. These two loops are a bit ugly if you ask me, but they are carried\nover from preexisting code. So this is good enough.\n\n> +\treturn 0;\n> +}\n> +\n> +static intmax_t parse_count(int argc, const char **argv, int pos)\n> +{\n> +\treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n> +}\n\nWe can use `git_parse_signed()` instead.\n\n> diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\n> index 0b27d75fe3..b2032a799d 100755\n> --- a/t/t0613-reftable-write-options.sh\n> +++ b/t/t0613-reftable-write-options.sh\n> @@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n>  \t(\n>  \t\tcd repo &&\n>  \t\ttest_commit A &&\n> -\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n> +\t\ttest-tool genbytes 61 500 >logmsg &&\n>  \t\tcat >expect <<-EOF &&\n>  \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n>  \t\tEOF\n\nI wonder about the regression in readability though. Maybe that's just\nme, but I don't happen to know all ASCII values by heart, so if I were\nto read this I would now wonder what's so special about 61. And the next\nthing to wonder about is whether this is supposed to encode \"=\" (decimal\n61) or \"a\" (hexadecimal 61), and without reading the code of genbytes I\nwould not be able to tell.\n\nSo can we maybe make the underpinning logic a bit more intelligent, e.g.\nby accepting \"\\NNN\" or \"\\xNN\" syntax instead of two hex characters? That\nmight also make it more extensible going forward if the next developer\nwants to make genbytes take and repeat strings of arbitrary length\nas-is.\n\nPatrick\n\n"},{"id":"557879","messageId":"CAL71e4PkS0dRvqSkL81hSPO3M3K2n5ZTumFNvCx9MsBsFKbnmw@mail.gmail.com","threadId":"66800","inReplyTo":"xmqqeccu3hqq.fsf@gitster.g","subject":"Re: [PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-09T07:10:45Z","receivedAt":"2026-11-09T07:10:45Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 9 Nov 2026 at 02:39, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> > diff --git a/t/helper/test-genbytes.c b/t/helper/test-genbytes.c\n> > new file mode 100644\n> > index 0000000000..b760326b24\n> > --- /dev/null\n> > +++ b/t/helper/test-genbytes.c\n> > @@ -0,0 +1,61 @@\n> > +#define DISABLE_SIGN_COMPARE_WARNINGS\n> > +\n> > +#include \"test-tool.h\"\n> > +#include \"git-compat-util.h\"\n> > +#include \"hex-ll.h\"\n> > +\n> > +/*\n> > + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n> > + * \"count\" is negative.\n> > + */\n> > +static int generate_bytes(char byte, intmax_t count)\n> > +{\n> > +     char buf[256 * 1024];\n>\n> Allocating 256kB on stack may not be a great practice, but this may\n> sufficiently work on most reasonable systems, as this is an internal\n> of a \"run once and exit\" test helper function.\n>\n> Still, the stack memory for the primary thread of a process is\n> limited to 8MB or so on modern Linux systems, and I hear macOS also\n> uses 8MB.  I am not familiar with Windows, but some documentation\n> found on the Web seems to say that their linker gives 1MB to their\n> product by default, and among 1MB, 256kB is not an insignificant\n> fraction.\n>\n> Again, this is an internal of a \"run once and exit\" test helper, so\n> declaring this as \"static char buf[]\" should not pose an issue; it is\n> not like you would need to have multiple simultaneous invocations of\n> this function, each of which needs to fill this buffer with\n> different values.\n\nYes, I think we have a few options to choose between:\n\n  1. keep the previous static.  Technically this means that we\n     should also document it as not being thread-safe.\n     Currently it does not matter, since it's never called from\n     anywhere else, but there is a slight code smell to it\n     since the buffer no longer hosts a constant value.\n\n  2. remove static and keep the size.  Since the stack is not\n     complex or deep here it should be safe.  It would break\n     on platforms with a very small stack, but since it's only\n     use by CI and developers any breakage would be immediately\n     obvious and easy to fix.\n\n  3. remove static and shrink the buffer size.  From my (limited)\n     local experiment, it seems the buffer is larger than needed.\n     We could shrink it without losing performance.\n\nI don't have a strong opinion here; I went with (2) because it\nseemed like the smallest change but I'd be happy to change it.\n\nIf we go with static, I'd prefer to have a small comment\nabove the function definition:\n\n  /* not thread-safe (due to the mutable static 'buf') */\n\nThanks,\nKristofer\n\n"},{"id":"557882","messageId":"CAL71e4Mrw2h-D+yKLVE30q-7dnkT31CVsyVmktmS9Wx6Xu6ocQ@mail.gmail.com","threadId":"66800","inReplyTo":"avFvYD5K4SKlveiI@pks.im","subject":"Re: [PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-09T07:43:22Z","receivedAt":"2026-11-09T07:43:22Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 9 Nov 2026 at 07:59, Patrick Steinhardt <ps@pks.im> wrote:\n>\n> Okay. The diff is unfortunately quite useless as we do the file rename\n> plus all the changes in a single step. It might've been helpful to split\n> this up into:\n>\n>   1. A rename of the file.\n>\n>   2. A refactoring of `cmd_genzeros()` to use the write buffer.\n>\n>   3. A change to introduce `cmd_genbytes()`.\n>\n> But maybe I'm also overdoing this.\n\nv1 had the rename as a separate commit, and Junio preferred to have\nit squashed into the change.  I do not have a strong opinion either\nway, so I will wait for a consensus before sending v3.\n\n>\n> > +     return 0;\n> > +}\n> > +\n> > +static intmax_t parse_count(int argc, const char **argv, int pos)\n> > +{\n> > +     return pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n> > +}\n>\n> We can use `git_parse_signed()` instead.\n\nGood idea.  It would also reject invalid counts, which are currently\nsilently treated as 0.\n\n> I wonder about the regression in readability though. Maybe that's just\n> me, but I don't happen to know all ASCII values by heart, so if I were\n> to read this I would now wonder what's so special about 61. And the next\n> thing to wonder about is whether this is supposed to encode \"=\" (decimal\n> 61) or \"a\" (hexadecimal 61), and without reading the code of genbytes I\n> would not be able to tell.\n>\n> So can we maybe make the underpinning logic a bit more intelligent, e.g.\n> by accepting \"\\NNN\" or \"\\xNN\" syntax instead of two hex characters? That\n> might also make it more extensible going forward if the next developer\n> wants to make genbytes take and repeat strings of arbitrary length\n> as-is.\n\nThat made me look at what the callers actually need:\n\n  callers                     need\n  t1007 (4x), t5318           NUL bytes\n  t1051 (2x), t1450 (2x),     any byte, only the size matters\n    t4141\n  t0613, t4205, t7006, t9850  any byte except NUL and newline\n  t5300                       three different non-NUL bytes\n\nSo none of them needs a particular byte value, and the only special\ncase is NUL, which \"genzeros\" already covers.  For everything else, a\nplain character is the most readable:\n\nGiven that, I am now leaning towards keeping \"genzeros\" as it is and\nadding \"genbytes <char> [<count>]\" for the other callers, i.e. going\nback to v1 without its last patch (and without a separate rename\npatch).  That avoids inventing a syntax for NUL, keeps the call\nsites readable, and also leaves the existing \"genzeros\" callers\nalone, which addresses Johannes' concern about the churn.\n\nJunio, would that be acceptable, given your suggestion to use hex\ndigits so that NUL does not need special treatment?\n\nThe rename is perhaps less important to do if genzeros is kept\n(though it also gains the genbytes function).\n\nThanks,\nKristofer\n\n"},{"id":"557904","messageId":"ddea802a-d4f4-4de9-9108-e68201841bc4@kdbg.org","threadId":"66800","inReplyTo":"c8688e1a-17b6-4d0b-ba9a-fc1e0fa8938b@kdbg.org","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-11-09T09:33:03Z","receivedAt":"2026-11-09T09:33:03Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 08.11.26 um 17:34 schrieb Johannes Sixt:\n> Am 08.11.26 um 13:24 schrieb Kristofer Karlsson:\n>> -       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n>> +       printf 000000000%02147483640d 0 >expect &&\n>>\n>> I tried this out and it works on Linux but fails on macOS[1].  What\n>> happens is that printf only outputs the nine literal zeros\n>> (\"000000000\") there, and silently ignores the very large padding.\n> \n> Fair enough.\n> \n>> [1] https://github.com/gitgitgadget/git/actions/runs/42475956442/job/128717309586\n\nWell, this patch passed for me[2], but I have no idea whether the\nparticular tests were skipped due to the EXPENSIVE prerequisite or not.\n\n[2] https://github.com/j6t/git/actions/runs/42583623620/job/129048078411\n\n\n\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 01b97c8888..5dedb9054e 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1182,7 +1182,8 @@ unfold () {\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n-\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+\t# 3 * 715827883 == 2147483649\n+\tprintf %0715827883d 0 0 0 >expect &&\n \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n '\n \n\n\n"},{"id":"557931","messageId":"CAL71e4PSOJgKFfBLrTXJSAewnZTJpNEiAMC-_FbW_9cmpEvLmg@mail.gmail.com","threadId":"66800","inReplyTo":"ddea802a-d4f4-4de9-9108-e68201841bc4@kdbg.org","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-09T13:05:25Z","receivedAt":"2026-11-09T13:05:25Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Mon, 9 Nov 2026 at 10:33, Johannes Sixt <j6t@kdbg.org> wrote:\n>\n> Am 08.11.26 um 17:34 schrieb Johannes Sixt:\n> > Am 08.11.26 um 13:24 schrieb Kristofer Karlsson:\n> >> -       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n> >> +       printf 000000000%02147483640d 0 >expect &&\n> >>\n> >> I tried this out and it works on Linux but fails on macOS[1].  What\n> >> happens is that printf only outputs the nine literal zeros\n> >> (\"000000000\") there, and silently ignores the very large padding.\n> >\n> > Fair enough.\n> >\n> >> [1] https://github.com/gitgitgadget/git/actions/runs/42475956442/job/128717309586\n>\n> Well, this patch passed for me[2], but I have no idea whether the\n> particular tests were skipped due to the EXPENSIVE prerequisite or not.\n>\n> [2] https://github.com/j6t/git/actions/runs/42583623620/job/129048078411\n>\n>\n>\n> diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> index 01b97c8888..5dedb9054e 100755\n> --- a/t/t4205-log-pretty-formats.sh\n> +++ b/t/t4205-log-pretty-formats.sh\n> @@ -1182,7 +1182,8 @@ unfold () {\n>  '\n>\n>  test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n> -       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n> +       # 3 * 715827883 == 2147483649\n> +       printf %0715827883d 0 0 0 >expect &&\n>         huge_commit=$(git commit-tree -F expect HEAD^{tree})\n\nI tested this and it passes on all the macOS jobs.  t4205 goes\nfrom 800-900s to 170-280s there, so your idea works well.\n\nTwo caveats: the printf builtin in bash 3.2 (/bin/sh on macOS)\nsilently prints nothing for a field wider than INT_MAX - 15,\nand it builds each field in memory, so this peaks at about 2.1GB\nRSS.  Both go away by writing in 1MiB chunks, at the same speed:\n\n    gen_ascii_zeros () {\n        _n=$1\n        while test \"$_n\" -gt 0\n        do\n            _chunk=$(( _n > 1048576 ? 1048576 : _n ))\n            printf \"%0${_chunk}d\" 0 || return 1\n            _n=$(( _n - _chunk ))\n        done\n    }\n\nas a local helper in t4205.  That fixes the slow test without a\nnew test-tool, at the cost of yet another way of generating test\ndata alongside genzeros and \"genzeros | tr\".\n\nI have it ready locally as a single patch, as an alternative to\nthis series, but perhaps it's best to wait for more people to\nchime in first.\n\nThanks,\nKristofer\n\n"},{"id":"557954","messageId":"7b2e7ddf-dc8b-4538-b736-92217e20c8f0@kdbg.org","threadId":"66800","inReplyTo":"CAL71e4PSOJgKFfBLrTXJSAewnZTJpNEiAMC-_FbW_9cmpEvLmg@mail.gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2026-11-09T15:38:48Z","receivedAt":"2026-11-09T15:38:48Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 09.11.26 um 14:05 schrieb Kristofer Karlsson:\n> On Mon, 9 Nov 2026 at 10:33, Johannes Sixt <j6t@kdbg.org> wrote:\n>>  test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n>> -       test-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n>> +       # 3 * 715827883 == 2147483649\n>> +       printf %0715827883d 0 0 0 >expect &&\n>>         huge_commit=$(git commit-tree -F expect HEAD^{tree})\n> \n> I tested this and it passes on all the macOS jobs.  t4205 goes\n> from 800-900s to 170-280s there, so your idea works well.\n> \n> Two caveats: the printf builtin in bash 3.2 (/bin/sh on macOS)\n> silently prints nothing for a field wider than INT_MAX - 15,\n> and it builds each field in memory, so this peaks at about 2.1GB\n> RSS.  Both go away by writing in 1MiB chunks, at the same speed:\n> \n>     gen_ascii_zeros () {\n>         _n=$1\n>         while test \"$_n\" -gt 0\n>         do\n>             _chunk=$(( _n > 1048576 ? 1048576 : _n ))\n>             printf \"%0${_chunk}d\" 0 || return 1\n>             _n=$(( _n - _chunk ))\n>         done\n>     }\n\nI wouldn't do that (and pay the memory price instead), because the next\nshell doesn't have printf as a builtin and spawns 2000 printf processes.\n\nNow let's stop painting bikeshes. I'd be fine with your previous\nsuggestion to have both `test-tool genzeros` and `test-tool genbytes`.\n\n-- Hannes\n\n\n"},{"id":"557958","messageId":"xmqqh5hq106x.fsf@gitster.g","threadId":"66800","inReplyTo":"CAL71e4PkS0dRvqSkL81hSPO3M3K2n5ZTumFNvCx9MsBsFKbnmw@mail.gmail.com","subject":"Re: [PATCH v2 1/2] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-09T15:41:58Z","receivedAt":"2026-11-09T15:41:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n>   2. remove static and keep the size.  Since the stack is not\n>      complex or deep here it should be safe.  It would break\n>      on platforms with a very small stack, but since it's only\n>      use by CI and developers any breakage would be immediately\n>      obvious and easy to fix.\n>\n>   3. remove static and shrink the buffer size.  From my (limited)\n>      local experiment, it seems the buffer is larger than needed.\n>      We could shrink it without losing performance.\n>\n> I don't have a strong opinion here; I went with (2) because it\n> seemed like the smallest change but I'd be happy to change it.\n>\n> If we go with static, I'd prefer to have a small comment\n> above the function definition:\n>\n>   /* not thread-safe (due to the mutable static 'buf') */\n\nI do not think such a comment is not warranted in this case, though.\nAfter all this is more or less throw-away code used only by CI and\ndevelopers ;-)\n\n"},{"id":"558021","messageId":"8733t9z0mi.fsf@gmail.com","threadId":"66800","inReplyTo":"CAL71e4NFyAShDZPW9JRB7Bpg3aLY5mY4wdn5m_ZYoLQjHZw_jA@mail.gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Collin Funk","fromEmail":"collin.funk1@gmail.com","sentAt":"2026-11-10T06:00:53Z","receivedAt":"2026-11-10T06:00:53Z","isPatch":true,"sender":{"key":"collin.funk1@gmail.com","avatar":"https://avatars.githubusercontent.com/u/65689063?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> Nice trick, and it would work for t4205, which does not care about\n> the byte value.  I tried it for the 2147483649 bytes there on Linux:\n>\n>   printf           time   peak memory\n>   dash             6.5s   2.1GB\n>   bash             6.0s   3MB\n>   coreutils        3.7s   2.1GB\n>\n> It is definitely fast enough (on the variants I tested).\n> The annoying thing (but not a real blocker) is the extra memory\n> usage required and I am not sure I fully trust that all\n> implementations will behave the same for absurdly large padding.\n>\n> (Particularly for paddings larger than 2^31, as you noted too.)\n>\n> It feels a bit like a hack, and tweaking the existing test-tool\n> feels like a more sane option that is easier to reason about.\n>\n> A gotcha here is that the printf hack only works for producing\n> \"0\", so for the cases we truly need NUL characters it does not\n> help us, and I think that is also a point in favor of\n> simply generalizing test-tool genzeros instead.\n\nI've mentioned this idea on coreutils or gnulib's mailing lists before,\nbut I forget where and when. I've wanted to create a printf variant that\ndoesn't allocate much memory and will just use putc_unlocked for each\npadding character, which would avoid the memory issue you bring up.\n\nMost platforms will be limited to INT_MAX there, since that is what\nPOSIX specifies is supplied with '*' [1]:\n\n    A field width, or precision, or both, may be indicated by an\n    <asterisk> ('*'). In this case an argument of type int supplies the\n    field width or precision.\n\nE.g., in an invocation like this where the field width is supplied with\nanother argument:\n\n    $ printf '%0*d\\n' 5 1\n    00001\n\nI think my original reason for wanting a different printf variant was\nescaping that limit.\n\nFor generating bytes where the values don't matter, in GNU coreutils we\nfrequently use 'yes' for that. As a result, the implementation is very\noverengineered. E.g., it will use zero-copy since coreutils-9.11. Before\nthat, it would reuse ARGV if it was contiguous and the arguments used to\ndetermine the printed string were long enough such that we weren't calling\nread/write on a tiny string, or it would generate a large buffer before\nusing read and write. Here is the performance from before and after\nzero-copy was introduced on a beefy Power10 machine, stolen from the\ncommit message [2]:\n\n   IBM,9043-MRX $ taskset 1 yes | taskset 2 pv > /dev/null\n   ... [11.6GiB/s]\n   IBM,9043-MRX $ taskset 1 src/yes | taskset 2 pv > /dev/null\n   ... [175GiB/s]\n\nThis was introduced around the time of the copy-fail vulnerability and\nvarious zero-copy vulnerabilities, where there was serious consideration\ngiven to nerfing splice and vmsplice. Luckily for us, that never\nhappened.\n\nFreeBSD, and therefore, I assume, macOS use something similar to our\nprevious read/write method, which is fast enough. I think OpenBSD and\nNetBSD use puts on ARGV or the \"y\" string, so it's relatively slow.\n\nGNU 'tr' doesn't handle multi-byte characters, so you can adjust that\noutput if needed and remain relatively quick:\n\n    $ timeout 10 yes 0 | LC_ALL=C tr -d '\\n' | pv -r > /dev/null\n    [ 616MiB/s]\n\nI'm not sure if macOS 'tr' handles multibyte characters. But it might be\nworth checking if they do, and if they keep a fast case for single-byte\nlocales. Doing that without duplicating code is quite tricky, which is\npart of the reason why we haven't done it in GNU 'tr' yet (but recent\nreleases have improved multi-byte character support for other utilities,\nso hopefully it is handled soon).\n\nProbably more information than you need, but hopefully it helps you\ngenerate slop sometimes needed for testing. :)\n\nCollin\n\n[1] https://pubs.opengroup.org/onlinepubs/9799919799/functions/printf.html\n[2] https://github.com/coreutils/coreutils/commit/2b1c059e6a06eebbb721d010b1221ec54200cc33\n\n"},{"id":"558044","messageId":"CAL71e4PR0KOAE0Q42S08zQbwukw4mM3Mffbekm6mJUAWONnDYA@mail.gmail.com","threadId":"66800","inReplyTo":"8733t9z0mi.fsf@gmail.com","subject":"Re: [PATCH 2/3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-10T08:09:21Z","receivedAt":"2026-11-10T08:09:21Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Tue, 10 Nov 2026 at 07:08, Collin Funk <collin.funk1@gmail.com> wrote:\n>\n> I'm not sure if macOS 'tr' handles multibyte characters. But it might be\n> worth checking if they do, and if they keep a fast case for single-byte\n> locales. Doing that without duplicating code is quite tricky, which is\n> part of the reason why we haven't done it in GNU 'tr' yet (but recent\n> releases have improved multi-byte character support for other utilities,\n> so hopefully it is handled soon).\n\nThanks, this was an interesting read (even if it may not solve\nthe problem at hand).\n\nTo answer your question about macOS tr: it does not seem to have a\nsingle-byte fast path.  Apple's tr (text_cmds tr/tr.c) goes\nthrough getwchar()/putwchar() for every byte even with LC_ALL=C,\nwhich is why \"genzeros | tr\" takes about 2 minutes for 2GB there,\ncompared to a few seconds with GNU tr.\n\n\"yes\" is nicely optimized, but without an option to drop the\nnewlines we would still need to filter its output, and filtering\nis exactly what is slow on macOS.  I suppose the primitive I would\nideally want is something like \"repeat <string> [<count>]\".\n\nIn the end portability is the main problem: with tr and printf\nbehaving this differently across platforms, a tiny test-tool\nhelper is easier to reason about than any combination of shell\ntools.\n\nThanks,\nKristofer\n\n>\n> Collin\n>\n> [1] https://pubs.opengroup.org/onlinepubs/9799919799/functions/printf.html\n> [2] https://github.com/coreutils/coreutils/commit/2b1c059e6a06eebbb721d010b1221ec54200cc33\n>\n\n"},{"id":"558124","messageId":"pull.2301.v3.git.1794335168170.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-10T18:26:08Z","receivedAt":"2026-11-10T18:26:08Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nSeveral tests create fixture data consisting of a single repeated\ncharacter with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\nworks well on platforms where tr(1) is implemented efficiently.\n\nThe implementation of tr(1) on macOS supports multi-byte characters,\nand processes its input one character at a time through\ngetwchar() and putwchar(), even in the C locale.  For small inputs\nthat does not matter, but t4205 generates more than 2GB of data this\nway, which makes that test much slower on macOS than on Linux: on the\nmacOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\nit, compared to about 6 seconds on Linux.\n\nFix this by adding \"test-tool genbytes <char> [<count>]\" next to\n\"genzeros\", sharing the same code, which writes the requested\ncharacter directly.  Use it in all tests that piped \"genzeros\"\nthrough tr(1), not just t4205, so that it is the obvious choice for\nnew tests.  Callers that want NUL bytes keep using \"genzeros\".\n\nWhile at it, parse the count with git_parse_signed(), so that an\ninvalid count is reported as a usage error instead of being silently\ntreated as 0.\n\nThis makes t4205 much faster on macOS in CI.  The Linux jobs, shown\nfor comparison, show smaller differences, which may partly be due to\nnormal run-to-run variation:\n\n  job                before    after   change\n  osx-gcc            821.1s   243.3s   -70.4%\n  osx-reftable       911.0s   194.0s   -78.7%\n  linux-asan-ubsan   226.7s   222.1s    -2.0%\n  linux-leaks        320.2s   298.8s    -6.7%\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n    t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n    \n    While looking for slow tests in CI, t4205 stood out on macOS: its\n    EXPENSIVE tests take several times longer there than on Linux. A CI\n    experiment showed that much of the time was spent in tr(1), which is\n    very slow on macOS for the 2GB of data that t4205 generates.\n    \n    This adds \"test-tool genbytes []\" and uses it instead of \"genzeros |\n    tr\", which is where the speedup comes from.\n    \n    Changes since v2:\n    \n     * \"genbytes\" takes a literal character again (\"genbytes a\") instead of\n       two hex digits (\"genbytes 61\"), which Patrick found hard to read at\n       the call sites.\n    \n     * \"genzeros\" and its callers that want NUL bytes are kept as they are,\n       and the patch that converted them to \"genbytes 00\" is dropped. This\n       avoids churn in tests that do not need to change, as discussed with\n       Johannes. Keeping \"genzeros\" also means that \"genbytes\" does not need\n       special handling for NUL bytes.\n    \n     * test-genzeros.c is no longer renamed, and \"genbytes\" lives next to\n       \"genzeros\" in the same file. Since \"genzeros\" stays, the current file\n       name is still valid and not worth churning.\n    \n     * The buffer is static again instead of 256 KiB on the stack, as\n       suggested by Junio, to avoid using a large part of the limited stack\n       space. There is no risk of concurrent use in a run-once helper.\n    \n     * The count is parsed with git_parse_signed(), so that an invalid count\n       is a usage error instead of being silently treated as 0, as suggested\n       by Patrick. This applies to \"genzeros\" as well.\n    \n    Changes in v2 (subsequently reverted in v3):\n    \n     * Squashed the rename of test-genzeros.c into the patch that changes\n       it, as suggested by Junio.\n    \n     * \"genbytes\" took the byte as two lowercase hexadecimal digits instead\n       of a literal character, as suggested by Junio. This also worked for\n       NUL bytes, so \"genbytes 00\" replaced special-casing \"NUL\".\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2301%2Fspkrka%2Ftest-tool-genbytes-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2301/spkrka/test-tool-genbytes-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/2301\n\nRange-diff vs v2:\n\n 1:  54139a3a97 ! 1:  17e5baacdb t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n     @@ Commit message\n          macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\n          it, compared to about 6 seconds on Linux.\n      \n     -    Fix this by adding \"test-tool genbytes <hex-byte> [<count>]\", which\n     -    writes the requested byte, given as two lowercase hexadecimal\n     -    digits, directly.  Turn \"genzeros\" into a thin wrapper around the\n     -    same code, and rename the file to match.  Use the new helper in all\n     -    tests that piped \"genzeros\" through tr(1), not just t4205, so that\n     -    it is the obvious choice for new tests.\n     +    Fix this by adding \"test-tool genbytes <char> [<count>]\" next to\n     +    \"genzeros\", sharing the same code, which writes the requested\n     +    character directly.  Use it in all tests that piped \"genzeros\"\n     +    through tr(1), not just t4205, so that it is the obvious choice for\n     +    new tests.  Callers that want NUL bytes keep using \"genzeros\".\n     +\n     +    While at it, parse the count with git_parse_signed(), so that an\n     +    invalid count is reported as a usage error instead of being silently\n     +    treated as 0.\n      \n          This makes t4205 much faster on macOS in CI.  The Linux jobs, shown\n     -    for comparison, only save the few seconds that GNU tr(1) took, which\n     -    is within the usual noise:\n     +    for comparison, show smaller differences, which may partly be due to\n     +    normal run-to-run variation:\n      \n            job                before    after   change\n            osx-gcc            821.1s   243.3s   -70.4%\n     @@ Commit message\n      \n          Signed-off-by: Kristofer Karlsson <krka@spotify.com>\n      \n     - ## Makefile ##\n     -@@ Makefile: TEST_BUILTINS_OBJS += test-env-helper.o\n     - TEST_BUILTINS_OBJS += test-example-tap.o\n     - TEST_BUILTINS_OBJS += test-find-pack.o\n     - TEST_BUILTINS_OBJS += test-fsmonitor-client.o\n     -+TEST_BUILTINS_OBJS += test-genbytes.o\n     - TEST_BUILTINS_OBJS += test-genrandom.o\n     --TEST_BUILTINS_OBJS += test-genzeros.o\n     - TEST_BUILTINS_OBJS += test-getcwd.o\n     - TEST_BUILTINS_OBJS += test-hash-speed.o\n     - TEST_BUILTINS_OBJS += test-hash.o\n     -\n     - ## t/helper/meson.build ##\n     -@@ t/helper/meson.build: test_tool_sources = [\n     -   'test-example-tap.c',\n     -   'test-find-pack.c',\n     -   'test-fsmonitor-client.c',\n     -+  'test-genbytes.c',\n     -   'test-genrandom.c',\n     --  'test-genzeros.c',\n     -   'test-getcwd.c',\n     -   'test-hash-speed.c',\n     -   'test-hash.c',\n     -\n     - ## t/helper/test-genbytes.c (new) ##\n     + ## t/helper/test-genzeros.c ##\n      @@\n     -+#define DISABLE_SIGN_COMPARE_WARNINGS\n     -+\n     -+#include \"test-tool.h\"\n     -+#include \"git-compat-util.h\"\n     -+#include \"hex-ll.h\"\n     -+\n     + \n     + #include \"test-tool.h\"\n     + #include \"git-compat-util.h\"\n     ++#include \"parse.h\"\n     + \n     +-int cmd__genzeros(int argc, const char **argv)\n      +/*\n      + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n      + * \"count\" is negative.\n      + */\n     -+static int generate_bytes(char byte, intmax_t count)\n     -+{\n     -+\tchar buf[256 * 1024];\n     -+\tssize_t n;\n     -+\n     ++static int generate_bytes(unsigned char byte, intmax_t count)\n     + {\n     +-\t/* static, so that it is NUL-initialized */\n     +-\tstatic const char zeros[256 * 1024];\n     +-\tintmax_t count;\n     ++\tstatic char buf[256 * 1024];\n     + \tssize_t n;\n     + \n     +-\tif (argc > 2) {\n     +-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n     +-\t\treturn 1;\n     +-\t}\n     +-\n     +-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n      +\tmemset(buf, byte, sizeof(buf));\n     -+\n     + \n     +-\t/* Writing out individual NUL bytes is slow... */\n      +\t/* Writing out individual bytes is slow... */\n     -+\twhile (count < 0)\n     + \twhile (count < 0)\n     +-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n      +\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n     -+\t\t\tdie_errno(\"write error\");\n     -+\n     -+\twhile (count > 0) {\n     + \t\t\tdie_errno(\"write error\");\n     + \n     + \twhile (count > 0) {\n     +-\t\tn = xwrite(1, zeros,\n     +-\t\t\t   count < ARRAY_SIZE(zeros)\n     +-\t\t\t   ? count : ARRAY_SIZE(zeros));\n      +\t\tn = xwrite(1, buf,\n      +\t\t\t   count < ARRAY_SIZE(buf)\n      +\t\t\t   ? count : ARRAY_SIZE(buf));\n     -+\n     -+\t\tif (n < 0)\n     -+\t\t\tdie_errno(\"write error\");\n     -+\n     -+\t\tcount -= n;\n     -+\t}\n     -+\n     -+\treturn 0;\n     -+}\n     -+\n     -+static intmax_t parse_count(int argc, const char **argv, int pos)\n     -+{\n     -+\treturn pos < argc ? strtoimax(argv[pos], NULL, 0) : -1;\n     -+}\n     + \n     + \t\tif (n < 0)\n     + \t\t\tdie_errno(\"write error\");\n     +@@ t/helper/test-genzeros.c: int cmd__genzeros(int argc, const char **argv)\n     + \n     + \treturn 0;\n     + }\n      +\n      +int cmd__genbytes(int argc, const char **argv)\n      +{\n     -+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 2 ||\n     -+\t    strspn(argv[1], \"0123456789abcdef\") != 2) {\n     -+\t\tfprintf(stderr, \"usage: %s <hex-byte> [<count>]\\n\", argv[0]);\n     ++\tintmax_t count = -1;\n     ++\n     ++\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1 ||\n     ++\t    (argc == 3 && !git_parse_signed(argv[2], &count, INTMAX_MAX))) {\n     ++\t\tfprintf(stderr, \"usage: %s <char> [<count>]\\n\", argv[0]);\n      +\t\treturn 1;\n      +\t}\n      +\n     -+\treturn generate_bytes(hex2chr(argv[1]), parse_count(argc, argv, 2));\n     ++\treturn generate_bytes(argv[1][0], count);\n      +}\n      +\n      +int cmd__genzeros(int argc, const char **argv)\n      +{\n     -+\tif (argc > 2) {\n     ++\tintmax_t count = -1;\n     ++\n     ++\tif (argc > 2 ||\n     ++\t    (argc == 2 && !git_parse_signed(argv[1], &count, INTMAX_MAX))) {\n      +\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n      +\t\treturn 1;\n      +\t}\n      +\n     -+\treturn generate_bytes('\\0', parse_count(argc, argv, 1));\n     ++\treturn generate_bytes('\\0', count);\n      +}\n      \n     - ## t/helper/test-genzeros.c (deleted) ##\n     -@@\n     --#define DISABLE_SIGN_COMPARE_WARNINGS\n     --\n     --#include \"test-tool.h\"\n     --#include \"git-compat-util.h\"\n     --\n     --int cmd__genzeros(int argc, const char **argv)\n     --{\n     --\t/* static, so that it is NUL-initialized */\n     --\tstatic const char zeros[256 * 1024];\n     --\tintmax_t count;\n     --\tssize_t n;\n     --\n     --\tif (argc > 2) {\n     --\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n     --\t\treturn 1;\n     --\t}\n     --\n     --\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n     --\n     --\t/* Writing out individual NUL bytes is slow... */\n     --\twhile (count < 0)\n     --\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n     --\t\t\tdie_errno(\"write error\");\n     --\n     --\twhile (count > 0) {\n     --\t\tn = xwrite(1, zeros,\n     --\t\t\t   count < ARRAY_SIZE(zeros)\n     --\t\t\t   ? count : ARRAY_SIZE(zeros));\n     --\n     --\t\tif (n < 0)\n     --\t\t\tdie_errno(\"write error\");\n     --\n     --\t\tcount -= n;\n     --\t}\n     --\n     --\treturn 0;\n     --}\n     -\n       ## t/helper/test-tool.c ##\n      @@ t/helper/test-tool.c: static struct test_cmd cmds[] = {\n       \t{ \"example-tap\", cmd__example_tap },\n     @@ t/t0613-reftable-write-options.sh: test_expect_success 'small block size fails w\n       \t\tcd repo &&\n       \t\ttest_commit A &&\n      -\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n     -+\t\ttest-tool genbytes 61 500 >logmsg &&\n     ++\t\ttest-tool genbytes a 500 >logmsg &&\n       \t\tcat >expect <<-EOF &&\n       \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n       \t\tEOF\n     @@ t/t4205-log-pretty-formats.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'lo\n       \n       test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n      -\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n     -+\ttest-tool genbytes 31 2147483649 >expect &&\n     ++\ttest-tool genbytes 1 2147483649 >expect &&\n       \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n       '\n       \n     @@ t/t5300-pack-object.sh: test_description='git pack-object'\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n      -\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n     -+\ttest-tool genbytes 61 4096 >a &&\n     -+\ttest-tool genbytes 62 4096 >b &&\n     -+\ttest-tool genbytes 63 4096 >c &&\n     ++\ttest-tool genbytes a 4096 >a &&\n     ++\ttest-tool genbytes b 4096 >b &&\n     ++\ttest-tool genbytes c 4096 >c &&\n       \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n       \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n       \tgit update-index --add a a_big b b_big c &&\n     @@ t/t7006-pager.sh: test_expect_success 'setup trace2' '\n       test_expect_success 'setup large log output' '\n      -\ttest-tool genzeros 50000 |\n      -\ttr \"\\000\" \"a\" |\n     -+\ttest-tool genbytes 61 50000 |\n     ++\ttest-tool genbytes a 50000 |\n       \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n       \tgit commit --allow-empty -F commit-msg\n       '\n     @@ t/t9850-shell.sh: test_expect_success 'shell allows interactive command' '\n       \n       test_expect_success 'shell complains of overlong commands' '\n      -\ttest-tool genzeros | tr \"\\000\" \"a\" |\n     -+\ttest-tool genbytes 61 |\n     ++\ttest-tool genbytes a |\n       \ttest_must_fail git shell 2>err &&\n       \ttest_grep \"too long\" err\n       '\n 2:  f49af0e33f < -:  ---------- t: replace \"test-tool genzeros\" with \"genbytes 00\"\n\n\n t/helper/test-genzeros.c          | 54 ++++++++++++++++++++++---------\n t/helper/test-tool.c              |  1 +\n t/helper/test-tool.h              |  1 +\n t/t0613-reftable-write-options.sh |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 ++--\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 8 files changed, 48 insertions(+), 23 deletions(-)\n\ndiff --git a/t/helper/test-genzeros.c b/t/helper/test-genzeros.c\nindex b895436a32..e8d83d5012 100644\n--- a/t/helper/test-genzeros.c\n+++ b/t/helper/test-genzeros.c\n@@ -2,30 +2,28 @@\n \n #include \"test-tool.h\"\n #include \"git-compat-util.h\"\n+#include \"parse.h\"\n \n-int cmd__genzeros(int argc, const char **argv)\n+/*\n+ * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n+ * \"count\" is negative.\n+ */\n+static int generate_bytes(unsigned char byte, intmax_t count)\n {\n-\t/* static, so that it is NUL-initialized */\n-\tstatic const char zeros[256 * 1024];\n-\tintmax_t count;\n+\tstatic char buf[256 * 1024];\n \tssize_t n;\n \n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n-\n-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n+\tmemset(buf, byte, sizeof(buf));\n \n-\t/* Writing out individual NUL bytes is slow... */\n+\t/* Writing out individual bytes is slow... */\n \twhile (count < 0)\n-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n+\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n \t\t\tdie_errno(\"write error\");\n \n \twhile (count > 0) {\n-\t\tn = xwrite(1, zeros,\n-\t\t\t   count < ARRAY_SIZE(zeros)\n-\t\t\t   ? count : ARRAY_SIZE(zeros));\n+\t\tn = xwrite(1, buf,\n+\t\t\t   count < ARRAY_SIZE(buf)\n+\t\t\t   ? count : ARRAY_SIZE(buf));\n \n \t\tif (n < 0)\n \t\t\tdie_errno(\"write error\");\n@@ -35,3 +33,29 @@ int cmd__genzeros(int argc, const char **argv)\n \n \treturn 0;\n }\n+\n+int cmd__genbytes(int argc, const char **argv)\n+{\n+\tintmax_t count = -1;\n+\n+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1 ||\n+\t    (argc == 3 && !git_parse_signed(argv[2], &count, INTMAX_MAX))) {\n+\t\tfprintf(stderr, \"usage: %s <char> [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes(argv[1][0], count);\n+}\n+\n+int cmd__genzeros(int argc, const char **argv)\n+{\n+\tintmax_t count = -1;\n+\n+\tif (argc > 2 ||\n+\t    (argc == 2 && !git_parse_signed(argv[1], &count, INTMAX_MAX))) {\n+\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes('\\0', count);\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex b71a22b43b..af70e2bec7 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -33,6 +33,7 @@ static struct test_cmd cmds[] = {\n \t{ \"example-tap\", cmd__example_tap },\n \t{ \"find-pack\", cmd__find_pack },\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n+\t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n \t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex f2885b33d5..c35e7412af 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -26,6 +26,7 @@ int cmd__env_helper(int argc, const char **argv);\n int cmd__example_tap(int argc, const char **argv);\n int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n+int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex 0b27d75fe3..0b74ff6157 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n \t(\n \t\tcd repo &&\n \t\ttest_commit A &&\n-\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n+\t\ttest-tool genbytes a 500 >logmsg &&\n \t\tcat >expect <<-EOF &&\n \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n \t\tEOF\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 01b97c8888..894d88c39b 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1182,7 +1182,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n-\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+\ttest-tool genbytes 1 2147483649 >expect &&\n \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n '\n \ndiff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\nindex e5ce478484..4fb6a22145 100755\n--- a/t/t5300-pack-object.sh\n+++ b/t/t5300-pack-object.sh\n@@ -9,9 +9,9 @@ test_description='git pack-object'\n \n test_expect_success 'setup' '\n \trm -f .git/index* &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n+\ttest-tool genbytes a 4096 >a &&\n+\ttest-tool genbytes b 4096 >b &&\n+\ttest-tool genbytes c 4096 >c &&\n \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n \tgit update-index --add a a_big b b_big c &&\ndiff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\nindex 75ca7a25da..8f1f71bf12 100755\n--- a/t/t7006-pager.sh\n+++ b/t/t7006-pager.sh\n@@ -662,8 +662,7 @@ test_expect_success 'setup trace2' '\n '\n \n test_expect_success 'setup large log output' '\n-\ttest-tool genzeros 50000 |\n-\ttr \"\\000\" \"a\" |\n+\ttest-tool genbytes a 50000 |\n \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n \tgit commit --allow-empty -F commit-msg\n '\ndiff --git a/t/t9850-shell.sh b/t/t9850-shell.sh\nindex 27cc49bbd1..d56edebda7 100755\n--- a/t/t9850-shell.sh\n+++ b/t/t9850-shell.sh\n@@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '\n '\n \n test_expect_success 'shell complains of overlong commands' '\n-\ttest-tool genzeros | tr \"\\000\" \"a\" |\n+\ttest-tool genbytes a |\n \ttest_must_fail git shell 2>err &&\n \ttest_grep \"too long\" err\n '\n\nbase-commit: fbd2d1e765da4d2931cadc4c62d3ca5d5b590e7e\n-- \ngitgitgadget\n\n"},{"id":"558135","messageId":"xmqqo6bwqy9n.fsf@gitster.g","threadId":"66800","inReplyTo":"pull.2301.v3.git.1794335168170.gitgitgadget@gmail.com","subject":"Re: [PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-10T19:31:16Z","receivedAt":"2026-11-10T19:31:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\n>      * test-genzeros.c is no longer renamed, and \"genbytes\" lives next to\n>        \"genzeros\" in the same file. Since \"genzeros\" stays, the current file\n>        name is still valid and not worth churning.\n\nYes, it is a great realization that the file will have both genbytes\nand genzeros in it in the end, and there is no strong reason why it\nneeds to be renamed.\n\n> diff --git a/t/helper/test-genzeros.c b/t/helper/test-genzeros.c\n> index b895436a32..e8d83d5012 100644\n> --- a/t/helper/test-genzeros.c\n> +++ b/t/helper/test-genzeros.c\n> @@ -2,30 +2,28 @@\n>  \n>  #include \"test-tool.h\"\n>  #include \"git-compat-util.h\"\n> +#include \"parse.h\"\n>  \n> +/*\n> + * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n> + * \"count\" is negative.\n> + */\n> +static int generate_bytes(unsigned char byte, intmax_t count)\n>  {\n> +\tstatic char buf[256 * 1024];\n>  \tssize_t n;\n>  \n> +\tmemset(buf, byte, sizeof(buf));\n>  \n> +\t/* Writing out individual bytes is slow... */\n>  \twhile (count < 0)\n> +\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n>  \t\t\tdie_errno(\"write error\");\n\nA curious infinite loop but that is OK.\n\n>  \twhile (count > 0) {\n> +\t\tn = xwrite(1, buf,\n> +\t\t\t   count < ARRAY_SIZE(buf)\n> +\t\t\t   ? count : ARRAY_SIZE(buf));\n>  \n>  \t\tif (n < 0)\n>  \t\t\tdie_errno(\"write error\");\n> @@ -35,3 +33,29 @@ int cmd__genzeros(int argc, const char **argv)\n>  \n>  \treturn 0;\n>  }\n\n> +int cmd__genbytes(int argc, const char **argv)\n> +{\n> +\tintmax_t count = -1;\n> +\n> +\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1 ||\n> +\t    (argc == 3 && !git_parse_signed(argv[2], &count, INTMAX_MAX))) {\n> +\t\tfprintf(stderr, \"usage: %s <char> [<count>]\\n\", argv[0]);\n> +\t\treturn 1;\n> +\t}\n> +\n> +\treturn generate_bytes(argv[1][0], count);\n> +}\n> +\n> +int cmd__genzeros(int argc, const char **argv)\n> +{\n> +\tintmax_t count = -1;\n> +\n> +\tif (argc > 2 ||\n> +\t    (argc == 2 && !git_parse_signed(argv[1], &count, INTMAX_MAX))) {\n> +\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n> +\t\treturn 1;\n> +\t}\n> +\n> +\treturn generate_bytes('\\0', count);\n> +}\n\nThe callers are so straight-forward.\n\n> diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\n> diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\n> diff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\n\nAll conversions of these consumers are so trivially correct.\n\n> diff --git a/t/t9850-shell.sh b/t/t9850-shell.sh\n> index 27cc49bbd1..d56edebda7 100755\n> --- a/t/t9850-shell.sh\n> +++ b/t/t9850-shell.sh\n> @@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '\n>  '\n>  \n>  test_expect_success 'shell complains of overlong commands' '\n> -\ttest-tool genzeros | tr \"\\000\" \"a\" |\n> +\ttest-tool genbytes a |\n>  \ttest_must_fail git shell 2>err &&\n>  \ttest_grep \"too long\" err\n>  '\n\nOK, so that's how an infinitely long sequence of characters is used.\n\nLooking good.  Let's mark the topic as ready for 'next'.\n\n"},{"id":"558179","messageId":"avRJPDEJWTGWySKp@pks.im","threadId":"66800","inReplyTo":"pull.2301.v3.git.1794335168170.gitgitgadget@gmail.com","subject":"Re: [PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-11T10:54:00Z","receivedAt":"2026-11-11T10:54:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Nov 10, 2026 at 06:26:08PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> diff --git a/t/helper/test-genzeros.c b/t/helper/test-genzeros.c\n> index b895436a32..e8d83d5012 100644\n> --- a/t/helper/test-genzeros.c\n> +++ b/t/helper/test-genzeros.c\n> @@ -35,3 +33,29 @@ int cmd__genzeros(int argc, const char **argv)\n>  \n>  \treturn 0;\n>  }\n> +\n> +int cmd__genbytes(int argc, const char **argv)\n> +{\n> +\tintmax_t count = -1;\n> +\n> +\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1 ||\n> +\t    (argc == 3 && !git_parse_signed(argv[2], &count, INTMAX_MAX))) {\n> +\t\tfprintf(stderr, \"usage: %s <char> [<count>]\\n\", argv[0]);\n> +\t\treturn 1;\n> +\t}\n> +\n> +\treturn generate_bytes(argv[1][0], count);\n> +}\n\nOkay, so instead of interpreting `argv[0]` we now just take it as-is and\nverify that it's a single character, only. That makes users way easier\nto reason about because you don't have to wonder about how the argument\nis encoded. It's also extensible because we can easily introduce longer\nstrings or \"\\x00\" and \"\\000\" eventually without breaking any existing\ncallers.\n\n> +int cmd__genzeros(int argc, const char **argv)\n> +{\n> +\tintmax_t count = -1;\n> +\n> +\tif (argc > 2 ||\n> +\t    (argc == 2 && !git_parse_signed(argv[1], &count, INTMAX_MAX))) {\n> +\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n> +\t\treturn 1;\n> +\t}\n> +\n> +\treturn generate_bytes('\\0', count);\n> +}\n\nAnd this one here is trivially adapted.\n\n> diff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\n> index 0b27d75fe3..0b74ff6157 100755\n> --- a/t/t0613-reftable-write-options.sh\n> +++ b/t/t0613-reftable-write-options.sh\n> @@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n>  \t(\n>  \t\tcd repo &&\n>  \t\ttest_commit A &&\n> -\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n> +\t\ttest-tool genbytes a 500 >logmsg &&\n>  \t\tcat >expect <<-EOF &&\n>  \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n>  \t\tEOF\n\nAnd all of the callers are easier to understand now.\n\nThanks, this looks great!\n\nPatrick\n\n"},{"id":"558229","messageId":"xmqqse17nztl.fsf@gitster.g","threadId":"66800","inReplyTo":"pull.2301.v3.git.1794335168170.gitgitgadget@gmail.com","subject":"Re: [PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-11T15:38:14Z","receivedAt":"2026-11-11T15:38:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> From: Kristofer Karlsson <krka@spotify.com>\n>\n> Several tests create fixture data consisting of a single repeated\n> character with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\n> works well on platforms where tr(1) is implemented efficiently.\n\nThis looks good as-is, but once we merge this down, we will find\nthat there is a new use of the pattern that pipes test-tool genzeros\ninto tr in t/t1600-index.sh recently merged as part of the Git\n2.50.2 update.\n\nPerhaps we should rebase to catch that callsite?\n\nThanks.\n\n\n"},{"id":"558232","messageId":"CAL71e4N2T0XLZgnUVPEk1q8JjRiWbRo5B47DN_hfs1CreYH6Jg@mail.gmail.com","threadId":"66800","inReplyTo":"xmqqse17nztl.fsf@gitster.g","subject":"Re: [PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-11-11T15:47:41Z","receivedAt":"2026-11-11T15:47:41Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"On Wed, 11 Nov 2026 at 16:38, Junio C Hamano <gitster@pobox.com> wrote:\n>\n> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n> > From: Kristofer Karlsson <krka@spotify.com>\n> >\n> > Several tests create fixture data consisting of a single repeated\n> > character with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\n> > works well on platforms where tr(1) is implemented efficiently.\n>\n> This looks good as-is, but once we merge this down, we will find\n> that there is a new use of the pattern that pipes test-tool genzeros\n> into tr in t/t1600-index.sh recently merged as part of the Git\n> 2.50.2 update.\n>\n> Perhaps we should rebase to catch that callsite?\n>\n> Thanks.\n>\n\nOh that was unlucky timing, but easy to fix.  Not only is this\nnew test using tr, it's also using very large values.\nI'll submit a v4 rebased on latest master, which should fix it.\n\nThanks,\nKristofer\n\n"},{"id":"558233","messageId":"pull.2301.v4.git.1794412751763.gitgitgadget@gmail.com","threadId":"66800","inReplyTo":"pull.2301.git.1794063565.gitgitgadget@gmail.com","subject":"[PATCH v4] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-11-11T15:59:11Z","receivedAt":"2026-11-11T15:59:11Z","isPatch":true,"sender":{"key":"krka@spotify.com","avatar":"https://avatars.githubusercontent.com/u/5349266?v=4"},"body":"From: Kristofer Karlsson <krka@spotify.com>\n\nSeveral tests create fixture data consisting of a single repeated\ncharacter with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\nworks well on platforms where tr(1) is implemented efficiently.\n\nThe implementation of tr(1) on macOS supports multi-byte characters,\nand processes its input one character at a time through\ngetwchar() and putwchar(), even in the C locale.  For small inputs\nthat does not matter, but t4205 generates more than 2GB of data this\nway, which makes that test much slower on macOS than on Linux: on the\nmacOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on\nit, compared to about 6 seconds on Linux.\n\nFix this by adding \"test-tool genbytes <char> [<count>]\" next to\n\"genzeros\", sharing the same code, which writes the requested\ncharacter directly.  Use it in all tests that piped \"genzeros\"\nthrough tr(1), not just t4205, so that it is the obvious choice for\nnew tests.  Callers that want NUL bytes keep using \"genzeros\".\n\nWhile at it, parse the count with git_parse_signed(), so that an\ninvalid count is reported as a usage error instead of being silently\ntreated as 0.\n\nThis makes t4205 much faster on macOS in CI.  The Linux jobs, shown\nfor comparison, show smaller differences, which may partly be due to\nnormal run-to-run variation:\n\n  job                before    after   change\n  osx-gcc            821.1s   243.3s   -70.4%\n  osx-reftable       911.0s   194.0s   -78.7%\n  linux-asan-ubsan   226.7s   222.1s    -2.0%\n  linux-leaks        320.2s   298.8s    -6.7%\n\nSigned-off-by: Kristofer Karlsson <krka@spotify.com>\n---\n    t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n    \n    While looking for slow tests in CI, t4205 stood out on macOS: its\n    EXPENSIVE tests take several times longer there than on Linux. A CI\n    experiment showed that much of the time was spent in tr(1), which is\n    very slow on macOS for the 2GB of data that t4205 generates.\n    \n    This adds \"test-tool genbytes []\" and uses it instead of \"genzeros |\n    tr\", which is where the speedup comes from.\n    \n    Changes since v3:\n    \n     * Rebased on latest master to also convert a new large \"genzeros | tr\"\n       use in t1600.\n    \n    Changes since v2:\n    \n     * \"genbytes\" takes a literal character again (\"genbytes a\") instead of\n       two hex digits (\"genbytes 61\"), which Patrick found hard to read at\n       the call sites.\n    \n     * \"genzeros\" and its callers that want NUL bytes are kept as they are,\n       and the patch that converted them to \"genbytes 00\" is dropped. This\n       avoids churn in tests that do not need to change, as discussed with\n       Johannes. Keeping \"genzeros\" also means that \"genbytes\" does not need\n       special handling for NUL bytes.\n    \n     * test-genzeros.c is no longer renamed, and \"genbytes\" lives next to\n       \"genzeros\" in the same file. Since \"genzeros\" stays, the current file\n       name is still valid and not worth churning.\n    \n     * The buffer is static again instead of 256 KiB on the stack, as\n       suggested by Junio, to avoid using a large part of the limited stack\n       space. There is no risk of concurrent use in a run-once helper.\n    \n     * The count is parsed with git_parse_signed(), so that an invalid count\n       is a usage error instead of being silently treated as 0, as suggested\n       by Patrick. This applies to \"genzeros\" as well.\n    \n    Changes in v2 (subsequently reverted in v3):\n    \n     * Squashed the rename of test-genzeros.c into the patch that changes\n       it, as suggested by Junio.\n    \n     * \"genbytes\" took the byte as two lowercase hexadecimal digits instead\n       of a literal character, as suggested by Junio. This also worked for\n       NUL bytes, so \"genbytes 00\" replaced special-casing \"NUL\".\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2301%2Fspkrka%2Ftest-tool-genbytes-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2301/spkrka/test-tool-genbytes-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/2301\n\nRange-diff vs v3:\n\n 1:  17e5baacdb ! 1:  c94a8e0ca7 t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS\n     @@ t/t0613-reftable-write-options.sh: test_expect_success 'small block size fails w\n       \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n       \t\tEOF\n      \n     + ## t/t1600-index.sh ##\n     +@@ t/t1600-index.sh: test_expect_success 'index.maxPathLen rejects out-of-range values' '\n     + make_large_tree () {\n     + \t{\n     + \t\tprintf \"100644 \" &&\n     +-\t\ttest-tool genzeros \"$1\" | tr \"\\000\" a &&\n     ++\t\ttest-tool genbytes a \"$1\" &&\n     + \t\tprintf \"\\0\" &&\n     + \t\ttest-tool genzeros \"$(test_oid rawsz)\"\n     + \t} >\"tree-$1\" &&\n     +\n       ## t/t4205-log-pretty-formats.sh ##\n      @@ t/t4205-log-pretty-formats.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes\n       '\n\n\n t/helper/test-genzeros.c          | 54 ++++++++++++++++++++++---------\n t/helper/test-tool.c              |  1 +\n t/helper/test-tool.h              |  1 +\n t/t0613-reftable-write-options.sh |  2 +-\n t/t1600-index.sh                  |  2 +-\n t/t4205-log-pretty-formats.sh     |  2 +-\n t/t5300-pack-object.sh            |  6 ++--\n t/t7006-pager.sh                  |  3 +-\n t/t9850-shell.sh                  |  2 +-\n 9 files changed, 49 insertions(+), 24 deletions(-)\n\ndiff --git a/t/helper/test-genzeros.c b/t/helper/test-genzeros.c\nindex b895436a32..e8d83d5012 100644\n--- a/t/helper/test-genzeros.c\n+++ b/t/helper/test-genzeros.c\n@@ -2,30 +2,28 @@\n \n #include \"test-tool.h\"\n #include \"git-compat-util.h\"\n+#include \"parse.h\"\n \n-int cmd__genzeros(int argc, const char **argv)\n+/*\n+ * Write \"count\" copies of \"byte\" to stdout, or an endless stream of them if\n+ * \"count\" is negative.\n+ */\n+static int generate_bytes(unsigned char byte, intmax_t count)\n {\n-\t/* static, so that it is NUL-initialized */\n-\tstatic const char zeros[256 * 1024];\n-\tintmax_t count;\n+\tstatic char buf[256 * 1024];\n \tssize_t n;\n \n-\tif (argc > 2) {\n-\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n-\t\treturn 1;\n-\t}\n-\n-\tcount = argc > 1 ? strtoimax(argv[1], NULL, 0) : -1;\n+\tmemset(buf, byte, sizeof(buf));\n \n-\t/* Writing out individual NUL bytes is slow... */\n+\t/* Writing out individual bytes is slow... */\n \twhile (count < 0)\n-\t\tif (xwrite(1, zeros, ARRAY_SIZE(zeros)) < 0)\n+\t\tif (xwrite(1, buf, ARRAY_SIZE(buf)) < 0)\n \t\t\tdie_errno(\"write error\");\n \n \twhile (count > 0) {\n-\t\tn = xwrite(1, zeros,\n-\t\t\t   count < ARRAY_SIZE(zeros)\n-\t\t\t   ? count : ARRAY_SIZE(zeros));\n+\t\tn = xwrite(1, buf,\n+\t\t\t   count < ARRAY_SIZE(buf)\n+\t\t\t   ? count : ARRAY_SIZE(buf));\n \n \t\tif (n < 0)\n \t\t\tdie_errno(\"write error\");\n@@ -35,3 +33,29 @@ int cmd__genzeros(int argc, const char **argv)\n \n \treturn 0;\n }\n+\n+int cmd__genbytes(int argc, const char **argv)\n+{\n+\tintmax_t count = -1;\n+\n+\tif (argc < 2 || argc > 3 || strlen(argv[1]) != 1 ||\n+\t    (argc == 3 && !git_parse_signed(argv[2], &count, INTMAX_MAX))) {\n+\t\tfprintf(stderr, \"usage: %s <char> [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes(argv[1][0], count);\n+}\n+\n+int cmd__genzeros(int argc, const char **argv)\n+{\n+\tintmax_t count = -1;\n+\n+\tif (argc > 2 ||\n+\t    (argc == 2 && !git_parse_signed(argv[1], &count, INTMAX_MAX))) {\n+\t\tfprintf(stderr, \"usage: %s [<count>]\\n\", argv[0]);\n+\t\treturn 1;\n+\t}\n+\n+\treturn generate_bytes('\\0', count);\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex 83e5fee730..768d1c7e35 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -35,6 +35,7 @@ static struct test_cmd cmds[] = {\n \t{ \"example-tap\", cmd__example_tap },\n \t{ \"find-pack\", cmd__find_pack },\n \t{ \"fsmonitor-client\", cmd__fsmonitor_client },\n+\t{ \"genbytes\", cmd__genbytes },\n \t{ \"genrandom\", cmd__genrandom },\n \t{ \"genzeros\", cmd__genzeros },\n \t{ \"getcwd\", cmd__getcwd },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex b020e24fc4..8dc54150ef 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -28,6 +28,7 @@ int cmd__env_helper(int argc, const char **argv);\n int cmd__example_tap(int argc, const char **argv);\n int cmd__find_pack(int argc, const char **argv);\n int cmd__fsmonitor_client(int argc, const char **argv);\n+int cmd__genbytes(int argc, const char **argv);\n int cmd__genrandom(int argc, const char **argv);\n int cmd__genzeros(int argc, const char **argv);\n int cmd__getcwd(int argc, const char **argv);\ndiff --git a/t/t0613-reftable-write-options.sh b/t/t0613-reftable-write-options.sh\nindex 0b27d75fe3..0b74ff6157 100755\n--- a/t/t0613-reftable-write-options.sh\n+++ b/t/t0613-reftable-write-options.sh\n@@ -146,7 +146,7 @@ test_expect_success 'small block size fails with large reflog message' '\n \t(\n \t\tcd repo &&\n \t\ttest_commit A &&\n-\t\ttest-tool genzeros 500 | tr \"\\000\" \"a\" >logmsg &&\n+\t\ttest-tool genbytes a 500 >logmsg &&\n \t\tcat >expect <<-EOF &&\n \t\tfatal: update_ref failed for ref ${SQ}refs/heads/logme${SQ}: reftable: transaction failure: entry too large\n \t\tEOF\ndiff --git a/t/t1600-index.sh b/t/t1600-index.sh\nindex 9c51cc04e1..dc4d68c1fb 100755\n--- a/t/t1600-index.sh\n+++ b/t/t1600-index.sh\n@@ -161,7 +161,7 @@ test_expect_success 'index.maxPathLen rejects out-of-range values' '\n make_large_tree () {\n \t{\n \t\tprintf \"100644 \" &&\n-\t\ttest-tool genzeros \"$1\" | tr \"\\000\" a &&\n+\t\ttest-tool genbytes a \"$1\" &&\n \t\tprintf \"\\0\" &&\n \t\ttest-tool genzeros \"$(test_oid rawsz)\"\n \t} >\"tree-$1\" &&\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 01b97c8888..894d88c39b 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1182,7 +1182,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit mes\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n-\ttest-tool genzeros 2147483649 | tr \"\\000\" \"1\" >expect &&\n+\ttest-tool genbytes 1 2147483649 >expect &&\n \thuge_commit=$(git commit-tree -F expect HEAD^{tree})\n '\n \ndiff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh\nindex e5ce478484..4fb6a22145 100755\n--- a/t/t5300-pack-object.sh\n+++ b/t/t5300-pack-object.sh\n@@ -9,9 +9,9 @@ test_description='git pack-object'\n \n test_expect_success 'setup' '\n \trm -f .git/index* &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"a\" >a &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"b\" >b &&\n-\ttest-tool genzeros 4096 | tr \"\\000\" \"c\" >c &&\n+\ttest-tool genbytes a 4096 >a &&\n+\ttest-tool genbytes b 4096 >b &&\n+\ttest-tool genbytes c 4096 >c &&\n \ttest-tool genrandom \"seed a\" 2097152 >a_big &&\n \ttest-tool genrandom \"seed b\" 2097152 >b_big &&\n \tgit update-index --add a a_big b b_big c &&\ndiff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\nindex 75ca7a25da..8f1f71bf12 100755\n--- a/t/t7006-pager.sh\n+++ b/t/t7006-pager.sh\n@@ -662,8 +662,7 @@ test_expect_success 'setup trace2' '\n '\n \n test_expect_success 'setup large log output' '\n-\ttest-tool genzeros 50000 |\n-\ttr \"\\000\" \"a\" |\n+\ttest-tool genbytes a 50000 |\n \tsed \"s/a/this is a long commit message/g\" >commit-msg &&\n \tgit commit --allow-empty -F commit-msg\n '\ndiff --git a/t/t9850-shell.sh b/t/t9850-shell.sh\nindex 27cc49bbd1..d56edebda7 100755\n--- a/t/t9850-shell.sh\n+++ b/t/t9850-shell.sh\n@@ -30,7 +30,7 @@ test_expect_success 'shell allows interactive command' '\n '\n \n test_expect_success 'shell complains of overlong commands' '\n-\ttest-tool genzeros | tr \"\\000\" \"a\" |\n+\ttest-tool genbytes a |\n \ttest_must_fail git shell 2>err &&\n \ttest_grep \"too long\" err\n '\n\nbase-commit: 1891de46f79dc444a47db1c87bd0c8c76b5463f2\n-- \ngitgitgadget\n\n"},{"id":"558242","messageId":"xmqq4idnntct.fsf@gitster.g","threadId":"66800","inReplyTo":"CAL71e4N2T0XLZgnUVPEk1q8JjRiWbRo5B47DN_hfs1CreYH6Jg@mail.gmail.com","subject":"Re: [PATCH v3] t: add \"test-tool genbytes\" to avoid slow tr(1) on macOS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-11T17:57:54Z","receivedAt":"2026-11-11T17:57:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kristofer Karlsson <krka@spotify.com> writes:\n\n> On Wed, 11 Nov 2026 at 16:38, Junio C Hamano <gitster@pobox.com> wrote:\n>>\n>> \"Kristofer Karlsson via GitGitGadget\" <gitgitgadget@gmail.com>\n>> writes:\n>>\n>> > From: Kristofer Karlsson <krka@spotify.com>\n>> >\n>> > Several tests create fixture data consisting of a single repeated\n>> > character with \"test-tool genzeros <count> | tr '\\000' <char>\".  This\n>> > works well on platforms where tr(1) is implemented efficiently.\n>>\n>> This looks good as-is, but once we merge this down, we will find\n>> that there is a new use of the pattern that pipes test-tool genzeros\n>> into tr in t/t1600-index.sh recently merged as part of the Git\n>> 2.50.2 update.\n>>\n>> Perhaps we should rebase to catch that callsite?\n>>\n>> Thanks.\n>>\n>\n> Oh that was unlucky timing, but easy to fix.  Not only is this\n> new test using tr, it's also using very large values.\n> I'll submit a v4 rebased on latest master, which should fix it.\n\nThanks for a prompt response.\n\n"}]}