From: Kristofer Karlsson Date: Sun, 08 Nov 2026 09:18:27 GMT Subject: Re: [PATCH 2/3] t: add "test-tool genbytes" to avoid slow tr(1) on macOS Message-ID: In-Reply-To: <50b6c35f-e9c3-47ef-8fca-aebd2620623a@kdbg.org> On Sun, 8 Nov 2026 at 09:54, Johannes Sixt wrote: > > Am 07.11.26 um 15:59 schrieb Kristofer Karlsson via GitGitGadget: > > Several tests create fixture data consisting of a single repeated > > character with "test-tool genzeros | tr '\000' ". This > > works well on platforms where tr(1) is implemented efficiently. > > > > The implementation of tr(1) on macOS supports multi-byte characters, > > and processes its input one character at a time through > > getwchar() and putwchar(), even in the C locale. For small inputs > > that does not matter, but t4205 generates more than 2GB of data this > > way, which makes that test much slower on macOS than on Linux: on the > > macOS CI runners, tr(1) alone spends 95 to 123 seconds of CPU time on > > it, compared to about 6 seconds on Linux. > > > > Fix this by adding "test-tool genbytes []", which writes > > the requested byte directly, and turn "genzeros" into a thin > > wrapper around the same code. > > All cases that do not need a particular character can use > > printf %0.500d 0 > > Though, on my Linux box, > > printf %0.2147483649d 0 > > produces nothing, but > > printf 000000000%0.2147483640d 0 > > has 2147483649 bytes of output. > > If this isn't insanely slow on macOS, use this pattern instead of a > highly engineered helper program? ;-) > > -- Hannes > Nice trick, and it would work for t4205, which does not care about the byte value. I tried it for the 2147483649 bytes there on Linux: printf time peak memory dash 6.5s 2.1GB bash 6.0s 3MB coreutils 3.7s 2.1GB It is definitely fast enough (on the variants I tested). The annoying thing (but not a real blocker) is the extra memory usage required and I am not sure I fully trust that all implementations will behave the same for absurdly large padding. (Particularly for paddings larger than 2^31, as you noted too.) It feels a bit like a hack, and tweaking the existing test-tool feels like a more sane option that is easier to reason about. A gotcha here is that the printf hack only works for producing "0", so for the cases we truly need NUL characters it does not help us, and I think that is also a point in favor of simply generalizing test-tool genzeros instead. Thanks, Kristofer