Re: [PATCH 3/9] t4141: fix inefficient use of dd(1)
- From
Patrick Steinhardt <ps@pks.im>
- Date
- Jul 3, 2026, 05:42 UTC
- Message-ID
- <akdLqNHW3pGThQat@pks.im>
- In-Reply-To
- <20260702211614.GB2051171@coredump.intra.peff.net>
On Thu, Jul 02, 2026 at 05:16:14PM -0400, Jeff King wrote:
Show 21 quoted lines
> On Thu, Jul 02, 2026 at 02:00:56PM +0200, Patrick Steinhardt wrote: > > > This test setup is extremely expensive, as `test_copy_bytes` is > > implemented via `dd ibs=1 count="$1"`, which copies data one byte at a > > time. So as we write 1GB of data, we end up doing 1 billion reads and > > writes. This naturally takes a while: it takes 6 minutes on my system, > > and around 40 minutes in some CI jobs! > > > > We can do much better though, as genzeros already knows to handle an > > optional limit of how much data it is supposed to write, which allows us > > to remove the call to `test_copy_bytes`. Furthermore, it has already > > been optimized to generate the data fast. > > Seems like a good fix for this case, where we can skip the extra process > entirely. > > It feels like test_copy_bytes should be able to do much better in > general. The obvious thing to reach for is "head -c", but the function > was originally added because that wasn't portable. The "-c" option is > not in POSIX, though the original comment claims IRIX was the problem, > so I wonder if "head -c" is de facto portable these days.
An alternative could be to implement a simple helper as part of our test-tool. But I doubt it's really worth it: almost all callers only want to copy a small number of bytes. The only exception seems to be t0021, where we copy up to 65kB. But that whole test suite still only takes ~3 seconds, so optimizing that feels like wasted time to me.
Patrick