Re: [RFC PATCH 0/3] Introduce clar testing framework

From: Patrick Steinhardt <ps@pks.im>
Date: Aug 1, 2024, 09:31 UTC
Message-ID: <ZqtV_G3ZNNFR5dDf@tanuki>
In-Reply-To: <avbtyqklmtwmz63vbpevkz4xyv3dboyghwhkodxjlb3ur6zt36@afi5ptrsvpz2>

On Wed, Jul 31, 2024 at 11:33:55AM -0700, Josh Steadmon wrote:

Show 8 quoted lines

> On 2024.07.31 11:04, Patrick Steinhardt wrote:
> >   - I prefer that the proposed solution results in a single binary as
> >     compared to one binary per test system.
> 
> Does clar allow running test functions in parallel? With multiple
> binaries, we can at least run independent tests in parallel (although
> right now the unit tests are fewer and so much faster than the shell
> tests that it's hardly noticeable).

Ah, that's something I didn't think of. clar does not support running tests in parallel.

As you say, I guess for now that is fine and I'd claim that it likely is faster to just run all tests sequentially with a single binary if you also include build times. If that claim isn't true, or if we eventually grow a huge body of tests, then we should likely revert to having separate binaries.

Show 6 quoted lines

> As part of the original unit-test series, I wrote a comparison between
> different frameworks: Documentation/technical/unit-tests.txt, poorly
> rendered at [1]. Could you add a row to the table evaluating clar on the
> individual points there?
> 
> [1] https://git-scm.com/docs/unit-tests#framework-selection

Ah, I wasn't aware of this document. I can update it depending on how the discussion goes overall :)

Patrick