Re: [PATCH 0/4] meson: parse TAP output generated by our tests

From: Phillip Wood <phillip.wood123@gmail.com>
Date: May 26, 2025, 13:31 UTC
Message-ID: <74c03ad4-b132-4cad-b65d-821324380343@gmail.com>
In-Reply-To: <aDRiLdUCEVQHq26z@pks.im>

On 26/05/2025 13:44, Patrick Steinhardt wrote:

Show 5 quoted lines

> 
> I don't think it's inherently a bad thing to fail on unexpected passes.
> After all, it shows that our assumption that the test fails is broken,
> and that we should have a look why that is. But I can see arguments both
> ways.

Personally I'd be very happy if our test suite failed on an unexpected pass. Currently it is easy to miss, especially if the unexpected pass occurs in a CI run. Missing an unexpected pass means we don't change 'test_expect_failure' to 'test_expect_pass' and a future regression that causes the test to fail again will go unnoticed.

Best Wishes

Phillip

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.