Re: [PATCH 0/3] [RFC] tests: add test_todo() for known failures

From: Junio C Hamano <gitster@pobox.com>
Date: Oct 7, 2022, 17:08 UTC
Message-ID: <xmqq8rlrh880.fsf@gitster.g>
In-Reply-To: <8df2260c-7a35-5b50-7273-fbd9894a614c@dunelm.org.uk>

Phillip Wood <phillip.wood123@gmail.com> writes:

> I think there is a question of whether we need a new toplevel
> test_expect_todo - why would we add it if we can just reuse
> test_expect_success? That way when a test failure is fixed all that
> needs to be done is to remove the test_todo calls.

Yup, that is one of the reasons why I favor test_todo inside the normal test_expect_success. A patch that fixes a breakage would come with a change to the tests that flips test_expect_failure to test_expect_success often had the step that were expected to fail outside the post context and did not make it immediately obvious what was fixed, and use of a more focused test_todo would make it more clear. Unless we gain a clear advantage by using a different top-level (e.g. some of the limitations like "not in a subshell" can be lifted?), I do not think we want to add one.