Re: [PATCH 5/5] test-lib: add support for GIT_TODO_TESTS

From: Junio C Hamano <gitster@pobox.com>
Date: Nov 29, 2018, 06:42 UTC
Message-ID: <xmqqk1kwo04u.fsf@gitster-ct.c.googlers.com>
In-Reply-To: <20181127225445.30045-5-avarab@gmail.com>

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

Show 6 quoted lines

> -To skip tests, set the GIT_SKIP_TESTS variable. Individual tests can
> -be skipped:
> +To skip tests, set either the GIT_SKIP_TESTS or GIT_TODO_TESTS
> +variables. The difference is that with SKIP the tests won't be run at
> +all, whereas they will be run with TODO, but in success or failure
> +annotated as a TODO test.

It is entirely unclear what "They will be run with TODO" means.

I know what you want to achieve from the code change; instead of just skipping, you want to cause as much side effect the skipped test piece wants to make until it fails and dies, without taking the remainder of the test down. And I kind of agree that sometimes such a mode would be very useful (i.e. when you do not want to bother separating such a failing test properly into the setup part that would affect later tests and the one-thing-it-wants-to-test part that can now be safely skipped). From the cursory look, the code change in this patch is sensible realization of that idea.

Having said all that, I won't picking this up until next month ;-) I really want to see that everybody is concentrating on making sure that 2.20 is solid before playing with shiny new toys.

Thanks.