Re: [PATCH 1/3] [RFC] tests: add test_todo() to mark known breakages

From: Junio C Hamano <gitster@pobox.com>
Date: Dec 9, 2022, 01:09 UTC
Message-ID: <xmqqedt92xjk.fsf@gitster.g>
In-Reply-To: <1b5fc712-9659-1bd6-493c-197b003d21d6@dunelm.org.uk>

Phillip Wood <phillip.wood123@gmail.com> writes:

Show 5 quoted lines

> I added the restrictions to 'test_todo' to try and stop it being
> misused but I'm happy to relax them if needed. I'm keen that test_todo
> is able to distinguish between an expected failure and a failure due
> to the wrapped command being misused e.g. 'test_todo grep
> --invalid-option' should report an error.

Hmm, but it is not useful if the failure is from "you cannot use the system command grep with test_todo", as the (implicitly) encouraged "fix" the developer who wrote the test would pick would be to use "! grep --invalid-option" which would still fail for a wrong reason.

If a "git" command is expected to run to a completion but is currently broken and produces a wrong output, it would be very useful to be able to write

	git command --args >actual &&
	test_todo grep -e "$expected_token" actual

to say "when 'git command' is fixed, the output should contains this, but we know it currently is broken".

> I think you've convinced be to remove the restrictions on what can be
> wrapped by 'test_todo' when I re-roll.
>
> Thanks for your thoughtful comments

Thanks.