Re: [RFC PATCH 1/2] Add C TAP harness

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: May 2, 2023, 16:34 UTC
Message-ID: <230502.86wn1qhemd.gmgdl@evledraar.gmail.com>
In-Reply-To: <afd3e80f-99d9-7472-36e7-335f86263d09@gmail.com>

On Thu, Apr 27 2023, Phillip Wood wrote:

Show 25 quoted lines

> Hi Calvin
>
> On 27/04/2023 18:50, Calvin Wan wrote:
>> Introduces the C TAP harness from https://github.com/rra/c-tap-harness/
>> There is also more complete documentation at
>> https://www.eyrie.org/~eagle/software/c-tap-harness/
>
> I'm afraid this reply is rather briefer than I'd like but I'm short of
> time and about to go off-list for a couple of weeks. My ideal unit
> test library would
>
>  - print the file and line number of failed assertions
>  - allow the test plan to be omitted by calling test_done() at the end
>    of the test file as we do in our main test suite.
>  - support the TODO directive
>  - allow named tests (this maybe more trouble that it is worth as I
>    think it inevitably leads to more boilerplate code calling the named
>    tests)
>
> Unfortunately this library doesn't seem to offer any of those
> features. It does support a lazy test plan but uses atexit() so will
> not detect if the test program exits before all the tests have run. I
> think it would be useful to add some unit tests to our test suite and
> maybe this library could form the basis of that but I think printing
> the file and line number of failed assertions is pretty essential.

Other things aside, I prefer our explicit "test_done", but I don't see why you think an atexit() isn't enough to catch incomplete tests.

For a C program you'd just do something like this (somewhat pseudocode,
I didn't check if it compiled etc):
	
        static int done; /* read by atexit() handler */

        void on_atexit(void)
	{
		if (!done)
			BUG();
        	print_plan_line();
	}

	int main(void)
	{
                int ret;

	        setup_atexit(a_handler);
		ret = do_tests();
	        done = 1;

                return ret;
	}

If I'm understanding you correctly you're concerned that if some user code within do_test() calls exit() we won't return from "do_test()", but we *would* call print_plan_line().

That's a valid concern, we want to distinguish such "early return" from cases where we run to completion, that's why we use "test_done" in the shell code.

But in the C case I think just using something like the "done" variable pattern above should cover that, without the need for an explicit "test_done".

But maybe I'm missing something.