Re: [PATCH 1/6] t0080: move expected output to a file

From: René Scharfe <l.s.r@web.de>
Date: Jul 1, 2024, 19:51 UTC
Message-ID: <177e98da-803e-4839-9484-fd24828bd21b@web.de>
In-Reply-To: <20240701032047.GA610406@coredump.intra.peff.net>

Am 01.07.24 um 05:20 schrieb Jeff King:

Show 31 quoted lines

> On Sat, Jun 29, 2024 at 05:35:31PM +0200, René Scharfe wrote:
>
>> Provide the expected output of "test-tool example-tap" verbatim instead
>> of as a here-doc, to avoid distractions due to quoting, variables
>> containing quotes and indentation.
>
> I'm not really opposed to this patch, but I wondered...
>
>>  test_expect_success 'TAP output from unit tests' '
>> -	cat >expect <<-EOF &&
>> -	ok 1 - passing test
>> -	ok 2 - passing test and assertion return 1
>
> If you could take the test input on stdin, like so:
>
>   test_expect_success 'TAP output from unit tests' - <<-\EOT
> 	cat >expect <<-\EOF
> 	ok 1 - passing test
> 	ok 2 - passing test and assertion return 1
> 	[...]
> 	# check "'a' == '\n'" failed at t/helper/test-example-tap.c:64
> 	#    left: 'a'
> 	#   right: '\012'
> 	[...]
> 	EOF
>   EOT
>
> would that be preferable to moving it to its own file? I kind of like
> keeping everything in the test scripts themselves so related changes can
> happen side-by-side, though I admit in this case it is intimately tied
> to the separate test-example-tap.c source anyway.

I can't think of an example where we keep test definitions in the same file as the code to be tested. It would be somewhat cool to empower the unit test framework to test itself, but I suspect that this nesting ability would be hard to achieve and not very useful otherwise. And would we be able to trust such a self-test?

We could cheese it by putting the expected output into a special comment before (or after) the TEST invocations and letting the test script piece them together to build the expect file, something like:

	/* expect
	ok 1 - passing test
	 */
	test_res = TEST(check_res = check_int(1, ==, 1), "passing test");

That would be a bit annoying if we change something because some messages contain line numbers and the comments would affect those due to their existence alone. And there would be a ripple effect if we change the number of output lines of a test to the output of later tests.

The only downside of keeping the expected output of t0080 separate that I can think of is that it might get confusing if we'd ever add more test_expect_success calls to it, but I can't imagine why we'd want to do that.

> But I do have such an "EOT" patch which I've been meaning to send out,
> since it makes many of these quoting annoyances go away (though of
> course it leaves the indentation).

Being able to pass the test code to test_expect_success as a here-doc or file to avoid nested shell quoting sounds useful in general. For t0080 we could achieve the same effect already by creating the expect file before calling test_expect_success. That has the downside of passing even when the disk is full and the files are created empty, but we can throw in a "test -s" to rule it out.

René