Re: [PATCH v8 2/3] unit tests: add TAP unit test framework
- From
Christian Couder <christian.couder@gmail.com>
- Date
- Oct 27, 2023, 20:15 UTC
- Message-ID
- <CAP8UFD3eY_i36YO0OcpAp9ey5KO0q-PrwvjSLRXKYQb=iZ8JCA@mail.gmail.com>
- In-Reply-To
- <00d3c95a81449bf49c4ce992d862d7a858691840.1696889530.git.steadmon@google.com>
On Tue, Oct 10, 2023 at 12:22 AM Josh Steadmon <steadmon@google.com> wrote:
Show 38 quoted lines
>
> From: Phillip Wood <phillip.wood@dunelm.org.uk>
>
> This patch contains an implementation for writing unit tests with TAP
> output. Each test is a function that contains one or more checks. The
> test is run with the TEST() macro and if any of the checks fail then the
> test will fail. A complete program that tests STRBUF_INIT would look
> like
>
> #include "test-lib.h"
> #include "strbuf.h"
>
> static void t_static_init(void)
> {
> struct strbuf buf = STRBUF_INIT;
>
> check_uint(buf.len, ==, 0);
> check_uint(buf.alloc, ==, 0);
> check_char(buf.buf[0], ==, '\0');
> }
>
> int main(void)
> {
> TEST(t_static_init(), "static initialization works);
>
> return test_done();
> }
>
> The output of this program would be
>
> ok 1 - static initialization works
> 1..1
>
> If any of the checks in a test fail then they print a diagnostic message
> to aid debugging and the test will be reported as failing. For example a
> failing integer check would look like
>
> # check "x >= 3" failed at my-test.c:102I wonder if it would be a bit better to say that the test was an integer test for example with "check_int(x >= 3) failed ..."
> # left: 2 > # right: 3
I like "expected" and "actual" better than "left" and "right", not sure how it's possible to have that in a way consistent with the shell tests though.
Show 7 quoted lines
> not ok 1 - x is greater than or equal to three > > There are a number of check functions implemented so far. check() checks > a boolean condition, check_int(), check_uint() and check_char() take two > values to compare and a comparison operator. check_str() will check if > two strings are equal. Custom checks are simple to implement as shown in > the comments above test_assert() in test-lib.h.
Yeah, nice.
> Tests can be skipped with test_skip() which can be supplied with a > reason for skipping which it will print. Tests can print diagnostic > messages with test_msg(). Checks that are known to fail can be wrapped > in TEST_TODO().
Maybe TEST_TOFIX() would be a bit more clear, but "TODO" is something that is more likely to be searched for than "TOFIX", so Ok.
Show 10 quoted lines
> There are a couple of example test programs included in this > patch. t-basic.c implements some self-tests and demonstrates the > diagnostic output for failing test. The output of this program is > checked by t0080-unit-test-output.sh. t-strbuf.c shows some example > unit tests for strbuf.c > > The unit tests will be built as part of the default "make all" target, > to avoid bitrot. If you wish to build just the unit tests, you can run > "make build-unit-tests". To run the tests, you can use "make unit-tests" > or run the test binaries directly, as in "./t/unit-tests/bin/t-strbuf".
Nice!
> +unit-tests-prove: > + @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS)
Nice, but DEFAULT_TEST_TARGET=prove isn't used. So not sure how important or relevant the 'prove' related sections are in the Documentation/technical/unit-tests.txt file introduced by the previous patch.
Show 8 quoted lines
> +int test_assert(const char *location, const char *check, int ok)
> +{
> + assert(ctx.running);
> +
> + if (ctx.result == RESULT_SKIP) {
> + test_msg("skipping check '%s' at %s", check, location);
> + return 1;
> + } else if (!ctx.todo) {I think it would be a bit clearer without the "else" above and with the "if (!ctx.todo) {" starting on a new line.
Show 10 quoted lines
> + if (ok) {
> + test_pass();
> + } else {
> + test_msg("check \"%s\" failed at %s", check, location);
> + test_fail();
> + }
> + }
> +
> + return !!ok;
> +}Otherwise it looks good to me.