Re: [PATCH v3 3/7] unit-tests: add for_test

From: René Scharfe <l.s.r@web.de>
Date: Jul 31, 2024, 16:48 UTC
Message-ID: <85c4f785-de27-488c-aace-c93c2c2a84ac@web.de>
In-Reply-To: <ZqnJcawtpzvguxtq@tanuki>

Am 31.07.24 um 07:19 schrieb Patrick Steinhardt:

Show 9 quoted lines

> On Tue, Jul 30, 2024 at 04:00:07PM +0200, René Scharfe wrote:
>>
>> From the point of view of a "minimal C unit testing framework" surely
>> the implementation language with the best dependency story would be C.
>> Perhaps it could then also test itself.  On the other hand, writing a
>> string handling program in 2024 in C is probably not the smartest idea.
>
> I certainly wouldn't want to do it ;) I think Perl would be perfectly
> fine given that we already rely on it rather extensively in Git.

Right.

Show 39 quoted lines

> clar comes with
> some niceties like being able to run only some tests, all tests of a
> particular suite or to exclude certain tests:
>
>     # Run only submodule::add tests.
>     $ ./libgit2_tests -t -ssubmodule::add::
>     TAP version 13
>     # start of suite 1: submodule::add
>     ok 1 - submodule::add::url_absolute
>     ok 2 - submodule::add::url_relative
>     ok 3 - submodule::add::url_relative_to_origin
>     ok 4 - submodule::add::url_relative_to_workdir
>     ok 5 - submodule::add::path_exists_in_index
>     ok 6 - submodule::add::file_exists_in_index
>     ok 7 - submodule::add::submodule_clone
>     ok 8 - submodule::add::submodule_clone_into_nonempty_dir_succeeds
>     ok 9 - submodule::add::submodule_clone_twice_fails
>     1..9
>
>     # Run only a single test in the submodule::add suite.
>     $ ./libgit2_tests -t -ssubmodule::add::url_absolute
>     TAP version 13
>     # start of suite 1: submodule::add
>     ok 1 - submodule::add::url_absolute
>     1..1
>
>     # Run all submodule tests, but exclude submodule::add.
>     $ ./libgit2_tests -t -ssubmodule::  -xsubmodule::add::
>     TAP version 13
>     # start of suite 1: submodule::escape
>     ok 1 - submodule::escape::from_gitdir
>     ok 2 - submodule::escape::from_gitdir_windows
>     # start of suite 2: submodule::init
>     ok 3 - submodule::init::absolute_url
>     ...
>
> So I'm not sure whether it's all that important to be able to recognize
> things like #if and #ifdef given that you can pick tests to run on the
> command line.

My only reference are the regular (shell-based) tests, and those have to deal with prerequisites like PTHREADS, PCRE or LIBCURL. Such tests wouldn't even compile, and would thus be #if'd out. We'd have to move those into separate files instead and skip the whole file, it seems.

> The alternative would be of course to hook up LLVM and properly parse
> the C sources. But that feels like overthinking it ;)

It's as simple as calling clang (or gcc) with the option -E. We'd need to disable -Wmissing-prototypes as well, though, as there will be no declarations before we generate clar-decls.h. Not sure how to do that all in a portable way.

Anyway, another consequence is that adding, removing or renaming a single test function requires rebuilding all unit tests, because clar-decls.h changes and all of them include it. Not a problem right now. If we ever get to the same number of test files as in the regular test suite (ca. 1000) then this could become annoying.

Apropos, I wonder how a Clar-style t-ctype would look like, but didn't dare to even start thinking about it, yet.

René