git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/3] t/Makefile: get UNIT_TESTS list from C sources

From
PWPhillip Wood <phillip.wood123@gmail.com>
Date
Feb 1, 2024, 10:50 UTC
Message-ID
<08303f49-63dd-4141-be34-0d6b5fa28a0b@gmail.com>
In-Reply-To
<20240130054037.GC166699@coredump.intra.peff.net>
Hi Peff
On 30/01/2024 05:40, Jeff King wrote:
Show 24 quoted lines
> We decide on the set of unit tests to run by asking make to expand the
> wildcard "t/unit-tests/bin/*". One unfortunate outcome of this is that
> we'll run anything in that directory, even if it is leftover cruft from
> a previous build. This isn't _quite_ as bad as it sounds, since in
> theory the unit tests executables are self-contained (so if they passed
> before, they'll pass even though they now have nothing to do with the
> checked out version of Git). But at the very least it's wasteful, and if
> they _do_ fail it can be quite confusing to understand why they are
> being run at all.
> 
> This wildcarding presumably came from our handling of the regular
> shell-script tests, which use $(wildcard t[0-9][0-9][0-9][0-9]-*.sh).
> But the difference there is that those are actual tracked files. So if
> you checkout a different commit, they'll go away. Whereas the contents
> of unit-tests/bin are ignored (so not only do they stick around, but you
> are not even warned of the stale files via "git status").
> 
> This patch fixes the situation by looking for the actual unit-test
> source files and then massaging those names into the final executable
> names. This has two additional benefits:
> 
>    1. It will notice if we failed to build one or more unit-tests for
>       some reason (wheras the current code just runs whatever made it to
>       the bin/ directory).

The downside to this is that if there are any cruft C files lying about t/unit-tests we'll fail to run the unit tests. In the past we've avoided using wildcard rules on C sources to avoid problems like this[1]. This change may well be the lesser of two evils but a test run that fails due to a cruft C file cannot be fixed by "make clean && make" whereas that will fix the problem of a stale test executable.

Show 10 quoted lines
>    2. The wildcard should avoid other build cruft, like the pdb files we
>       worked around in 0df903d402 (unit-tests: do not mistake `.pdb`
>       files for being executable, 2023-09-25).
> 
> Our new wildcard does make an assumption that unit tests are built from
> C sources. It would be a bit cleaner if we consulted UNIT_TEST_PROGRAMS
> from the top-level Makefile. But doing so is tricky unless we reorganize
> that Makefile to split the source file lists into include-able subfiles.
> That might be worth doing in general, but in the meantime, the
> assumptions made by the wildcard here seems reasonable.
Using UNIT_TEST_PROGRAMS would definitely be a nicer approach long term.
Best Wishes
Phillip
[1] https://lore.kernel.org/git/xmqqtugl102l.fsf@gitster.g/
Previous: Junio C HamanoNext: Junio C Hamano
Message 20 of 24 in “some unit-test Makefile polishing”
  1. 0/2 some unit-test Makefile polishingJeff King, Jan 29, 2024
  2. 1/2 Makefile: use order-only prereq for UNIT_TEST_BINJeff King, Jan 29, 2024
  3. SZEDER GáborJan 29, 2024
  4. Junio C HamanoJan 29, 2024
  5. Jeff KingJan 30, 2024
  6. 2/2 t/Makefile: get UNIT_TESTS list from C sourcesJeff King, Jan 29, 2024
  7. Patrick SteinhardtJan 29, 2024
  8. Jeff KingJan 29, 2024
  9. Adam DinwoodieJan 29, 2024
  10. Junio C HamanoJan 30, 2024
  11. Jeff KingJan 30, 2024
  12. Adam DinwoodieJan 31, 2024
  13. Jeff KingJan 30, 2024
  14. Junio C HamanoJan 29, 2024
  15. 0/3 some unit-test Makefile polishingJeff King, Jan 30, 2024
  16. 1/3 Makefile: use mkdir_p_parent_template for UNIT_TEST_BINJeff King, Jan 30, 2024
  17. 2/3 Makefile: remove UNIT_TEST_BIN directory with "make clean"Jeff King, Jan 30, 2024
  18. 3/3 t/Makefile: get UNIT_TESTS list from C sourcesJeff King, Jan 30, 2024
  19. Junio C HamanoJan 31, 2024
  20. Phillip WoodFeb 1, 2024
  21. Junio C HamanoFeb 2, 2024
  22. Johannes SchindelinFeb 2, 2024
  23. Junio C HamanoFeb 3, 2024
  24. Jeff KingFeb 4, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.