git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 0/7] test-tool: add unit test suite runner

From
Josh Steadmon <steadmon@google.com>
Date
Apr 30, 2024, 19:55 UTC
Message-ID
<cover.1714506612.git.steadmon@google.com>
In-Reply-To
<cover.1705443632.git.steadmon@google.com>

For various reasons (see discussion at [1]) we would like an alternative to `prove` for running test suites (including the unit tests) on Windows.

[1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/
This series extends the existing `test-tool run-command testsuite` to
support running unit tests. In addition, it includes some small
cleanups:
* move t-basic out of the unit-tests directory
* don't hardcode the shell for running tests in `test-tool ... testsuite`
* don't hardcode a test name filter in `test-tool ... testsuite`
* add a test wrapper script to allow unit tests and the shell test suite
  to run in a single `prove` process
Changes in V5:
* Style fix in t/run-test.sh
Changes in V4:
* Set TEST_SHELL_PATH when running the `prove` test target, and error
  out of run-test.sh if TEST_SHELL_PATH is not set.
Changes in V3:
* Added new patch (#7) to use the new test-tool runner for unit tests in
  Windows CI.
* Restored the explicit sort call in t/Makefile, for backwards
  compatibility with older GNU Make versions.
* Rebased the series on the latest jk/unit-tests-buildfix branch.
* Fixed header include order in patch #1.
* Removed a paragraph in patch #1's commit message that is obsolete now
  that we're building the list of test files from the sources rather
  than by globbing.
* Added a note in patch #2 that setting a NULL suite.shell_path will be
  used in a later commit.
* Clarified up some sloppy wording in commit messages and comments in
  t/helper/test-run-command.c.
Changes in V2:
* Rebased the series on the latest jk/unit-tests-buildfix branch.
* Patch 1: move t-basic to a test-tool subcommand rather than a new
  executable under t/t0080/
* New patch 2: get the shell path from TEST_SHELL_PATH in
  `test-tool run-command testsuite`
* New patch 3: remove the hardcoded filename filter in
  `test-tool run-command testsuite`
* Patch 4 (previously 2): simplified now that we no longer need to add
  any command-line flags to support unit tests
* Patch 5 (previously 3): avoid trying to run cmake *.pdb files by using
  the unit test list built in the makefile in jk/unit-tests-buildfix.
Jeff King (1):
  t/Makefile: run unit tests alongside shell tests
Josh Steadmon (6):
  t0080: turn t-basic unit test into a helper
  test-tool run-command testsuite: get shell from env
  test-tool run-command testsuite: remove hardcoded filter
  test-tool run-command testsuite: support unit tests
  unit tests: add rule for running with test-tool
  ci: use test-tool as unit test runner on Windows
 Makefile                                      |  6 ++--
 ci/run-build-and-tests.sh                     |  2 --
 ci/run-test-slice.sh                          |  2 +-
 t/Makefile                                    | 14 ++++++++--
 .../t-basic.c => helper/test-example-tap.c}   |  5 ++--
 t/helper/test-run-command.c                   | 28 +++++++++++++++----
 t/helper/test-tool.c                          |  1 +
 t/helper/test-tool.h                          |  1 +
 t/run-test.sh                                 | 18 ++++++++++++
 t/t0080-unit-test-output.sh                   | 24 ++++++++--------
 10 files changed, 72 insertions(+), 29 deletions(-)
 rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%)
 create mode 100755 t/run-test.sh
Range-diff against v4:
1:  6777451100 = 1:  6777451100 t0080: turn t-basic unit test into a helper
2:  24f47f8fc7 = 2:  24f47f8fc7 test-tool run-command testsuite: get shell from env
3:  4a16a3ec24 = 3:  4a16a3ec24 test-tool run-command testsuite: remove hardcoded filter
4:  abc9a7afe8 = 4:  abc9a7afe8 test-tool run-command testsuite: support unit tests
5:  a8bbff2c6b = 5:  a8bbff2c6b unit tests: add rule for running with test-tool
6:  0e32de1afe ! 6:  c6606446c4 t/Makefile: run unit tests alongside shell tests
    @@ t/run-test.sh (new)
     +
     +case "$1" in
     +*.sh)
    -+  if test -z "${TEST_SHELL_PATH+set}" ; then
    -+          echo "ERROR: TEST_SHELL_PATH is not set" >&2
    ++  if test -z "${TEST_SHELL_PATH+set}"
    ++  then
    ++          echo >&2 "ERROR: TEST_SHELL_PATH is empty or not set"
     +          exit 1
     +  fi
     +  exec ${TEST_SHELL_PATH} "$@"
7:  c562515293 = 7:  4a92131ab9 ci: use test-tool as unit test runner on Windows
base-commit: 4904a4d08cc085716df12ce713ae7ee3d5ecb75a
-- 
2.45.0.rc0.197.gbae5840b3b-goog
Previous: Josh SteadmonNext: Josh Steadmon
Message 79 of 99 in “test-tool: add unit test suite runner”
  1. 0/4 test-tool: add unit test suite runnerJosh Steadmon, Jan 16, 2024
  2. 1/4 t0080: turn t-basic unit test into a helperJosh Steadmon, Jan 16, 2024
  3. Junio C HamanoJan 16, 2024
  4. Jeff KingJan 23, 2024
  5. Josh SteadmonFeb 1, 2024
  6. 2/4 test-tool run-command testsuite: support unit testsJosh Steadmon, Jan 16, 2024
  7. Junio C HamanoJan 16, 2024
  8. Junio C HamanoJan 16, 2024
  9. Jeff KingJan 23, 2024
  10. Josh SteadmonFeb 1, 2024
  11. Junio C HamanoFeb 1, 2024
  12. Josh SteadmonFeb 1, 2024
  13. Josh SteadmonFeb 1, 2024
  14. 3/4 unit tests: add rule for running with test-toolJosh Steadmon, Jan 16, 2024
  15. 4/4 t/Makefile: run unit tests alongside shell testsJosh Steadmon, Jan 16, 2024
  16. Junio C HamanoJan 16, 2024
  17. 0/6 test-tool: add unit test suite runnerJosh Steadmon, Feb 3, 2024
  18. 1/6 t0080: turn t-basic unit test into a helperJosh Steadmon, Feb 3, 2024
  19. Junio C HamanoFeb 7, 2024
  20. Josh SteadmonFeb 12, 2024
  21. Jeff KingFeb 7, 2024
  22. Junio C HamanoFeb 8, 2024
  23. Josh SteadmonFeb 12, 2024
  24. Junio C HamanoFeb 12, 2024
  25. Jeff KingFeb 13, 2024
  26. rsbecker@nexbridge.comFeb 13, 2024
  27. Josh SteadmonFeb 22, 2024
  28. rsbecker@nexbridge.comFeb 23, 2024
  29. Junio C HamanoFeb 13, 2024
  30. Josh SteadmonFeb 22, 2024
  31. 2/6 test-tool run-command testsuite: get shell from envJosh Steadmon, Feb 3, 2024
  32. Junio C HamanoFeb 7, 2024
  33. Josh SteadmonFeb 12, 2024
  34. 3/6 test-tool run-command testsuite: remove hardcoded filterJosh Steadmon, Feb 3, 2024
  35. Junio C HamanoFeb 7, 2024
  36. Josh SteadmonFeb 12, 2024
  37. Junio C HamanoFeb 12, 2024
  38. 4/6 test-tool run-command testsuite: support unit testsJosh Steadmon, Feb 3, 2024
  39. phillip.wood123@gmail.comFeb 5, 2024
  40. Josh SteadmonFeb 12, 2024
  41. Junio C HamanoFeb 7, 2024
  42. Josh SteadmonFeb 23, 2024
  43. 5/6 unit tests: add rule for running with test-toolJosh Steadmon, Feb 3, 2024
  44. Junio C HamanoFeb 7, 2024
  45. 6/6 t/Makefile: run unit tests alongside shell testsJosh Steadmon, Feb 3, 2024
  46. Junio C HamanoFeb 7, 2024
  47. Jeff KingFeb 7, 2024
  48. Junio C HamanoFeb 7, 2024
  49. Junio C HamanoFeb 3, 2024
  50. Josh SteadmonFeb 12, 2024
  51. Junio C HamanoFeb 7, 2024
  52. 0/7 test-tool: add unit test suite runnerJosh Steadmon, Feb 23, 2024
  53. 1/7 t0080: turn t-basic unit test into a helperJosh Steadmon, Feb 23, 2024
  54. 2/7 test-tool run-command testsuite: get shell from envJosh Steadmon, Feb 23, 2024
  55. 3/7 test-tool run-command testsuite: remove hardcoded filterJosh Steadmon, Feb 23, 2024
  56. 4/7 test-tool run-command testsuite: support unit testsJosh Steadmon, Feb 23, 2024
  57. 5/7 unit tests: add rule for running with test-toolJosh Steadmon, Feb 23, 2024
  58. 6/7 t/Makefile: run unit tests alongside shell testsJosh Steadmon, Feb 23, 2024
  59. Jeff KingMar 27, 2024
  60. Josh SteadmonApr 11, 2024
  61. Jeff KingApr 12, 2024
  62. Josh SteadmonApr 24, 2024
  63. 7/7 ci: use test-tool as unit test runner on WindowsJosh Steadmon, Feb 23, 2024
  64. Junio C HamanoMar 26, 2024
  65. Jeff KingMar 27, 2024
  66. 0/7 test-tool: add unit test suite runnerJosh Steadmon, Apr 24, 2024
  67. 1/7 t0080: turn t-basic unit test into a helperJosh Steadmon, Apr 24, 2024
  68. 2/7 test-tool run-command testsuite: get shell from envJosh Steadmon, Apr 24, 2024
  69. 3/7 test-tool run-command testsuite: remove hardcoded filterJosh Steadmon, Apr 24, 2024
  70. 4/7 test-tool run-command testsuite: support unit testsJosh Steadmon, Apr 24, 2024
  71. 5/7 unit tests: add rule for running with test-toolJosh Steadmon, Apr 24, 2024
  72. 6/7 t/Makefile: run unit tests alongside shell testsJosh Steadmon, Apr 24, 2024
  73. Junio C HamanoApr 24, 2024
  74. Josh SteadmonApr 30, 2024
  75. Jeff KingMay 3, 2024
  76. Junio C HamanoMay 3, 2024
  77. Josh SteadmonMay 6, 2024
  78. 7/7 ci: use test-tool as unit test runner on WindowsJosh Steadmon, Apr 24, 2024
  79. 0/7 test-tool: add unit test suite runnerJosh Steadmon, Apr 30, 2024
  80. 1/7 t0080: turn t-basic unit test into a helperJosh Steadmon, Apr 30, 2024
  81. 2/7 test-tool run-command testsuite: get shell from envJosh Steadmon, Apr 30, 2024
  82. 3/7 test-tool run-command testsuite: remove hardcoded filterJosh Steadmon, Apr 30, 2024
  83. 4/7 test-tool run-command testsuite: support unit testsJosh Steadmon, Apr 30, 2024
  84. 5/7 unit tests: add rule for running with test-toolJosh Steadmon, Apr 30, 2024
  85. 6/7 t/Makefile: run unit tests alongside shell testsJosh Steadmon, Apr 30, 2024
  86. Junio C HamanoApr 30, 2024
  87. Josh SteadmonMay 6, 2024
  88. 7/7 ci: use test-tool as unit test runner on WindowsJosh Steadmon, Apr 30, 2024
  89. Junio C HamanoApr 30, 2024
  90. Josh SteadmonMay 6, 2024
  91. 0/7 test-tool: add unit test suite runnerJosh Steadmon, May 6, 2024
  92. 1/7 t0080: turn t-basic unit test into a helperJosh Steadmon, May 6, 2024
  93. 2/7 test-tool run-command testsuite: get shell from envJosh Steadmon, May 6, 2024
  94. 3/7 test-tool run-command testsuite: remove hardcoded filterJosh Steadmon, May 6, 2024
  95. 4/7 test-tool run-command testsuite: support unit testsJosh Steadmon, May 6, 2024
  96. 5/7 unit tests: add rule for running with test-toolJosh Steadmon, May 6, 2024
  97. 6/7 t/Makefile: run unit tests alongside shell testsJosh Steadmon, May 6, 2024
  98. 7/7 ci: use test-tool as unit test runner on WindowsJosh Steadmon, May 6, 2024
  99. Junio C HamanoMay 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.