git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 1/5] cmake: make it easier to diagnose regressions in CTest runs

From
Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>
Date
Oct 18, 2022, 10:59 UTC
Message-ID
<356b2e9a1007bcd1382f26f333926ff0d5b9abe2.1666090745.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1320.v3.git.1666090745.gitgitgadget@gmail.com>
From: Johannes Schindelin <johannes.schindelin@gmx.de>

When a test script fails in Git's test suite, the usual course of action is to re-run it using options to increase the verbosity of the output, e.g. `-v` and `-x`.

Like in Git's CI runs, when running the tests in Visual Studio via the CTest route, it is cumbersome or at least requires a very unintuitive approach to pass options to the test scripts: the CMakeLists.txt file would have to be modified, passing the desired options to _all_ test scripts, and then the CMake Cache would have to be reconfigured before running the test in question individually. Unintuitive at best, and opposite to the niceties IDE users expect.

So let's just pass those options by default: This will not clutter any output window but the log that is written to a log file will have information necessary to figure out test failures.

While at it, also imitate what the Windows jobs in Git's CI runs do to accelerate running the test scripts: pass the `--no-bin-wrappers` and `--no-chain-lint` options.

This makes the test runs noticeably faster because the `bin-wrappers/` scripts as well as the `chain-lint` code make heavy use of POSIX shell scripting, which is really, really slow on Windows due to the need to emulate POSIX behavior via the MSYS2 runtime. In a test by Eric Sunshine, it added two minutes (!) just to perform the chain-lint task.

The idea of adding a CMake config option (á la `GIT_TEST_OPTS`) was considered during the development of this patch, but then dropped: such a setting is global, across _all_ tests, where e.g. `--run=...` would not make sense. Users wishing to override these new defaults are better advised running the test script manually, in a Git Bash, with full control over the command line.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 contrib/buildsystems/CMakeLists.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..6ac20bc5054 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -1088,7 +1088,7 @@ file(GLOB test_scipts "${CMAKE_SOURCE_DIR}/t/t[0-9]*.sh")
 #test
 foreach(tsh ${test_scipts})
 	add_test(NAME ${tsh}
-		COMMAND ${SH_EXE} ${tsh}
+		COMMAND ${SH_EXE} ${tsh} --no-bin-wrappers --no-chain-lint -vx
 		WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/t)
 endforeach()
 
-- 
gitgitgadget
Previous: Johannes Schindelin via GitGitGadgetNext: Ævar Arnfjörð Bjarmason
Message 49 of 58 in “Some fixes and an improvement for using CTest on Windows”
  1. 0/5 Some fixes and an improvement for using CTest on WindowsJohannes Schindelin via GitGitGadget, Aug 10, 2022
  2. 1/5 cmake: align CTest definition with Git's CI runsJohannes Schindelin via GitGitGadget, Aug 10, 2022
  3. Junio C HamanoAug 10, 2022
  4. Johannes SchindelinAug 16, 2022
  5. Junio C HamanoAug 16, 2022
  6. Johannes SchindelinAug 19, 2022
  7. Ævar Arnfjörð BjarmasonAug 11, 2022
  8. 2/5 cmake: copy the merge tools for testingJohannes Schindelin via GitGitGadget, Aug 10, 2022
  9. 3/5 tests: explicitly skip `chmod` calls on WindowsJohannes Schindelin via GitGitGadget, Aug 10, 2022
  10. Ævar Arnfjörð BjarmasonAug 11, 2022
  11. Johannes SchindelinAug 22, 2022
  12. Johannes SchindelinAug 23, 2022
  13. 4/5 add -p: avoid ambiguous signed/unsigned comparisonJohannes Schindelin via GitGitGadget, Aug 10, 2022
  14. Junio C HamanoAug 10, 2022
  15. Johannes SchindelinAug 16, 2022
  16. Junio C HamanoAug 16, 2022
  17. Johannes SchindelinAug 19, 2022
  18. Phillip WoodAug 11, 2022
  19. Johannes SchindelinAug 16, 2022
  20. Phillip WoodAug 16, 2022
  21. Johannes SchindelinAug 19, 2022
  22. 5/5 cmake: avoid editing t/test-lib.shJohannes Schindelin via GitGitGadget, Aug 10, 2022
  23. Ævar Arnfjörð BjarmasonAug 11, 2022
  24. Johannes SchindelinOct 18, 2022
  25. Phillip WoodAug 11, 2022
  26. Johannes SchindelinAug 16, 2022
  27. Phillip WoodAug 16, 2022
  28. 0/5 Some fixes and an improvement for using CTest on WindowsJohannes Schindelin via GitGitGadget, Aug 23, 2022
  29. 4/5 cmake: avoid editing t/test-lib.shJohannes Schindelin via GitGitGadget, Aug 23, 2022
  30. Ævar Arnfjörð BjarmasonSep 8, 2022
  31. Johannes SchindelinOct 18, 2022
  32. Ævar Arnfjörð BjarmasonOct 18, 2022
  33. Victoria DyeSep 8, 2022
  34. Victoria DyeSep 8, 2022
  35. Junio C HamanoSep 8, 2022
  36. Johannes SchindelinOct 18, 2022
  37. 5/5 cmake: increase time-out for a long-running testJohannes Schindelin via GitGitGadget, Aug 23, 2022
  38. Ævar Arnfjörð BjarmasonSep 8, 2022
  39. Victoria DyeSep 8, 2022
  40. 2/5 cmake: copy the merge tools for testingJohannes Schindelin via GitGitGadget, Aug 23, 2022
  41. 1/5 cmake: make it easier to diagnose regressions in CTest runsJohannes Schindelin via GitGitGadget, Aug 23, 2022
  42. Victoria DyeSep 7, 2022
  43. Johannes SchindelinOct 18, 2022
  44. Ævar Arnfjörð BjarmasonSep 8, 2022
  45. Eric SunshineSep 28, 2022
  46. 3/5 add -p: avoid ambiguous signed/unsigned comparisonJohannes Schindelin via GitGitGadget, Aug 23, 2022
  47. Victoria DyeSep 8, 2022
  48. 0/5 Some fixes and an improvement for using CTest on WindowsJohannes Schindelin via GitGitGadget, Oct 18, 2022
  49. 1/5 cmake: make it easier to diagnose regressions in CTest runsJohannes Schindelin via GitGitGadget, Oct 18, 2022
  50. Ævar Arnfjörð BjarmasonOct 18, 2022
  51. 2/5 cmake: copy the merge tools for testingJohannes Schindelin via GitGitGadget, Oct 18, 2022
  52. Ævar Arnfjörð BjarmasonOct 18, 2022
  53. 3/5 add -p: avoid ambiguous signed/unsigned comparisonJohannes Schindelin via GitGitGadget, Oct 18, 2022
  54. Ævar Arnfjörð BjarmasonOct 18, 2022
  55. 4/5 cmake: avoid editing t/test-lib.shJohannes Schindelin via GitGitGadget, Oct 18, 2022
  56. Ævar Arnfjörð BjarmasonOct 18, 2022
  57. Johannes SchindelinOct 18, 2022
  58. 5/5 cmake: increase time-out for a long-running testJohannes Schindelin via GitGitGadget, Oct 18, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.