git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 00/10] meson: parse TAP output generated by our tests

From
Patrick Steinhardt <ps@pks.im>
Date
May 30, 2025, 13:31 UTC
Message-ID
<20250530-pks-meson-tap-v3-0-676f5e41f2e4@pks.im>
In-Reply-To
<20250506-pks-meson-tap-v1-0-5aaab2942a4c@pks.im>
Hi,

this patch series starts to parse TAP output generated by our tests when executing them via Meson. This has the benefit that Meson starts to understand skipped tests and reports how many subtests have been executed:

    ```
    $ meson test t002*
    ninja: Entering directory `/home/pks/Development/git/build'
     1/10 t0024-crlf-archive                  OK              0.17s   2 subtests passed
     2/10 t0022-crlf-rename                   OK              0.18s   2 subtests passed
     3/10 t0029-core-unsetenvvars             SKIP            0.15s
     4/10 t0023-crlf-am                       OK              0.18s   2 subtests passed
     5/10 t0025-crlf-renormalize              OK              0.21s   3 subtests passed
     6/10 t0026-eol-config                    OK              0.25s   5 subtests passed
     7/10 t0020-crlf                          OK              0.81s   36 subtests passed
     8/10 t0028-working-tree-encoding         OK              0.85s   22 subtests passed
     9/10 t0021-conversion                    OK              3.45s   38 subtests passed
    10/10 t0027-auto-crlf                     OK             26.35s   2600 subtests passed
    
    Ok:                9
    Fail:              0
    Skipped:           1
    ```

This new feature is only enabled with Meson 1.8 and newer, which contains a bugfix that we have upstreamed [1] to make the TAP parser work in `meson test --interactive` mode.

Despite the changes to Meson itself, this patch series also contains a couple of fixes for our test suite that caused us to not generate proper TAP output.

Changes in v2:
  - Add a patch to fix an unexpectedly passing test on macOS.
  - A couple more fixes for broken TAP output.
  - Link to v1: https://lore.kernel.org/r/20250506-pks-meson-tap-v1-0-5aaab2942a4c@pks.im
Changes in v3:
  - Split up the patch that silences output into multiple patches and
    rework them a bit.
  - Remove redirect that was retained by accident from an earlier
    version.
  - Slight rewording of a commit message.
  - Treat unexpected passes as failure in prove(1) and when executing
    the test directly.
  - Link to v2: https://lore.kernel.org/r/20250527-pks-meson-tap-v2-0-ae360f77786e@pks.im
Thanks!
Patrick
[1]: https://github.com/mesonbuild/meson/pull/13980
---
Patrick Steinhardt (10):
      t: stop announcing prereqs
      t: silence output from `test_create_repo()`
      t9822: use prereq to check for ISO-8859-1 support
      t983*: use prereq to check for Python-specific git-b4(1) support
      t/test-lib: don't print shell traces to stdout
      t/test-lib: fix TAP format for BASH_XTRACEFD warning
      t7815: fix unexpectedly passing test on macOS
      test-lib: fail on unexpectedly passing tests
      meson: introduce kwargs variable for tests
      meson: parse TAP output generated by our tests
 contrib/credential/netrc/meson.build         |  2 +-
 contrib/subtree/meson.build                  |  2 +-
 meson.build                                  | 12 +++++++++
 t/meson.build                                |  6 ++---
 t/t0000-basic.sh                             | 39 +++++++++++++++-------------
 t/t0050-filesystem.sh                        | 30 +++++----------------
 t/t1007-hash-object.sh                       |  2 +-
 t/t3600-rm.sh                                |  5 ----
 t/t4000-diff-format.sh                       |  2 +-
 t/t4041-diff-submodule-option.sh             | 22 +++++++++-------
 t/t4060-diff-submodule-option-diff-format.sh |  9 ++++---
 t/t7401-submodule-summary.sh                 | 18 ++++++++-----
 t/t7815-grep-binary.sh                       |  2 +-
 t/t9500-gitweb-standalone-no-errors.sh       | 16 +++++-------
 t/t9822-git-p4-path-encoding.sh              | 13 +++++++---
 t/t9835-git-p4-metadata-encoding-python2.sh  | 24 +++++++++--------
 t/t9836-git-p4-metadata-encoding-python3.sh  | 24 +++++++++--------
 t/t9903-bash-prompt.sh                       |  4 ---
 t/test-lib.sh                                | 18 ++++++++++---
 19 files changed, 133 insertions(+), 117 deletions(-)
Range-diff versus v2:
 1:  b5ae3aba1ad <  -:  ----------- t: fix cases where output breaks TAP format
 -:  ----------- >  1:  15702b96125 t: stop announcing prereqs
 -:  ----------- >  2:  444a5e8a72f t: silence output from `test_create_repo()`
 -:  ----------- >  3:  dd24e16f93c t9822: use prereq to check for ISO-8859-1 support
 -:  ----------- >  4:  5ea96164181 t983*: use prereq to check for Python-specific git-b4(1) support
 2:  69d4b420eb9 =  5:  783cf673a22 t/test-lib: don't print shell traces to stdout
 3:  ec921bbb183 =  6:  726795c9a11 t/test-lib: fix TAP format for BASH_XTRACEFD warning
 4:  e463fb29a8a =  7:  df1be586474 t7815: fix unexpectedly passing test on macOS
 -:  ----------- >  8:  e0b06f9ffcb test-lib: fail on unexpectedly passing tests
 5:  58173827436 =  9:  0f34503b28b meson: introduce kwargs variable for tests
 6:  1242bbf74f3 = 10:  d28b9306b46 meson: parse TAP output generated by our tests

--- base-commit: 845c48a16a7f7b2c44d8cb137b16a4a1f0140229 change-id: 20250429-pks-meson-tap-1eed604a02a3

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 47 of 79 in “meson: parse TAP output generated by our tests”
  1. 0/4 meson: parse TAP output generated by our testsPatrick Steinhardt, May 6, 2025
  2. 1/4 t: fix cases where output breaks TAP formatPatrick Steinhardt, May 6, 2025
  3. Phillip WoodMay 6, 2025
  4. Patrick SteinhardtMay 7, 2025
  5. Phillip WoodMay 7, 2025
  6. Karthik NayakMay 14, 2025
  7. 2/4 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 6, 2025
  8. 3/4 meson: introduce kwargs variable for testsPatrick Steinhardt, May 6, 2025
  9. Karthik NayakMay 15, 2025
  10. 4/4 meson: parse TAP output generated by our testsPatrick Steinhardt, May 6, 2025
  11. Karthik NayakMay 15, 2025
  12. Patrick SteinhardtMay 15, 2025
  13. Karthik NayakMay 15, 2025
  14. Patrick SteinhardtMay 6, 2025
  15. Patrick SteinhardtMay 7, 2025
  16. Patrick SteinhardtMay 21, 2025
  17. Hridoy AhmedMay 21, 2025
  18. Junio C HamanoMay 21, 2025
  19. Junio C HamanoMay 21, 2025
  20. Patrick SteinhardtMay 23, 2025
  21. Patrick SteinhardtMay 23, 2025
  22. Junio C HamanoMay 23, 2025
  23. Ramsay JonesMay 23, 2025
  24. Ramsay JonesMay 23, 2025
  25. Junio C HamanoMay 23, 2025
  26. Patrick SteinhardtMay 26, 2025
  27. Phillip WoodMay 26, 2025
  28. Todd ZullingerMay 26, 2025
  29. Eli SchwartzMay 26, 2025
  30. Patrick SteinhardtMay 26, 2025
  31. Junio C HamanoMay 27, 2025
  32. Patrick SteinhardtMay 28, 2025
  33. Junio C HamanoMay 27, 2025
  34. 0/6 meson: parse TAP output generated by our testsPatrick Steinhardt, May 27, 2025
  35. 1/6 t: fix cases where output breaks TAP formatPatrick Steinhardt, May 27, 2025
  36. Eric SunshineMay 27, 2025
  37. Patrick SteinhardtMay 28, 2025
  38. Eric SunshineMay 28, 2025
  39. Patrick SteinhardtMay 30, 2025
  40. 2/6 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 27, 2025
  41. Junio C HamanoMay 27, 2025
  42. Patrick SteinhardtMay 28, 2025
  43. 3/6 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, May 27, 2025
  44. 4/6 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, May 27, 2025
  45. 6/6 meson: parse TAP output generated by our testsPatrick Steinhardt, May 27, 2025
  46. 5/6 meson: introduce kwargs variable for testsPatrick Steinhardt, May 27, 2025
  47. 00/10 meson: parse TAP output generated by our testsPatrick Steinhardt, May 30, 2025
  48. 01/10 t: stop announcing prereqsPatrick Steinhardt, May 30, 2025
  49. Karthik NayakMay 31, 2025
  50. 03/10 t9822: use prereq to check for ISO-8859-1 supportPatrick Steinhardt, May 30, 2025
  51. 02/10 t: silence output from `test_create_repo()`Patrick Steinhardt, May 30, 2025
  52. Eric SunshineMay 30, 2025
  53. 04/10 t983*: use prereq to check for Python-specific git-b4(1) supportPatrick Steinhardt, May 30, 2025
  54. Todd ZullingerMay 30, 2025
  55. Patrick SteinhardtMay 30, 2025
  56. 05/10 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 30, 2025
  57. Karthik NayakMay 31, 2025
  58. 06/10 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, May 30, 2025
  59. Karthik NayakMay 31, 2025
  60. 07/10 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, May 30, 2025
  61. Karthik NayakMay 31, 2025
  62. Kristoffer HaugsbakkJun 1, 2025
  63. Patrick SteinhardtJun 2, 2025
  64. 08/10 test-lib: fail on unexpectedly passing testsPatrick Steinhardt, May 30, 2025
  65. 09/10 meson: introduce kwargs variable for testsPatrick Steinhardt, May 30, 2025
  66. 10/10 meson: parse TAP output generated by our testsPatrick Steinhardt, May 30, 2025
  67. Karthik NayakMay 31, 2025
  68. 00/10 meson: parse TAP output generated by our testsPatrick Steinhardt, Jun 2, 2025
  69. 01/10 t: stop announcing prereqsPatrick Steinhardt, Jun 2, 2025
  70. 02/10 t: silence output from `test_create_repo()`Patrick Steinhardt, Jun 2, 2025
  71. 03/10 t9822: use prereq to check for ISO-8859-1 supportPatrick Steinhardt, Jun 2, 2025
  72. 04/10 t983*: use prereq to check for Python-specific git-p4(1) supportPatrick Steinhardt, Jun 2, 2025
  73. 06/10 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, Jun 2, 2025
  74. 05/10 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, Jun 2, 2025
  75. 08/10 test-lib: fail on unexpectedly passing testsPatrick Steinhardt, Jun 2, 2025
  76. 07/10 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, Jun 2, 2025
  77. 09/10 meson: introduce kwargs variable for testsPatrick Steinhardt, Jun 2, 2025
  78. 10/10 meson: parse TAP output generated by our testsPatrick Steinhardt, Jun 2, 2025
  79. Karthik NayakJun 2, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.