git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/6 + 1] test-lib: make --verbose output valid TAP

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 9, 2021, 16:02 UTC
Message-ID
<20210309160219.13779-1-avarab@gmail.com>
In-Reply-To
<87r1kzj7xi.fsf@evledraar.gmail.com>
This makes the --verbose valid TAP.

This is a non-RFC version of an RFC patch I sent in https://lore.kernel.org/git/87tupwj5y5.fsf@evledraar.gmail.com/

There's a 7/7 patch that's an RFC at the end. It works for me, but since I haven't used the JUnit output it's likely subtly broken in some way (e.g. the new output doesn't include a timestamp attribute), but it shows how easy it is to add new output targets.

The eventual goal not included in this series is to have multiple output targets, and e.g. convert ci/print-test-failures.sh to use a TAP parser.

Machine-readable "TAP --verbose -x" output can bring us a lot of nice things down the line, I have some local WIP code that's a smarter version of ci/print-test-failures.sh that knows how to spew out only the output relevant to the failing test(s).

Future improvements based on that could be showing what specific line in a &&-chain failed, as well as e.g. running a debug version of a failing "grep" if we detect that the failing line line was 'grep <pat> <filename>'.

Ævar Arnfjörð Bjarmason (7):
  test-lib: remove test_external
  test-lib: add say_color_tap helper to emit TAP format
  test-lib: color "ok" TAP directives green under --verbose (or -x)
  test-lib: add tee with TAP support to test-tool
  test-lib: indent and format GIT_TEST_TEE_OUTPUT_FILE code
  test-lib: make --verbose output valid TAP
  test-lib: generate JUnit output via TAP
 Makefile                                      |   2 +-
 .../netrc/t-git-credential-netrc.sh           |   7 +-
 t/README                                      |  35 +--
 t/helper/test-date.c                          |  12 -
 t/helper/test-path-utils.c                    |  21 --
 t/helper/test-tee.c                           |  85 +++++++
 t/helper/test-tool.c                          |   2 +-
 t/helper/test-tool.h                          |   2 +-
 t/helper/test-xml-encode.c                    |  80 -------
 t/t0000-basic.sh                              |   8 +
 t/t0202-gettext-perl.sh                       |  10 +-
 t/t9700-perl-git.sh                           |  10 +-
 t/test-lib-functions.sh                       |  89 +------
 t/test-lib.sh                                 | 226 +++++-------------
 14 files changed, 177 insertions(+), 412 deletions(-)
 create mode 100644 t/helper/test-tee.c
 delete mode 100644 t/helper/test-xml-encode.c
-- 
2.31.0.rc1.210.g0f8085a843c
Previous: Ævar Arnfjörð BjarmasonNext: SZEDER Gábor
Message 13 of 47 in “Prove "Tests out of sequence" Error”
  1. Lars SchneiderOct 21, 2016
  2. Stefan BellerOct 21, 2016
  3. Jeff KingOct 21, 2016
  4. Jeff KingOct 21, 2016
  5. 0/3 fix travis TAP/--verbose conflictJeff King, Oct 21, 2016
  6. 1/3 test-lib: handle TEST_OUTPUT_DIRECTORY with spacesJeff King, Oct 21, 2016
  7. 2/3 test-lib: add --verbose-log optionJeff King, Oct 21, 2016
  8. Junio C HamanoOct 21, 2016
  9. Jeff KingOct 21, 2016
  10. test-lib: make --verbose work under proveÆvar Arnfjörð Bjarmason, Feb 28, 2021
  11. Jeff KingMar 1, 2021
  12. Ævar Arnfjörð BjarmasonMar 1, 2021
  13. 0/6 test-lib: make --verbose output valid TAPÆvar Arnfjörð Bjarmason, Mar 9, 2021
  14. SZEDER GáborMar 9, 2021
  15. Ævar Arnfjörð BjarmasonMar 9, 2021
  16. SZEDER GáborMar 9, 2021
  17. 3/7 test-lib: color "ok" TAP directives green under --verbose (or -x)Ævar Arnfjörð Bjarmason, Mar 9, 2021
  18. 4/7 test-lib: add tee with TAP support to test-toolÆvar Arnfjörð Bjarmason, Mar 9, 2021
  19. 2/7 test-lib: add say_color_tap helper to emit TAP formatÆvar Arnfjörð Bjarmason, Mar 9, 2021
  20. Junio C HamanoMar 10, 2021
  21. 1/7 test-lib: remove test_externalÆvar Arnfjörð Bjarmason, Mar 9, 2021
  22. Junio C HamanoMar 10, 2021
  23. Ævar Arnfjörð BjarmasonMar 10, 2021
  24. 5/7 test-lib: indent and format GIT_TEST_TEE_OUTPUT_FILE codeÆvar Arnfjörð Bjarmason, Mar 9, 2021
  25. 7/7 test-lib: generate JUnit output via TAPÆvar Arnfjörð Bjarmason, Mar 9, 2021
  26. Johannes SchindelinMar 19, 2021
  27. Ævar Arnfjörð BjarmasonMar 21, 2021
  28. Johannes SchindelinMar 22, 2021
  29. 6/7 test-lib: make --verbose output valid TAPÆvar Arnfjörð Bjarmason, Mar 9, 2021
  30. SZEDER GáborMar 9, 2021
  31. Ævar Arnfjörð BjarmasonMar 9, 2021
  32. SZEDER GáborMar 9, 2021
  33. Ævar Arnfjörð BjarmasonMar 10, 2021
  34. Ævar Arnfjörð BjarmasonMar 16, 2021
  35. SZEDER GáborMar 9, 2021
  36. Junio C HamanoMar 10, 2021
  37. Chris TorekMar 10, 2021
  38. 3/3 travis: use --verbose-log test optionJeff King, Oct 21, 2016
  39. Stefan BellerOct 21, 2016
  40. Lars SchneiderOct 24, 2016
  41. Jacob KellerOct 21, 2016
  42. Jeff KingOct 21, 2016
  43. Jacob KellerOct 21, 2016
  44. Jeff KingOct 21, 2016
  45. Jacob KellerOct 21, 2016
  46. 4/3 test-lib: bail out when "-v" used under "prove"Jeff King, Oct 22, 2016
  47. Jacob KellerOct 22, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.