git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/6] t: fix cases where output breaks TAP format

From
Patrick Steinhardt <ps@pks.im>
Date
May 28, 2025, 15:55 UTC
Message-ID
<aDcx4dXe12tRUyYd@pks.im>
In-Reply-To
<CAPig+cSYhY+LQ5pD+a1O16Rxwo_js45WqfcW8wtC2daYmNyMCQ@mail.gmail.com>
On Tue, May 27, 2025 at 03:47:16PM -0400, Eric Sunshine wrote:
Show 17 quoted lines
> On Tue, May 27, 2025 at 10:03 AM Patrick Steinhardt <ps@pks.im> wrote:
> > The TAP format does not allow arbitrary output outside of a specific
> > test case. If a test suite wants to print any such diagnostic output,
> > then this output has to be prefixed with "#" to mark it accordingly.
> > A bunch of our tests generate output outside of `test_expect_*`
> > testcases anyway without such a mark, which breaks strict TAP parsers.
> >
> > Upon further inspection, all of the output generated by such tests is
> > rather uninteresting. Refactor them so that we don't break the TAP
> > format.
> 
> Nit: Can we avoid the word "refactor" for changes such as those made
> by this patch which clearly are not refactoring[*].
> 
> [*]: From Wikipedia: "... code refactoring is the process of
> restructuring existing source code—changing the factoring—without
> changing its external behavior."
Fair. We can say "adapt" instead.
Show 16 quoted lines
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> > diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh
> > @@ -30,7 +30,7 @@ setup_repo() {
> >  test_repo=test
> >  push_repo() {
> > -       test_create_repo $test_repo
> > +       test_create_repo $test_repo >/dev/null
> >         cd $test_repo
> >         setup_repo
> 
> Yuck, but certainly the simplest "fix" in this particular case
> considering that, ultimately, this entire script ought to be reworked
> since it cd's around outside of tests with abandon. It would be nice
> to see this script get overhauled eventually but such an undertaking
> doesn't need to be part of this patch series.
Yeah, a bunch of test scripts fall into this category indeed.
Show 27 quoted lines
> > diff --git a/t/t4041-diff-submodule-option.sh b/t/t4041-diff-submodule-option.sh
> > @@ -48,7 +48,7 @@ commit_file () {
> > -test_create_repo sm1 &&
> > +test_create_repo sm1 >/dev/null &&
> >  add_file . foo >/dev/null
> >
> >  head1=$(add_file sm1 foo1 foo2)
> 
> Unlike the case with t1007, in which the entire script needs an
> overhaul, it is much easier to fix the problems in this script without
> papering over them via ">/dev/null". In particular, it would be
> preferable to resolve the issue by wrapping test_expect_success around
> the code which currently resides outside of any test. So, for example,
> the above could become:
> 
>     test_expect_success 'setup submodule 1' '
>         test_create_repo sm1 &&
>         add_file . foo &&
>         head1=$(add_file sm1 foo1 foo2) &&
>         fullhead1=$(cd sm1; git rev-parse --verify HEAD)
>     '
> 
> Note that I also dropped the ">/dev/null" redirect from the add_file()
> invocation.
> 
> The same comment applies to similar changes made by this patch to
> other scripts, such as t4060, t7401.

Yes, it isn't particularly hard. But it does result in a bunch of shuffling that makes the patch way harder to read.

Show 14 quoted lines
> > diff --git a/t/t9822-git-p4-path-encoding.sh b/t/t9822-git-p4-path-encoding.sh
> > @@ -7,12 +7,17 @@ test_description='Clone repositories with non ASCII paths'
> > -ISO8859="$(printf "$ISO8859_ESCAPED")" &&
> > -echo content123 >"$ISO8859" &&
> > -rm "$ISO8859" || {
> > +test_lazy_prereq FS_ACCEPTS_ISO_8859_1 '
> > +       ISO8859="$(printf "$ISO8859_ESCAPED")" &&
> > +       echo content123 >"$ISO8859" 2>/dev/null &&
> > +       rm "$ISO8859"
> > +'
> 
> Was the problem here that the `echo content123 > "$..."` was
> potentially spitting out an error message to stderr, thus you had to
> redirect it to /dev/null to silence it?

Ah, this redirect is not required anymore. I had it in a previous version due to the exact problem that you mentioned, that echo spit out an error.

> If so, did the file get created in the error case? What I'm wondering
> is whether you also should use `rm -f` when removing the file.

The idea here is that some systems fail to create the file in the first place, which will cause the echo to fail. In that case, the file has not been created either, so there is no need to remove it.

Patrick
Previous: Eric SunshineNext: Eric Sunshine
Message 37 of 79 in “meson: parse TAP output generated by our tests”
  1. 0/4 meson: parse TAP output generated by our testsPatrick Steinhardt, May 6, 2025
  2. 1/4 t: fix cases where output breaks TAP formatPatrick Steinhardt, May 6, 2025
  3. Phillip WoodMay 6, 2025
  4. Patrick SteinhardtMay 7, 2025
  5. Phillip WoodMay 7, 2025
  6. Karthik NayakMay 14, 2025
  7. 2/4 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 6, 2025
  8. 3/4 meson: introduce kwargs variable for testsPatrick Steinhardt, May 6, 2025
  9. Karthik NayakMay 15, 2025
  10. 4/4 meson: parse TAP output generated by our testsPatrick Steinhardt, May 6, 2025
  11. Karthik NayakMay 15, 2025
  12. Patrick SteinhardtMay 15, 2025
  13. Karthik NayakMay 15, 2025
  14. Patrick SteinhardtMay 6, 2025
  15. Patrick SteinhardtMay 7, 2025
  16. Patrick SteinhardtMay 21, 2025
  17. Hridoy AhmedMay 21, 2025
  18. Junio C HamanoMay 21, 2025
  19. Junio C HamanoMay 21, 2025
  20. Patrick SteinhardtMay 23, 2025
  21. Patrick SteinhardtMay 23, 2025
  22. Junio C HamanoMay 23, 2025
  23. Ramsay JonesMay 23, 2025
  24. Ramsay JonesMay 23, 2025
  25. Junio C HamanoMay 23, 2025
  26. Patrick SteinhardtMay 26, 2025
  27. Phillip WoodMay 26, 2025
  28. Todd ZullingerMay 26, 2025
  29. Eli SchwartzMay 26, 2025
  30. Patrick SteinhardtMay 26, 2025
  31. Junio C HamanoMay 27, 2025
  32. Patrick SteinhardtMay 28, 2025
  33. Junio C HamanoMay 27, 2025
  34. 0/6 meson: parse TAP output generated by our testsPatrick Steinhardt, May 27, 2025
  35. 1/6 t: fix cases where output breaks TAP formatPatrick Steinhardt, May 27, 2025
  36. Eric SunshineMay 27, 2025
  37. Patrick SteinhardtMay 28, 2025
  38. Eric SunshineMay 28, 2025
  39. Patrick SteinhardtMay 30, 2025
  40. 2/6 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 27, 2025
  41. Junio C HamanoMay 27, 2025
  42. Patrick SteinhardtMay 28, 2025
  43. 3/6 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, May 27, 2025
  44. 4/6 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, May 27, 2025
  45. 6/6 meson: parse TAP output generated by our testsPatrick Steinhardt, May 27, 2025
  46. 5/6 meson: introduce kwargs variable for testsPatrick Steinhardt, May 27, 2025
  47. 00/10 meson: parse TAP output generated by our testsPatrick Steinhardt, May 30, 2025
  48. 01/10 t: stop announcing prereqsPatrick Steinhardt, May 30, 2025
  49. Karthik NayakMay 31, 2025
  50. 03/10 t9822: use prereq to check for ISO-8859-1 supportPatrick Steinhardt, May 30, 2025
  51. 02/10 t: silence output from `test_create_repo()`Patrick Steinhardt, May 30, 2025
  52. Eric SunshineMay 30, 2025
  53. 04/10 t983*: use prereq to check for Python-specific git-b4(1) supportPatrick Steinhardt, May 30, 2025
  54. Todd ZullingerMay 30, 2025
  55. Patrick SteinhardtMay 30, 2025
  56. 05/10 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, May 30, 2025
  57. Karthik NayakMay 31, 2025
  58. 06/10 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, May 30, 2025
  59. Karthik NayakMay 31, 2025
  60. 07/10 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, May 30, 2025
  61. Karthik NayakMay 31, 2025
  62. Kristoffer HaugsbakkJun 1, 2025
  63. Patrick SteinhardtJun 2, 2025
  64. 08/10 test-lib: fail on unexpectedly passing testsPatrick Steinhardt, May 30, 2025
  65. 09/10 meson: introduce kwargs variable for testsPatrick Steinhardt, May 30, 2025
  66. 10/10 meson: parse TAP output generated by our testsPatrick Steinhardt, May 30, 2025
  67. Karthik NayakMay 31, 2025
  68. 00/10 meson: parse TAP output generated by our testsPatrick Steinhardt, Jun 2, 2025
  69. 01/10 t: stop announcing prereqsPatrick Steinhardt, Jun 2, 2025
  70. 02/10 t: silence output from `test_create_repo()`Patrick Steinhardt, Jun 2, 2025
  71. 03/10 t9822: use prereq to check for ISO-8859-1 supportPatrick Steinhardt, Jun 2, 2025
  72. 04/10 t983*: use prereq to check for Python-specific git-p4(1) supportPatrick Steinhardt, Jun 2, 2025
  73. 06/10 t/test-lib: fix TAP format for BASH_XTRACEFD warningPatrick Steinhardt, Jun 2, 2025
  74. 05/10 t/test-lib: don't print shell traces to stdoutPatrick Steinhardt, Jun 2, 2025
  75. 08/10 test-lib: fail on unexpectedly passing testsPatrick Steinhardt, Jun 2, 2025
  76. 07/10 t7815: fix unexpectedly passing test on macOSPatrick Steinhardt, Jun 2, 2025
  77. 09/10 meson: introduce kwargs variable for testsPatrick Steinhardt, Jun 2, 2025
  78. 10/10 meson: parse TAP output generated by our testsPatrick Steinhardt, Jun 2, 2025
  79. Karthik NayakJun 2, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.