git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/3] making test-suite tracing more useful

From
Jeff King <peff@peff.net>
Date
Oct 19, 2017, 21:01 UTC
Message-ID
<20171019210140.64lb52cqtgdh22ew@sigill.intra.peff.net>

I sometimes run git's test suite as part of an automated testing process. I was hoping to add "-x" support to get more details when a test fails (since failures are sometimes hard to reproduce). But I hit a few small snags:

  - you have to run with bash, since BASH_XTRACEFD is required to avoid
    failures in some tests when we capture the stderr of shell functions
    or subshells (which get polluted with the "set -x" outupt). This
    requirement isn't a big deal for me, but it showed some other
    issues.
  - the output with BASH_XTRACEFD is a little confusing; fixed by patch
    1.
  - there's one test that _only_ fails with BASH_XTRACEFD. That's fixed
    in patch 2.
  - I wanted to use "prove" since the output of "make -j64 test" is
    unreadable. But that means using "--verbose-log", which was
    incompatible with "-x". That's patch 3.
With these patches, I can now do:
  make -j64 test \
    SHELL_PATH=/bin/bash \
    GIT_TEST_OPTS="--verbose-log -x" \
    DEFAULT_TEST_TARGET=prove \
    GIT_PROVE_OPTS=-j64 || {
          echo "Failing tests:"
          echo "--------------"
	  grep -l '[^0]' t/test-results/*.exit |
	  while read failed; do
	          base=${failed%.exit}
		  name=${base#t/test-results/}
		  echo "==> $name"
		  cat "$base.out"
	  done
	  exit 1
    }

and get fairly readable output (a nice summary from prove, and then a dump of any failing test output).

Johannes, I've seen that you do "-x" in the tests that the git-for-windows bot uses to comment on GitHub. You may have seen the bogus failure in t5615, which this series should fix (you may also have seen the "set +x" cruft at the end of each test, which is fixed here, too).

Lars, I think with this it should be possible to turn on "-x" for the Travis build.

  [1/3]: test-lib: silence "-x" cleanup under bash
  [2/3]: t5615: avoid re-using descriptor 4
  [3/3]: test-lib: make "-x" work with "--verbose-log"
 t/t5615-alternate-env.sh |  6 +++---
 t/test-lib.sh            | 44 ++++++++++++++++++++++++++++++++------------
 2 files changed, 35 insertions(+), 15 deletions(-)
-Peff
Next: Jeff King
Message 1 of 36 in “making test-suite tracing more useful”
  1. 0/3 making test-suite tracing more usefulJeff King, Oct 19, 2017
  2. 1/3 test-lib: silence "-x" cleanup under bashJeff King, Oct 19, 2017
  3. 2/3 t5615: avoid re-using descriptor 4Jeff King, Oct 19, 2017
  4. Stefan BellerOct 19, 2017
  5. Jeff KingOct 19, 2017
  6. Jeff KingOct 20, 2017
  7. Stefan BellerOct 20, 2017
  8. Jeff KingOct 20, 2017
  9. Simon RuderichOct 21, 2017
  10. Junio C HamanoOct 21, 2017
  11. Jeff KingOct 21, 2017
  12. 3/3 test-lib: make "-x" work with "--verbose-log"Jeff King, Oct 19, 2017
  13. Johannes SchindelinOct 23, 2017
  14. 4/3 t/Makefile: introduce TEST_SHELL_PATHJeff King, Oct 20, 2017
  15. SZEDER GáborOct 20, 2017
  16. Jeff KingOct 21, 2017
  17. Johannes SchindelinOct 23, 2017
  18. Jeff KingOct 24, 2017
  19. Johannes SchindelinOct 25, 2017
  20. Jeff KingOct 25, 2017
  21. Johannes SchindelinOct 27, 2017
  22. Johannes SchindelinOct 23, 2017
  23. Jeff KingOct 24, 2017
  24. 0/4 making test-suite tracing more usefulJeff King, Dec 8, 2017
  25. 1/4 test-lib: silence "-x" cleanup under bashJeff King, Dec 8, 2017
  26. 2/4 t5615: avoid re-using descriptor 4Jeff King, Dec 8, 2017
  27. 3/4 test-lib: make "-x" work with "--verbose-log"Jeff King, Dec 8, 2017
  28. 4/4 t/Makefile: introduce TEST_SHELL_PATHJeff King, Dec 8, 2017
  29. Johannes SchindelinDec 8, 2017
  30. Jeff KingDec 8, 2017
  31. Johannes SchindelinDec 9, 2017
  32. Jeff KingDec 10, 2017
  33. Junio C HamanoDec 11, 2017
  34. Jeff KingDec 15, 2017
  35. Junio C HamanoDec 15, 2017
  36. Jeff KingDec 21, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.