git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 00/15] tests: don't ignore "git" exit codes

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 2, 2022, 17:27 UTC
Message-ID
<cover-00.15-00000000000-20220302T171755Z-avarab@gmail.com>

This series fixes issues where we ignored the exit code of "git" due to it being on the LHS of a pipe, or because we interpolated its output with $() in a "test" construct, or had missing &&- chains in helper functions etc.

This series is not made by string-replacing things in our test suite, if it was it would be much larger. These are all tests I've seen real hide real failures under SANITIZE=leak, either on current "master", or in combination with various local leak fixes I've got unsubmitted.

In cases where I was starting to fix a pattern in a file I'd fix the rest of the file if it was easy, but otherwise these are all cases where I ran SANITIZE=leak, had a test pass, but having ASAN_OPTIONS log to a file revealed that we had memory leaks within that test.

As an aside we still have various potential issues with hidden segfaults etc. in the test suite after this that are tricked to solve, because:

 * Our tests will (mostly) catch segfaults and abort(), but if we
   invoke a command that invokes another command it needs to ferry the
   exit code up to us.
 * run-command.c notably does not do that, so for e.g. "git push"
   tests where we expect a failure and an underlying "git" command
   fails we won't ferry up the segfault or abort exit code.
 * We have gitweb.perl and some other perl code ignoring return values
   from close(), i.e. ignoring exit codes from "git rev-parse" et al.
 * We have in-tree shellscripts like "git-merge-one-file.sh" invoking
   git commands, and if they fail returning "1", not ferrying up the
   segfault or abort() exit code.
Ævar Arnfjörð Bjarmason (15):
  tests: change some 'test $(git) = "x"' to test_cmp
  tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"
  read-tree tests: check "diff-files" exit code on failure
  diff tests: don't ignore "git diff" exit code
  diff tests: don't ignore "git diff" exit code in "read" loop
  apply tests: use "test_must_fail" instead of ad-hoc pattern
  merge tests: use "test_must_fail" instead of ad-hoc pattern
  rev-parse tests: don't ignore "git reflog" exit code
  notes tests: don't ignore "git" exit code
  diff tests: don't ignore "git rev-list" exit code
  rev-list tests: don't hide abort() in "test_expect_failure"
  gettext tests: don't ignore "test-tool regex" exit code
  apply tests: don't ignore "git ls-files" exit code, drop sub-shell
  checkout tests: don't ignore "git <cmd>" exit code
  rev-list simplify tests: don't ignore "git" exit code
 t/t0002-gitfile.sh                     |   6 +-
 t/t1001-read-tree-m-2way.sh            |   6 +-
 t/t1002-read-tree-m-u-2way.sh          |   6 +-
 t/t1503-rev-parse-verify.sh            |   5 +-
 t/t2012-checkout-last.sh               |  51 ++++++---
 t/t2200-add-update.sh                  |  33 ++++--
 t/t3302-notes-index-expensive.sh       |   6 +-
 t/t3303-notes-subtrees.sh              |   9 +-
 t/t3305-notes-fanout.sh                |  14 +--
 t/t4020-diff-external.sh               | 153 ++++++++++++-------------
 t/t4027-diff-submodule.sh              |   7 +-
 t/t4123-apply-shrink.sh                |  18 +--
 t/t4128-apply-root.sh                  |  36 +++---
 t/t6005-rev-list-count.sh              |  43 ++++---
 t/t6012-rev-list-simplify.sh           |  12 +-
 t/t6102-rev-list-unexpected-objects.sh |  13 ++-
 t/t6407-merge-binary.sh                |  22 +---
 t/t7103-reset-bare.sh                  |   7 +-
 t/t7812-grep-icase-non-ascii.sh        |  11 +-
 19 files changed, 240 insertions(+), 218 deletions(-)
-- 
2.35.1.1226.g8b497615d32
Next: Ævar Arnfjörð Bjarmason
Message 1 of 48 in “tests: don't ignore "git" exit codes”
  1. 00/15 tests: don't ignore "git" exit codesÆvar Arnfjörð Bjarmason, Mar 2, 2022
  2. 07/15 merge tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 2, 2022
  3. 08/15 rev-parse tests: don't ignore "git reflog" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  4. 02/15 tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"Ævar Arnfjörð Bjarmason, Mar 2, 2022
  5. 05/15 diff tests: don't ignore "git diff" exit code in "read" loopÆvar Arnfjörð Bjarmason, Mar 2, 2022
  6. 01/15 tests: change some 'test $(git) = "x"' to test_cmpÆvar Arnfjörð Bjarmason, Mar 2, 2022
  7. 12/15 gettext tests: don't ignore "test-tool regex" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  8. Junio C HamanoMar 2, 2022
  9. Ævar Arnfjörð BjarmasonMar 3, 2022
  10. Junio C HamanoMar 3, 2022
  11. 13/15 apply tests: don't ignore "git ls-files" exit code, drop sub-shellÆvar Arnfjörð Bjarmason, Mar 2, 2022
  12. 14/15 checkout tests: don't ignore "git <cmd>" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  13. 15/15 rev-list simplify tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  14. 11/15 rev-list tests: don't hide abort() in "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 2, 2022
  15. Junio C HamanoMar 2, 2022
  16. Ævar Arnfjörð BjarmasonMar 3, 2022
  17. 10/15 diff tests: don't ignore "git rev-list" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  18. 09/15 notes tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  19. 03/15 read-tree tests: check "diff-files" exit code on failureÆvar Arnfjörð Bjarmason, Mar 2, 2022
  20. Junio C HamanoMar 2, 2022
  21. 06/15 apply tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 2, 2022
  22. Junio C HamanoMar 2, 2022
  23. 04/15 diff tests: don't ignore "git diff" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  24. Junio C HamanoMar 2, 2022
  25. Derrick StoleeMar 3, 2022
  26. Ævar Arnfjörð BjarmasonMar 3, 2022
  27. Phillip WoodMar 3, 2022
  28. Ævar Arnfjörð BjarmasonMar 3, 2022
  29. Junio C HamanoMar 4, 2022
  30. Ævar Arnfjörð BjarmasonMar 4, 2022
  31. Shubham MishraMar 3, 2022
  32. Ævar Arnfjörð BjarmasonMar 3, 2022
  33. 00/15 tests: don't ignore "git" exit codesÆvar Arnfjörð Bjarmason, Mar 7, 2022
  34. 01/15 tests: change some 'test $(git) = "x"' to test_cmpÆvar Arnfjörð Bjarmason, Mar 7, 2022
  35. 02/15 tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"Ævar Arnfjörð Bjarmason, Mar 7, 2022
  36. 03/15 read-tree tests: check "diff-files" exit code on failureÆvar Arnfjörð Bjarmason, Mar 7, 2022
  37. 04/15 diff tests: don't ignore "git diff" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  38. 05/15 diff tests: don't ignore "git diff" exit code in "read" loopÆvar Arnfjörð Bjarmason, Mar 7, 2022
  39. 06/15 apply tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 7, 2022
  40. 07/15 merge tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 7, 2022
  41. 08/15 rev-parse tests: don't ignore "git reflog" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  42. 09/15 notes tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  43. 10/15 diff tests: don't ignore "git rev-list" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  44. 12/15 gettext tests: don't ignore "test-tool regex" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  45. 13/15 apply tests: don't ignore "git ls-files" exit code, drop sub-shellÆvar Arnfjörð Bjarmason, Mar 7, 2022
  46. 14/15 checkout tests: don't ignore "git <cmd>" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  47. 11/15 rev-list tests: don't hide abort() in "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 7, 2022
  48. 15/15 rev-list simplify tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.