git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 11/15] rev-list tests: don't hide abort() in "test_expect_failure"

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 7, 2022, 12:49 UTC
Message-ID
<patch-v2-11.15-52397b3575a-20220307T124817Z-avarab@gmail.com>
In-Reply-To
<cover-v2-00.15-00000000000-20220307T124817Z-avarab@gmail.com>

Change a couple of uses of "test_expect_failure" to use a "test_expect_success" to positively assert the current behavior, and replace the intent of "test_expect_failure" with a "TODO" comment int the description.

As noted in [1] the "test_expect_failure" feature is overly eager to accept any failure as OK, and thus by design hides segfaults, abort() etc. Because of that I didn't notice in dd9cede9136 (leak tests: mark some rev-list tests as passing with SANITIZE=leak, 2021-10-31) that this test leaks memory under SANITIZE=leak.

I have some larger local changes to add a better "test_expect_failure", which would work just like "test_expect_success", but would allow us say "test_todo" here (and "success" would emit a "not ok [...] # TODO", not "ok [...]".

So even though using "test_expect_success" here comes with its own problems[2], let's use it as a narrow change to fix the problem at hand here and stop conflating the current "success" with actual SANITIZE=leak failures.

1. https://lore.kernel.org/git/87tuhmk19c.fsf@evledraar.gmail.com/
2. https://lore.kernel.org/git/xmqq4k9kj15p.fsf@gitster.g/
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t6102-rev-list-unexpected-objects.sh | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/t/t6102-rev-list-unexpected-objects.sh b/t/t6102-rev-list-unexpected-objects.sh
index 6f0902b8638..cf0195e8263 100755
--- a/t/t6102-rev-list-unexpected-objects.sh
+++ b/t/t6102-rev-list-unexpected-objects.sh
@@ -17,8 +17,13 @@ test_expect_success 'setup unexpected non-blob entry' '
 	broken_tree="$(git hash-object -w --literally -t tree broken-tree)"
 '
 
-test_expect_failure 'traverse unexpected non-blob entry (lone)' '
-	test_must_fail git rev-list --objects $broken_tree
+test_expect_success !SANITIZE_LEAK 'TODO (should fail!): traverse unexpected non-blob entry (lone)' '
+	sed "s/Z$//" >expect <<-EOF &&
+	$broken_tree Z
+	$tree foo
+	EOF
+	git rev-list --objects $broken_tree >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'traverse unexpected non-blob entry (seen)' '
@@ -116,8 +121,8 @@ test_expect_success 'setup unexpected non-blob tag' '
 	tag=$(git hash-object -w --literally -t tag broken-tag)
 '
 
-test_expect_failure 'traverse unexpected non-blob tag (lone)' '
-	test_must_fail git rev-list --objects $tag
+test_expect_success !SANITIZE_LEAK 'TODO (should fail!): traverse unexpected non-blob tag (lone)' '
+	git rev-list --objects $tag
 '
 
 test_expect_success 'traverse unexpected non-blob tag (seen)' '

-- 
2.35.1.1242.gfeba0eae32b
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 47 of 48 in “tests: don't ignore "git" exit codes”
  1. 00/15 tests: don't ignore "git" exit codesÆvar Arnfjörð Bjarmason, Mar 2, 2022
  2. 07/15 merge tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 2, 2022
  3. 08/15 rev-parse tests: don't ignore "git reflog" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  4. 02/15 tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"Ævar Arnfjörð Bjarmason, Mar 2, 2022
  5. 05/15 diff tests: don't ignore "git diff" exit code in "read" loopÆvar Arnfjörð Bjarmason, Mar 2, 2022
  6. 01/15 tests: change some 'test $(git) = "x"' to test_cmpÆvar Arnfjörð Bjarmason, Mar 2, 2022
  7. 12/15 gettext tests: don't ignore "test-tool regex" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  8. Junio C HamanoMar 2, 2022
  9. Ævar Arnfjörð BjarmasonMar 3, 2022
  10. Junio C HamanoMar 3, 2022
  11. 13/15 apply tests: don't ignore "git ls-files" exit code, drop sub-shellÆvar Arnfjörð Bjarmason, Mar 2, 2022
  12. 14/15 checkout tests: don't ignore "git <cmd>" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  13. 15/15 rev-list simplify tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  14. 11/15 rev-list tests: don't hide abort() in "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 2, 2022
  15. Junio C HamanoMar 2, 2022
  16. Ævar Arnfjörð BjarmasonMar 3, 2022
  17. 10/15 diff tests: don't ignore "git rev-list" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  18. 09/15 notes tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  19. 03/15 read-tree tests: check "diff-files" exit code on failureÆvar Arnfjörð Bjarmason, Mar 2, 2022
  20. Junio C HamanoMar 2, 2022
  21. 06/15 apply tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 2, 2022
  22. Junio C HamanoMar 2, 2022
  23. 04/15 diff tests: don't ignore "git diff" exit codeÆvar Arnfjörð Bjarmason, Mar 2, 2022
  24. Junio C HamanoMar 2, 2022
  25. Derrick StoleeMar 3, 2022
  26. Ævar Arnfjörð BjarmasonMar 3, 2022
  27. Phillip WoodMar 3, 2022
  28. Ævar Arnfjörð BjarmasonMar 3, 2022
  29. Junio C HamanoMar 4, 2022
  30. Ævar Arnfjörð BjarmasonMar 4, 2022
  31. Shubham MishraMar 3, 2022
  32. Ævar Arnfjörð BjarmasonMar 3, 2022
  33. 00/15 tests: don't ignore "git" exit codesÆvar Arnfjörð Bjarmason, Mar 7, 2022
  34. 01/15 tests: change some 'test $(git) = "x"' to test_cmpÆvar Arnfjörð Bjarmason, Mar 7, 2022
  35. 02/15 tests: use "test_stdout_line_count", not "test $(git [...] | wc -l)"Ævar Arnfjörð Bjarmason, Mar 7, 2022
  36. 03/15 read-tree tests: check "diff-files" exit code on failureÆvar Arnfjörð Bjarmason, Mar 7, 2022
  37. 04/15 diff tests: don't ignore "git diff" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  38. 05/15 diff tests: don't ignore "git diff" exit code in "read" loopÆvar Arnfjörð Bjarmason, Mar 7, 2022
  39. 06/15 apply tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 7, 2022
  40. 07/15 merge tests: use "test_must_fail" instead of ad-hoc patternÆvar Arnfjörð Bjarmason, Mar 7, 2022
  41. 08/15 rev-parse tests: don't ignore "git reflog" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  42. 09/15 notes tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  43. 10/15 diff tests: don't ignore "git rev-list" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  44. 12/15 gettext tests: don't ignore "test-tool regex" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  45. 13/15 apply tests: don't ignore "git ls-files" exit code, drop sub-shellÆvar Arnfjörð Bjarmason, Mar 7, 2022
  46. 14/15 checkout tests: don't ignore "git <cmd>" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022
  47. 11/15 rev-list tests: don't hide abort() in "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 7, 2022
  48. 15/15 rev-list simplify tests: don't ignore "git" exit codeÆvar Arnfjörð Bjarmason, Mar 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.