git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCHv6 01/16] test-lib: make test_expect_code a test command

From
Elijah Newren <newren@gmail.com>
Date
Oct 3, 2010, 19:59 UTC
Message-ID
<1286136014-7728-2-git-send-email-newren@gmail.com>
In-Reply-To
<1286136014-7728-1-git-send-email-newren@gmail.com>
From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

Change test_expect_code to be a normal test command instead of a top-level command.

As a top-level command it would fail in cases like:
    test_expect_code 1 'phoney' '
        foo && bar && (exit 1)
    '

Here the test might incorrectly succeed if "foo" or "bar" happened to fail with exit status 1. Instead we now do:

    test_expect_success 'phoney' '
        foo && bar && test_expect_code 1 "(exit 1)"
    '

Which will only succeed if "foo" and "bar" return status 0, and "(exit 1)" returns status 1. Note that test_expect_code has been made slightly noisier, as it reports the exit code it receives even upon success.

Some test code in t0000-basic.sh relied on the old semantics of test_expect_code to test the test_when_finished command. I've converted that code to use an external test similar to the TODO test I added in v1.7.3-rc0~2^2~3.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Acked-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
---
 t/README                |   16 +++++++------
 t/t0000-basic.sh        |   55 ++++++++++++++++++++++++++++++++++++++--------
 t/t1504-ceiling-dirs.sh |    5 ++-
 t/t6020-merge-df.sh     |    4 ++-
 t/test-lib.sh           |   40 ++++++++++++++++++---------------
 5 files changed, 82 insertions(+), 38 deletions(-)
diff --git a/t/README b/t/README
index a1eb7c8..ee4c0cf 100644
--- a/t/README
+++ b/t/README
@@ -395,13 +395,6 @@ library for your script to use.
    Like test_expect_success this function can optionally use a three
    argument invocation with a prerequisite as the first argument.
 
- - test_expect_code [<prereq>] <code> <message> <script>
-
-   Analogous to test_expect_success, but pass the test if it exits
-   with a given exit <code>
-
- test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
-
  - test_debug <script>
 
    This takes a single argument, <script>, and evaluates it only
@@ -482,6 +475,15 @@ library for your script to use.
 	    'Perl API' \
 	    "$PERL_PATH" "$TEST_DIRECTORY"/t9700/test.pl
 
+ - test_expect_code <exit-code> <command>
+
+   Run a command and ensure that it exits with the given exit code.
+   For example:
+
+	test_expect_success 'Merge with d/f conflicts' '
+		test_expect_code 1 git merge "merge msg" B master
+	'
+
  - test_must_fail <git-command>
 
    Run a git command and ensure it fails in a controlled way.  Use
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index f688bd3..c2f5f8d 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -130,22 +130,57 @@ test_expect_success 'tests clean up after themselves' '
     test_when_finished clean=yes
 '
 
-cleaner=no
-test_expect_code 1 'tests clean up even after a failure' '
-    test_when_finished cleaner=yes &&
-    (exit 1)
-'
-
-if test $clean$cleaner != yesyes
+if test $clean != yes
 then
-	say "bug in test framework: cleanup commands do not work reliably"
+	say "bug in test framework: basic cleanup command does not work reliably"
 	exit 1
 fi
 
-test_expect_code 2 'failure to clean up causes the test to fail' '
-    test_when_finished "(exit 2)"
+test_expect_success 'tests clean up even on failures' "
+    mkdir failing-cleanup &&
+    (cd failing-cleanup &&
+    cat >failing-cleanup.sh <<EOF &&
+#!$SHELL_PATH
+
+test_description='Failing tests with cleanup commands'
+
+# Point to the t/test-lib.sh, which isn't in ../ as usual
+TEST_DIRECTORY=\"$TEST_DIRECTORY\"
+. \"\$TEST_DIRECTORY\"/test-lib.sh
+
+test_expect_success 'tests clean up even after a failure' '
+    touch clean-after-failure &&
+    test_when_finished rm clean-after-failure &&
+    (exit 1)
 '
 
+test_expect_success 'failure to clean up causes the test to fail' '
+    test_when_finished \"(exit 2)\"
+'
+
+test_done
+EOF
+    chmod +x failing-cleanup.sh &&
+    test_must_fail ./failing-cleanup.sh >out 2>err &&
+    ! test -s err &&
+    ! test -f \"trash directory.failing-cleanup/clean-after-failure\" &&
+cat >expect <<EOF &&
+not ok - 1 tests clean up even after a failure
+#	
+#	    touch clean-after-failure &&
+#	    test_when_finished rm clean-after-failure &&
+#	    (exit 1)
+#	
+not ok - 2 failure to clean up causes the test to fail
+#	
+#	    test_when_finished \"(exit 2)\"
+#	
+# failed 2 among 2 test(s)
+1..2
+EOF
+    test_cmp expect out)
+"
+
 ################################################################
 # Basics of the basics
 
diff --git a/t/t1504-ceiling-dirs.sh b/t/t1504-ceiling-dirs.sh
index df5ad8c..cce87a5 100755
--- a/t/t1504-ceiling-dirs.sh
+++ b/t/t1504-ceiling-dirs.sh
@@ -9,8 +9,9 @@ test_prefix() {
 }
 
 test_fail() {
-	test_expect_code 128 "$1: prefix" \
-	"git rev-parse --show-prefix"
+	test_expect_success "$1: prefix" '
+		test_expect_code 128 git rev-parse --show-prefix
+	'
 }
 
 TRASH_ROOT="$PWD"
diff --git a/t/t6020-merge-df.sh b/t/t6020-merge-df.sh
index 490d397..5d91d05 100755
--- a/t/t6020-merge-df.sh
+++ b/t/t6020-merge-df.sh
@@ -20,7 +20,9 @@ echo "file dir" > dir &&
 git add dir &&
 git commit -m "File: dir"'
 
-test_expect_code 1 'Merge with d/f conflicts' 'git merge "merge msg" B master'
+test_expect_success 'Merge with d/f conflicts' '
+	test_expect_code 1 git merge "merge msg" B master
+'
 
 test_expect_success 'F/D conflict' '
 	git reset --hard &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 830e5e7..d86edcd 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -473,24 +473,6 @@ test_expect_success () {
 	echo >&3 ""
 }
 
-test_expect_code () {
-	test "$#" = 4 && { prereq=$1; shift; } || prereq=
-	test "$#" = 3 ||
-	error "bug in the test script: not 3 or 4 parameters to test-expect-code"
-	if ! test_skip "$@"
-	then
-		say >&3 "expecting exit code $1: $3"
-		test_run_ "$3"
-		if [ "$?" = 0 -a "$eval_ret" = "$1" ]
-		then
-			test_ok_ "$2"
-		else
-			test_failure_ "$@"
-		fi
-	fi
-	echo >&3 ""
-}
-
 # test_external runs external test scripts that provide continuous
 # test output about their progress, and succeeds/fails on
 # zero/non-zero exit code.  It outputs the test output on stdout even
@@ -658,6 +640,28 @@ test_might_fail () {
 	return 0
 }
 
+# Similar to test_must_fail and test_might_fail, but check that a
+# given command exited with a given exit code. Meant to be used as:
+#
+#	test_expect_success 'Merge with d/f conflicts' '
+#		test_expect_code 1 git merge "merge msg" B master
+#	'
+
+test_expect_code () {
+	want_code=$1
+	shift
+	"$@"
+	exit_code=$?
+	if test $exit_code = $want_code
+	then
+		echo >&2 "test_expect_code: command exited with $exit_code: $*"
+		return 0
+	else
+		echo >&2 "test_expect_code: command exited with $exit_code, we wanted $want_code $*"
+		return 1
+	fi
+}
+
 # test_cmp is a helper function to compare actual and expected output.
 # You can use it like:
 #

-- 
1.7.3.1.66.gab790
Previous: Elijah NewrenNext: Junio C Hamano
Message 2 of 62 in “[PATCHv6 00/16] Add missing &&'s in the testsuite”
  1. Elijah NewrenOct 3, 2010
  2. 01/16 test-lib: make test_expect_code a test commandElijah Newren, Oct 3, 2010
  3. Junio C HamanoOct 4, 2010
  4. Ævar Arnfjörð BjarmasonOct 4, 2010
  5. Jonathan NiederOct 4, 2010
  6. Ævar Arnfjörð BjarmasonOct 4, 2010
  7. Jonathan NiederOct 4, 2010
  8. Ævar Arnfjörð BjarmasonOct 4, 2010
  9. Jonathan NiederOct 4, 2010
  10. Ævar Arnfjörð BjarmasonOct 4, 2010
  11. 02/16 t3020 (ls-files-error-unmatch): remove stray '1' from end of fileElijah Newren, Oct 3, 2010
  12. Junio C HamanoOct 4, 2010
  13. 03/16 t4017 (diff-retval): replace manual exit code check with test_expect_codeElijah Newren, Oct 3, 2010
  14. 04/16 t100[12] (read-tree-m-2way, read_tree_m_u_2way): add missing &&Elijah Newren, Oct 3, 2010
  15. 05/16 t4002 (diff-basic): use test_might_fail for commands that might failElijah Newren, Oct 3, 2010
  16. 06/16 t4202 (log): Replace '<git-command> || :' with test_might_failElijah Newren, Oct 3, 2010
  17. 07/16 t3600 (rm): add lots of missing &&Elijah Newren, Oct 3, 2010
  18. Jonathan NiederOct 3, 2010
  19. Junio C HamanoOct 3, 2010
  20. 08/16 t4019 (diff-wserror): add lots of missing &&Elijah Newren, Oct 3, 2010
  21. 09/16 t4026 (color): remove unneeded and unchained commandElijah Newren, Oct 3, 2010
  22. 10/16 t5602 (clone-remote-exec): add missing &&Elijah Newren, Oct 3, 2010
  23. 11/16 t6016 (rev-list-graph-simplify-history): add missing &&Elijah Newren, Oct 3, 2010
  24. 12/16 t7001 (mv): add missing &&Elijah Newren, Oct 3, 2010
  25. 13/16 t7601 (merge-pull-config): add missing &&Elijah Newren, Oct 3, 2010
  26. 14/16 t7800 (difftool): add missing &&Elijah Newren, Oct 3, 2010
  27. 15/16 Add missing &&'s throughout the testsuiteElijah Newren, Oct 3, 2010
  28. Jonathan NiederOct 3, 2010
  29. Jonathan NiederOct 3, 2010
  30. tests: add missing &&Jonathan Nieder, Oct 31, 2010
  31. Junio C HamanoOct 31, 2010
  32. 00/10 Re: [PATCH en/cascade-tests] tests: add missing &&Jonathan Nieder, Oct 31, 2010
  33. 01/10 tests: add missing &&, batch 2Jonathan Nieder, Oct 31, 2010
  34. 02/10 test-lib: introduce test_line_count to measure filesJonathan Nieder, Oct 31, 2010
  35. Junio C HamanoNov 9, 2010
  36. Ævar Arnfjörð BjarmasonNov 9, 2010
  37. 03/10 t6022 (renaming merge): chain test commands with &&Jonathan Nieder, Oct 31, 2010
  38. 04/10 t1502 (rev-parse --parseopt): test exit code from "-h"Jonathan Nieder, Oct 31, 2010
  39. 05/10 t1400 (update-ref): use test_must_failJonathan Nieder, Oct 31, 2010
  40. 06/10 t3301 (notes): use test_expect_code for clarityJonathan Nieder, Oct 31, 2010
  41. 07/10 t3404 (rebase -i): unroll test_commit loopsJonathan Nieder, Oct 31, 2010
  42. 08/10 t3404 (rebase -i): move comment to descriptionJonathan Nieder, Oct 31, 2010
  43. Junio C HamanoNov 17, 2010
  44. 09/10 t3404 (rebase -i): introduce helper to check position of HEADJonathan Nieder, Oct 31, 2010
  45. Junio C HamanoNov 17, 2010
  46. 10/10 t4124 (apply --whitespace): use test_might_failJonathan Nieder, Oct 31, 2010
  47. Junio C HamanoNov 9, 2010
  48. Elijah NewrenNov 5, 2010
  49. 16/16 Introduce portable_unset and use it to ensure proper && chainingElijah Newren, Oct 3, 2010
  50. Ævar Arnfjörð BjarmasonOct 4, 2010
  51. Jonathan NiederOct 4, 2010
  52. Junio C HamanoOct 4, 2010
  53. Jonathan NiederOct 4, 2010
  54. Jonathan NiederOct 3, 2010
  55. Ævar Arnfjörð BjarmasonOct 4, 2010
  56. Jonathan NiederOct 4, 2010
  57. Ævar Arnfjörð BjarmasonOct 4, 2010
  58. yj2133011Oct 4, 2010
  59. test-lib: &&-chaining testerJonathan Nieder, Oct 6, 2010
  60. Matthieu MoyOct 6, 2010
  61. Johannes SixtOct 6, 2010
  62. Sverre RabbelierOct 6, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.