{"thread":{"id":"24189","subject":"[PATCH v4 0/5] TAP support for Git","startedAt":"2010-06-24T17:44:44Z","lastAt":"2010-06-27T18:27:30Z","messageCount":16,"participants":["Ævar Arnfjörð Bjarmason","Jakub Narebski","Junio C Hamano"],"isPatch":true,"patchVersion":4,"patchTotal":5},"messages":[{"id":"144166","messageId":"1277401489-27885-1-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":null,"subject":"[PATCH v4 0/5] TAP support for Git","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:44Z","receivedAt":"2010-06-24T17:44:44Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This is v4 of the TAP series for Git. This version fixes the failure\nJunio had with it (see <7v6319duky.fsf@alter.siamese.dyndns.org>).\n\nIt does this by reverting the removal of the test_external_*\nfunctions. Now they're made TAP aware instead with a global\n`test_external_has_tap' that can be set by tests who wish to output\ntheir own TAP.\n\nThus t9700-perl-git.sh will again fail if it has any output on stderr,\nbut since it sets `test_external_has_tap=1' we'll let it finalize its\nown TAP plan.\n\nNot removing the test_external_* makes the impact of this series\nsmaller, and preserves compatibility with other topics in-flight, such\nas the Gettext series.\n\nThis is the diff --stat from v3:\n\n t/t9700-perl-git.sh | 11 ++++--\n t/test-lib.sh | 97 ++++++++++++++++++++++++++++++++++++++++++++++++--\n 2 files changed, 101 insertions(+), 7 deletions(-)\n\nMost of the additions are the restorations of the test_external_*\nfunctions, in slightly altered form.\n\nÆvar Arnfjörð Bjarmason (5):\n test-lib: Adjust output to be valid TAP format\n test-lib: Make the test_external_* functions TAP-aware\n test-lib: output a newline before \"ok\" under a TAP harness\n tests: Skip tests in a way that makes sense under TAP\n tests: Say \"pass\" rather than \"ok\" on empty lines for TAP\n\n t/README | 55 ++++++++++++++++-------\n t/lib-git-svn.sh | 4 +-\n t/lib-httpd.sh | 3 +-\n t/t1020-subdirectory.sh | 12 +++---\n t/t1304-default-acl.sh | 9 +++-\n t/t1509-root-worktree.sh | 6 +-\n t/t2007-checkout-symlink.sh | 2 +-\n t/t2102-update-index-symlinks.sh | 2 +-\n t/t3300-funny-names.sh | 2 +-\n t/t3302-notes-index-expensive.sh | 2 +-\n t/t3600-rm.sh | 2 +-\n t/t3700-add.sh | 12 +++---\n t/t3701-add-interactive.sh | 4 +-\n t/t3902-quoted.sh | 2 +-\n t/t4004-diff-rename-symlink.sh | 2 +-\n t/t4011-diff-symlink.sh | 2 +-\n t/t4016-diff-quote.sh | 2 +-\n t/t4023-diff-rename-typechange.sh | 2 +-\n t/t4114-apply-typechange.sh | 2 +-\n t/t4115-apply-symlink.sh | 2 +-\n t/t4122-apply-symlink-inside.sh | 2 +-\n t/t5302-pack-index.sh | 2 +-\n t/t5503-tagfollow.sh | 2 +-\n t/t5522-pull-symlink.sh | 2 +-\n t/t5540-http-push.sh | 2 +-\n t/t5541-http-push.sh | 2 +-\n t/t5550-http-fetch.sh | 2 +-\n t/t5551-http-fetch.sh | 2 +-\n t/t5561-http-backend.sh | 2 +-\n t/t5705-clone-2gb.sh | 2 +-\n t/t6035-merge-dir-to-symlink.sh | 2 +-\n t/t7004-tag.sh | 2 +-\n t/t7006-pager.sh | 2 +-\n t/t7800-difftool.sh | 2 +-\n t/t9001-send-email.sh | 4 +-\n t/t9100-git-svn-basic.sh | 2 +-\n t/t9119-git-svn-info.sh | 2 +-\n t/t9129-git-svn-i18n-commitencoding.sh | 2 +-\n t/t9143-git-svn-gc.sh | 2 +-\n t/t9200-git-cvsexportcommit.sh | 4 +-\n t/t9400-git-cvsserver-server.sh | 6 +-\n t/t9401-git-cvsserver-crlf.sh | 6 +-\n t/t9600-cvsimport.sh | 2 +-\n t/t9700-perl-git.sh | 7 ++-\n t/t9700/test.pl | 11 +++++\n t/test-lib.sh | 76 +++++++++++++++++++++++--------\n 46 files changed, 178 insertions(+), 103 deletions(-)\n"},{"id":"144167","messageId":"1277401489-27885-2-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH v4 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:45Z","receivedAt":"2010-06-24T17:44:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"TAP, the Test Anything Protocol, is a simple text-based interface\nbetween testing modules in a test harness. test-lib.sh's output was\nalready very close to being valid TAP. This change brings it all the\nway there. Before:\n\n $ ./t0005-signals.sh\n * ok 1: sigchain works\n * passed all 1 test(s)\n\nAnd after:\n\n $ ./t0005-signals.sh\n ok 1 - sigchain works\n # passed all 1 test(s)\n 1..1\n\nThe advantage of using TAP is that any program that reads the format\n(a \"test harness\") can run the tests. The most popular of these is the\nprove(1) utility that comes with Perl. It can run tests in parallel,\ndisplay colored output, format the output to console, file, HTML etc.,\nand much more. An example:\n\n $ prove ./t0005-signals.sh\n ./t0005-signals.sh .. ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)\n Result: PASS\n\nprove(1) gives you human readable output without being too\nverbose. Running the test suite in parallel with `make test -j15`\nproduces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`\nmakes it easy to follow what's going on.\n\nAll this patch does is re-arrange the output a bit so that it conforms\nwith the TAP spec, everything that the test suite did before continues\nto work. That includes aggregating results in t/test-results/, the\n--verbose, --debug and other options for tests, and the test color\noutput.\n\nTAP harnesses ignore everything that they don't know about, so running\nthe tests with --verbose works:\n\n $ prove ./t0005-signals.sh :: --verbose --debug\n ./t0005-signals.sh .. Terminated\n ./t0005-signals.sh .. ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)\n Result: PASS\n\nJust supply the -v option to prove itself to get all the verbose\noutput that it suppresses:\n\n $ prove -v ./t0005-signals.sh :: --verbose --debug\n ./t0005-signals.sh ..\n Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/\n expecting success:\n test-sigchain >actual\n case \"$?\" in\n 143) true ;; # POSIX w/ SIGTERM=15\n 3) true ;; # Windows\n *) false ;;\n esac &&\n test_cmp expect actual\n Terminated\n ok 1 - sigchain works\n # passed all 1 test(s)\n 1..1\n ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)\n Result: PASS\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/README | 55 +++++++++++++++++++++++++++++++++++++++----------------\n t/test-lib.sh | 30 ++++++++++++++++++------------\n 2 files changed, 57 insertions(+), 28 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex 0e4e8d8..aa4ed28 100644\n--- a/t/README\n+++ b/t/README\n@@ -18,25 +18,48 @@ The easiest way to run tests is to say \"make\". This runs all\n the tests.\n \n *** t0000-basic.sh ***\n- * ok 1: .git/objects should be empty after git-init in an empty repo.\n- * ok 2: .git/objects should have 256 subdirectories.\n- * ok 3: git-update-index without --add should fail adding.\n+ ok 1 - .git/objects should be empty after git init in an empty repo.\n+ ok 2 - .git/objects should have 3 subdirectories.\n+ ok 3 - success is reported like this\n ...\n- * ok 23: no diff after checkout and git-update-index --refresh.\n- * passed all 23 test(s)\n- *** t0100-environment-names.sh ***\n- * ok 1: using old names should issue warnings.\n- * ok 2: using old names but having new names should not issue warnings.\n- ...\n-\n-Or you can run each test individually from command line, like\n-this:\n-\n- $ sh ./t3001-ls-files-killed.sh\n- * ok 1: git-update-index --add to add various paths.\n- * ok 2: git-ls-files -k to show killed files.\n- * ok 3: validate git-ls-files -k output.\n- * passed all 3 test(s)\n+ ok 43 - very long name in the index handled sanely\n+ # fixed 1 known breakage(s)\n+ # still have 1 known breakage(s)\n+ # passed all remaining 42 test(s)\n+ 1..43\n+ *** t0001-init.sh ***\n+ ok 1 - plain\n+ ok 2 - plain with GIT_WORK_TREE\n+ ok 3 - plain bare\n+\n+Since the tests all output TAP (see http://testanything.org) they can\n+be run with any TAP harness. Here's an example of paralell testing\n+powered by a recent version of prove(1):\n+\n+ $ prove --timer --jobs 15 ./t[0-9]*.sh\n+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms\n+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms\n+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms\n+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms\n+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms\n+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===\n+\n+prove and other harnesses come with a lot of useful options. The\n+--state option in particular is very useful:\n+\n+ # Repeat until no more failures\n+ $ prove -j 15 --state=failed,save ./t[0-9]*.sh\n+\n+You can also run each test individually from command line, like this:\n+\n+ $ sh ./t3010-ls-files-killed-modified.sh\n+ ok 1 - git update-index --add to add various paths.\n+ ok 2 - git ls-files -k to show killed files.\n+ ok 3 - validate git ls-files -k output.\n+ ok 4 - git ls-files -m to show modified files.\n+ ok 5 - validate git ls-files -m output.\n+ # passed all 5 test(s)\n+ 1..5\n \n You can pass --verbose (or -v), --debug (or -d), and --immediate\n (or -i) command line argument to the test, or by setting GIT_TEST_OPTS\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 367f053..e97645c 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -160,7 +160,7 @@ if test -n \"$color\"; then\n \t\t\t*) test -n \"$quiet\" && return;;\n \t\tesac\n \t\tshift\n-\t\tprintf \"* %s\" \"$*\"\n+\t\tprintf \"%s\" \"$*\"\n \t\ttput sgr0\n \t\techo\n \t\t)\n@@ -169,7 +169,7 @@ else\n \tsay_color() {\n \t\ttest -z \"$1\" && test -n \"$quiet\" && return\n \t\tshift\n-\t\techo \"* $*\"\n+\t\techo \"$*\"\n \t}\n fi\n \n@@ -339,25 +339,25 @@ test_have_prereq () {\n \n test_ok_ () {\n \ttest_success=$(($test_success + 1))\n-\tsay_color \"\" \" ok $test_count: $@\"\n+\tsay_color \"\" \"ok $test_count - $@\"\n }\n \n test_failure_ () {\n \ttest_failure=$(($test_failure + 1))\n-\tsay_color error \"FAIL $test_count: $1\"\n+\tsay_color error \"not ok - $test_count $1\"\n \tshift\n-\techo \"$@\" | sed -e 's/^/\t/'\n+\techo \"$@\" | sed -e 's/^/#\t/'\n \ttest \"$immediate\" = \"\" || { GIT_EXIT_OK=t; exit 1; }\n }\n \n test_known_broken_ok_ () {\n \ttest_fixed=$(($test_fixed+1))\n-\tsay_color \"\" \" FIXED $test_count: $@\"\n+\tsay_color \"\" \"ok $test_count - $@ # TODO known breakage\"\n }\n \n test_known_broken_failure_ () {\n \ttest_broken=$(($test_broken+1))\n-\tsay_color skip \" still broken $test_count: $@\"\n+\tsay_color skip \"not ok $test_count - $@ # TODO known breakage\"\n }\n \n test_debug () {\n@@ -390,7 +390,7 @@ test_skip () {\n \tcase \"$to_skip\" in\n \tt)\n \t\tsay_color skip >&3 \"skipping test: $@\"\n-\t\tsay_color skip \"skip $test_count: $1\"\n+\t\tsay_color skip \"ok $test_count: # skip $1\"\n \t\t: true\n \t\t;;\n \t*)\n@@ -620,18 +620,22 @@ test_done () {\n \n \tif test \"$test_fixed\" != 0\n \tthen\n-\t\tsay_color pass \"fixed $test_fixed known breakage(s)\"\n+\t\tsay_color pass \"# fixed $test_fixed known breakage(s)\"\n \tfi\n \tif test \"$test_broken\" != 0\n \tthen\n-\t\tsay_color error \"still have $test_broken known breakage(s)\"\n+\t\tsay_color error \"# still have $test_broken known breakage(s)\"\n \t\tmsg=\"remaining $(($test_count-$test_broken)) test(s)\"\n \telse\n \t\tmsg=\"$test_count test(s)\"\n \tfi\n \tcase \"$test_failure\" in\n \t0)\n-\t\tsay_color pass \"passed all $msg\"\n+\t\t# Maybe print SKIP message\n+\t\t[ -z \"$skip_all\" ] || skip_all=\" # SKIP $skip_all\"\n+\n+\t\tsay_color pass \"# passed all $msg\"\n+\t\tsay \"1..$test_count$skip_all\"\n \n \t\ttest -d \"$remove_trash\" &&\n \t\tcd \"$(dirname \"$remove_trash\")\" &&\n@@ -640,7 +644,9 @@ test_done () {\n \t\texit 0 ;;\n \n \t*)\n-\t\tsay_color error \"failed $test_failure among $msg\"\n+\t\tsay_color error \"# failed $test_failure among $msg\"\n+\t\tsay \"1..$test_count\"\n+\n \t\texit 1 ;;\n \n \tesac\n-- \n1.7.1.251.g92a7\n"},{"id":"144168","messageId":"1277401489-27885-3-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH v4 2/5] test-lib: Make the test_external_* functions TAP-aware","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:46Z","receivedAt":"2010-06-24T17:44:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Before TAP we just ran the Perl test and assumed that it failed if\nnothing was printed on STDERR. Continue doing that, but introduce a\n`test_external_has_tap' variable which tests can set to indicate that\nthey're outputting TAP.\n\nIf it's set we won't output a test plan, but trust the external test\nto do so. That way we can make external tests work with a TAP harness,\nbut still maintain compatibility with test-lib's own way of tracking\ntests through the test-results directory.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t9700-perl-git.sh | 3 +++\n t/t9700/test.pl | 11 +++++++++++\n t/test-lib.sh | 51 +++++++++++++++++++++++++++++++++++++++------------\n 3 files changed, 53 insertions(+), 12 deletions(-)\n\ndiff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh\nindex 8686086..0b92726 100755\n--- a/t/t9700-perl-git.sh\n+++ b/t/t9700-perl-git.sh\n@@ -46,6 +46,9 @@ test_expect_success \\\n git config --add test.int 2k\n '\n \n+# The external test will outputs its own plan\n+test_external_has_tap=1\n+\n test_external_without_stderr \\\n 'Perl API' \\\n \"$PERL_PATH\" \"$TEST_DIRECTORY\"/t9700/test.pl\ndiff --git a/t/t9700/test.pl b/t/t9700/test.pl\nindex 666722d..e5d4b03 100755\n--- a/t/t9700/test.pl\n+++ b/t/t9700/test.pl\n@@ -7,6 +7,13 @@ use strict;\n \n use Test::More qw(no_plan);\n \n+BEGIN {\n+\t# t9700-perl-git.sh kicks off our testing, so we have to go from\n+\t# there.\n+\t$Test::Builder::Test->{Curr_Test} = 1;\n+\t$Test::Builder::Test->{No_Ending} = 1;\n+}\n+\n use Cwd;\n use File::Basename;\n \n@@ -105,3 +112,7 @@ my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));\n like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');\n my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');\n isnt($last_commit, $dir_commit, 'log . does not show last commit');\n+\n+printf \"1..%d\\n\", $Test::Builder::Test->{Curr_Test};\n+\n+exit($Test::Builder::Test->{Is_Passing} ? 0 : 1);\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex e97645c..7077210 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -206,6 +206,8 @@ test_fixed=0\n test_broken=0\n test_success=0\n \n+test_external_has_tap=0\n+\n die () {\n \tcode=$?\n \tif test -n \"$GIT_EXIT_OK\"\n@@ -456,7 +458,7 @@ test_expect_code () {\n # test_external runs external test scripts that provide continuous\n # test output about their progress, and succeeds/fails on\n # zero/non-zero exit code. It outputs the test output on stdout even\n-# in non-verbose mode, and announces the external script with \"* run\n+# in non-verbose mode, and announces the external script with \"# run\n # : ...\" before running it. When providing relative paths, keep in\n # mind that all scripts run in \"trash directory\".\n # Usage: test_external description command arguments...\n@@ -471,7 +473,7 @@ test_external () {\n \tthen\n \t\t# Announce the script to reduce confusion about the\n \t\t# test output that follows.\n-\t\tsay_color \"\" \" run $test_count: $descr ($*)\"\n+\t\tsay_color \"\" \"# run $test_count: $descr ($*)\"\n \t\t# Export TEST_DIRECTORY, TRASH_DIRECTORY and GIT_TEST_LONG\n \t\t# to be able to use them in script\n \t\texport TEST_DIRECTORY TRASH_DIRECTORY GIT_TEST_LONG\n@@ -481,9 +483,19 @@ test_external () {\n \t\t\"$@\" 2>&4\n \t\tif [ \"$?\" = 0 ]\n \t\tthen\n-\t\t\ttest_ok_ \"$descr\"\n+\t\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\t\ttest_ok_ \"$descr\"\n+\t\t\telse\n+\t\t\t\tsay_color \"\" \"# test_external test $descr was ok\"\n+\t\t\t\ttest_success=$(($test_success + 1))\n+\t\t\tfi\n \t\telse\n-\t\t\ttest_failure_ \"$descr\" \"$@\"\n+\t\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\t\ttest_failure_ \"$descr\" \"$@\"\n+\t\t\telse\n+\t\t\t\tsay_color error \"# test_external test $descr failed: $@\"\n+\t\t\t\ttest_failure=$(($test_failure + 1))\n+\t\t\tfi\n \t\tfi\n \tfi\n }\n@@ -499,19 +511,30 @@ test_external_without_stderr () {\n \t[ -f \"$stderr\" ] || error \"Internal error: $stderr disappeared.\"\n \tdescr=\"no stderr: $1\"\n \tshift\n-\tsay >&3 \"expecting no stderr from previous command\"\n+\tsay >&3 \"# expecting no stderr from previous command\"\n \tif [ ! -s \"$stderr\" ]; then\n \t\trm \"$stderr\"\n-\t\ttest_ok_ \"$descr\"\n+\n+\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\ttest_ok_ \"$descr\"\n+\t\telse\n+\t\t\tsay_color \"\" \"# test_external_without_stderr test $descr was ok\"\n+\t\t\ttest_success=$(($test_success + 1))\n+\t\tfi\n \telse\n \t\tif [ \"$verbose\" = t ]; then\n-\t\t\toutput=`echo; echo Stderr is:; cat \"$stderr\"`\n+\t\t\toutput=`echo; echo \"# Stderr is:\"; cat \"$stderr\"`\n \t\telse\n \t\t\toutput=\n \t\tfi\n \t\t# rm first in case test_failure exits.\n \t\trm \"$stderr\"\n-\t\ttest_failure_ \"$descr\" \"$@\" \"$output\"\n+\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\ttest_failure_ \"$descr\" \"$@\" \"$output\"\n+\t\telse\n+\t\t\tsay_color error \"# test_external_without_stderr test $descr failed: $@: $output\"\n+\t\t\ttest_failure=$(($test_failure + 1))\n+\t\tfi\n \tfi\n }\n \n@@ -634,8 +657,10 @@ test_done () {\n \t\t# Maybe print SKIP message\n \t\t[ -z \"$skip_all\" ] || skip_all=\" # SKIP $skip_all\"\n \n-\t\tsay_color pass \"# passed all $msg\"\n-\t\tsay \"1..$test_count$skip_all\"\n+\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\tsay_color pass \"# passed all $msg\"\n+\t\t\tsay \"1..$test_count$skip_all\"\n+\t\tfi\n \n \t\ttest -d \"$remove_trash\" &&\n \t\tcd \"$(dirname \"$remove_trash\")\" &&\n@@ -644,8 +669,10 @@ test_done () {\n \t\texit 0 ;;\n \n \t*)\n-\t\tsay_color error \"# failed $test_failure among $msg\"\n-\t\tsay \"1..$test_count\"\n+\t\tif test $test_external_has_tap -eq 0; then\n+\t\t\tsay_color error \"# failed $test_failure among $msg\"\n+\t\t\tsay \"1..$test_count\"\n+\t\tfi\n \n \t\texit 1 ;;\n \n-- \n1.7.1.251.g92a7\n"},{"id":"144169","messageId":"1277401489-27885-4-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH v4 3/5] test-lib: output a newline before \"ok\" under a TAP harness","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:47Z","receivedAt":"2010-06-24T17:44:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Some tests in the testsuite will emit a line that doesn't end with a\nnewline, right before we're about to output \"ok\" or \"not ok\". This\nbreaks the TAP output with \"Tests out of sequence\" errors since a TAP\nharness can't understand this:\n\n ok 1 - A test\n [some output here]ok 2 - Another test\n ok 3 - Yet another test\n\nWork around it by emitting an empty line before we're about to say\n\"ok\" or \"not ok\", but only if we're running under --verbose and\nHARNESS_ACTIVE=1 is set, which'll only be the case when running under\na harnesses like prove(1).\n\nI think it's better to do this than fix each tests by adding `&& echo'\neverywhere. More tests might be added that break TAP in the future,\nand a human isn't going to look at the extra whitespace, since\nHARNESS_ACTIVE=1 always means a harness is reading it.\n\nThe tests that had issues were:\n\n t1007, t3410, t3413, t3409, t3414, t3415, t3416, t3412, t3404,\n t5407, t7402, t7003, t9001\n\nWith this workaround the entire test suite runs without errors under:\n\n prove -j 10 ./t[0-9]*.sh :: --verbose\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/test-lib.sh | 3 +++\n 1 files changed, 3 insertions(+), 0 deletions(-)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 7077210..ac496aa 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -371,6 +371,9 @@ test_run_ () {\n \teval >&3 2>&4 \"$1\"\n \teval_ret=$?\n \teval >&3 2>&4 \"$test_cleanup\"\n+\tif test \"$verbose\" = \"t\" && test -n \"$HARNESS_ACTIVE\"; then\n+\t\techo \"\"\n+\tfi\n \treturn 0\n }\n \n-- \n1.7.1.251.g92a7\n"},{"id":"144171","messageId":"1277401489-27885-5-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH v4 4/5] tests: Skip tests in a way that makes sense under TAP","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:48Z","receivedAt":"2010-06-24T17:44:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"SKIP messages are now part of the TAP plan. A TAP harness now knows\nwhy a particular test was skipped and can report that information. The\nnon-TAP harness built into Git's test-lib did nothing special with\nthese messages, and is unaffected by these changes.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/lib-git-svn.sh | 4 ++--\n t/lib-httpd.sh | 3 +--\n t/t1304-default-acl.sh | 9 ++++++---\n t/t1509-root-worktree.sh | 6 +++---\n t/t2007-checkout-symlink.sh | 2 +-\n t/t3300-funny-names.sh | 2 +-\n t/t3302-notes-index-expensive.sh | 2 +-\n t/t3600-rm.sh | 2 +-\n t/t3701-add-interactive.sh | 4 ++--\n t/t3902-quoted.sh | 2 +-\n t/t4004-diff-rename-symlink.sh | 2 +-\n t/t4011-diff-symlink.sh | 2 +-\n t/t4016-diff-quote.sh | 2 +-\n t/t4023-diff-rename-typechange.sh | 2 +-\n t/t4114-apply-typechange.sh | 2 +-\n t/t4115-apply-symlink.sh | 2 +-\n t/t4122-apply-symlink-inside.sh | 2 +-\n t/t5302-pack-index.sh | 2 +-\n t/t5503-tagfollow.sh | 2 +-\n t/t5522-pull-symlink.sh | 2 +-\n t/t5540-http-push.sh | 2 +-\n t/t5541-http-push.sh | 2 +-\n t/t5550-http-fetch.sh | 2 +-\n t/t5551-http-fetch.sh | 2 +-\n t/t5561-http-backend.sh | 2 +-\n t/t5705-clone-2gb.sh | 2 +-\n t/t6035-merge-dir-to-symlink.sh | 2 +-\n t/t7004-tag.sh | 2 +-\n t/t7006-pager.sh | 2 +-\n t/t7800-difftool.sh | 2 +-\n t/t9001-send-email.sh | 4 ++--\n t/t9100-git-svn-basic.sh | 2 +-\n t/t9119-git-svn-info.sh | 2 +-\n t/t9129-git-svn-i18n-commitencoding.sh | 2 +-\n t/t9143-git-svn-gc.sh | 2 +-\n t/t9200-git-cvsexportcommit.sh | 4 ++--\n t/t9400-git-cvsserver-server.sh | 6 +++---\n t/t9401-git-cvsserver-crlf.sh | 6 +++---\n t/t9600-cvsimport.sh | 2 +-\n t/t9700-perl-git.sh | 4 ++--\n 40 files changed, 56 insertions(+), 54 deletions(-)\n\ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 0f7f35c..344785d 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -5,11 +5,11 @@ git_svn_id=git\"\"-svn-id\n \n if test -n \"$NO_SVN_TESTS\"\n then\n-\tsay 'skipping git svn tests, NO_SVN_TESTS defined'\n+\tskip_all='skipping git svn tests, NO_SVN_TESTS defined'\n \ttest_done\n fi\n if ! test_have_prereq PERL; then\n-\tsay 'skipping git svn tests, perl not available'\n+\tskip_all='skipping git svn tests, perl not available'\n \ttest_done\n fi\n \ndiff --git a/t/lib-httpd.sh b/t/lib-httpd.sh\nindex da4b8d5..a0944d6 100644\n--- a/t/lib-httpd.sh\n+++ b/t/lib-httpd.sh\n@@ -5,8 +5,7 @@\n \n if test -z \"$GIT_TEST_HTTPD\"\n then\n-\tsay \"skipping test, network testing disabled by default\"\n-\tsay \"(define GIT_TEST_HTTPD to enable)\"\n+\tskip_all=\"Network testing disabled (define GIT_TEST_HTTPD to enable)\"\n \ttest_done\n fi\n \ndiff --git a/t/t1304-default-acl.sh b/t/t1304-default-acl.sh\nindex 055ad00..97ab02a 100755\n--- a/t/t1304-default-acl.sh\n+++ b/t/t1304-default-acl.sh\n@@ -15,9 +15,12 @@ umask 077\n # is a good candidate: exists on all unices, and it has permission\n # anyway, so we don't create a security hole running the testsuite.\n \n-if ! setfacl -m u:root:rwx .; then\n- say \"Skipping ACL tests: unable to use setfacl\"\n- test_done\n+setfacl_out=\"$(setfacl -m u:root:rwx . 2>&1)\"\n+setfacl_ret=$?\n+\n+if [ $setfacl_ret != 0 ]; then\n+\tskip_all=\"Skipping ACL tests: unable to use setfacl (output: '$setfacl_out'; return code: '$setfacl_ret')\"\n+\ttest_done\n fi\n \n check_perms_and_acl () {\ndiff --git a/t/t1509-root-worktree.sh b/t/t1509-root-worktree.sh\nindex 5322a3b..7f60fd0 100755\n--- a/t/t1509-root-worktree.sh\n+++ b/t/t1509-root-worktree.sh\n@@ -99,17 +99,17 @@ test_foobar_foobar() {\n }\n \n if ! test_have_prereq POSIXPERM || ! [ -w / ]; then\n-\tsay \"Dangerous test skipped. Read this test if you want to execute it\"\n+\tskip_all=\"Dangerous test skipped. Read this test if you want to execute it\"\n \ttest_done\n fi\n \n if [ \"$IKNOWWHATIAMDOING\" != \"YES\" ]; then\n-\tsay \"You must set env var IKNOWWHATIAMDOING=YES in order to run this test\"\n+\tskip_all=\"You must set env var IKNOWWHATIAMDOING=YES in order to run this test\"\n \ttest_done\n fi\n \n if [ \"$UID\" = 0 ]; then\n-\tsay \"No you can't run this with root\"\n+\tskip_all=\"No you can't run this with root\"\n \ttest_done\n fi\n \ndiff --git a/t/t2007-checkout-symlink.sh b/t/t2007-checkout-symlink.sh\nindex 27e2127..05cc8fd 100755\n--- a/t/t2007-checkout-symlink.sh\n+++ b/t/t2007-checkout-symlink.sh\n@@ -8,7 +8,7 @@ test_description='git checkout to switch between branches with symlink<->dir'\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay \"symbolic links not supported - skipping tests\"\n+\tskip_all=\"symbolic links not supported - skipping tests\"\n \ttest_done\n fi\n \ndiff --git a/t/t3300-funny-names.sh b/t/t3300-funny-names.sh\nindex db46d53..a99e4d8 100755\n--- a/t/t3300-funny-names.sh\n+++ b/t/t3300-funny-names.sh\n@@ -26,7 +26,7 @@ echo 'Foo Bar Baz' >\"$p2\"\n \n test -f \"$p1\" && cmp \"$p0\" \"$p1\" || {\n \t# since FAT/NTFS does not allow tabs in filenames, skip this test\n-\tsay 'Your filesystem does not allow tabs in filenames, test skipped.'\n+\tskip_all='Your filesystem does not allow tabs in filenames, test skipped.'\n \ttest_done\n }\n \ndiff --git a/t/t3302-notes-index-expensive.sh b/t/t3302-notes-index-expensive.sh\nindex ee84fc4..361a10a 100755\n--- a/t/t3302-notes-index-expensive.sh\n+++ b/t/t3302-notes-index-expensive.sh\n@@ -8,7 +8,7 @@ test_description='Test commit notes index (expensive!)'\n . ./test-lib.sh\n \n test -z \"$GIT_NOTES_TIMING_TESTS\" && {\n-\tsay Skipping timing tests\n+\tskip_all=\"Skipping timing tests\"\n \ttest_done\n \texit\n }\ndiff --git a/t/t3600-rm.sh b/t/t3600-rm.sh\nindex 0aaf0ad..b514cbb 100755\n--- a/t/t3600-rm.sh\n+++ b/t/t3600-rm.sh\n@@ -39,7 +39,7 @@ if test -f test-file\n then\n \ttest_set_prereq RO_DIR\n else\n-\tsay 'skipping removal failure test (perhaps running as root?)'\n+\tskip_all='skipping removal failure test (perhaps running as root?)'\n fi\n chmod 775 .\n rm -f test-file\ndiff --git a/t/t3701-add-interactive.sh b/t/t3701-add-interactive.sh\nindex b6eba6a..7ad8465 100755\n--- a/t/t3701-add-interactive.sh\n+++ b/t/t3701-add-interactive.sh\n@@ -4,7 +4,7 @@ test_description='add -i basic tests'\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping git add -i tests, perl not available'\n+\tskip_all='skipping git add -i tests, perl not available'\n \ttest_done\n fi\n \n@@ -154,7 +154,7 @@ rm -f .gitignore\n \n if test \"$(git config --bool core.filemode)\" = false\n then\n-\tsay 'skipping filemode tests (filesystem does not properly support modes)'\n+\tsay '# skipping filemode tests (filesystem does not properly support modes)'\n else\n \ttest_set_prereq FILEMODE\n fi\ndiff --git a/t/t3902-quoted.sh b/t/t3902-quoted.sh\nindex 29103f6..147e634 100755\n--- a/t/t3902-quoted.sh\n+++ b/t/t3902-quoted.sh\n@@ -17,7 +17,7 @@ DQ='\"'\n echo foo 2>/dev/null > \"Name and an${HT}HT\"\n test -f \"Name and an${HT}HT\" || {\n \t# since FAT/NTFS does not allow tabs in filenames, skip this test\n-\tsay 'Your filesystem does not allow tabs in filenames, test skipped.'\n+\tskip_all='Your filesystem does not allow tabs in filenames, test skipped.'\n \ttest_done\n }\n \ndiff --git a/t/t4004-diff-rename-symlink.sh b/t/t4004-diff-rename-symlink.sh\nindex a4da119..1a09e8d 100755\n--- a/t/t4004-diff-rename-symlink.sh\n+++ b/t/t4004-diff-rename-symlink.sh\n@@ -14,7 +14,7 @@ by an edit for them.\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t4011-diff-symlink.sh b/t/t4011-diff-symlink.sh\nindex e12fbea..918a21a 100755\n--- a/t/t4011-diff-symlink.sh\n+++ b/t/t4011-diff-symlink.sh\n@@ -11,7 +11,7 @@ test_description='Test diff of symlinks.\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t4016-diff-quote.sh b/t/t4016-diff-quote.sh\nindex 55eb5f8..34e5144 100755\n--- a/t/t4016-diff-quote.sh\n+++ b/t/t4016-diff-quote.sh\n@@ -14,7 +14,7 @@ P2='pathname with SP'\n P3='pathname\n with LF'\n : 2>/dev/null >\"$P1\" && test -f \"$P1\" && rm -f \"$P1\" || {\n-\tsay 'Your filesystem does not allow tabs in filenames, test skipped.'\n+\tskip_all='Your filesystem does not allow tabs in filenames, test skipped.'\n \ttest_done\n }\n \ndiff --git a/t/t4023-diff-rename-typechange.sh b/t/t4023-diff-rename-typechange.sh\nindex 9bdf659..40a95a1 100755\n--- a/t/t4023-diff-rename-typechange.sh\n+++ b/t/t4023-diff-rename-typechange.sh\n@@ -6,7 +6,7 @@ test_description='typechange rename detection'\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t4114-apply-typechange.sh b/t/t4114-apply-typechange.sh\nindex 99ec13d..164d58c 100755\n--- a/t/t4114-apply-typechange.sh\n+++ b/t/t4114-apply-typechange.sh\n@@ -11,7 +11,7 @@ test_description='git apply should not get confused with type changes.\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t4115-apply-symlink.sh b/t/t4115-apply-symlink.sh\nindex b852e58..aff4348 100755\n--- a/t/t4115-apply-symlink.sh\n+++ b/t/t4115-apply-symlink.sh\n@@ -11,7 +11,7 @@ test_description='git apply symlinks and partial files\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t4122-apply-symlink-inside.sh b/t/t4122-apply-symlink-inside.sh\nindex 0d3c1d5..923fcab 100755\n--- a/t/t4122-apply-symlink-inside.sh\n+++ b/t/t4122-apply-symlink-inside.sh\n@@ -5,7 +5,7 @@ test_description='apply to deeper directory without getting fooled with symlink'\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh\nindex 4360e77..fb3a270 100755\n--- a/t/t5302-pack-index.sh\n+++ b/t/t5302-pack-index.sh\n@@ -74,7 +74,7 @@ if msg=$(git verify-pack -v \"test-3-${pack3}.pack\" 2>&1) ||\n then\n \ttest_set_prereq OFF64_T\n else\n-\tsay \"skipping tests concerning 64-bit offsets\"\n+\tsay \"# skipping tests concerning 64-bit offsets\"\n fi\n \n test_expect_success OFF64_T \\\ndiff --git a/t/t5503-tagfollow.sh b/t/t5503-tagfollow.sh\nindex d5db75d..bab1a53 100755\n--- a/t/t5503-tagfollow.sh\n+++ b/t/t5503-tagfollow.sh\n@@ -6,7 +6,7 @@ test_description='test automatic tag following'\n \n case $(uname -s) in\n *MINGW*)\n-\tsay \"GIT_DEBUG_SEND_PACK not supported - skipping tests\"\n+\tskip_all=\"GIT_DEBUG_SEND_PACK not supported - skipping tests\"\n \ttest_done\n esac\n \ndiff --git a/t/t5522-pull-symlink.sh b/t/t5522-pull-symlink.sh\nindex 7206817..298200f 100755\n--- a/t/t5522-pull-symlink.sh\n+++ b/t/t5522-pull-symlink.sh\n@@ -6,7 +6,7 @@ test_description='pulling from symlinked subdir'\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t5540-http-push.sh b/t/t5540-http-push.sh\nindex 37fe875..a266ca5 100755\n--- a/t/t5540-http-push.sh\n+++ b/t/t5540-http-push.sh\n@@ -11,7 +11,7 @@ This test runs various sanity checks on http-push.'\n \n if git http-push > /dev/null 2>&1 || [ $? -eq 128 ]\n then\n-\tsay \"skipping test, USE_CURL_MULTI is not defined\"\n+\tskip_all=\"skipping test, USE_CURL_MULTI is not defined\"\n \ttest_done\n fi\n \ndiff --git a/t/t5541-http-push.sh b/t/t5541-http-push.sh\nindex 17e1bdc..504884b 100755\n--- a/t/t5541-http-push.sh\n+++ b/t/t5541-http-push.sh\n@@ -7,7 +7,7 @@ test_description='test smart pushing over http via http-backend'\n . ./test-lib.sh\n \n if test -n \"$NO_CURL\"; then\n-\tsay 'skipping test, git built without http support'\n+\tskip_all='skipping test, git built without http support'\n \ttest_done\n fi\n \ndiff --git a/t/t5550-http-fetch.sh b/t/t5550-http-fetch.sh\nindex fc675b5..2fb48d0 100755\n--- a/t/t5550-http-fetch.sh\n+++ b/t/t5550-http-fetch.sh\n@@ -4,7 +4,7 @@ test_description='test dumb fetching over http via static file'\n . ./test-lib.sh\n \n if test -n \"$NO_CURL\"; then\n-\tsay 'skipping test, git built without http support'\n+\tskip_all='skipping test, git built without http support'\n \ttest_done\n fi\n \ndiff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh\nindex 7faa31a..fd19121 100755\n--- a/t/t5551-http-fetch.sh\n+++ b/t/t5551-http-fetch.sh\n@@ -4,7 +4,7 @@ test_description='test smart fetching over http via http-backend'\n . ./test-lib.sh\n \n if test -n \"$NO_CURL\"; then\n-\tsay 'skipping test, git built without http support'\n+\tskip_all='skipping test, git built without http support'\n \ttest_done\n fi\n \ndiff --git a/t/t5561-http-backend.sh b/t/t5561-http-backend.sh\nindex 8c6d0b2..b5d7fbc 100755\n--- a/t/t5561-http-backend.sh\n+++ b/t/t5561-http-backend.sh\n@@ -4,7 +4,7 @@ test_description='test git-http-backend'\n . ./test-lib.sh\n \n if test -n \"$NO_CURL\"; then\n-\tsay 'skipping test, git built without http support'\n+\tskip_all='skipping test, git built without http support'\n \ttest_done\n fi\n \ndiff --git a/t/t5705-clone-2gb.sh b/t/t5705-clone-2gb.sh\nindex 8afbdd4..e4d1b6a 100755\n--- a/t/t5705-clone-2gb.sh\n+++ b/t/t5705-clone-2gb.sh\n@@ -4,7 +4,7 @@ test_description='Test cloning a repository larger than 2 gigabyte'\n . ./test-lib.sh\n \n test -z \"$GIT_TEST_CLONE_2GB\" &&\n-say \"Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t\" &&\n+skip_all=\"Skipping expensive 2GB clone test; enable it with GIT_TEST_CLONE_2GB=t\" &&\n test_done &&\n exit\n \ndiff --git a/t/t6035-merge-dir-to-symlink.sh b/t/t6035-merge-dir-to-symlink.sh\nindex 3202e1d..cd3190c 100755\n--- a/t/t6035-merge-dir-to-symlink.sh\n+++ b/t/t6035-merge-dir-to-symlink.sh\n@@ -5,7 +5,7 @@ test_description='merging when a directory was replaced with a symlink'\n \n if ! test_have_prereq SYMLINKS\n then\n-\tsay 'Symbolic links not supported, skipping tests.'\n+\tskip_all='Symbolic links not supported, skipping tests.'\n \ttest_done\n fi\n \ndiff --git a/t/t7004-tag.sh b/t/t7004-tag.sh\nindex 73dbc43..ac943f5 100755\n--- a/t/t7004-tag.sh\n+++ b/t/t7004-tag.sh\n@@ -583,7 +583,7 @@ test_expect_success \\\n # subsequent tests require gpg; check if it is available\n gpg --version >/dev/null 2>/dev/null\n if [ $? -eq 127 ]; then\n-\tsay \"gpg not found - skipping tag signing and verification tests\"\n+\tsay \"# gpg not found - skipping tag signing and verification tests\"\n else\n \t# As said here: http://www.gnupg.org/documentation/faqs.html#q6.19\n \t# the gpg version 1.0.6 didn't parse trust packets correctly, so for\ndiff --git a/t/t7006-pager.sh b/t/t7006-pager.sh\nindex c2a3c8e..eb9651d 100755\n--- a/t/t7006-pager.sh\n+++ b/t/t7006-pager.sh\n@@ -37,7 +37,7 @@ then\n \t}\n \ttest_set_prereq TTY\n else\n-\tsay no usable terminal, so skipping some tests\n+\tsay \"# no usable terminal, so skipping some tests\"\n fi\n \n test_expect_success 'setup' '\ndiff --git a/t/t7800-difftool.sh b/t/t7800-difftool.sh\nindex 1de83ef..196827e 100755\n--- a/t/t7800-difftool.sh\n+++ b/t/t7800-difftool.sh\n@@ -11,7 +11,7 @@ Testing basic diff tool invocation\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping difftool tests, perl not available'\n+\tskip_all='skipping difftool tests, perl not available'\n \ttest_done\n fi\n \ndiff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh\nindex 0b8a591..37610db 100755\n--- a/t/t9001-send-email.sh\n+++ b/t/t9001-send-email.sh\n@@ -4,7 +4,7 @@ test_description='git send-email'\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping git send-email tests, perl not available'\n+\tskip_all='skipping git send-email tests, perl not available'\n \ttest_done\n fi\n \n@@ -58,7 +58,7 @@ test_no_confirm () {\n # Exit immediately to prevent hang if a no-confirm test fails\n check_no_confirm () {\n \ttest -f no_confirm_okay || {\n-\t\tsay 'No confirm test failed; skipping remaining tests to prevent hanging'\n+\t\tskip_all='confirm test failed; skipping remaining tests to prevent hanging'\n \t\ttest_done\n \t}\n }\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 570e035..13766ab 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -15,7 +15,7 @@ case \"$GIT_SVN_LC_ALL\" in\n \ttest_set_prereq UTF8\n \t;;\n *)\n-\tsay \"UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n+\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n \t;;\n esac\n \ndiff --git a/t/t9119-git-svn-info.sh b/t/t9119-git-svn-info.sh\nindex a9a558d..5fb94fb 100755\n--- a/t/t9119-git-svn-info.sh\n+++ b/t/t9119-git-svn-info.sh\n@@ -13,7 +13,7 @@ case $v in\n 1.[456].*)\n \t;;\n *)\n-\tsay \"skipping svn-info test (SVN version: $v not supported)\"\n+\tskip_all=\"skipping svn-info test (SVN version: $v not supported)\"\n \ttest_done\n \t;;\n esac\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 1e9a2eb..8cfdfe7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -23,7 +23,7 @@ if test -n \"$a_utf8_locale\"\n then\n \ttest_set_prereq UTF8\n else\n-\tsay \"UTF-8 locale not available, some tests are skipped\"\n+\tsay \"# UTF-8 locale not available, some tests are skipped\"\n fi\n \n compare_svn_head_with () {\ndiff --git a/t/t9143-git-svn-gc.sh b/t/t9143-git-svn-gc.sh\nindex 99f69c6..337ea59 100755\n--- a/t/t9143-git-svn-gc.sh\n+++ b/t/t9143-git-svn-gc.sh\n@@ -43,7 +43,7 @@ then\n \t\t gunzip .git/svn/refs/remotes/git-svn/unhandled.log.gz\n \t\t'\n else\n-\tsay \"Perl Compress::Zlib unavailable, skipping gunzip test\"\n+\tsay \"# Perl Compress::Zlib unavailable, skipping gunzip test\"\n fi\n \n test_expect_success 'git svn gc does not change unhandled.log files' '\ndiff --git a/t/t9200-git-cvsexportcommit.sh b/t/t9200-git-cvsexportcommit.sh\nindex 61bcb8f..ee39b36 100755\n--- a/t/t9200-git-cvsexportcommit.sh\n+++ b/t/t9200-git-cvsexportcommit.sh\n@@ -7,14 +7,14 @@ test_description='Test export of commits to CVS'\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping git cvsexportcommit tests, perl not available'\n+\tskip_all='skipping git cvsexportcommit tests, perl not available'\n \ttest_done\n fi\n \n cvs >/dev/null 2>&1\n if test $? -ne 1\n then\n- say 'skipping git cvsexportcommit tests, cvs not found'\n+ skip_all='skipping git cvsexportcommit tests, cvs not found'\n test_done\n fi\n \ndiff --git a/t/t9400-git-cvsserver-server.sh b/t/t9400-git-cvsserver-server.sh\nindex 8639506..36c457e 100755\n--- a/t/t9400-git-cvsserver-server.sh\n+++ b/t/t9400-git-cvsserver-server.sh\n@@ -11,17 +11,17 @@ cvs CLI client via git-cvsserver server'\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping git cvsserver tests, perl not available'\n+\tskip_all='skipping git cvsserver tests, perl not available'\n \ttest_done\n fi\n cvs >/dev/null 2>&1\n if test $? -ne 1\n then\n- say 'skipping git-cvsserver tests, cvs not found'\n+ skip_all='skipping git-cvsserver tests, cvs not found'\n test_done\n fi\n \"$PERL_PATH\" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {\n- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'\n+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'\n test_done\n }\n \ndiff --git a/t/t9401-git-cvsserver-crlf.sh b/t/t9401-git-cvsserver-crlf.sh\nindex ed7b513..925bd0f 100755\n--- a/t/t9401-git-cvsserver-crlf.sh\n+++ b/t/t9401-git-cvsserver-crlf.sh\n@@ -41,16 +41,16 @@ not_present() {\n cvs >/dev/null 2>&1\n if test $? -ne 1\n then\n- say 'skipping git-cvsserver tests, cvs not found'\n+ skip_all='skipping git-cvsserver tests, cvs not found'\n test_done\n fi\n if ! test_have_prereq PERL\n then\n- say 'skipping git-cvsserver tests, perl not available'\n+ skip_all='skipping git-cvsserver tests, perl not available'\n test_done\n fi\n \"$PERL_PATH\" -e 'use DBI; use DBD::SQLite' >/dev/null 2>&1 || {\n- say 'skipping git-cvsserver tests, Perl SQLite interface unavailable'\n+ skip_all='skipping git-cvsserver tests, Perl SQLite interface unavailable'\n test_done\n }\n \ndiff --git a/t/t9600-cvsimport.sh b/t/t9600-cvsimport.sh\nindex b572ce3..2eff9cd 100755\n--- a/t/t9600-cvsimport.sh\n+++ b/t/t9600-cvsimport.sh\n@@ -4,7 +4,7 @@ test_description='git cvsimport basic tests'\n . ./lib-cvs.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping git cvsimport tests, perl not available'\n+\tskip_all='skipping git cvsimport tests, perl not available'\n \ttest_done\n fi\n \ndiff --git a/t/t9700-perl-git.sh b/t/t9700-perl-git.sh\nindex 0b92726..3787186 100755\n--- a/t/t9700-perl-git.sh\n+++ b/t/t9700-perl-git.sh\n@@ -7,12 +7,12 @@ test_description='perl interface (Git.pm)'\n . ./test-lib.sh\n \n if ! test_have_prereq PERL; then\n-\tsay 'skipping perl interface tests, perl not available'\n+\tskip_all='skipping perl interface tests, perl not available'\n \ttest_done\n fi\n \n \"$PERL_PATH\" -MTest::More -e 0 2>/dev/null || {\n-\tsay \"Perl Test::More unavailable, skipping test\"\n+\tskip_all=\"Perl Test::More unavailable, skipping test\"\n \ttest_done\n }\n \n-- \n1.7.1.251.g92a7\n"},{"id":"144170","messageId":"1277401489-27885-6-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH v4 5/5] tests: Say \"pass\" rather than \"ok\" on empty lines for TAP","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T17:44:49Z","receivedAt":"2010-06-24T17:44:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Lines that begin with \"ok\" confuse the TAP harness because it can't\ndistinguish them from a test counter. Work around the issue by saying\n\"pass\" instead, which isn't a reserved TAP word.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t1020-subdirectory.sh | 12 ++++++------\n t/t2102-update-index-symlinks.sh | 2 +-\n t/t3700-add.sh | 12 ++++++------\n 3 files changed, 13 insertions(+), 13 deletions(-)\n\ndiff --git a/t/t1020-subdirectory.sh b/t/t1020-subdirectory.sh\nindex 210e594..5687499 100755\n--- a/t/t1020-subdirectory.sh\n+++ b/t/t1020-subdirectory.sh\n@@ -24,18 +24,18 @@ test_expect_success 'update-index and ls-files' '\n \tcd \"$HERE\" &&\n \tgit update-index --add one &&\n \tcase \"`git ls-files`\" in\n-\tone) echo ok one ;;\n+\tone) echo pass one ;;\n \t*) echo bad one; exit 1 ;;\n \tesac &&\n \tcd dir &&\n \tgit update-index --add two &&\n \tcase \"`git ls-files`\" in\n-\ttwo) echo ok two ;;\n+\ttwo) echo pass two ;;\n \t*) echo bad two; exit 1 ;;\n \tesac &&\n \tcd .. &&\n \tcase \"`git ls-files`\" in\n-\tdir/two\"$LF\"one) echo ok both ;;\n+\tdir/two\"$LF\"one) echo pass both ;;\n \t*) echo bad; exit 1 ;;\n \tesac\n '\n@@ -58,17 +58,17 @@ test_expect_success 'diff-files' '\n \techo a >>one &&\n \techo d >>dir/two &&\n \tcase \"`git diff-files --name-only`\" in\n-\tdir/two\"$LF\"one) echo ok top ;;\n+\tdir/two\"$LF\"one) echo pass top ;;\n \t*) echo bad top; exit 1 ;;\n \tesac &&\n \t# diff should not omit leading paths\n \tcd dir &&\n \tcase \"`git diff-files --name-only`\" in\n-\tdir/two\"$LF\"one) echo ok subdir ;;\n+\tdir/two\"$LF\"one) echo pass subdir ;;\n \t*) echo bad subdir; exit 1 ;;\n \tesac &&\n \tcase \"`git diff-files --name-only .`\" in\n-\tdir/two) echo ok subdir limited ;;\n+\tdir/two) echo pass subdir limited ;;\n \t*) echo bad subdir limited; exit 1 ;;\n \tesac\n '\ndiff --git a/t/t2102-update-index-symlinks.sh b/t/t2102-update-index-symlinks.sh\nindex 1ed44ee..4d0d0a3 100755\n--- a/t/t2102-update-index-symlinks.sh\n+++ b/t/t2102-update-index-symlinks.sh\n@@ -24,7 +24,7 @@ git update-index symlink'\n test_expect_success \\\n 'the index entry must still be a symbolic link' '\n case \"`git ls-files --stage --cached symlink`\" in\n-120000\" \"*symlink) echo ok;;\n+120000\" \"*symlink) echo pass;;\n *) echo fail; git ls-files --stage --cached symlink; (exit 1);;\n esac'\n \ndiff --git a/t/t3700-add.sh b/t/t3700-add.sh\nindex 525c9a8..6f031af 100755\n--- a/t/t3700-add.sh\n+++ b/t/t3700-add.sh\n@@ -26,7 +26,7 @@ test_expect_success \\\n \t chmod 755 xfoo1 &&\n \t git add xfoo1 &&\n \t case \"`git ls-files --stage xfoo1`\" in\n-\t 100644\" \"*xfoo1) echo ok;;\n+\t 100644\" \"*xfoo1) echo pass;;\n \t *) echo fail; git ls-files --stage xfoo1; (exit 1);;\n \t esac'\n \n@@ -35,7 +35,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym\n \tln -s foo xfoo1 &&\n \tgit add xfoo1 &&\n \tcase \"`git ls-files --stage xfoo1`\" in\n-\t120000\" \"*xfoo1) echo ok;;\n+\t120000\" \"*xfoo1) echo pass;;\n \t*) echo fail; git ls-files --stage xfoo1; (exit 1);;\n \tesac\n '\n@@ -47,7 +47,7 @@ test_expect_success \\\n \t chmod 755 xfoo2 &&\n \t git update-index --add xfoo2 &&\n \t case \"`git ls-files --stage xfoo2`\" in\n-\t 100644\" \"*xfoo2) echo ok;;\n+\t 100644\" \"*xfoo2) echo pass;;\n \t *) echo fail; git ls-files --stage xfoo2; (exit 1);;\n \t esac'\n \n@@ -56,7 +56,7 @@ test_expect_success SYMLINKS 'git add: filemode=0 should not get confused by sym\n \tln -s foo xfoo2 &&\n \tgit update-index --add xfoo2 &&\n \tcase \"`git ls-files --stage xfoo2`\" in\n-\t120000\" \"*xfoo2) echo ok;;\n+\t120000\" \"*xfoo2) echo pass;;\n \t*) echo fail; git ls-files --stage xfoo2; (exit 1);;\n \tesac\n '\n@@ -67,7 +67,7 @@ test_expect_success SYMLINKS \\\n \t ln -s xfoo2 xfoo3 &&\n \t git update-index --add xfoo3 &&\n \t case \"`git ls-files --stage xfoo3`\" in\n-\t 120000\" \"*xfoo3) echo ok;;\n+\t 120000\" \"*xfoo3) echo pass;;\n \t *) echo fail; git ls-files --stage xfoo3; (exit 1);;\n \t esac'\n \n@@ -172,7 +172,7 @@ test_expect_success 'git add --refresh' '\n \ttest -z \"`git diff-index HEAD -- foo`\" &&\n \tgit read-tree HEAD &&\n \tcase \"`git diff-index HEAD -- foo`\" in\n-\t:100644\" \"*\"M\tfoo\") echo ok;;\n+\t:100644\" \"*\"M\tfoo\") echo pass;;\n \t*) echo fail; (exit 1);;\n \tesac &&\n \tgit add --refresh -- foo &&\n-- \n1.7.1.251.g92a7\n"},{"id":"144176","messageId":"m3mxuknumn.fsf@localhost.localdomain","threadId":"24189","inReplyTo":"1277401489-27885-2-git-send-email-avarab@gmail.com","subject":"Re: [PATCH v4 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2010-06-24T18:39:31Z","receivedAt":"2010-06-24T18:39:31Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> TAP, the Test Anything Protocol, is a simple text-based interface\n> between testing modules in a test harness. test-lib.sh's output was\n> already very close to being valid TAP. This change brings it all the\n> way there. Before:\n> \n> $ ./t0005-signals.sh\n> * ok 1: sigchain works\n> * passed all 1 test(s)\n> \n> And after:\n> \n> $ ./t0005-signals.sh\n> ok 1 - sigchain works\n> # passed all 1 test(s)\n> 1..1\n\nThis is incomplete information. It only tells us how the output is\nchanged for the case when all test_expect_success tests passes. This\ncommit message doesn't tell us how failing tests looks like, and how\ntest_expect_failure results gets translated.\n\nTake for example a following test script:\n-- >8 --\n#!/bin/sh\n\ntest_description='this is a sample test.\n\nThis test is here to see various test outputs.'\n\n. ./test-lib.sh\n\nsay 'diagnostic message'\n\ntest_expect_success 'true test' 'true'\ntest_expect_success 'false test' 'false'\n\ntest_expect_failure 'true test (todo)' 'true'\ntest_expect_failure 'false test (todo)' 'false'\n\ntest_debug 'echo \"debug message\"'\n\ntest_done\n-- 8< --\n\nThis test script output looks like the following (the comments are not\npart of output, but denote color of given line of test output):\n----\n* diagnostic message # yellow\n* ok 1: true test\n* FAIL 2: false test # bold red\n false\n* FIXED 3: true test (todo)\n* still broken 4: false test (todo) # bold green\n* fixed 1 known breakage(s) # green\n* still have 1 known breakage(s) # bold red\n* failed 1 among remaining 3 test(s) # bold red\n----\n\nHow would the output of this test look like after TAP-ification?\nWould it still provide color output when run on terminal?\n\nThis test also generates summary of test run in a file in\ntest-results/ subdirectory. Currently such file can be e.g. named\ntest-results/test_test-8030, and consist of\n-- >8 --\ntotal 4\nsuccess 1\nfixed 1\nbroken 1\nfailed 1\n\n-- 8< --\n\nWould TAP-ification change that? Even if there is no change, this is\nworth a sentence or a few words in a commit message.\n\nP.S. With current output pass / fail results of test_expect_failure\nare justified on ':' separator. This is not the case of TAP output.\n-- \nJakub Narebski\nPoland\nShadeHawk on #git\n"},{"id":"144180","messageId":"AANLkTinVp6kivNzbdUj-7EXjKrolBqvtsNROo-dCSydn@mail.gmail.com","threadId":"24189","inReplyTo":"m3mxuknumn.fsf@localhost.localdomain","subject":"Re: [PATCH v4 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T19:17:55Z","receivedAt":"2010-06-24T19:17:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, Jun 24, 2010 at 18:39, Jakub Narebski wrote:\n> Ævar Arnfjörð Bjarmason   writes:\n>\n>> TAP, the Test Anything Protocol, is a simple text-based interface\n>> between testing modules in a test harness. test-lib.sh's output was\n>> already very close to being valid TAP. This change brings it all the\n>> way there. Before:\n>>\n>>    $ ./t0005-signals.sh\n>>    *   ok 1: sigchain works\n>>    * passed all 1 test(s)\n>>\n>> And after:\n>>\n>>    $ ./t0005-signals.sh\n>>    ok 1 - sigchain works\n>>    # passed all 1 test(s)\n>>    1..1\n>\n> This is incomplete information.  It only tells us how the output is\n> changed for the case when all test_expect_success tests passes.  This\n> commit message doesn't tell us how failing tests looks like, and how\n> test_expect_failure results gets translated.\n>\n> Take for example a following test script:\n> -- >8 --\n> #!/bin/sh\n>\n> test_description='this is a sample test.\n>\n> This test is here to see various test outputs.'\n>\n> . ./test-lib.sh\n>\n> say 'diagnostic message'\n>\n> test_expect_success 'true  test' 'true'\n> test_expect_success 'false test' 'false'\n>\n> test_expect_failure 'true  test (todo)' 'true'\n> test_expect_failure 'false test (todo)' 'false'\n>\n> test_debug 'echo \"debug message\"'\n>\n> test_done\n> -- 8< --\n>\n> This test script output looks like the following (the comments are not\n> part of output, but denote color of given line of test output):\n> ----\n> * diagnostic message                      # yellow\n> *   ok 1: true  test\n> * FAIL 2: false test                      # bold red\n>        false\n> *   FIXED 3: true  test (todo)\n> *   still broken 4: false test (todo)     # bold green\n> * fixed 1 known breakage(s)               # green\n> * still have 1 known breakage(s)          # bold red\n> * failed 1 among remaining 3 test(s)      # bold red\n> ----\n>\n> How would the output of this test look like after TAP-ification?\n> Would it still provide color output when run on terminal?\n>\n> This test also generates summary of test run in a file in\n> test-results/ subdirectory.  Currently such file can be e.g. named\n> test-results/test_test-8030, and consist of\n> -- >8 --\n> total 4\n> success 1\n> fixed 1\n> broken 1\n> failed 1\n>\n> -- 8< --\n>\n> Would TAP-ification change that?  Even if there is no change, this is\n> worth a sentence or a few words in a commit message.\n\nTo all of the above: I could be more verbose in the commit message in\nneeded, but after you raised similar questions in a previous version\nof this series I added this to the above commit message:\n\n All this patch does is re-arrange the output a bit so that it conforms\n with the TAP spec, everything that the test suite did before continues\n to work. That includes aggregating results in t/test-results/, the\n --verbose, --debug and other options for tests, and the test color\n output.\n\nI.e. aside from turning \"FAIL\" into \"not ok\" and removing the \"* \"\nprefix from lines everything else works exactly as before, that\nincludes color output and the aggregation in t/test-results/.\n\nI can expand on that paragraph with more examples if you think it's necessary.\n\nHere's how the output from the above test script looks under TAP:\n\n diagnostic message\n ok 1 - true test\n not ok - 2 false test\n # false\n ok 3 - true test (todo) # TODO known breakage\n not ok 4 - false test (todo) # TODO known breakage\n # fixed 1 known breakage(s)\n # still have 1 known breakage(s)\n # failed 1 among remaining 3 test(s)\n 1..4\n\n> P.S. With current output pass / fail results of test_expect_failure\n> are justified on ':' separator.  This is not the case of TAP output.\n\nTAP output is conventionally not justified, unlike the existing\noutput. I don't know if anything would break if it were, i.e.:\n\n ok 1 - foo\n not ok 2 - bar\n"},{"id":"144213","messageId":"1277416332-12072-1-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"AANLkTinVp6kivNzbdUj-7EXjKrolBqvtsNROo-dCSydn@mail.gmail.com","subject":"[PATCH v5 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-24T21:52:12Z","receivedAt":"2010-06-24T21:52:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"TAP, the Test Anything Protocol, is a simple text-based interface\nbetween testing modules in a test harness. test-lib.sh's output was\nalready very close to being valid TAP. This change brings it all the\nway there. Before:\n\n $ ./t0005-signals.sh\n * ok 1: sigchain works\n * passed all 1 test(s)\n\nAnd after:\n\n $ ./t0005-signals.sh\n ok 1 - sigchain works\n # passed all 1 test(s)\n 1..1\n\nThe advantage of using TAP is that any program that reads the format\n(a \"test harness\") can run the tests. The most popular of these is the\nprove(1) utility that comes with Perl. It can run tests in parallel,\ndisplay colored output, format the output to console, file, HTML etc.,\nand much more. An example:\n\n $ prove ./t0005-signals.sh\n ./t0005-signals.sh .. ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.01 cusr 0.02 csys = 0.06 CPU)\n Result: PASS\n\nprove(1) gives you human readable output without being too\nverbose. Running the test suite in parallel with `make test -j15`\nproduces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`\nmakes it easy to follow what's going on.\n\nAll this patch does is re-arrange the output a bit so that it conforms\nwith the TAP spec, everything that the test suite did before continues\nto work. That includes aggregating results in t/test-results/, the\n--verbose, --debug and other options for tests, and the test color\noutput.\n\nTAP harnesses ignore everything that they don't know about, so running\nthe tests with --verbose works:\n\n $ prove ./t0005-signals.sh :: --verbose --debug\n ./t0005-signals.sh .. Terminated\n ./t0005-signals.sh .. ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.01 cusr 0.01 csys = 0.05 CPU)\n Result: PASS\n\nJust supply the -v option to prove itself to get all the verbose\noutput that it suppresses:\n\n $ prove -v ./t0005-signals.sh :: --verbose --debug\n ./t0005-signals.sh ..\n Initialized empty Git repository in /home/avar/g/git/t/trash directory.t0005-signals/.git/\n expecting success:\n test-sigchain >actual\n case \"$?\" in\n 143) true ;; # POSIX w/ SIGTERM=15\n 3) true ;; # Windows\n *) false ;;\n esac &&\n test_cmp expect actual\n Terminated\n ok 1 - sigchain works\n # passed all 1 test(s)\n 1..1\n ok\n All tests successful.\n Files=1, Tests=1, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.01 cusr 0.01 csys = 0.04 CPU)\n Result: PASS\n\nAs a further example, consider this test script that uses a lot of\ntest-lib.sh features by Jakub Narebski:\n\n #!/bin/sh\n\n test_description='this is a sample test.\n\n This test is here to see various test outputs.'\n\n . ./test-lib.sh\n\n say 'diagnostic message'\n\n test_expect_success 'true test' 'true'\n test_expect_success 'false test' 'false'\n\n test_expect_failure 'true test (todo)' 'true'\n test_expect_failure 'false test (todo)' 'false'\n\n test_debug 'echo \"debug message\"'\n\n test_done\n\nThe output of that was previously:\n\n * diagnostic message # yellow\n * ok 1: true test\n * FAIL 2: false test # bold red\n false\n * FIXED 3: true test (todo)\n * still broken 4: false test (todo) # bold green\n * fixed 1 known breakage(s) # green\n * still have 1 known breakage(s) # bold red\n * failed 1 among remaining 3 test(s) # bold red\n\nBut is now:\n\n diagnostic message # yellow\n ok 1 - true test\n not ok - 2 false test # bold red\n # false\n ok 3 - true test (todo) # TODO known breakage\n not ok 4 - false test (todo) # TODO known breakage # bold green\n # fixed 1 known breakage(s) # green\n # still have 1 known breakage(s) # bold red\n # failed 1 among remaining 3 test(s) # bold red\n 1..4\n\nAll the coloring is preserved when the test is run manually. Under\nprove(1) the test performs as expected, even with --debug and\n--verbose options:\n\n $ prove ./example.sh :: --debug --verbose\n ./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)\n Failed 1/4 subtests\n (1 TODO test unexpectedly succeeded)\n\n Test Summary Report\n -------------------\n ./example.sh (Wstat: 256 Tests: 4 Failed: 1)\n Failed test: 2\n TODO passed: 3\n Non-zero exit status: 1\n Files=1, Tests=4, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.00 cusr 0.01 csys = 0.03 CPU)\n Result: FAIL\n\nThe TAP harness itself doesn't get confused by the color output, they\naren't used by test-lib.sh stdout isn't open to a terminal (test -t 1).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n\n> To all of the above: I could be more verbose in the commit message in\n> needed.\n\nHere's v5 of this patch which does that. I used the example Jakub\nprovided and explained how it works with regards to colors and\ninteraction with a TAP harness.\n\n t/README | 55 +++++++++++++++++++++++++++++++++++++++----------------\n t/test-lib.sh | 30 ++++++++++++++++++------------\n 2 files changed, 57 insertions(+), 28 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex 0e4e8d8..aa4ed28 100644\n--- a/t/README\n+++ b/t/README\n@@ -18,25 +18,48 @@ The easiest way to run tests is to say \"make\". This runs all\n the tests.\n \n *** t0000-basic.sh ***\n- * ok 1: .git/objects should be empty after git-init in an empty repo.\n- * ok 2: .git/objects should have 256 subdirectories.\n- * ok 3: git-update-index without --add should fail adding.\n+ ok 1 - .git/objects should be empty after git init in an empty repo.\n+ ok 2 - .git/objects should have 3 subdirectories.\n+ ok 3 - success is reported like this\n ...\n- * ok 23: no diff after checkout and git-update-index --refresh.\n- * passed all 23 test(s)\n- *** t0100-environment-names.sh ***\n- * ok 1: using old names should issue warnings.\n- * ok 2: using old names but having new names should not issue warnings.\n- ...\n-\n-Or you can run each test individually from command line, like\n-this:\n-\n- $ sh ./t3001-ls-files-killed.sh\n- * ok 1: git-update-index --add to add various paths.\n- * ok 2: git-ls-files -k to show killed files.\n- * ok 3: validate git-ls-files -k output.\n- * passed all 3 test(s)\n+ ok 43 - very long name in the index handled sanely\n+ # fixed 1 known breakage(s)\n+ # still have 1 known breakage(s)\n+ # passed all remaining 42 test(s)\n+ 1..43\n+ *** t0001-init.sh ***\n+ ok 1 - plain\n+ ok 2 - plain with GIT_WORK_TREE\n+ ok 3 - plain bare\n+\n+Since the tests all output TAP (see http://testanything.org) they can\n+be run with any TAP harness. Here's an example of paralell testing\n+powered by a recent version of prove(1):\n+\n+ $ prove --timer --jobs 15 ./t[0-9]*.sh\n+ [19:17:33] ./t0005-signals.sh ................................... ok 36 ms\n+ [19:17:33] ./t0022-crlf-rename.sh ............................... ok 69 ms\n+ [19:17:33] ./t0024-crlf-archive.sh .............................. ok 154 ms\n+ [19:17:33] ./t0004-unwritable.sh ................................ ok 289 ms\n+ [19:17:33] ./t0002-gitfile.sh ................................... ok 480 ms\n+ ===( 102;0 25/? 6/? 5/? 16/? 1/? 4/? 2/? 1/? 3/? 1... )===\n+\n+prove and other harnesses come with a lot of useful options. The\n+--state option in particular is very useful:\n+\n+ # Repeat until no more failures\n+ $ prove -j 15 --state=failed,save ./t[0-9]*.sh\n+\n+You can also run each test individually from command line, like this:\n+\n+ $ sh ./t3010-ls-files-killed-modified.sh\n+ ok 1 - git update-index --add to add various paths.\n+ ok 2 - git ls-files -k to show killed files.\n+ ok 3 - validate git ls-files -k output.\n+ ok 4 - git ls-files -m to show modified files.\n+ ok 5 - validate git ls-files -m output.\n+ # passed all 5 test(s)\n+ 1..5\n \n You can pass --verbose (or -v), --debug (or -d), and --immediate\n (or -i) command line argument to the test, or by setting GIT_TEST_OPTS\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 367f053..e97645c 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -160,7 +160,7 @@ if test -n \"$color\"; then\n \t\t\t*) test -n \"$quiet\" && return;;\n \t\tesac\n \t\tshift\n-\t\tprintf \"* %s\" \"$*\"\n+\t\tprintf \"%s\" \"$*\"\n \t\ttput sgr0\n \t\techo\n \t\t)\n@@ -169,7 +169,7 @@ else\n \tsay_color() {\n \t\ttest -z \"$1\" && test -n \"$quiet\" && return\n \t\tshift\n-\t\techo \"* $*\"\n+\t\techo \"$*\"\n \t}\n fi\n \n@@ -339,25 +339,25 @@ test_have_prereq () {\n \n test_ok_ () {\n \ttest_success=$(($test_success + 1))\n-\tsay_color \"\" \" ok $test_count: $@\"\n+\tsay_color \"\" \"ok $test_count - $@\"\n }\n \n test_failure_ () {\n \ttest_failure=$(($test_failure + 1))\n-\tsay_color error \"FAIL $test_count: $1\"\n+\tsay_color error \"not ok - $test_count $1\"\n \tshift\n-\techo \"$@\" | sed -e 's/^/\t/'\n+\techo \"$@\" | sed -e 's/^/#\t/'\n \ttest \"$immediate\" = \"\" || { GIT_EXIT_OK=t; exit 1; }\n }\n \n test_known_broken_ok_ () {\n \ttest_fixed=$(($test_fixed+1))\n-\tsay_color \"\" \" FIXED $test_count: $@\"\n+\tsay_color \"\" \"ok $test_count - $@ # TODO known breakage\"\n }\n \n test_known_broken_failure_ () {\n \ttest_broken=$(($test_broken+1))\n-\tsay_color skip \" still broken $test_count: $@\"\n+\tsay_color skip \"not ok $test_count - $@ # TODO known breakage\"\n }\n \n test_debug () {\n@@ -390,7 +390,7 @@ test_skip () {\n \tcase \"$to_skip\" in\n \tt)\n \t\tsay_color skip >&3 \"skipping test: $@\"\n-\t\tsay_color skip \"skip $test_count: $1\"\n+\t\tsay_color skip \"ok $test_count: # skip $1\"\n \t\t: true\n \t\t;;\n \t*)\n@@ -620,18 +620,22 @@ test_done () {\n \n \tif test \"$test_fixed\" != 0\n \tthen\n-\t\tsay_color pass \"fixed $test_fixed known breakage(s)\"\n+\t\tsay_color pass \"# fixed $test_fixed known breakage(s)\"\n \tfi\n \tif test \"$test_broken\" != 0\n \tthen\n-\t\tsay_color error \"still have $test_broken known breakage(s)\"\n+\t\tsay_color error \"# still have $test_broken known breakage(s)\"\n \t\tmsg=\"remaining $(($test_count-$test_broken)) test(s)\"\n \telse\n \t\tmsg=\"$test_count test(s)\"\n \tfi\n \tcase \"$test_failure\" in\n \t0)\n-\t\tsay_color pass \"passed all $msg\"\n+\t\t# Maybe print SKIP message\n+\t\t[ -z \"$skip_all\" ] || skip_all=\" # SKIP $skip_all\"\n+\n+\t\tsay_color pass \"# passed all $msg\"\n+\t\tsay \"1..$test_count$skip_all\"\n \n \t\ttest -d \"$remove_trash\" &&\n \t\tcd \"$(dirname \"$remove_trash\")\" &&\n@@ -640,7 +644,9 @@ test_done () {\n \t\texit 0 ;;\n \n \t*)\n-\t\tsay_color error \"failed $test_failure among $msg\"\n+\t\tsay_color error \"# failed $test_failure among $msg\"\n+\t\tsay \"1..$test_count\"\n+\n \t\texit 1 ;;\n \n \tesac\n-- \n1.7.1.251.g92a7\n"},{"id":"144256","messageId":"7v6317koy2.fsf@alter.siamese.dyndns.org","threadId":"24189","inReplyTo":"1277416332-12072-1-git-send-email-avarab@gmail.com","subject":"Re: [PATCH v5 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-25T17:21:57Z","receivedAt":"2010-06-25T17:21:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> diff --git a/t/README b/t/README\n> index 0e4e8d8..aa4ed28 100644\n> --- a/t/README\n> +++ b/t/README\n> @@ -18,25 +18,48 @@ The easiest way to run tests is to say \"make\". This runs all\n> ...\n\nThis describes the general framework of tests from the point of view of a\nperson who runs them and reads their output well. I'll replace ab/tap\ntopic with these five patches.\n\nI'd however like to ask you to update the sections in t/README to help\ntest writers. You have gained enough experience with patches 4 and 5 (and\nwith patch 2 for Perl tests) to know what kind of script constructs, used\nby people who are used to the old way of writing our tests, would cause\nthe TAP stuff to break, and are in the ideal position to rewrite that\nsection. Two examples of what is drastically different from the old world\norder are \"do not print to the standard output strings that begin with XX\n(or partial lines)\" and \"here is how to skip the remainder of the tests\",\nbut you may have a handful of more rules. I expect that \"Writing Tests\"\nthrough \"Test harness library\" sections would be affected.\n\nEarlier I kept wondering if this series is worth the trouble. Having to\nlearn and follow the set of these new rules is exactly what I meant by\n\"trouble\". Let's make sure that the new rules are clear, small, and easy\nto follow for other contributors.\n\nThanks.\n"},{"id":"144266","messageId":"AANLkTilYoGtTIr9lIQD4sx4PRjlWYjIwSDMquREzv6eq@mail.gmail.com","threadId":"24189","inReplyTo":"7v6317koy2.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v5 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-25T21:24:47Z","receivedAt":"2010-06-25T21:24:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Fri, Jun 25, 2010 at 17:21, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> diff --git a/t/README b/t/README\n>> index 0e4e8d8..aa4ed28 100644\n>> --- a/t/README\n>> +++ b/t/README\n>> @@ -18,25 +18,48 @@ The easiest way to run tests is to say \"make\".  This runs all\n>> ...\n>\n> This describes the general framework of tests from the point of view of a\n> person who runs them and reads their output well.  I'll replace ab/tap\n> topic with these five patches.\n>\n> I'd however like to ask you to update the sections in t/README to help\n> test writers.  You have gained enough experience with patches 4 and 5 (and\n> with patch 2 for Perl tests) to know what kind of script constructs, used\n> by people who are used to the old way of writing our tests, would cause\n> the TAP stuff to break, and are in the ideal position to rewrite that\n> section.  Two examples of what is drastically different from the old world\n> order are \"do not print to the standard output strings that begin with XX\n> (or partial lines)\" and \"here is how to skip the remainder of the tests\",\n> but you may have a handful of more rules.  I expect that \"Writing Tests\"\n> through \"Test harness library\" sections would be affected.\n>\n> Earlier I kept wondering if this series is worth the trouble.  Having to\n> learn and follow the set of these new rules is exactly what I meant by\n> \"trouble\".  Let's make sure that the new rules are clear, small, and easy\n> to follow for other contributors.\n\nI'll write those docs, but I probably won't have time in the next few\ndays so I'll submit a patch for that it separately if that's OK.\n"},{"id":"144268","messageId":"7v6316kd6g.fsf@alter.siamese.dyndns.org","threadId":"24189","inReplyTo":"AANLkTilYoGtTIr9lIQD4sx4PRjlWYjIwSDMquREzv6eq@mail.gmail.com","subject":"Re: [PATCH v5 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-25T21:36:07Z","receivedAt":"2010-06-25T21:36:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> On Fri, Jun 25, 2010 at 17:21, Junio C Hamano wrote:\n>> Ævar Arnfjörð Bjarmason writes:\n>>\n>>> diff --git a/t/README b/t/README\n>>> index 0e4e8d8..aa4ed28 100644\n>>> --- a/t/README\n>>> +++ b/t/README\n>>> @@ -18,25 +18,48 @@ The easiest way to run tests is to say \"make\".  This runs all\n>>> ...\n>>\n>> This describes the general framework of tests from the point of view of a\n>> person who runs them and reads their output well.  I'll replace ab/tap\n>> topic with these five patches.\n>> ...\n>> I'd however like to ask you to update the sections in t/README to help\n>> test writers....\n>>\n>> Earlier I kept wondering if this series is worth the trouble.  Having to\n>> learn and follow the set of these new rules is exactly what I meant by\n>> \"trouble\".  Let's make sure that the new rules are clear, small, and easy\n>> to follow for other contributors.\n>\n> I'll write those docs, but I probably won't have time in the next few\n> days so I'll submit a patch for that it separately if that's OK.\n\nOh, absolutely. I meant to say that v4/v5 round already has a good\ndocumentation for test users/readers; a follow-up patch to help test\nwriters would make the series more complete.\n\nThanks.\n"},{"id":"144269","messageId":"AANLkTikLoLqqn6JQ8CHtmL_L9aXfzUH58ECW0RREERIk@mail.gmail.com","threadId":"24189","inReplyTo":"7v6316kd6g.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v5 1/5] test-lib: Adjust output to be valid TAP format","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-25T21:39:49Z","receivedAt":"2010-06-25T21:39:49Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Fri, Jun 25, 2010 at 21:36, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n>> I'll write those docs, but I probably won't have time in the next few\n>> days so I'll submit a patch for that it separately if that's OK.\n>\n> Oh, absolutely.  I meant to say that v4/v5 round already has a good\n> documentation for test users/readers; a follow-up patch to help test\n> writers would make the series more complete.\n\nSweet, yeah I'll make the t/README more awesomer later, good to hear\nthat v4/v5 is good enough.\n\nThanks!\n"},{"id":"144298","messageId":"1277554872-6402-1-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277401489-27885-1-git-send-email-avarab@gmail.com","subject":"[PATCH] t9700: Use Test::More->builder, not $Test::Builder::Test","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-26T12:21:12Z","receivedAt":"2010-06-26T12:21:12Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"$Test::Builder::Test was only made into an `our' variable in 0.94\nreleased in September 2009, older distros are more likely to have 0.92\nor earlier. Use the singleton Test::More->builder constructor instead,\nthe test now works on 0.92 too.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n\nThis patch fixes the TAP output in pu for older Test::More\nlibraries. It could be applied as a fixup to \"test-lib: Make the\ntest_external_* functions TAP-aware\", but it's probably useful to note\nthis in the version history.\n\n t/t9700/test.pl | 8 ++++----\n 1 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/t/t9700/test.pl b/t/t9700/test.pl\nindex e5d4b03..3e6a27c 100755\n--- a/t/t9700/test.pl\n+++ b/t/t9700/test.pl\n@@ -10,8 +10,8 @@ use Test::More qw(no_plan);\n BEGIN {\n \t# t9700-perl-git.sh kicks off our testing, so we have to go from\n \t# there.\n-\t$Test::Builder::Test->{Curr_Test} = 1;\n-\t$Test::Builder::Test->{No_Ending} = 1;\n+\tTest::More->builder->{Curr_Test} = 1;\n+\tTest::More->builder->{No_Ending} = 1;\n }\n \n use Cwd;\n@@ -113,6 +113,6 @@ like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');\n my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');\n isnt($last_commit, $dir_commit, 'log . does not show last commit');\n \n-printf \"1..%d\\n\", $Test::Builder::Test->{Curr_Test};\n+printf \"1..%d\\n\", Test::More->builder->{Curr_Test};\n \n-exit($Test::Builder::Test->{Is_Passing} ? 0 : 1);\n+exit(Test::More->builder->{Is_Passing} ? 0 : 1);\n-- \n1.7.1.251.g92a7\n"},{"id":"144299","messageId":"1277556161-8718-1-git-send-email-avarab@gmail.com","threadId":"24189","inReplyTo":"1277554872-6402-1-git-send-email-avarab@gmail.com","subject":"[PATCH v2] t9700: Use Test::More->builder, not $Test::Builder::Test","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-26T12:42:41Z","receivedAt":"2010-06-26T12:42:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"$Test::Builder::Test was only made into an `our' variable in 0.94\nreleased in September 2009, older distros are more likely to have 0.92\nor earlier. Use the singleton Test::More->builder constructor instead.\n\nThe exit() call was also unportable to <0.94. Just output a meaningful\nexit code if the ->is_passing method exists. The t9700-perl-git.sh\ntest only cares about stderr output, so this doesn't affect test\nresults when using older Test::More modules.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n\nThe previous patch didn't take into account that the exit() call was\nalso unportable. Oops.\n\n t/t9700/test.pl | 9 +++++----\n 1 files changed, 5 insertions(+), 4 deletions(-)\n\ndiff --git a/t/t9700/test.pl b/t/t9700/test.pl\nindex e5d4b03..f2820d2 100755\n--- a/t/t9700/test.pl\n+++ b/t/t9700/test.pl\n@@ -10,8 +10,8 @@ use Test::More qw(no_plan);\n BEGIN {\n \t# t9700-perl-git.sh kicks off our testing, so we have to go from\n \t# there.\n-\t$Test::Builder::Test->{Curr_Test} = 1;\n-\t$Test::Builder::Test->{No_Ending} = 1;\n+\tTest::More->builder->{Curr_Test} = 1;\n+\tTest::More->builder->{No_Ending} = 1;\n }\n \n use Cwd;\n@@ -113,6 +113,7 @@ like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');\n my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');\n isnt($last_commit, $dir_commit, 'log . does not show last commit');\n \n-printf \"1..%d\\n\", $Test::Builder::Test->{Curr_Test};\n+printf \"1..%d\\n\", Test::More->builder->{Curr_Test};\n \n-exit($Test::Builder::Test->{Is_Passing} ? 0 : 1);\n+my $is_passing = eval { Test::More->is_passing };\n+exit($is_passing ? 0 : 1) unless $@ =~ /Can't locate object method/;\n-- \n1.7.1.251.g92a7\n"},{"id":"144354","messageId":"7vvd94fi0d.fsf@alter.siamese.dyndns.org","threadId":"24189","inReplyTo":"1277556161-8718-1-git-send-email-avarab@gmail.com","subject":"Re: [PATCH v2] t9700: Use Test::More->builder, not $Test::Builder::Test","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-27T18:27:30Z","receivedAt":"2010-06-27T18:27:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> $Test::Builder::Test was only made into an `our' variable in 0.94\n> released in September 2009, older distros are more likely to have 0.92\n> or earlier. Use the singleton Test::More->builder constructor instead.\n\nAh, that is what it was. I was wondering what it was that showed 1..1 and\nthen 1..40.\n\nThanks; will queue on top of ab/tap.\n"}]}