Re: [PATCH v4 1/5] test-lib: Adjust output to be valid TAP format

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

This is incomplete information. It only tells us how the output is changed for the case when all test_expect_success tests passes. This commit message doesn't tell us how failing tests looks like, and how test_expect_failure results gets translated.

Take for example a following test script: -- >8 -- #!/bin/sh

test_description='this is a sample test.

This test is here to see various test outputs.'

. ./test-lib.sh

say 'diagnostic message'

test_expect_success 'true test' 'true' test_expect_success 'false test' 'false'

test_expect_failure 'true test (todo)' 'true' test_expect_failure 'false test (todo)' 'false'

test_debug 'echo "debug message"'

test_done -- 8< --

This test script output looks like the following (the comments are not
part of output, but denote color of given line of test output):
----
* diagnostic message                      # yellow
*   ok 1: true  test
* FAIL 2: false test                      # bold red
        false
*   FIXED 3: true  test (todo)
*   still broken 4: false test (todo)     # bold green
* fixed 1 known breakage(s)               # green
* still have 1 known breakage(s)          # bold red
* failed 1 among remaining 3 test(s)      # bold red
----

How would the output of this test look like after TAP-ification? Would it still provide color output when run on terminal?

This test also generates summary of test run in a file in test-results/ subdirectory. Currently such file can be e.g. named test-results/test_test-8030, and consist of -- >8 -- total 4 success 1 fixed 1 broken 1 failed 1

-- 8< --

Would TAP-ification change that? Even if there is no change, this is worth a sentence or a few words in a commit message.

P.S. With current output pass / fail results of test_expect_failure are justified on ':' separator. This is not the case of TAP output.