Re: [PATCH v2 0/3] Make the Git tests emit TAP format
- From
Andreas Ericsson <ae@op5.se>
- Date
- Jun 15, 2010, 16:42 UTC
- Message-ID
- <4C17AD93.1020402@op5.se>
- In-Reply-To
- <AANLkTimxE0mqmitRzlXjSAO6v7IOEg4guUnRlzJUyIF1@mail.gmail.com>
On 06/15/2010 05:17 PM, Ævar Arnfjörð Bjarmason wrote:
Show 46 quoted lines
> On Mon, Jun 14, 2010 at 22:10, Ævar Arnfjörð Bjarmason<avarab@gmail.com> wrote: >> On Mon, Jun 14, 2010 at 21:49, Jakub Narebski<jnareb@gmail.com> wrote: >>> Ævar Arnfjörð Bjarmason<avarab@gmail.com> writes: >>> >>>> All this series does is slightly adjust the raw text output of our >>>> tests so that it conforms. to the TAP standard, i.e. instead of this: >>>> >>>> $ ./t0005-signals.sh >>>> * ok 1: sigchain works >>>> * passed all 1 test(s) >>>> >>>> We get this: >>>> >>>> $ ./t0005-signals.sh >>>> ok 1 - sigchain works >>>> # passed all 1 test(s) >>>> 1..1 >>>> >>>> Changing the output format like this gives us the ability to run the >>>> Git tests with any TAP tool (like prove(1)) at no extra cost. Every >>>> other existing way of running the tests continues to work, it's just >>>> easier for machines to read the output now. >> >> I'll re-submit a version of the patch with a better commit message >> which addresses all of the below. >> >>> This doesn't tell us if the result of running test suite with >>> '--verbose' and/or with '--debug' changes, and if changes how? Is it >>> compatibile with TAP format so that TAP parsers understand it? >> >> It just changes in that the lines that previously said e.g. "* ok 1: >> sigchain works" now say "ok 1 - sigchain works". >> >> TAP parsers still understand it, since ignoring unknown garbage is >> part of the TAP standard. > > Actually it doesn't work for all the tests. The following crops up on > prove -j 10 ./t[0-9]*.sh :: --verbose: > > ./t1007-hash-object.sh (Wstat: 0 > Tests: 19 Failed: 0) > Parse errors: Tests out of sequence. Found (12) but expected (11) > Tests out of sequence. Found (13) but expected (12) > Tests out of sequence. Found (14) but expected (13) > Tests out of sequence. Found (15) but expected (14) > Tests out of sequence. Found (16) but expected (15)
Why are you using a plan at all? I absolutely detest using planned tap series, and last I heard was that the majority of TAP-fanatics actually agreed that using a plan to run tests was an extraordinarily lousy idea, since it makes it harder to add tests properly.
-- Andreas Ericsson andreas.ericsson@op5.se OP5 AB www.op5.se Tel: +46 8-230225 Fax: +46 8-230231 Considering the successes of the wars on alcohol, poverty, drugs and terror, I think we should give some serious thought to declaring war on peace.