git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/6] Make test-lib.sh emit valid TAP format

From
Jakub Narebski <jnareb@gmail.com>
Date
May 7, 2010, 20:17 UTC
Message-ID
<m37hnffow2.fsf@localhost.localdomain>
In-Reply-To
<1273261025-31523-1-git-send-email-avarab@gmail.com>
Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
Show 17 quoted lines
> TAP, the the Test Anything Protocol, is a simple text-based interface
> between testing modules in a test harness. test-lib.sh's output was
> already very close to being valid TAP. This change brings it all the
> way there.
> 
> The advantage of using TAP is that any program that reads the format
> (a "test harness") can run the tests. The most popular of these is the
> prove(1) utility that comes with Perl. It can run tests in parallel,
> display colored output, format the output to console, file, HTML etc.,
> and much more.
> 
> On my quad Xeon server running the test suite with `make test` takes
> 21 minutes. Running it with `prove -j 15 ./t[0-9]*.sh` takes just over
> 5 minutes.
> 
> With parallel tests the whole test suite doesn't have to stall because
> tests like t3404-rebase-interactive.sh take a long time.

I would have thought that it would be better for git test suite to enable TAP output with --tap switch. On the other hand changing output to TAP, replacing old format, would be less code to maintain.

I am not sure if testing with 'prove' and TAP output is compatibile with all current git test suite options, i.e. --debug, --tee, --verbose and --immediate, and with GIT_SKIP_TESTS environmental variable. Also valuable way of checking where the error occurs in the test, namely 'sh -x ./tXXXX-test.sh' would not work, I think, with 'prove'.

Note also that having Perl (and 'prove') installed is not requirement for git runnig, and I think it should not be requirement for git development.

P.S. I think that such series would be better with the cover letter.
-- 
Jakub Narebski
Poland
ShadeHawk on #git
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 16 of 17 in “Make test-lib.sh emit valid TAP format”
  1. 1/6 Make test-lib.sh emit valid TAP formatÆvar Arnfjörð Bjarmason, May 7, 2010
  2. 2/6 Move t6000lib.sh to lib-*Ævar Arnfjörð Bjarmason, May 7, 2010
  3. Jakub NarebskiMay 7, 2010
  4. 3/6 Skip tests in a way that makes sense under TAPÆvar Arnfjörð Bjarmason, May 7, 2010
  5. 4/6 Turn setup code in t2007-checkout-symlink.sh into a testÆvar Arnfjörð Bjarmason, May 7, 2010
  6. Jakub NarebskiMay 7, 2010
  7. 5/6 We use TAP so the Perl test can run without scaffoldingÆvar Arnfjörð Bjarmason, May 7, 2010
  8. 6/6 Add test_harness make target for testing with prove(1)Ævar Arnfjörð Bjarmason, May 7, 2010
  9. Jakub NarebskiMay 7, 2010
  10. Junio C HamanoMay 8, 2010
  11. Tor ArntsenMay 9, 2010
  12. Sverre RabbelierMay 7, 2010
  13. Ævar Arnfjörð BjarmasonMay 7, 2010
  14. 1/6 Make test-lib.sh emit valid TAP formatÆvar Arnfjörð Bjarmason, May 7, 2010
  15. Ævar Arnfjörð BjarmasonMay 15, 2010
  16. Jakub NarebskiMay 7, 2010
  17. Ævar Arnfjörð BjarmasonMay 7, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.