git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4 7/7] t/README: Document the do's and don'ts of tests

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Jul 6, 2010, 12:50 UTC
Message-ID
<AANLkTimvyxNq3GAnwNZJEm6U5Jn7TlaedLZk12O9g8xe@mail.gmail.com>
In-Reply-To
<7vaaq58hhb.fsf@alter.siamese.dyndns.org>
On Tue, Jul 6, 2010 at 02:35, Junio C Hamano <gitster@pobox.com> wrote:
Show 67 quoted lines
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> +Do's, don'ts & things to keep in mind
>> +-------------------------------------
>> +
>> +Here's a few examples of things you probably should and shouldn't do
>> +when writing tests.
>
> "Here are" perhaps?
>
>> +Do:
>> +
>> + - Put as much code as possible inside test_expect_success and other
>> +   assertions.
>> +
>> +   Even code that isn't a test per se, but merely some setup code
>> +   should be inside a test assertion if at all possible. Test scripts
>> +   should only have trivial code outside of their assertions.
>
> Let's make it even stronger; "should only have trivial" -> "shouldn't have
> any ... unless there is a good reason."
>
>> +Don't:
>> +
>> + - exit() within a <script> part.
>> +
>> +   The harness will catch this as a programming error of the test.
>> +   Use test_done instead if you need to stop the tests early (see
>> +   "Skipping tests" below).
>> +
>> + - Break the TAP output
>> +
>> +   The raw output from your test might be interpreted by a TAP
>> +   harness. You usually don't have to worry about that. TAP harnesses
>
> I'd recommend dropping "You usually...about that"  You do care, but the
> limitation may be not so severe.
>
>> +   will ignore everything they don't know about, but don't step on
>> +   their toes in these areas:
>> +
>> +   - Don't print lines like "$x..$y" where $x and $y are integers.
>> +
>> +   - Don't print lines that begin with "ok" or "not ok".
>> +
>> +   A TAP harness expect a line that begins with either "ok" and "not
>> +   ok" to signal a test passed or failed (and our harness already
>> +   produces such lines), so your script shouldn't emit such lines to
>> +   their output.
>> +
>> +   You can glean some further possible issues from the TAP grammar
>> +   (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar)
>> +   but the best indication is to just run the tests with prove(1),
>> +   it'll complain if anything is amiss.
>> +
>> +Keep in mind:
>> +
>> + - That what you print to stderr and stdout is usually ignored
>> +
>> +   Inside <script> part, the standard output and standard error
>
> Splitting the above into two sentences (or a header and a body) makes it
> unclear that your "usually" comes from the earlier "Do Put as much code
> inside test_expect_success...".  I think you can simply drop "That what
> you print ... ignored".
>
> Everything else in the series looked good.  Thanks.

All of these rewordings (the "t/README: proposed rewording..." commit in pu) look good.

    Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Previous: Jakub Narebski
Message 13 of 13 in “Improvements for t/README”
  1. 0/7 Improvements for t/READMEÆvar Arnfjörð Bjarmason, Jul 2, 2010
  2. 1/7 t/README: The trash is in 't/trash directory.$name'Ævar Arnfjörð Bjarmason, Jul 2, 2010
  3. 2/7 t/README: Typo: paralell -> parallelÆvar Arnfjörð Bjarmason, Jul 2, 2010
  4. 3/7 t/README: Document the prereq functions, and 3-arg test_*Ævar Arnfjörð Bjarmason, Jul 2, 2010
  5. 4/7 t/README: Document test_external*Ævar Arnfjörð Bjarmason, Jul 2, 2010
  6. 5/7 t/README: Document test_expect_codeÆvar Arnfjörð Bjarmason, Jul 2, 2010
  7. 6/7 t/README: Add a section about skipping testsÆvar Arnfjörð Bjarmason, Jul 2, 2010
  8. 7/7 t/README: Document the do's and don'ts of testsÆvar Arnfjörð Bjarmason, Jul 2, 2010
  9. Junio C HamanoJul 6, 2010
  10. Jakub NarebskiJul 6, 2010
  11. Ævar Arnfjörð BjarmasonJul 6, 2010
  12. Jakub NarebskiJul 6, 2010
  13. Ævar Arnfjörð BjarmasonJul 6, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.