git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/3] test-lib: reduce verbosity of skipped tests

From
Elijah Newren <newren@gmail.com>
Date
Oct 13, 2020, 17:56 UTC
Message-ID
<CABPp-BFj0S1Z7rgNGsUuRurNgKFOqtD_Nm0Dr3fZt-RFMTwoDw@mail.gmail.com>
In-Reply-To
<20201013154439.GB1913367@nand.local>
On Tue, Oct 13, 2020 at 8:44 AM Taylor Blau <me@ttaylorr.com> wrote:
Show 57 quoted lines
>
> On Mon, Oct 12, 2020 at 11:26:04PM +0000, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
> >
> > When using the --run flag to run just two or three tests from a test
> > file which contains several dozen tests, having every skipped test print
> > out dozens of lines of output for the test code for that skipped test
> > adds up to hundreds or thousands of lines of irrelevant output that make
> > it very hard to fish out the relevant results you were looking for.
> > Simplify the output for skipped tests down to just showing the one-line
> > descriptions.
> >
> > Signed-off-by: Elijah Newren <newren@gmail.com>
> > ---
> >  t/test-lib.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/t/test-lib.sh b/t/test-lib.sh
> > index 2aca398e1e..7602bbe9e9 100644
> > --- a/t/test-lib.sh
> > +++ b/t/test-lib.sh
> > @@ -1059,7 +1059,7 @@ test_skip () {
> >                               "      <skipped message=\"$message\" />"
> >               fi
> >
> > -             say_color skip >&3 "skipping test: $@"
> > +             say_color skip >&3 "skipping test: $1"
>
> I would be comfortable going further than this and removing this line
> entirely. We need the "ok $test_count # skip" below since it's part of
> our TAP output, but now the output looks somewhat redundant.
>
> With this patch running a test that I'm working on with `--run=...`, I
> get output that looks like:
>
>   skipping test: writing bitmaps via command-line can duplicate .keep objects
>   ok 2 # skip writing bitmaps via command-line can duplicate .keep objects (--run)
>
>   skipping test: writing bitmaps via config can duplicate .keep objects
>   ok 3 # skip writing bitmaps via config can duplicate .keep objects (--run)
>
> Scanning over the same test description twice per skipped test makes the
> output difficult (but still much easier than before) to scan. What do
> you think about either of the following:
>
>   skipping test: writing bitmaps via command-line can duplicate .keep objects
>   ok 2 # skip (--run)
>
> or:
>
>   ok 2 # skip writing bitmaps via command-line can duplicate .keep objects (--run)
>
> I have a slight preference towards the latter, since it keeps more of
> the information in the TAP line, and it cuts the total line count of
> output from skipped tests in half.
>
> For what it's worth, I'd also be fine with the patch as-is.

I was worried that shortening it as much as I did would run into objections for some obscure reason. But if no one objects, I think I'd also prefer your latter suggestion for shortening it even more. It looks like the dual output for skipped tests comes from commit 04ece59399 ("GIT_SKIP_TESTS: allow users to omit tests that are known to break", 2006-12-28) by Junio, so it'd be nice to hear his opinion on how much we shorten it.

Previous: Taylor BlauNext: Junio C Hamano
Message 9 of 71 in “Make test selection easier by specifying description substrings instead of just numeric counters”
  1. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 12, 2020
  2. 1/3 test-lib: allow selecting tests by substring/regex with --runElijah Newren via GitGitGadget, Oct 12, 2020
  3. Taylor BlauOct 13, 2020
  4. Elijah NewrenOct 13, 2020
  5. Taylor BlauOct 13, 2020
  6. Elijah NewrenOct 13, 2020
  7. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 12, 2020
  8. Taylor BlauOct 13, 2020
  9. Elijah NewrenOct 13, 2020
  10. Junio C HamanoOct 13, 2020
  11. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 12, 2020
  12. Phillip WoodOct 13, 2020
  13. Johannes SchindelinOct 15, 2020
  14. Phillip WoodOct 15, 2020
  15. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 13, 2020
  16. 1/3 test-lib: allow selecting tests by substring/regex with --runElijah Newren via GitGitGadget, Oct 13, 2020
  17. Jeff KingOct 14, 2020
  18. Junio C HamanoOct 14, 2020
  19. Jeff KingOct 14, 2020
  20. Junio C HamanoOct 14, 2020
  21. Elijah NewrenOct 14, 2020
  22. Jeff KingOct 14, 2020
  23. Junio C HamanoOct 15, 2020
  24. Junio C HamanoOct 15, 2020
  25. Elijah NewrenOct 14, 2020
  26. Junio C HamanoOct 14, 2020
  27. Elijah NewrenOct 14, 2020
  28. Junio C HamanoOct 15, 2020
  29. Jeff KingOct 14, 2020
  30. Elijah NewrenOct 14, 2020
  31. Junio C HamanoOct 15, 2020
  32. Jeff KingOct 15, 2020
  33. Junio C HamanoOct 15, 2020
  34. Jeff KingOct 16, 2020
  35. Junio C HamanoOct 16, 2020
  36. Junio C HamanoOct 16, 2020
  37. Jeff KingOct 16, 2020
  38. Junio C HamanoOct 16, 2020
  39. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 13, 2020
  40. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 13, 2020
  41. Jeff KingOct 14, 2020
  42. Elijah NewrenOct 14, 2020
  43. Jeff KingOct 14, 2020
  44. Elijah NewrenOct 14, 2020
  45. Taylor BlauOct 13, 2020
  46. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 14, 2020
  47. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 14, 2020
  48. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 14, 2020
  49. 1/3 test-lib: allow selecting tests by substring/glob with --runElijah Newren via GitGitGadget, Oct 14, 2020
  50. Phillip WoodOct 16, 2020
  51. Elijah NewrenOct 16, 2020
  52. Phillip WoodOct 16, 2020
  53. Junio C HamanoOct 16, 2020
  54. Jeff KingOct 16, 2020
  55. Jeff KingOct 15, 2020
  56. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 16, 2020
  57. 1/3 test-lib: allow selecting tests by substring/glob with --runElijah Newren via GitGitGadget, Oct 16, 2020
  58. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 16, 2020
  59. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 16, 2020
  60. Junio C HamanoOct 16, 2020
  61. Elijah NewrenOct 16, 2020
  62. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 16, 2020
  63. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 16, 2020
  64. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 16, 2020
  65. 1/3 test-lib: allow selecting tests by substring/glob with --runElijah Newren via GitGitGadget, Oct 16, 2020
  66. Andrei RybakOct 16, 2020
  67. Elijah NewrenOct 17, 2020
  68. 0/3 Make test selection easier by specifying description substrings instead of just numeric countersElijah Newren via GitGitGadget, Oct 18, 2020
  69. 1/3 test-lib: allow selecting tests by substring/glob with --runElijah Newren via GitGitGadget, Oct 18, 2020
  70. 2/3 t6006, t6012: adjust tests to use 'setup' instead of synonymsElijah Newren via GitGitGadget, Oct 18, 2020
  71. 3/3 test-lib: reduce verbosity of skipped testsElijah Newren via GitGitGadget, Oct 18, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.