git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/7] test-lib-functions: a better "test_expect_failure"

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 18, 2022, 00:33 UTC
Message-ID
<cover-0.7-00000000000-20220318T002951Z-avarab@gmail.com>

As explained in detail in 1/7 there's several major shortcomings to "test_expect_failure", this series adds a "test_expect_todo".

It allows us to assert exactly how something that fails .. fails (no more conflating segfaults for "exit 1"), but also emits all the right output & metadata about the test being "TODO" that "test_expect_failure" does.

The rest of the series then expands on that to convert several more tests, along with adding helpers like:

    todo_test_cmp want expect actual

Where the "want" is what we want once we'd mark the test as "test_expect_success", but the "expect" is what we get right now with the failing TODO test.

This series has a (rather trivial) textual conflict with js/ci-github-workflow-markup in "seen".

Ævar Arnfjörð Bjarmason (7):
  test-lib: add a "test_expect_todo", similar to "test_expect_failure"
  test-lib-functions: add and use a "test_todo" helper
  tests: allow test_* in "test_must_fail_acceptable" for "test_todo"
  test-lib-functions: add and use a "todo_test_cmp" helper
  test-lib-functions: add and use a "todo_test_path" helper
  test-lib-functions: make test_todo support a --reset
  sparse tests: convert a TODO test to use "test_expect_todo"
 t/README                           |  20 +++-
 t/t0000-basic.sh                   |  48 +++++++-
 t/t1021-rerere-in-workdir.sh       |   9 +-
 t/t1060-object-corruption.sh       |   4 +-
 t/t1091-sparse-checkout-builtin.sh |  17 ++-
 t/t1309-early-config.sh            |  14 ++-
 t/t1410-reflog.sh                  |   4 +-
 t/t3600-rm.sh                      |  23 +++-
 t/t6403-merge-file.sh              |  32 +++++-
 t/t7814-grep-recurse-submodules.sh |  60 ++++++----
 t/t7815-grep-binary.sh             |   4 +-
 t/test-lib-functions.sh            | 172 ++++++++++++++++++++++++++++-
 t/test-lib.sh                      |  32 +++++-
 13 files changed, 379 insertions(+), 60 deletions(-)
-- 
2.35.1.1436.g756b814e59f
Next: Ævar Arnfjörð Bjarmason
Message 1 of 18 in “test-lib-functions: a better "test_expect_failure"”
  1. 0/7 test-lib-functions: a better "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 18, 2022
  2. 1/7 test-lib: add a "test_expect_todo", similar to "test_expect_failure"Ævar Arnfjörð Bjarmason, Mar 18, 2022
  3. Junio C HamanoMar 18, 2022
  4. Junio C HamanoMar 18, 2022
  5. Ævar Arnfjörð BjarmasonMar 18, 2022
  6. Junio C HamanoMar 19, 2022
  7. Ævar Arnfjörð BjarmasonMar 19, 2022
  8. Phillip WoodMar 20, 2022
  9. Junio C HamanoMar 20, 2022
  10. Derrick StoleeMar 22, 2022
  11. Junio C HamanoMar 23, 2022
  12. Phillip WoodMar 24, 2022
  13. 2/7 test-lib-functions: add and use a "test_todo" helperÆvar Arnfjörð Bjarmason, Mar 18, 2022
  14. 3/7 tests: allow test_* in "test_must_fail_acceptable" for "test_todo"Ævar Arnfjörð Bjarmason, Mar 18, 2022
  15. 4/7 test-lib-functions: add and use a "todo_test_cmp" helperÆvar Arnfjörð Bjarmason, Mar 18, 2022
  16. 6/7 test-lib-functions: make test_todo support a --resetÆvar Arnfjörð Bjarmason, Mar 18, 2022
  17. 5/7 test-lib-functions: add and use a "todo_test_path" helperÆvar Arnfjörð Bjarmason, Mar 18, 2022
  18. 7/7 sparse tests: convert a TODO test to use "test_expect_todo"Ævar Arnfjörð Bjarmason, Mar 18, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.