{"thread":{"id":"65743","subject":"[PATCH v2 1/4] t7527: fix broken TAP output","startedAt":"2026-06-03T05:39:59Z","lastAt":"2026-06-03T05:40:07Z","messageCount":5,"participants":["Patrick Steinhardt"],"isPatch":true,"patchVersion":2,"patchTotal":4},"messages":[{"id":"544572","messageId":"20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im","threadId":"65743","inReplyTo":"20260602-pks-t7527-fix-tap-output-v1-0-db3da2a1b137@pks.im","subject":"[PATCH v2 0/4] t: fix broken TAP output","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-06-03T05:39:43Z","receivedAt":"2026-06-03T05:39:58Z","isPatch":true,"body":"Hi,\n\nthis small patch series fixes another instance of broken TAP output that\nhas landed via 4d11b9c218 (Merge branch 'pt/fsmonitor-linux', 2026-05-31).\n\nAs this has happened multiple times by now I decided to have a look at\nwhether we can fix this class of issues a bit more holistically. So this\nseries also contains a change that makes prove bail out when it sees\ninvalid TAP output, which uncovers a small set of preexisting issues in\nour test suite.\n\nChanges in v2:\n  - Fix waiting for p4d, and deduplicate the logic that does this.\n  - Link to v1: https://patch.msgid.link/20260602-pks-t7527-fix-tap-output-v1-0-db3da2a1b137@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (4):\n      t7527: fix broken TAP output\n      t/test-lib: silence EBUSY errors on Windows during test cleanup\n      t/lib-git-p4: silence output when killing p4d and its watchdog\n      t: let prove fail when parsing invalid TAP output\n\n t/lib-git-p4.sh              |  4 ++--\n t/t7527-builtin-fsmonitor.sh |  7 ++++---\n t/test-lib.sh                | 10 ++++++++--\n 3 files changed, 14 insertions(+), 7 deletions(-)\n\nRange-diff versus v1:\n\n1:  09977059d1 = 1:  18b4fc7b81 t7527: fix broken TAP output\n2:  162d3d42d8 = 2:  8dd921534b t/test-lib: silence EBUSY errors on Windows during test cleanup\n3:  4ecb8cb1ce < -:  ---------- t/lib-git-p4: silence output when killing p4d and its watchdog\n-:  ---------- > 3:  8b343176fe t/lib-git-p4: silence output when killing p4d and its watchdog\n4:  95fb0d07ae = 4:  e69aa0ab79 t: let prove fail when parsing invalid TAP output\n\n---\nbase-commit: 1666c1265231b0bc5f613fbbf3f0a9896cdef76e\nchange-id: 20260601-pks-t7527-fix-tap-output-105da1d73df0\n\n"},{"id":"544571","messageId":"20260603-pks-t7527-fix-tap-output-v2-1-cf3af5694e20@pks.im","threadId":"65743","inReplyTo":"20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im","subject":"[PATCH v2 1/4] t7527: fix broken TAP output","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-06-03T05:39:44Z","receivedAt":"2026-06-03T05:39:59Z","isPatch":true,"body":"Before running the tests in t7527 we first verify whether the fsmonitor\neven works, which seems to depend on the actual filesystem that is in\nuse. The verification executes outside of any prerequisite or test body,\nso its stdout/stderr is not being redirected.\n\nThe consequence of this is that any command that prints to stdout/stderr\nmay break the TAP specification by printing invalid lines. And in fact\nwe already do that, as git-init(1) prints the path to the created Git\nrepository by default.\n\nFix this issue by moving the logic into a lazy prerequisite.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n t/t7527-builtin-fsmonitor.sh | 7 ++++---\n 1 file changed, 4 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh\nindex b63c162f9b..d881e27466 100755\n--- a/t/t7527-builtin-fsmonitor.sh\n+++ b/t/t7527-builtin-fsmonitor.sh\n@@ -25,7 +25,8 @@ maybe_timeout () {\n \t\t\"$@\"\n \tfi\n }\n-verify_fsmonitor_works () {\n+\n+test_lazy_prereq FSMONITOR_WORKS '\n \tgit init test_fsmonitor_smoke || return 1\n \n \tGIT_TRACE_FSMONITOR=\"$PWD/smoke.trace\" &&\n@@ -50,9 +51,9 @@ verify_fsmonitor_works () {\n \tret=$?\n \trm -rf test_fsmonitor_smoke smoke.trace\n \treturn $ret\n-}\n+'\n \n-if ! verify_fsmonitor_works\n+if ! test_have_prereq FSMONITOR_WORKS\n then\n \tskip_all=\"filesystem does not deliver fsmonitor events (container/overlayfs?)\"\n \ttest_done\n\n-- \n2.54.0.1064.gd145956f57.dirty\n\n"},{"id":"544573","messageId":"20260603-pks-t7527-fix-tap-output-v2-2-cf3af5694e20@pks.im","threadId":"65743","inReplyTo":"20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im","subject":"[PATCH v2 2/4] t/test-lib: silence EBUSY errors on Windows during test cleanup","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-06-03T05:39:45Z","receivedAt":"2026-06-03T05:40:01Z","isPatch":true,"body":"When tests have finished we clean up the trash directory via `rm -rf`.\nOn Windows this can fail with EBUSY in cases where a process still holds\nsome of the files open, for example when we have spawned a daemonized\nprocess that wasn't properly terminated. We thus retry several times,\nbut every failure will result in error messages being printed, and that\nin turn breaks the TAP output format.\n\nOne such case where this is causing issues is in t921x, which contains\ntests related to Scalar. Some tests spawn the fsmonitor daemon, and we\nnever properly terminate it.\n\nThe obvious fix would be to ensure that we never leak any processes, but\nthat gets ugly fast. Instead, let's work around the issue by silencing\nerror messages printed by the `rm -rf` calls. We already know to print\nan error when the retry loop fails, so we don't loose much.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n t/test-lib.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 4a7357b547..d1d24c4124 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1299,10 +1299,10 @@ test_done () {\n \t\t\terror \"Tests passed but trash directory already removed before test cleanup; aborting\"\n \n \t\t\tcd \"$TRASH_DIRECTORY/..\" &&\n-\t\t\trm -fr \"$TRASH_DIRECTORY\" || {\n+\t\t\trm -fr \"$TRASH_DIRECTORY\" 2>/dev/null || {\n \t\t\t\t# try again in a bit\n \t\t\t\tsleep 5;\n-\t\t\t\trm -fr \"$TRASH_DIRECTORY\"\n+\t\t\t\trm -fr \"$TRASH_DIRECTORY\" 2>/dev/null\n \t\t\t} ||\n \t\t\terror \"Tests passed but test cleanup failed; aborting\"\n \t\tfi\n\n-- \n2.54.0.1064.gd145956f57.dirty\n\n"},{"id":"544574","messageId":"20260603-pks-t7527-fix-tap-output-v2-3-cf3af5694e20@pks.im","threadId":"65743","inReplyTo":"20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im","subject":"[PATCH v2 3/4] t/lib-git-p4: silence output when killing p4d and its watchdog","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-06-03T05:39:46Z","receivedAt":"2026-06-03T05:40:06Z","isPatch":true,"body":"When stopping the p4d watchdog process via \"kill -9\", the shell may\nprint a job-control notification like:\n\n  ./test-lib.sh: line 1269: 57960 Killed: 9               while true; do\n      if test $nr_tries_left -eq 0; then\n          kill -9 $p4d_pid; exit 1;\n      fi; sleep 1; nr_tries_left=$(($nr_tries_left - 1));\n  done 2> /dev/null 4>&2  (wd: ~)\n\nThis message is printed asynchronously by the shell when it reaps the\nprocess. While harmless right now, this will cause breakage once we\nenable strict parsing of the TAP protocol in a subsequent commit.\n\nFix this by using `wait` so that we can synchronously reap the watchdog\nprocess and swallow the diagnostic.\n\nWhile at it, deduplicate the logic we have in `stop_p4d_and_watchdog ()`\nand `stop_and_cleanup_p4d ()`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n t/lib-git-p4.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh\nindex d22e9c684a..9108868187 100644\n--- a/t/lib-git-p4.sh\n+++ b/t/lib-git-p4.sh\n@@ -65,6 +65,7 @@ pidfile=\"$TRASH_DIRECTORY/p4d.pid\"\n \n stop_p4d_and_watchdog () {\n \tkill -9 $p4d_pid $watchdog_pid\n+\twait $p4d_pid $watchdog_pid 2>/dev/null\n }\n \n # git p4 submit generates a temp file, which will\n@@ -174,8 +175,7 @@ retry_until_success () {\n }\n \n stop_and_cleanup_p4d () {\n-\tkill -9 $p4d_pid $watchdog_pid\n-\twait $p4d_pid\n+\tstop_p4d_and_watchdog\n \trm -rf \"$db\" \"$cli\" \"$pidfile\"\n }\n \n\n-- \n2.54.0.1064.gd145956f57.dirty\n\n"},{"id":"544575","messageId":"20260603-pks-t7527-fix-tap-output-v2-4-cf3af5694e20@pks.im","threadId":"65743","inReplyTo":"20260603-pks-t7527-fix-tap-output-v2-0-cf3af5694e20@pks.im","subject":"[PATCH v2 4/4] t: let prove fail when parsing invalid TAP output","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-06-03T05:39:47Z","receivedAt":"2026-06-03T05:40:07Z","isPatch":true,"body":"To make the result of our tests accessible we use the TAP protocol. This\nprotocol is parsed by either prove or by Meson. Unfortunately, these two\ntools differ when it comes to their strictness when parsing the\nprotocol:\n\n  - Prove by default happily accepts lines not specified by the\n    protocol.\n\n  - Meson will also accept such lines, but prints a big and ugly warning\n    message.\n\nWe have fixed our test suite in the past to not print invalid TAP lines\nanymore via b1dc2e796e (Merge branch 'ps/meson-tap-parse', 2025-06-17).\nBut as none of our tools perform a strict check it's still possible for\nbroken tests to sneak back in, like for example in 362f69547f (Merge\nbranch 'ps/t1006-tap-fix', 2025-07-16). This doesn't hurt at all when\nusing prove, but it's quite annoying when using Meson due to the\ngenerated warnings.\n\nUnfortunately, there doesn't seem to be a portable way to make all tools\ncomplain about violations of the TAP format. The TAP 14 specification\nhas added pragmas to the protocol that would allow us to say `pragma\n+strict`, and the effect of that would be to treat invalid TAP lines as\na test failure. But the release of TAP 14 is still rather recent, and\nTest-Harness for example only gained support for it in version 3.48,\nwhich was released in 2023.\n\nIn fact though, this pragma was already introduced as an inofficial\nextension of the TAP protocol with Test-Harness 3.10, released in 2008.\nSo while not all tools understand the pragma, at least prove does for a\nlong time.\n\nUnconditionally enable the pragma when using prove so that we'll detect\ntests that emit broken TAP output right away. This would have detected\nthe issues fixed in preceding commits:\n\n    $ prove t7527-builtin-fsmonitor.sh\n    t7527-builtin-fsmonitor.sh .. All 69 subtests passed\n            (less 6 skipped subtests: 63 okay)\n\n    Test Summary Report\n    -------------------\n    t7527-builtin-fsmonitor.sh (Wstat: 0 Tests: 69 Failed: 0)\n      Parse errors: Unknown TAP token: \"Initialized empty Git repository in /tmp/git/test_fsmonitor_smoke/.git/\"\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n t/test-lib.sh | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex d1d24c4124..ceefb99bff 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1532,6 +1532,12 @@ then\n \tBAIL_OUT 'You need to build test-tool; Run \"make t/helper/test-tool\" in the source (toplevel) directory'\n fi\n \n+if test -n \"$HARNESS_ACTIVE\"\n+then\n+\tsay \"TAP version 13\"\n+\tsay \"pragma +strict\"\n+fi\n+\n # Are we running this test at all?\n remove_trash=\n this_test=${0##*/}\n\n-- \n2.54.0.1064.gd145956f57.dirty\n\n"}]}