{"thread":{"id":"61713","subject":"[PATCH 0/2] here-doc test bodies","startedAt":"2024-07-01T22:08:17Z","lastAt":"2024-08-21T17:02:56Z","messageCount":65,"participants":["Jeff King","Eric Sunshine","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"497937","messageId":"20240701220815.GA20293@coredump.intra.peff.net","threadId":"61713","inReplyTo":null,"subject":"[PATCH 0/2] here-doc test bodies","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-01T22:08:15Z","receivedAt":"2024-07-01T22:08:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"This is a re-post of an idea from 2021:\n\n  https://lore.kernel.org/git/YHDUg6ZR5vu93kGm@coredump.intra.peff.net/\n\nthat people seemed mostly positive on, and I just never got around to\nfollowing up. Mostly because it's not life-changing, but I think it is a\nsmall quality of life improvement, and it came up again recently in:\n\n  https://lore.kernel.org/git/20240701032047.GA610406@coredump.intra.peff.net/\n\nSo I thought it was worth considering again.\n\n  [1/2]: test-lib: allow test snippets as here-docs\n  [2/2]: t: convert some here-doc test bodies\n\n t/README                     |   8 ++\n t/t0600-reffiles-backend.sh  |  38 +++----\n t/t1404-update-ref-errors.sh | 196 +++++++++++++++++------------------\n t/test-lib-functions.sh      |  32 +++++-\n 4 files changed, 152 insertions(+), 122 deletions(-)\n\n-Peff\n"},{"id":"497938","messageId":"20240701220840.GA20631@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240701220815.GA20293@coredump.intra.peff.net","subject":"[PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-01T22:08:40Z","receivedAt":"2024-07-01T22:08:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Most test snippets are wrapped in single quotes, like:\n\n  test_expect_success 'some description' '\n          do_something\n  '\n\nThis sometimes makes the snippets awkward to write, because you can't\neasily use single quotes within them. We sometimes work around this with\n$SQ, or by loosening regexes to use \".\" instead of a literal quote, or\nby using double quotes when we'd prefer to use single-quotes (and just\nadding extra backslash-escapes to avoid interpolation).\n\nThis commit adds another option: feeding the snippet via the function's\nstdin. This doesn't conflict with anything the snippet would want to do,\nbecause we always redirect its stdin from /dev/null anyway (which we'll\ncontinue to do).\n\nA few notes on the implementation:\n\n  - it would be nice to push this down into test_run_, but we can't, as\n    test_expect_success and test_expect_failure want to see the actual\n    script content to report it for verbose-mode. A helper function\n    limits the amount of duplication in those callers here.\n\n  - The helper function is a little awkward to call, as you feed it the\n    name of the variable you want to set. The more natural thing in\n    shell would be command substitution like:\n\n      body=$(body_or_stdin \"$2\")\n\n    but that loses trailing whitespace. There are tricks around this,\n    like:\n\n      body=$(body_or_stdin \"$2\"; printf .)\n      body=${body%.}\n\n    but we'd prefer to keep such tricks in the helper, not in each\n    caller.\n\n  - I implemented the helper using a sequence of \"read\" calls. Together\n    with \"-r\" and unsetting the IFS, this preserves incoming whitespace.\n    An alternative is to use \"cat\" (which then requires the gross \".\"\n    trick above). But this saves us a process, which is probably a good\n    thing. The \"read\" builtin does use more read() syscalls than\n    necessary (one per byte), but that is almost certainly a win over a\n    separate process.\n\n    Both are probably slower than passing a single-quoted string, but\n    the difference is lost in the noise for a script that I converted as\n    an experiment.\n\n  - I handle test_expect_success and test_expect_failure here. If we\n    like this style, we could easily extend it to other spots (e.g.,\n    lazy_prereq bodies) on top of this patch.\n\n  - even though we are using \"local\", we have to be careful about our\n    variable names. Within test_expect_success, any variable we declare\n    with local will be seen as local by the test snippets themselves (so\n    it wouldn't persist between tests like normal variables would).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/README                |  8 ++++++++\n t/test-lib-functions.sh | 32 +++++++++++++++++++++++++++-----\n 2 files changed, 35 insertions(+), 5 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex d9e0e07506..dec644f997 100644\n--- a/t/README\n+++ b/t/README\n@@ -906,6 +906,14 @@ see test-lib-functions.sh for the full list and their options.\n \t    'git-write-tree should be able to write an empty tree.' \\\n \t    'tree=$(git-write-tree)'\n \n+   If <script> is `-` (a single dash), then the script to run is read\n+   from stdin. This lets you more easily use single quotes within the\n+   script by using a here-doc. For example:\n+\n+\ttest_expect_success 'output contains expected string' - <<\\EOT\n+\t\tgrep \"this string has 'quotes' in it\" output\n+\tEOT\n+\n    If you supply three parameters the first will be taken to be a\n    prerequisite; see the test_set_prereq and test_have_prereq\n    documentation below:\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 427b375b39..803ed2df39 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -872,6 +872,24 @@ test_verify_prereq () {\n \tBUG \"'$test_prereq' does not look like a prereq\"\n }\n \n+# assign the variable named by \"$1\" with the contents of \"$2\";\n+# if \"$2\" is \"-\", then read stdin into \"$1\" instead\n+test_body_or_stdin () {\n+\tif test \"$2\" != \"-\"\n+\tthen\n+\t\teval \"$1=\\$2\"\n+\t\treturn\n+\tfi\n+\n+\t# start with a newline, to match hanging newline from open-quote style\n+\teval \"$1=\\$LF\"\n+\tlocal test_line\n+\twhile IFS= read -r test_line\n+\tdo\n+\t\teval \"$1=\\${$1}\\${test_line}\\${LF}\"\n+\tdone\n+}\n+\n test_expect_failure () {\n \ttest_start_ \"$@\"\n \ttest \"$#\" = 3 && { test_prereq=$1; shift; } || test_prereq=\n@@ -881,9 +899,11 @@ test_expect_failure () {\n \texport test_prereq\n \tif ! test_skip \"$@\"\n \tthen\n+\t\tlocal test_body\n+\t\ttest_body_or_stdin test_body \"$2\"\n \t\ttest -n \"$test_skip_test_preamble\" ||\n-\t\tsay >&3 \"checking known breakage of $TEST_NUMBER.$test_count '$1': $2\"\n-\t\tif test_run_ \"$2\" expecting_failure\n+\t\tsay >&3 \"checking known breakage of $TEST_NUMBER.$test_count '$1': $test_body\"\n+\t\tif test_run_ \"$test_body\" expecting_failure\n \t\tthen\n \t\t\ttest_known_broken_ok_ \"$1\"\n \t\telse\n@@ -902,13 +922,15 @@ test_expect_success () {\n \texport test_prereq\n \tif ! test_skip \"$@\"\n \tthen\n+\t\tlocal test_body\n+\t\ttest_body_or_stdin test_body \"$2\"\n \t\ttest -n \"$test_skip_test_preamble\" ||\n-\t\tsay >&3 \"expecting success of $TEST_NUMBER.$test_count '$1': $2\"\n-\t\tif test_run_ \"$2\"\n+\t\tsay >&3 \"expecting success of $TEST_NUMBER.$test_count '$1': $test_body\"\n+\t\tif test_run_ \"$test_body\"\n \t\tthen\n \t\t\ttest_ok_ \"$1\"\n \t\telse\n-\t\t\ttest_failure_ \"$@\"\n+\t\t\ttest_failure_ \"$1\" \"$test_body\"\n \t\tfi\n \tfi\n \ttest_finish_\n-- \n2.45.2.1165.ga18b536d12\n\n"},{"id":"497939","messageId":"20240701220852.GB20631@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240701220815.GA20293@coredump.intra.peff.net","subject":"[PATCH 2/2] t: convert some here-doc test bodies","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-01T22:08:52Z","receivedAt":"2024-07-01T22:08:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The t1404 script checks a lot of output from Git which contains single\nquotes. Because the test snippets are themselves wrapped in the same\nsingle-quotes, we have to resort to using $SQ to match them.  This is\nerror-prone and makes the tests harder to read.\n\nInstead, let's use the new here-doc feature added in the previous\ncommit, which lets us write anything in the test body we want (except\nthe here-doc end marker on a line by itself, of course).\n\nNote that we do use \"\\\" in our marker to avoid interpolation (which is\nthe whole point). But we don't use \"<<-\", as we want to preserve\nwhitespace in the snippet (and running with \"-v\" before and after shows\nthat we produce the exact same output, except with the ugly $SQ\nreferences fixed).\n\nI just converted every test here, even though only some of them use\n$SQ. But it would be equally correct to mix-and-match styles if we don't\nmind the inconsistency.\n\nI've also converted a few tests in t0600 which were moved from t1404 (I\nhad written this patch before they were moved, but it seemed worth\nporting over the changes rather than losing them).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/t0600-reffiles-backend.sh  |  38 +++----\n t/t1404-update-ref-errors.sh | 196 +++++++++++++++++------------------\n 2 files changed, 117 insertions(+), 117 deletions(-)\n\ndiff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh\nindex b2a771ff2b..20df336cc3 100755\n--- a/t/t0600-reffiles-backend.sh\n+++ b/t/t0600-reffiles-backend.sh\n@@ -91,82 +91,82 @@ test_expect_success 'empty directory should not fool 1-arg delete' '\n \tgit update-ref --stdin\n '\n \n-test_expect_success 'non-empty directory blocks create' '\n+test_expect_success 'non-empty directory blocks create' - <<\\EOT\n \tprefix=refs/ne-create &&\n \tmkdir -p .git/$prefix/foo/bar &&\n \t: >.git/$prefix/foo/bar/baz.lock &&\n \ttest_when_finished \"rm -f .git/$prefix/foo/bar/baz.lock\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: there is a non-empty directory $SQ.git/$prefix/foo$SQ blocking reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': there is a non-empty directory '.git/$prefix/foo' blocking reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'broken reference blocks create' '\n+test_expect_success 'broken reference blocks create' - <<\\EOT\n \tprefix=refs/broken-create &&\n \tmkdir -p .git/$prefix &&\n \techo \"gobbledigook\" >.git/$prefix/foo &&\n \ttest_when_finished \"rm -f .git/$prefix/foo\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'non-empty directory blocks indirect create' '\n+test_expect_success 'non-empty directory blocks indirect create' - <<\\EOT\n \tprefix=refs/ne-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tmkdir -p .git/$prefix/foo/bar &&\n \t: >.git/$prefix/foo/bar/baz.lock &&\n \ttest_when_finished \"rm -f .git/$prefix/foo/bar/baz.lock\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: there is a non-empty directory $SQ.git/$prefix/foo$SQ blocking reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': there is a non-empty directory '.git/$prefix/foo' blocking reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'broken reference blocks indirect create' '\n+test_expect_success 'broken reference blocks indirect create' - <<\\EOT\n \tprefix=refs/broken-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \techo \"gobbledigook\" >.git/$prefix/foo &&\n \ttest_when_finished \"rm -f .git/$prefix/foo\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n test_expect_success 'no bogus intermediate values during delete' '\n \tprefix=refs/slow-transaction &&\n@@ -224,7 +224,7 @@ test_expect_success 'no bogus intermediate values during delete' '\n \ttest_must_fail git rev-parse --verify --quiet $prefix/foo\n '\n \n-test_expect_success 'delete fails cleanly if packed-refs file is locked' '\n+test_expect_success 'delete fails cleanly if packed-refs file is locked' - <<\\EOT\n \tprefix=refs/locked-packed-refs &&\n \t# Set up a reference with differing loose and packed versions:\n \tgit update-ref $prefix/foo $C &&\n@@ -236,9 +236,9 @@ test_expect_success 'delete fails cleanly if packed-refs file is locked' '\n \ttest_when_finished \"rm -f .git/packed-refs.lock\" &&\n \ttest_must_fail git update-ref -d $prefix/foo >out 2>err &&\n \tgit for-each-ref $prefix >actual &&\n-\ttest_grep \"Unable to create $SQ.*packed-refs.lock$SQ: \" err &&\n+\ttest_grep \"Unable to create '.*packed-refs.lock': \" err &&\n \ttest_cmp unchanged actual\n-'\n+EOT\n \n test_expect_success 'delete fails cleanly if packed-refs.new write fails' '\n \t# Setup and expectations are similar to the test above.\ndiff --git a/t/t1404-update-ref-errors.sh b/t/t1404-update-ref-errors.sh\nindex 67ebd81a4c..df90112618 100755\n--- a/t/t1404-update-ref-errors.sh\n+++ b/t/t1404-update-ref-errors.sh\n@@ -100,297 +100,297 @@ df_test() {\n \t\tprintf \"%s\\n\" \"delete $delname\" \"create $addname $D\"\n \tfi >commands &&\n \ttest_must_fail git update-ref --stdin <commands 2>output.err &&\n-\tgrep -E \"fatal:( cannot lock ref $SQ$addname$SQ:)? $SQ$delref$SQ exists; cannot create $SQ$addref$SQ\" output.err &&\n+\tgrep -E \"fatal:( cannot lock ref '$addname':)? '$delref' exists; cannot create '$addref'\" output.err &&\n \tprintf \"%s\\n\" \"$C $delref\" >expected-refs &&\n \tgit for-each-ref --format=\"%(objectname) %(refname)\" $prefix/r >actual-refs &&\n \ttest_cmp expected-refs actual-refs\n }\n \n-test_expect_success 'setup' '\n+test_expect_success 'setup' - <<\\EOT\n \n \tgit commit --allow-empty -m Initial &&\n \tC=$(git rev-parse HEAD) &&\n \tgit commit --allow-empty -m Second &&\n \tD=$(git rev-parse HEAD) &&\n \tgit commit --allow-empty -m Third &&\n \tE=$(git rev-parse HEAD)\n-'\n+EOT\n \n-test_expect_success 'existing loose ref is a simple prefix of new' '\n+test_expect_success 'existing loose ref is a simple prefix of new' - <<\\EOT\n \n \tprefix=refs/1l &&\n \ttest_update_rejected \"a c e\" false \"b c/x d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing packed ref is a simple prefix of new' '\n+test_expect_success 'existing packed ref is a simple prefix of new' - <<\\EOT\n \n \tprefix=refs/1p &&\n \ttest_update_rejected \"a c e\" true \"b c/x d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing loose ref is a deeper prefix of new' '\n+test_expect_success 'existing loose ref is a deeper prefix of new' - <<\\EOT\n \n \tprefix=refs/2l &&\n \ttest_update_rejected \"a c e\" false \"b c/x/y d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x/y$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x/y'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing packed ref is a deeper prefix of new' '\n+test_expect_success 'existing packed ref is a deeper prefix of new' - <<\\EOT\n \n \tprefix=refs/2p &&\n \ttest_update_rejected \"a c e\" true \"b c/x/y d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x/y$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x/y'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a simple prefix of existing loose' '\n+test_expect_success 'new ref is a simple prefix of existing loose' - <<\\EOT\n \n \tprefix=refs/3l &&\n \ttest_update_rejected \"a c/x e\" false \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a simple prefix of existing packed' '\n+test_expect_success 'new ref is a simple prefix of existing packed' - <<\\EOT\n \n \tprefix=refs/3p &&\n \ttest_update_rejected \"a c/x e\" true \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a deeper prefix of existing loose' '\n+test_expect_success 'new ref is a deeper prefix of existing loose' - <<\\EOT\n \n \tprefix=refs/4l &&\n \ttest_update_rejected \"a c/x/y e\" false \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x/y$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x/y' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a deeper prefix of existing packed' '\n+test_expect_success 'new ref is a deeper prefix of existing packed' - <<\\EOT\n \n \tprefix=refs/4p &&\n \ttest_update_rejected \"a c/x/y e\" true \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x/y$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x/y' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'one new ref is a simple prefix of another' '\n+test_expect_success 'one new ref is a simple prefix of another' - <<\\EOT\n \n \tprefix=refs/5 &&\n \ttest_update_rejected \"a e\" false \"b c c/x d\" \\\n-\t\t\"cannot process $SQ$prefix/c$SQ and $SQ$prefix/c/x$SQ at the same time\"\n+\t\t\"cannot process '$prefix/c' and '$prefix/c/x' at the same time\"\n \n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + delete short' '\n+test_expect_success 'D/F conflict prevents add long + delete short' - <<\\EOT\n \tdf_test refs/df-al-ds --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add short + delete long' '\n+test_expect_success 'D/F conflict prevents add short + delete long' - <<\\EOT\n \tdf_test refs/df-as-dl --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete long + add short' '\n+test_expect_success 'D/F conflict prevents delete long + add short' - <<\\EOT\n \tdf_test refs/df-dl-as --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete short + add long' '\n+test_expect_success 'D/F conflict prevents delete short + add long' - <<\\EOT\n \tdf_test refs/df-ds-al --del-add foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + delete short packed' '\n+test_expect_success 'D/F conflict prevents add long + delete short packed' - <<\\EOT\n \tdf_test refs/df-al-dsp --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add short + delete long packed' '\n+test_expect_success 'D/F conflict prevents add short + delete long packed' - <<\\EOT\n \tdf_test refs/df-as-dlp --pack --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete long packed + add short' '\n+test_expect_success 'D/F conflict prevents delete long packed + add short' - <<\\EOT\n \tdf_test refs/df-dlp-as --pack --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete short packed + add long' '\n+test_expect_success 'D/F conflict prevents delete short packed + add long' - <<\\EOT\n \tdf_test refs/df-dsp-al --pack --del-add foo foo/bar\n-'\n+EOT\n \n # Try some combinations involving symbolic refs...\n \n-test_expect_success 'D/F conflict prevents indirect add long + delete short' '\n+test_expect_success 'D/F conflict prevents indirect add long + delete short' - <<\\EOT\n \tdf_test refs/df-ial-ds --sym-add --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + indirect delete short' '\n+test_expect_success 'D/F conflict prevents indirect add long + indirect delete short' - <<\\EOT\n \tdf_test refs/df-ial-ids --sym-add --sym-del --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add short + indirect delete long' '\n+test_expect_success 'D/F conflict prevents indirect add short + indirect delete long' - <<\\EOT\n \tdf_test refs/df-ias-idl --sym-add --sym-del --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect delete long + indirect add short' '\n+test_expect_success 'D/F conflict prevents indirect delete long + indirect add short' - <<\\EOT\n \tdf_test refs/df-idl-ias --sym-add --sym-del --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + delete short packed' '\n+test_expect_success 'D/F conflict prevents indirect add long + delete short packed' - <<\\EOT\n \tdf_test refs/df-ial-dsp --sym-add --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + indirect delete short packed' '\n+test_expect_success 'D/F conflict prevents indirect add long + indirect delete short packed' - <<\\EOT\n \tdf_test refs/df-ial-idsp --sym-add --sym-del --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + indirect delete short packed' '\n+test_expect_success 'D/F conflict prevents add long + indirect delete short packed' - <<\\EOT\n \tdf_test refs/df-al-idsp --sym-del --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect delete long packed + indirect add short' '\n+test_expect_success 'D/F conflict prevents indirect delete long packed + indirect add short' - <<\\EOT\n \tdf_test refs/df-idlp-ias --sym-add --sym-del --pack --del-add foo/bar foo\n-'\n+EOT\n \n # Test various errors when reading the old values of references...\n \n-test_expect_success 'missing old value blocks update' '\n+test_expect_success 'missing old value blocks update' - <<\\EOT\n \tprefix=refs/missing-update &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks update' '\n+test_expect_success 'incorrect old value blocks update' - <<\\EOT\n \tprefix=refs/incorrect-update &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/foo': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks create' '\n+test_expect_success 'existing old value blocks create' - <<\\EOT\n \tprefix=refs/existing-create &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/foo': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"create $prefix/foo $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks delete' '\n+test_expect_success 'incorrect old value blocks delete' - <<\\EOT\n \tprefix=refs/incorrect-delete &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/foo': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"delete $prefix/foo $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'missing old value blocks indirect update' '\n+test_expect_success 'missing old value blocks indirect update' - <<\\EOT\n \tprefix=refs/missing-indirect-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect update' '\n+test_expect_success 'incorrect old value blocks indirect update' - <<\\EOT\n \tprefix=refs/incorrect-indirect-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks indirect create' '\n+test_expect_success 'existing old value blocks indirect create' - <<\\EOT\n \tprefix=refs/existing-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/symref': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"create $prefix/symref $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect delete' '\n+test_expect_success 'incorrect old value blocks indirect delete' - <<\\EOT\n \tprefix=refs/incorrect-indirect-delete &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"delete $prefix/symref $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'missing old value blocks indirect no-deref update' '\n+test_expect_success 'missing old value blocks indirect no-deref update' - <<\\EOT\n \tprefix=refs/missing-noderef-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference is missing but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': reference is missing but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect no-deref update' '\n+test_expect_success 'incorrect old value blocks indirect no-deref update' - <<\\EOT\n \tprefix=refs/incorrect-noderef-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks indirect no-deref create' '\n+test_expect_success 'existing old value blocks indirect no-deref create' - <<\\EOT\n \tprefix=refs/existing-noderef-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/symref': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"create $prefix/symref $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect no-deref delete' '\n+test_expect_success 'incorrect old value blocks indirect no-deref delete' - <<\\EOT\n \tprefix=refs/incorrect-noderef-delete &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"delete $prefix/symref $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n test_done\n-- \n2.45.2.1165.ga18b536d12\n"},{"id":"497945","messageId":"CAPig+cQdioTBinkacLFb7yktm168oB+qyP9MpZtTRE=1aBz5yg@mail.gmail.com","threadId":"61713","inReplyTo":"20240701220840.GA20631@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-01T22:45:19Z","receivedAt":"2024-07-01T22:45:31Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 1, 2024 at 6:08 PM Jeff King <peff@peff.net> wrote:\n> [...]\n> This commit adds another option: feeding the snippet via the function's\n> stdin. This doesn't conflict with anything the snippet would want to do,\n> because we always redirect its stdin from /dev/null anyway (which we'll\n> continue to do).\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n> diff --git a/t/README b/t/README\n> @@ -906,6 +906,14 @@ see test-lib-functions.sh for the full list and their options.\n> +   If <script> is `-` (a single dash), then the script to run is read\n> +   from stdin. This lets you more easily use single quotes within the\n> +   script by using a here-doc. For example:\n> +\n> +       test_expect_success 'output contains expected string' - <<\\EOT\n> +               grep \"this string has 'quotes' in it\" output\n> +       EOT\n\nWe lose `chainlint` functionality for test bodies specified in this manner.\n\nRestoring such functionality will require some (possibly)\nnot-so-subtle changes. There are at least a couple issues which need\nto be addressed:\n\n(1) chainlint.pl:ScriptParser::parse_cmd() only currently recognizes\n`test_expect_* [prereq] 'title' 'body'` but will now also need to\nrecognize `test_expect_success [prereq] 'title' - <body-as-here-doc>`.\n\n(2) Until now, chainlint.pl has never had to concern itself with the\nbody of a here-doc; it just throws them away. With this new calling\nconvention, here-doc bodies become relevant and must be returned by\nthe lexer. This may involve some not-so-minor surgery.\n"},{"id":"497947","messageId":"xmqq8qyku192.fsf@gitster.g","threadId":"61713","inReplyTo":"CAPig+cQdioTBinkacLFb7yktm168oB+qyP9MpZtTRE=1aBz5yg@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-01T23:43:37Z","receivedAt":"2024-07-01T23:43:40Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine <sunshine@sunshineco.com> writes:\n\n> We lose `chainlint` functionality for test bodies specified in this manner.\n\nOuch.\n\n> Restoring such functionality will require some (possibly)\n> not-so-subtle changes. There are at least a couple issues which need\n> to be addressed:\n>\n> (1) chainlint.pl:ScriptParser::parse_cmd() only currently recognizes\n> `test_expect_* [prereq] 'title' 'body'` but will now also need to\n> recognize `test_expect_success [prereq] 'title' - <body-as-here-doc>`.\n>\n> (2) Until now, chainlint.pl has never had to concern itself with the\n> body of a here-doc; it just throws them away. With this new calling\n> convention, here-doc bodies become relevant and must be returned by\n> the lexer. This may involve some not-so-minor surgery.\n"},{"id":"497950","messageId":"20240702005144.GA27170@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cQdioTBinkacLFb7yktm168oB+qyP9MpZtTRE=1aBz5yg@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-02T00:51:44Z","receivedAt":"2024-07-02T00:51:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 01, 2024 at 06:45:19PM -0400, Eric Sunshine wrote:\n\n> > @@ -906,6 +906,14 @@ see test-lib-functions.sh for the full list and their options.\n> > +   If <script> is `-` (a single dash), then the script to run is read\n> > +   from stdin. This lets you more easily use single quotes within the\n> > +   script by using a here-doc. For example:\n> > +\n> > +       test_expect_success 'output contains expected string' - <<\\EOT\n> > +               grep \"this string has 'quotes' in it\" output\n> > +       EOT\n> \n> We lose `chainlint` functionality for test bodies specified in this manner.\n> \n> Restoring such functionality will require some (possibly)\n> not-so-subtle changes. There are at least a couple issues which need\n> to be addressed:\n> \n> (1) chainlint.pl:ScriptParser::parse_cmd() only currently recognizes\n> `test_expect_* [prereq] 'title' 'body'` but will now also need to\n> recognize `test_expect_success [prereq] 'title' - <body-as-here-doc>`.\n> \n> (2) Until now, chainlint.pl has never had to concern itself with the\n> body of a here-doc; it just throws them away. With this new calling\n> convention, here-doc bodies become relevant and must be returned by\n> the lexer. This may involve some not-so-minor surgery.\n\nHmm. The patch below seems to work on a simple test.\n\nThe lexer stuffs the heredoc into a special variable. Which at first\nglance feels like a hack versus returning it from the token stream, but\nthe contents really _aren't_ part of that stream. They're a separate\nmagic thing that is found on the stdin of whatever command the tokens\nrepresent.\n\nAnd then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\ntoken isn't interesting, and that \"-\" means \"read the here-doc\".\n\nObviously we'd want to add to the chainlint tests here. It looks like\nthe current test infrastructure is focused on evaluating snippets, with\nthe test_expect_success part already handled.\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex 1bbd985b78..7eb904afaa 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -168,12 +168,15 @@ sub swallow_heredocs {\n \tmy $self = shift @_;\n \tmy $b = $self->{buff};\n \tmy $tags = $self->{heretags};\n+\t$self->{parser}->{heredoc} = '';\n \twhile (my $tag = shift @$tags) {\n \t\tmy $start = pos($$b);\n \t\tmy $indent = $$tag[0] =~ s/^\\t// ? '\\\\s*' : '';\n \t\t$$b =~ /(?:\\G|\\n)$indent\\Q$$tag[0]\\E(?:\\n|\\z)/gc;\n \t\tif (pos($$b) > $start) {\n \t\t\tmy $body = substr($$b, $start, pos($$b) - $start);\n+\t\t\t$self->{parser}->{heredoc} .=\n+\t\t\t\tsubstr($body, 0, length($body) - length($&));\n \t\t\t$self->{lineno} += () = $body =~ /\\n/sg;\n \t\t\tnext;\n \t\t}\n@@ -618,6 +621,9 @@ sub check_test {\n \tmy $self = shift @_;\n \tmy ($title, $body) = map(unwrap, @_);\n \t$self->{ntests}++;\n+\tif ($body eq '-') {\n+\t\t$body = $self->{heredoc};\n+\t}\n \tmy $parser = TestParser->new(\\$body);\n \tmy @tokens = $parser->parse();\n \tmy $problems = $parser->{problems};\n@@ -648,7 +654,7 @@ sub parse_cmd {\n \tmy @tokens = $self->SUPER::parse_cmd();\n \treturn @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n \tmy $n = $#tokens;\n-\t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n+\t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\||<<[A-Za-z]+)$/;\n \t$self->check_test($tokens[1], $tokens[2]) if $n == 2; # title body\n \t$self->check_test($tokens[2], $tokens[3]) if $n > 2;  # prereq title body\n \treturn @tokens;\n\n-Peff\n"},{"id":"497952","messageId":"20240702011350.GA31739@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240702005144.GA27170@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-02T01:13:50Z","receivedAt":"2024-07-02T01:13:52Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 01, 2024 at 08:51:45PM -0400, Jeff King wrote:\n\n> Hmm. The patch below seems to work on a simple test.\n> \n> The lexer stuffs the heredoc into a special variable. Which at first\n> glance feels like a hack versus returning it from the token stream, but\n> the contents really _aren't_ part of that stream. They're a separate\n> magic thing that is found on the stdin of whatever command the tokens\n> represent.\n> \n> And then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\n> token isn't interesting, and that \"-\" means \"read the here-doc\".\n\nBTW, there's one non-obvious thing here about why this works. You'd\nthink that:\n\n  test_expect_success 'foo' <<\\EOT\n\tcat <<-\\EOF\n\tthis is a here-doc\n\tEOF\n\techo ok\n  EOT\n\nwouldn't work, because the lexer only has a single here-doc store, and\nthe inner one is going to overwrite the outer. But we don't lex the\ninner contents of the test snippet until we've processed the\ntest_expect_success line, at which point we've copied it out.\n\nSo I dunno. It feels a bit hacky, but I think it's how you have to do it\nanyway.\n\n> @@ -648,7 +654,7 @@ sub parse_cmd {\n>  \tmy @tokens = $self->SUPER::parse_cmd();\n>  \treturn @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n>  \tmy $n = $#tokens;\n> -\t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n> +\t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\||<<[A-Za-z]+)$/;\n\nOne curiosity I noted is that the backslash of my \"<<\\EOT\" seems to be\neaten by the lexer (I guess because it doesn't know the special meaning\nof backslash here, and just does the usual \"take the next char\nliterally\"). I think that is OK for our purposes here, though we might\nin the long run want to raise a linting error if you accidentally used\nan interpolating here-doc (it's not strictly wrong to do so, but I think\nwe generally frown on it as a style thing).\n\n-Peff\n"},{"id":"498000","messageId":"20240702211913.GB120950@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240702005144.GA27170@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-02T21:19:13Z","receivedAt":"2024-07-02T21:19:14Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 01, 2024 at 08:51:44PM -0400, Jeff King wrote:\n\n> Obviously we'd want to add to the chainlint tests here. It looks like\n> the current test infrastructure is focused on evaluating snippets, with\n> the test_expect_success part already handled.\n\nSo doing this (with the patch I showed earlier):\n\ndiff --git a/t/Makefile b/t/Makefile\nindex b2eb9f770b..7c97aa3673 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -106,18 +106,28 @@ clean: clean-except-prove-cache\n clean-chainlint:\n \t$(RM) -r '$(CHAINLINTTMP_SQ)'\n \n+CHAINLINTTESTS_SRC = $(patsubst %,chainlint/%.test,$(CHAINLINTTESTS))\n check-chainlint:\n \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n \t\techo \"test_expect_success '$$i' '\" && \\\n \t\tsed -e '/^# LINT: /d' chainlint/$$i.test && \\\n \t\techo \"'\"; \\\n \tdone >'$(CHAINLINTTMP_SQ)'/tests && \\\n+\tfor i in $$(grep -L \"'\" $(CHAINLINTTESTS_SRC)); do \\\n+\t\techo \"test_expect_success '$$i' - <<\\\\\\\\EOT\" && \\\n+\t\tsed -e '/^# LINT: /d' $$i && \\\n+\t\techo \"EOT\"; \\\n+\tdone >>'$(CHAINLINTTMP_SQ)'/tests && \\\n \t{ \\\n \t\techo \"# chainlint: $(CHAINLINTTMP_SQ)/tests\" && \\\n \t\tfor i in $(CHAINLINTTESTS); do \\\n \t\t\techo \"# chainlint: $$i\" && \\\n \t\t\tcat chainlint/$$i.expect; \\\n+\t\tdone && \\\n+\t\tfor i in $$(grep -L \"'\" $(CHAINLINTTESTS_SRC)); do \\\n+\t\t\techo \"# chainlint: $$i\" && \\\n+\t\t\tcat $${i%.test}.expect; \\\n \t\tdone \\\n \t} >'$(CHAINLINTTMP_SQ)'/expect && \\\n \t$(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \\\n\ndoes pass. It's just running all of the tests inside an \"EOT\" block. But\nwe have to omit ones that have single quotes in them, because they are\nmaking the implicit assumption that they're inside a single-quoted block\n(so they do things like '\"$foo\"', or '\\'', etc, which behave differently\nin a here-doc).\n\nIt was a nice check that the output is the same in both cases, but it's\na bit limiting as a test suite, as there's no room to introduce test\ncases that vary the test_expect_success lines. I'm thinking the path\nforward may be:\n\n  1. Move the test_expect_success wrapping lines into each\n     chainlint/*.test file. It's a little bit of extra boilerplate, but\n     it makes them a bit easier to reason about on their own.\n\n  2. Add a few new tests that use here-docs with a few variations\n     (\"<<EOT\", \"<<\\EOT\", probably a here-doc inside the test here-doc).\n\nDoes that sound OK to you?\n\n-Peff\n"},{"id":"498003","messageId":"CAPig+cQ6PLZA=s6D1XsdcFeeg-=ffib9QZGFLycsHWLZZ1ibCg@mail.gmail.com","threadId":"61713","inReplyTo":"20240702005144.GA27170@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-02T21:25:48Z","receivedAt":"2024-07-02T21:26:00Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 1, 2024 at 8:51 PM Jeff King <peff@peff.net> wrote:\n> On Mon, Jul 01, 2024 at 06:45:19PM -0400, Eric Sunshine wrote:\n> > We lose `chainlint` functionality for test bodies specified in this manner.\n>\n> Hmm. The patch below seems to work on a simple test.\n>\n> The lexer stuffs the heredoc into a special variable. Which at first\n> glance feels like a hack versus returning it from the token stream, but\n> the contents really _aren't_ part of that stream. They're a separate\n> magic thing that is found on the stdin of whatever command the tokens\n> represent.\n\nI created a white-room fix for this issue, as well, before taking a\nlook at your patch. The two implementations bear a strong similarity\nwhich suggests that we agree upon the basic approach.\n\nMy implementation, however, takes a more formal and paranoid stance.\nRather than squirreling away only the most-recently-seen heredoc body,\nit stores each heredoc body along with the tag which introduced it.\nThis makes it robust against cases when multiple heredocs are\ninitiated on the same line (even within different parse contexts):\n\n    cat <<EOFA && x=$(cat <<EOFB &&\n    A body\n    EOFA\n    B body\n    EOFB\n\nOf course, that's not likely to come up in the context of\ntest_expect_* calls, but I prefer the added robustness over the more\nlax approach.\n\n> And then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\n> token isn't interesting, and that \"-\" means \"read the here-doc\".\n\nIn my implementation, the `<<` token is \"interesting\" because the\nheredoc tag is attached to it, and the tag is needed to pluck the\nheredoc body from the set of saved bodies (since my implementation\ndoesn't assume most-recently-seen body is the correct one).\n\n> Obviously we'd want to add to the chainlint tests here. It looks like\n> the current test infrastructure is focused on evaluating snippets, with\n> the test_expect_success part already handled.\n\nYes, the \"snippet\" approach is a throwback to the old chainlint.sed\nimplementation when there wasn't any actual parsing going on. As you\nnote, this unfortunately does not allow for testing parsing-related\naspects of the implementation, which is a limitation I most definitely\nfelt when chainlint.pl was implemented. It probably would be a good\nidea to update the infrastructure to allow for more broad testing but\nthat doesn't need to be part of the changes being discussed here.\n\n> diff --git a/t/chainlint.pl b/t/chainlint.pl\n> @@ -168,12 +168,15 @@ sub swallow_heredocs {\n>                 if (pos($$b) > $start) {\n>                         my $body = substr($$b, $start, pos($$b) - $start);\n> +                       $self->{parser}->{heredoc} .=\n> +                               substr($body, 0, length($body) - length($&));\n>                         $self->{lineno} += () = $body =~ /\\n/sg;\n\nIn my implementation, I use regex to strip off the ending tag before\nstoring the heredoc body. When I later looked at your implementation,\nI noticed that you used substr() -- which seems preferable -- but\ndiscovered that it strips too much in some cases. For instance, in\nt0600, I saw that:\n\n    cat >expected <<-\\EOF &&\n    HEAD\n    PSEUDO_WT_HEAD\n    refs/bisect/wt-random\n    refs/heads/main\n    refs/heads/wt-main\n    EOF\n\nwas getting stripped down to:\n\n    HEAD\n    PSEUDO_WT_HEAD\n    refs/bisect/wt-random\n    refs/heads/main\n    refs/heads/wt-ma{{missing-nl}}\n\nIt wasn't immediately obvious why this was happening, though I didn't\nspend a lot of time trying to debug it.\n\nAlthough I think my implementation is complete, I haven't submitted it\nyet because I discovered that the changes you made to t1404 are\ntriggering false-positives:\n\n    # chainlint: t1404-update-ref-errors.sh\n    # chainlint: existing loose ref is a simple prefix of new\n    120 prefix=refs/1l &&\n    121 test_update_rejected a c e false b c/x d \\\n    122   '$prefix/c' exists; ?!AMP?! cannot create '$prefix/c/x'\n\nUnfortunately, I ran out of time, thus haven't tracked down this\nproblem yet. I also haven't tested your implementation yet to\ndetermine if this is due to a change I made or due to a deeper\nexisting issue with chainlint.pl.\n"},{"id":"498005","messageId":"CAPig+cRJZeNTfOB_ygqo7yoYzko6-2wH11cxu2kHeNNBuxVn7g@mail.gmail.com","threadId":"61713","inReplyTo":"20240702011350.GA31739@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-02T21:37:39Z","receivedAt":"2024-07-02T21:37:51Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 1, 2024 at 9:13 PM Jeff King <peff@peff.net> wrote:\n> On Mon, Jul 01, 2024 at 08:51:45PM -0400, Jeff King wrote:\n> > And then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\n> > token isn't interesting, and that \"-\" means \"read the here-doc\".\n>\n> BTW, there's one non-obvious thing here about why this works. You'd\n> think that:\n>\n>   test_expect_success 'foo' <<\\EOT\n>         cat <<-\\EOF\n>         this is a here-doc\n>         EOF\n>         echo ok\n>   EOT\n>\n> wouldn't work, because the lexer only has a single here-doc store, and\n> the inner one is going to overwrite the outer. But we don't lex the\n> inner contents of the test snippet until we've processed the\n> test_expect_success line, at which point we've copied it out.\n>\n> So I dunno. It feels a bit hacky, but I think it's how you have to do it\n> anyway.\n\nIt wasn't non-obvious to me, but I suppose it's because I know the\nauthor, or I am the author, or something.\n\n> > -     $n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n> > +     $n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\||<<[A-Za-z]+)$/;\n>\n> One curiosity I noted is that the backslash of my \"<<\\EOT\" seems to be\n> eaten by the lexer (I guess because it doesn't know the special meaning\n> of backslash here, and just does the usual \"take the next char\n> literally\").\n\nThat's not the reason. It actively strips the backslash because it\nknows that it doesn't care about it after this point and, more\nimportantly, because it needs to extract the raw heredoc tag name\n(without the slash or other surrounding quotes) so that it can match\nupon that name (say, \"EOF\") to find the end of the heredoc body.\n\nIt's mostly an accident of implementation (and probably a throwback to\nchainlint.sed) that it strips the backslash early in\nLexer::scan_heredoc_tag() even though it doesn't actually have to be\nstripped until Lexer::swallow_heredocs() needs to match the tag name\nto find the end of the heredoc body. Thus, in retrospect, the\nimplementation could have retained the backslash (`\\EOF`) or quotes\n(`'EOF'` or `\"EOF\"`) and left it for swallow_heredocs() to strip them\nonly when needed.\n\nThere's another weird throwback to chainlint.sed in\nLexer::scan_heredoc_tag() where it transforms `<<-` into `<<\\t`, which\nis potentially more than a little confusing, especially since it is (I\nbelieve) totally unnecessary in the context of chainlint.pl.\n\n> I think that is OK for our purposes here, though we might\n> in the long run want to raise a linting error if you accidentally used\n> an interpolating here-doc (it's not strictly wrong to do so, but I think\n> we generally frown on it as a style thing).\n\nSuch a linting warning would probably have to be context-sensitive so\nit only triggers for test_expect_* calls.\n"},{"id":"498007","messageId":"CAPig+cQna9nDzr9it2=VN-4_hUctG+t4=97s5h9KqmNHS=cuuw@mail.gmail.com","threadId":"61713","inReplyTo":"20240702211913.GB120950@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-02T21:59:46Z","receivedAt":"2024-07-02T21:59:58Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jul 2, 2024 at 5:19 PM Jeff King <peff@peff.net> wrote:\n> On Mon, Jul 01, 2024 at 08:51:44PM -0400, Jeff King wrote:\n> > Obviously we'd want to add to the chainlint tests here. It looks like\n> > the current test infrastructure is focused on evaluating snippets, with\n> > the test_expect_success part already handled.\n>\n> So doing this (with the patch I showed earlier):\n>\n> diff --git a/t/Makefile b/t/Makefile\n> @@ -106,18 +106,28 @@ clean: clean-except-prove-cache\n> +       for i in $$(grep -L \"'\" $(CHAINLINTTESTS_SRC)); do \\\n> +               echo \"test_expect_success '$$i' - <<\\\\\\\\EOT\" && \\\n> +               sed -e '/^# LINT: /d' $$i && \\\n> +               echo \"EOT\"; \\\n> +       done >>'$(CHAINLINTTMP_SQ)'/tests && \\\n\nUnfortunately, `grep -L` is not POSIX.\n\n> does pass. It's just running all of the tests inside an \"EOT\" block. But\n> we have to omit ones that have single quotes in them, because they are\n> making the implicit assumption that they're inside a single-quoted block\n> (so they do things like '\"$foo\"', or '\\'', etc, which behave differently\n> in a here-doc).\n>\n> It was a nice check that the output is the same in both cases, but it's\n> a bit limiting as a test suite, as there's no room to introduce test\n> cases that vary the test_expect_success lines.\n\nAgreed. It feels rather hacky and awfully special-case, as it's only\n(additionally) checking that the `test_expect_* title - <<EOT` form\nworks, but doesn't help at all with testing other parsing-related\nbehaviors of chainlint.pl (which is something I definitely wanted to\nbe able to do when implementing the Perl version).\n\n> I'm thinking the path forward may be:\n>\n>   1. Move the test_expect_success wrapping lines into each\n>      chainlint/*.test file. It's a little bit of extra boilerplate, but\n>      it makes them a bit easier to reason about on their own.\n\nYes. This is exactly what I had in mind for moving forward. It's just\na one-time noise-patch cost but gives us much more flexibility in\nterms of testing.\n\nIt also makes spot-testing the chainlint self-test files much simpler.\nWe would be able to do this:\n\n    ./chainlint.pl chainlint/block.test\n\nrather than much more painful:\n\n    { echo \"test_expect_success foo '\" && cat chainlint/block.test &&\necho \"'\"; } >dummy && ./chainlint.pl dummy; rm dummy\n\nor something similar.\n\n>   2. Add a few new tests that use here-docs with a few variations\n>      (\"<<EOT\", \"<<\\EOT\", probably a here-doc inside the test here-doc).\n>\n> Does that sound OK to you?\n\nAbsolutely. I'm very much in favor of these changes.\n"},{"id":"498008","messageId":"CAPig+cTExD6_5N63Rs4mqp+DqgLVq+aXM84Kp2T0YicMfGX9Sw@mail.gmail.com","threadId":"61713","inReplyTo":"CAPig+cQ6PLZA=s6D1XsdcFeeg-=ffib9QZGFLycsHWLZZ1ibCg@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-02T22:36:44Z","receivedAt":"2024-07-02T22:36:56Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jul 2, 2024 at 5:25 PM Eric Sunshine <sunshine@sunshineco.com> wrote:\n> On Mon, Jul 1, 2024 at 8:51 PM Jeff King <peff@peff.net> wrote:\n> >                         my $body = substr($$b, $start, pos($$b) - $start);\n> > +                       $self->{parser}->{heredoc} .=\n> > +                               substr($body, 0, length($body) - length($&));\n> >                         $self->{lineno} += () = $body =~ /\\n/sg;\n>\n> In my implementation, I use regex to strip off the ending tag before\n> storing the heredoc body. When I later looked at your implementation,\n> I noticed that you used substr() -- which seems preferable -- but\n> discovered that it strips too much in some cases. [...]\n\nNevermind this part. I just looked again at the misbehaving code\n(which I had commented out but not deleted) and noticed that I botched\nthe implementation in two distinct ways. With those botches removed,\nthe substr() approach works just fine.\n"},{"id":"498009","messageId":"CAPig+cRphdkfrH7MrtXrbg1Pn7XnVN6oodZO5QuPqBbPPFXVyA@mail.gmail.com","threadId":"61713","inReplyTo":"CAPig+cQ6PLZA=s6D1XsdcFeeg-=ffib9QZGFLycsHWLZZ1ibCg@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-02T22:48:04Z","receivedAt":"2024-07-02T22:48:15Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jul 2, 2024 at 5:25 PM Eric Sunshine <sunshine@sunshineco.com> wrote:\n> Although I think my implementation is complete, I haven't submitted it\n> yet because I discovered that the changes you made to t1404 are\n> triggering false-positives:\n>\n>     # chainlint: t1404-update-ref-errors.sh\n>     # chainlint: existing loose ref is a simple prefix of new\n>     120 prefix=refs/1l &&\n>     121 test_update_rejected a c e false b c/x d \\\n>     122   '$prefix/c' exists; ?!AMP?! cannot create '$prefix/c/x'\n>\n> Unfortunately, I ran out of time, thus haven't tracked down this\n> problem yet.\n\nThis is also now fixed. It wasn't any deep problem, just a minor oversight.\n"},{"id":"498010","messageId":"20240702235034.88219-1-ericsunshine@charter.net","threadId":"61713","inReplyTo":"20240701220815.GA20293@coredump.intra.peff.net","subject":"[PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"ericsunshine@charter.net","sentAt":"2024-07-02T23:50:34Z","receivedAt":"2024-07-02T23:52:42Z","isPatch":true,"sender":{"key":"ericsunshine@charter.net","avatar":null},"body":"From: Eric Sunshine <sunshine@sunshineco.com>\n\nIn order to check tests for semantic problems, chainlint.pl scans test\nscripts, looking for tests defined as:\n\n    test_expect_success [prereq] title '\n        body\n    '\n\nwhere `body` is a single string which is then treated as a standalone\nchunk of code and \"linted\" to detect semantic issues. (The same happens\nfor `test_expect_failure` definitions.)\n\nThe introduction of test definitions in which the test body is instead\npresented via a heredoc rather than as a single string creates a blind\nspot in the linting process since such invocations are not recognized by\nchainlint.pl.\n\nAddress this shortcoming by also recognizing tests defined as:\n\n    test_expect_success [prereq] title - <<\\EOT\n        body\n    EOT\n\nA minor complication is that chainlint.pl has never considered heredoc\nbodies significant since it doesn't scan them for semantic problems,\nthus it has always simply thrown them away. However, with the new\n`test_expect_success` calling sequence, heredoc bodies become\nmeaningful, thus need to be captured.\n\nSigned-off-by: Eric Sunshine <sunshine@sunshineco.com>\n---\n\nThis is a clean-room implementation which serves the same purpose as a\nchange proposed[1] by Peff; it was created before I looked at Peff's\nproposal. The two independent implementations turned out quite similar,\nbut the one implemented by this patch takes a more formal and paranoid\nstance. In particular, unlike Peff's patch, it doesn't trust that the\nmost-recently-seen heredoc body is one associated with the\n`test_expect_success` invocation.\n\nThis patch can sit either at the top or bottom of Peff's series[2].\n\nThere was also related discussion of improving the chainlint self-test\ninfrastructure[3], however, such proposed changes needn't hold up Peff's\nseries[2]; such improvements can be applied after the dust settles. On\nthe other hand, Peff, if you plan to reroll for some reason, feel free\nto incorporate this patch into your series.\n\n[1]: https://lore.kernel.org/git/20240702005144.GA27170@coredump.intra.peff.net/\n[2]: https://lore.kernel.org/git/20240701220815.GA20293@coredump.intra.peff.net/\n[3]: https://lore.kernel.org/git/20240702211913.GB120950@coredump.intra.peff.net/\n\n t/chainlint.pl | 18 ++++++++++++++----\n 1 file changed, 14 insertions(+), 4 deletions(-)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex 1bbd985b78..eba509b8e1 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -174,6 +174,8 @@ sub swallow_heredocs {\n \t\t$$b =~ /(?:\\G|\\n)$indent\\Q$$tag[0]\\E(?:\\n|\\z)/gc;\n \t\tif (pos($$b) > $start) {\n \t\t\tmy $body = substr($$b, $start, pos($$b) - $start);\n+\t\t\t$self->{parser}->{heredocs}->{$$tag[0]} =\n+\t\t\t    substr($body, 0, length($body) - length($&));\n \t\t\t$self->{lineno} += () = $body =~ /\\n/sg;\n \t\t\tnext;\n \t\t}\n@@ -232,7 +234,8 @@ sub new {\n \tmy $self = bless {\n \t\tbuff => [],\n \t\tstop => [],\n-\t\toutput => []\n+\t\toutput => [],\n+\t\theredocs => {},\n \t} => $class;\n \t$self->{lexer} = Lexer->new($self, $s);\n \treturn $self;\n@@ -616,7 +619,9 @@ sub unwrap {\n \n sub check_test {\n \tmy $self = shift @_;\n-\tmy ($title, $body) = map(unwrap, @_);\n+\tmy $title = unwrap(shift @_);\n+\tmy $body = unwrap(shift @_);\n+\t$body = shift @_ if $body eq '-';\n \t$self->{ntests}++;\n \tmy $parser = TestParser->new(\\$body);\n \tmy @tokens = $parser->parse();\n@@ -649,8 +654,13 @@ sub parse_cmd {\n \treturn @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n \tmy $n = $#tokens;\n \t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n-\t$self->check_test($tokens[1], $tokens[2]) if $n == 2; # title body\n-\t$self->check_test($tokens[2], $tokens[3]) if $n > 2;  # prereq title body\n+\tmy $herebody;\n+\tif ($n >= 2 && $tokens[$n-1]->[0] eq '-' && $tokens[$n]->[0] =~ /^<<-?(.+)$/) {\n+\t\t$herebody = $self->{heredocs}->{$1};\n+\t\t$n--;\n+\t}\n+\t$self->check_test($tokens[1], $tokens[2], $herebody) if $n == 2; # title body\n+\t$self->check_test($tokens[2], $tokens[3], $herebody) if $n > 2;  # prereq title body\n \treturn @tokens;\n }\n \n-- \n2.45.2\n\n"},{"id":"498132","messageId":"20240706052317.GA698153@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cQna9nDzr9it2=VN-4_hUctG+t4=97s5h9KqmNHS=cuuw@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T05:23:17Z","receivedAt":"2024-07-06T05:23:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 02, 2024 at 05:59:46PM -0400, Eric Sunshine wrote:\n\n> > diff --git a/t/Makefile b/t/Makefile\n> > @@ -106,18 +106,28 @@ clean: clean-except-prove-cache\n> > +       for i in $$(grep -L \"'\" $(CHAINLINTTESTS_SRC)); do \\\n> > +               echo \"test_expect_success '$$i' - <<\\\\\\\\EOT\" && \\\n> > +               sed -e '/^# LINT: /d' $$i && \\\n> > +               echo \"EOT\"; \\\n> > +       done >>'$(CHAINLINTTMP_SQ)'/tests && \\\n> \n> Unfortunately, `grep -L` is not POSIX.\n\nYeah, this was just for illustration. Even if it were portable, I don't\nthink it's a good direction. :)\n\n> >   1. Move the test_expect_success wrapping lines into each\n> >      chainlint/*.test file. It's a little bit of extra boilerplate, but\n> >      it makes them a bit easier to reason about on their own.\n> \n> Yes. This is exactly what I had in mind for moving forward. It's just\n> a one-time noise-patch cost but gives us much more flexibility in\n> terms of testing.\n> \n> It also makes spot-testing the chainlint self-test files much simpler.\n> We would be able to do this:\n> \n>     ./chainlint.pl chainlint/block.test\n> \n> rather than much more painful:\n> \n>     { echo \"test_expect_success foo '\" && cat chainlint/block.test &&\n> echo \"'\"; } >dummy && ./chainlint.pl dummy; rm dummy\n\nOh, nice. Having just written new chainlint tests, this made checking\nthem _way_ easier.\n\n> >   2. Add a few new tests that use here-docs with a few variations\n> >      (\"<<EOT\", \"<<\\EOT\", probably a here-doc inside the test here-doc).\n> >\n> > Does that sound OK to you?\n> \n> Absolutely. I'm very much in favor of these changes.\n\nGreat! I have patches which I'll send out in a moment.\n\n-Peff\n"},{"id":"498133","messageId":"20240706053105.GB698153@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cQ6PLZA=s6D1XsdcFeeg-=ffib9QZGFLycsHWLZZ1ibCg@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T05:31:05Z","receivedAt":"2024-07-06T05:31:06Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 02, 2024 at 05:25:48PM -0400, Eric Sunshine wrote:\n\n> I created a white-room fix for this issue, as well, before taking a\n> look at your patch. The two implementations bear a strong similarity\n> which suggests that we agree upon the basic approach.\n> \n> My implementation, however, takes a more formal and paranoid stance.\n> Rather than squirreling away only the most-recently-seen heredoc body,\n> it stores each heredoc body along with the tag which introduced it.\n> This makes it robust against cases when multiple heredocs are\n> initiated on the same line (even within different parse contexts):\n> \n>     cat <<EOFA && x=$(cat <<EOFB &&\n>     A body\n>     EOFA\n>     B body\n>     EOFB\n> \n> Of course, that's not likely to come up in the context of\n> test_expect_* calls, but I prefer the added robustness over the more\n> lax approach.\n\nYes, that's so much better than what I wrote. I didn't engage my brain\nvery much when I read the in-code comments about multiple tags on the\nsame line, and I thought you meant:\n\n  cat <<FOO <<BAR\n  this is foo\n  FOO\n  this is bar\n  BAR\n\nwhich is...weird. It does \"work\" in the sense that \"FOO\" is a here-doc\nthat should be skipped past. But it is not doing anything useful; cat\nsees only \"this is bar\" on stdin. So even for this case, the appending\nbehavior that my patch does would not make sense.\n\nAnd of course for the actual useful thing, which you wrote above,\nappending is just nonsense. Recording and accessing by tag is the right\nthing.\n\n> > And then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\n> > token isn't interesting, and that \"-\" means \"read the here-doc\".\n> \n> In my implementation, the `<<` token is \"interesting\" because the\n> heredoc tag is attached to it, and the tag is needed to pluck the\n> heredoc body from the set of saved bodies (since my implementation\n> doesn't assume most-recently-seen body is the correct one).\n\nAh, OK. So it would probably not be that big of a deal to record a\nsingle bit for \"this heredoc is interpolated\". But until we have\nanything useful to do with that information, let's not worry about it\nfor now.\n\n> > diff --git a/t/chainlint.pl b/t/chainlint.pl\n> > @@ -168,12 +168,15 @@ sub swallow_heredocs {\n> >                 if (pos($$b) > $start) {\n> >                         my $body = substr($$b, $start, pos($$b) - $start);\n> > +                       $self->{parser}->{heredoc} .=\n> > +                               substr($body, 0, length($body) - length($&));\n> >                         $self->{lineno} += () = $body =~ /\\n/sg;\n> \n> In my implementation, I use regex to strip off the ending tag before\n> storing the heredoc body. When I later looked at your implementation,\n> I noticed that you used substr() -- which seems preferable -- but\n> discovered that it strips too much in some cases. For instance, in\n> t0600, I saw that:\n\nYeah, I was afraid of trying another regex, just because there are\noptional bits (like indentation) that we'd have to account for. Since $&\ncontains the match already, that's all taken care of by the existing\nregex. From your follow-up, it sounds like the substr() approach does\nwork (*phew*).\n\n-Peff\n"},{"id":"498134","messageId":"20240706053359.GA698567@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240706053105.GB698153@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T05:33:59Z","receivedAt":"2024-07-06T05:34:00Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jul 06, 2024 at 01:31:05AM -0400, Jeff King wrote:\n\n> > > And then ScriptParser::parse_cmd() just has to recognize that any \"<<\"\n> > > token isn't interesting, and that \"-\" means \"read the here-doc\".\n> > \n> > In my implementation, the `<<` token is \"interesting\" because the\n> > heredoc tag is attached to it, and the tag is needed to pluck the\n> > heredoc body from the set of saved bodies (since my implementation\n> > doesn't assume most-recently-seen body is the correct one).\n> \n> Ah, OK. So it would probably not be that big of a deal to record a\n> single bit for \"this heredoc is interpolated\". But until we have\n> anything useful to do with that information, let's not worry about it\n> for now.\n\nOh, oops. I attached this response to the wrong message (I read them all\nthrough before starting to respond). My response here was about the fact\nthat \"<<\\EOT\" does not record the \"\\\" anywhere from the lexer.\n\nBut yes, for your implementation, we do need to recognize \"<<\\EOT\", etc,\nto pull out \"EOT\".\n\n-Peff\n"},{"id":"498136","messageId":"20240706054415.GC698153@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cRJZeNTfOB_ygqo7yoYzko6-2wH11cxu2kHeNNBuxVn7g@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T05:44:15Z","receivedAt":"2024-07-06T05:44:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 02, 2024 at 05:37:39PM -0400, Eric Sunshine wrote:\n\n> > BTW, there's one non-obvious thing here about why this works. You'd\n> > think that:\n> >\n> >   test_expect_success 'foo' <<\\EOT\n> >         cat <<-\\EOF\n> >         this is a here-doc\n> >         EOF\n> >         echo ok\n> >   EOT\n> >\n> > wouldn't work, because the lexer only has a single here-doc store, and\n> > the inner one is going to overwrite the outer. But we don't lex the\n> > inner contents of the test snippet until we've processed the\n> > test_expect_success line, at which point we've copied it out.\n> >\n> > So I dunno. It feels a bit hacky, but I think it's how you have to do it\n> > anyway.\n> \n> It wasn't non-obvious to me, but I suppose it's because I know the\n> author, or I am the author, or something.\n\n:) I had a brief moment of panic where I thought \"wait, what I sent out\nis going to break in this case!\" and then was surprised when it worked.\n\n> > > -     $n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n> > > +     $n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\||<<[A-Za-z]+)$/;\n> >\n> > One curiosity I noted is that the backslash of my \"<<\\EOT\" seems to be\n> > eaten by the lexer (I guess because it doesn't know the special meaning\n> > of backslash here, and just does the usual \"take the next char\n> > literally\").\n> \n> That's not the reason. It actively strips the backslash because it\n> knows that it doesn't care about it after this point and, more\n> importantly, because it needs to extract the raw heredoc tag name\n> (without the slash or other surrounding quotes) so that it can match\n> upon that name (say, \"EOF\") to find the end of the heredoc body.\n> \n> It's mostly an accident of implementation (and probably a throwback to\n> chainlint.sed) that it strips the backslash early in\n> Lexer::scan_heredoc_tag() even though it doesn't actually have to be\n> stripped until Lexer::swallow_heredocs() needs to match the tag name\n> to find the end of the heredoc body. Thus, in retrospect, the\n> implementation could have retained the backslash (`\\EOF`) or quotes\n> (`'EOF'` or `\"EOF\"`) and left it for swallow_heredocs() to strip them\n> only when needed.\n\nOK. I think it does make things easier to normalize this a bit, so that\nScriptParser::parse_cmd() doesn't have to worry about all of the various\nspellings. If we recorded a single bit for \"this was quoted\" alongside\nthe heredoc contents, that would be plenty. But as I (erroneously) said\nelsewhere, we can worry about that later if we find something useful to\ndo with it.\n\n> There's another weird throwback to chainlint.sed in\n> Lexer::scan_heredoc_tag() where it transforms `<<-` into `<<\\t`, which\n> is potentially more than a little confusing, especially since it is (I\n> believe) totally unnecessary in the context of chainlint.pl.\n\nAh, I hadn't noticed that. Looks like we use it in swallow_heredocs() to\nread the tag data itself. But importantly the token stream still has\nthe correct original in it, which we need to correctly match in\nScriptParser::parse_cmd().\n\n> > I think that is OK for our purposes here, though we might\n> > in the long run want to raise a linting error if you accidentally used\n> > an interpolating here-doc (it's not strictly wrong to do so, but I think\n> > we generally frown on it as a style thing).\n> \n> Such a linting warning would probably have to be context-sensitive so\n> it only triggers for test_expect_* calls.\n\nYes, definitely.\n\n-Peff\n"},{"id":"498139","messageId":"20240706060143.GD698153@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240702235034.88219-1-ericsunshine@charter.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:01:43Z","receivedAt":"2024-07-06T06:01:44Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 02, 2024 at 07:50:34PM -0400, Eric Sunshine wrote:\n\n> This is a clean-room implementation which serves the same purpose as a\n> change proposed[1] by Peff; it was created before I looked at Peff's\n> proposal. The two independent implementations turned out quite similar,\n> but the one implemented by this patch takes a more formal and paranoid\n> stance. In particular, unlike Peff's patch, it doesn't trust that the\n> most-recently-seen heredoc body is one associated with the\n> `test_expect_success` invocation.\n\nThanks for working on this! I think this is better than the patch I\nshowed earlier. But I am still glad to have worked on that one, because\nthere is no way I'd be able to intelligently review that one without\nhaving poked at the code so much myself.\n\n> This patch can sit either at the top or bottom of Peff's series[2].\n> \n> There was also related discussion of improving the chainlint self-test\n> infrastructure[3], however, such proposed changes needn't hold up Peff's\n> series[2]; such improvements can be applied after the dust settles. On\n> the other hand, Peff, if you plan to reroll for some reason, feel free\n> to incorporate this patch into your series.\n\nIMHO we want it all to come together. We should not allow \"<<\\EOT\"\nwithout making sure we can chainlint the test bodies, and we should not\nmake such a big change to chainlint.pl without tests to make sure it\nworks.\n\nI'll post some patches in a moment:\n\n  [1/3]: chainlint.pl: fix line number reporting\n  [2/3]: t/chainlint: add test_expect_success call to test snippets\n  [3/3]: t/chainlint: add tests for test body in heredoc\n\nwith the idea that we'd apply your patch here on top of what Junio has\nqueued in jk/test-body-in-here-doc, and then these three on top. For\nJunio's sanity, I'll roll it all up into one series. But I wanted to\nshow it to you incrementally first, especially because I think the fixes\nfrom patch 1/3 above should probably just get squashed in (or even\nrewritten). I'll discuss the bugs they fix below.\n\n> diff --git a/t/chainlint.pl b/t/chainlint.pl\n> index 1bbd985b78..eba509b8e1 100755\n> --- a/t/chainlint.pl\n> +++ b/t/chainlint.pl\n> @@ -174,6 +174,8 @@ sub swallow_heredocs {\n>  \t\t$$b =~ /(?:\\G|\\n)$indent\\Q$$tag[0]\\E(?:\\n|\\z)/gc;\n>  \t\tif (pos($$b) > $start) {\n>  \t\t\tmy $body = substr($$b, $start, pos($$b) - $start);\n> +\t\t\t$self->{parser}->{heredocs}->{$$tag[0]} =\n> +\t\t\t    substr($body, 0, length($body) - length($&));\n>  \t\t\t$self->{lineno} += () = $body =~ /\\n/sg;\n>  \t\t\tnext;\n>  \t\t}\n\nOK, this part looks familiar. :)\n\n> @@ -232,7 +234,8 @@ sub new {\n>  \tmy $self = bless {\n>  \t\tbuff => [],\n>  \t\tstop => [],\n> -\t\toutput => []\n> +\t\toutput => [],\n> +\t\theredocs => {},\n>  \t} => $class;\n>  \t$self->{lexer} = Lexer->new($self, $s);\n>  \treturn $self;\n\nI think initializing is not strictly necessary here, since we'd only try\nto read tags if we saw a here-doc. But there might be some invalid cases\nwhere we could convince higher-level code to look for tags even though\nthere were none (and generate a perl warning about trying to dereference\nundef as a hashref).\n\nOn the flip side, what about cleaning up? The \"heretags\" array is\nemptied as we parse the heredocs in swallow_heredocs(). But I think once\na ShellParser's $self->{heredocs}->{FOO} is written, it will hang around\nforever (even though it's valid only for that one command). Probably not\na big deal, but there's probably some correct spot to reset it.\n\n> @@ -616,7 +619,9 @@ sub unwrap {\n>  \n>  sub check_test {\n>  \tmy $self = shift @_;\n> -\tmy ($title, $body) = map(unwrap, @_);\n> +\tmy $title = unwrap(shift @_);\n> +\tmy $body = unwrap(shift @_);\n> +\t$body = shift @_ if $body eq '-';\n>  \t$self->{ntests}++;\n>  \tmy $parser = TestParser->new(\\$body);\n>  \tmy @tokens = $parser->parse();\n\nThis has two problems related to line numbers. You can't see it in the\ncontext, but we later do:\n\n  my $lineno = $_[1]->[3];\n\nNow that we're shifting @_, that array item is gone.\n\nThe second is that the line number for the here-doc is actually one past\nthe initial line number of the test_expect_success. That works\nautomatically for hanging single-quotes, since the newline from that\nline is inside the quoted area. But for a here-doc, we have to account\nfor it manually. In my original patch I prepended \"\\n\", but you can also\njust increment $lineno (which is what I did in the fix I'm about to\nsend).\n\n> @@ -649,8 +654,13 @@ sub parse_cmd {\n>  \treturn @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n>  \tmy $n = $#tokens;\n>  \t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n> -\t$self->check_test($tokens[1], $tokens[2]) if $n == 2; # title body\n> -\t$self->check_test($tokens[2], $tokens[3]) if $n > 2;  # prereq title body\n> +\tmy $herebody;\n> +\tif ($n >= 2 && $tokens[$n-1]->[0] eq '-' && $tokens[$n]->[0] =~ /^<<-?(.+)$/) {\n> +\t\t$herebody = $self->{heredocs}->{$1};\n> +\t\t$n--;\n> +\t}\n> +\t$self->check_test($tokens[1], $tokens[2], $herebody) if $n == 2; # title body\n> +\t$self->check_test($tokens[2], $tokens[3], $herebody) if $n > 2;  # prereq title body\n>  \treturn @tokens;\n>  }\n\nOK, mostly as expected. I think the check for \"-\" here is redundant with\nwhat's in check_test(). We could just feed the heredoc body either way,\nand in the nonsense case of:\n\n  test_expect_success 'title' 'test body' <<EOT\n  nobody reads this!\n  EOT\n\nthe heredoc data would just be ignored.\n\nRequiring \"<<\" at the end is somewhat limiting. E.g. this is valid:\n\n  test_expect_success <<EOT 'title' -\n  the test body\n  EOT\n\nI don't expect anybody to do that, but it would be nice to be more\nrobust if we can. I think the tokens are still wrapped at this point, so\nwe could read through all of them looking for \"<<\" anywhere, without\ngetting confused by \"$(cat <<INNER_HEREDOC)\". I think, anyway (I didn't\ntest).\n\nI didn't address either of those comments in the patches I'm about to\nsend.\n\n-Peff\n"},{"id":"498140","messageId":"20240706060515.GA700151@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240706060143.GD698153@coredump.intra.peff.net","subject":"[PATCH 1/3] chainlint.pl: fix line number reporting","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:05:15Z","receivedAt":"2024-07-06T06:05:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The previous commit taught chainlint.pl to handle test bodies in\nheredocs, but there are two small bugs related to line numbers:\n\n  1. Prior to that commit, we'd leave the title and body untouched in\n     @_. So we could later pull the line number out of the $_[1] array\n     element. Now we shift off the front of the array, so we have to\n     remember that element to grab the line number. This is a regression\n     even for regular:\n\n       test_expect_success 'title' '\n               test body\n       '\n\n     invocations; the lines for ever test started fresh at 0.\n\n  2. For an invocation like the one above, if the test_expect_success\n     line is X, then \"test body\" would correctly start at X+1, since the\n     hanging newline at the start of the single-quoted test body\n     increments the count. But for a here-doc, there is an implicit\n     newline at the end of the token stream before the here-doc starts.\n     We have to increment \"lineno\" to account for this.\n\n     Actually, this is not _quite_ correct, as there could be multiple\n     here-docs, like:\n\n       test_expect_success \"$(cat <<END_OF_TITLE)\" - <<END_OF_TEST\n       this is the title\n       END_OF_TITLE\n       this is the test\n       END_OF_TEST\n\n     in which case we'd need to skip past END_OF_TITLE. Given how\n     unlikely it is for anybody to do this, and since it would only\n     affect line numbers, it's probably not worth caring about too much.\n     The solution would probably be to record the starting line number\n     of each here-doc section in the lexer/shellparser stage.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nNote to the maintainer: do not worry about applying these yet! The\nparent message describes where they'd go in the series, but I'll send a\nfull series once Eric and I have worked out the details. Review comments\nwelcome, of course. :)\n\nI actually suspect the \"record the heredoc line number\" thing would not\nbe too hard. I.e., turn ShellParser's \"heredoc\" hash to point to\nhashrefs like: \"{ content => ..., lineno => ... }\". And that would give\nus a good spot to stick an \"interpolate\" boolean later if we want.\n\n t/chainlint.pl | 10 +++++++---\n 1 file changed, 7 insertions(+), 3 deletions(-)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex eba509b8e1..c9ab79b6b0 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -620,15 +620,19 @@ sub unwrap {\n sub check_test {\n \tmy $self = shift @_;\n \tmy $title = unwrap(shift @_);\n-\tmy $body = unwrap(shift @_);\n-\t$body = shift @_ if $body eq '-';\n+\tmy $body = shift @_;\n+\tmy $lineno = $body->[3];\n+\t$body = unwrap($body);\n+\tif ($body eq '-') {\n+\t\t$body = shift @_;\n+\t\t$lineno++;\n+\t}\n \t$self->{ntests}++;\n \tmy $parser = TestParser->new(\\$body);\n \tmy @tokens = $parser->parse();\n \tmy $problems = $parser->{problems};\n \treturn unless $emit_all || @$problems;\n \tmy $c = main::fd_colors(1);\n-\tmy $lineno = $_[1]->[3];\n \tmy $start = 0;\n \tmy $checked = '';\n \tfor (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {\n-- \n2.45.2.1178.gaaad15bb7b\n\n"},{"id":"498141","messageId":"20240706060639.GB700151@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240706060143.GD698153@coredump.intra.peff.net","subject":"[PATCH 2/3] t/chainlint: add test_expect_success call to test snippets","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:06:39Z","receivedAt":"2024-07-06T06:06:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The chainlint tests are a series of individual files, each holding a\ntest body. The \"make check-chainlint\" target assembles them into a\nsingle file, adding a \"test_expect_success\" function call around each.\nLet's instead include that function call in the files themselves. This\nis a little more boilerplate, but has several advantages:\n\n  1. You can now run chainlint manually on snippets with just \"perl\n     chainlint.perl chainlint/foo.test\". This can make developing and\n     debugging a little easier.\n\n  2. Many of the tests implicitly relied on the syntax of the lines\n     added by the Makefile (in particular the use of single-quotes).\n     This assumption is much easier to see when the single-quotes are\n     alongside the test body.\n\n  3. We had no way to test how the chainlint program handled\n     various test_expect_success lines themselves. Now we'll be able to\n     (and the next patch will add some).\n\nThe change to the .test files was done mechanically, using the same\ntest names they would have been assigned by the Makefile (this is\nimportant to match the expected output). The Makefile has the minimal\nchange to drop the extra lines. A few more things we could do:\n\n  a. I left the for-loop in the Makefile for assembling the test file.\n     This could be collapsed to a single \"sed\" invocation, though we\n     might hit OS command-line limits (the current set of names is over\n     2k). And we need to loop anyway to assemble the expected output.\n     Although...\n\n  b. We could also stick the test names into the \".expect\" files, and\n     then that loop could go away in favor of \"cat\". That's more\n     boilerplate for little gain, though.\n\n  c. I didn't indent the test bodies themselves, as we would if writing\n     real tests. I don't think it really matters beyond aesthetics.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nI'm open to doing any of a-c if you feel strongly.\n\n t/Makefile                                              | 4 +---\n t/chainlint/arithmetic-expansion.test                   | 2 ++\n t/chainlint/bash-array.test                             | 2 ++\n t/chainlint/blank-line-before-esac.test                 | 2 ++\n t/chainlint/blank-line.test                             | 2 ++\n t/chainlint/block-comment.test                          | 2 ++\n t/chainlint/block.test                                  | 2 ++\n t/chainlint/broken-chain.test                           | 2 ++\n t/chainlint/case-comment.test                           | 2 ++\n t/chainlint/case.test                                   | 2 ++\n t/chainlint/chain-break-background.test                 | 2 ++\n t/chainlint/chain-break-continue.test                   | 2 ++\n t/chainlint/chain-break-false.test                      | 2 ++\n t/chainlint/chain-break-return-exit.test                | 2 ++\n t/chainlint/chain-break-status.test                     | 2 ++\n t/chainlint/chained-block.test                          | 2 ++\n t/chainlint/chained-subshell.test                       | 2 ++\n t/chainlint/close-nested-and-parent-together.test       | 2 ++\n t/chainlint/close-subshell.test                         | 2 ++\n t/chainlint/command-substitution-subsubshell.test       | 2 ++\n t/chainlint/command-substitution.test                   | 2 ++\n t/chainlint/comment.test                                | 2 ++\n t/chainlint/complex-if-in-cuddled-loop.test             | 2 ++\n t/chainlint/cuddled-if-then-else.test                   | 2 ++\n t/chainlint/cuddled-loop.test                           | 2 ++\n t/chainlint/cuddled.test                                | 2 ++\n t/chainlint/double-here-doc.test                        | 2 ++\n t/chainlint/dqstring-line-splice.test                   | 2 ++\n t/chainlint/dqstring-no-interpolate.test                | 2 ++\n t/chainlint/empty-here-doc.test                         | 2 ++\n t/chainlint/exclamation.test                            | 2 ++\n t/chainlint/exit-loop.test                              | 2 ++\n t/chainlint/exit-subshell.test                          | 2 ++\n t/chainlint/for-loop-abbreviated.test                   | 2 ++\n t/chainlint/for-loop.test                               | 2 ++\n t/chainlint/function.test                               | 2 ++\n t/chainlint/here-doc-close-subshell.test                | 2 ++\n t/chainlint/here-doc-indent-operator.test               | 2 ++\n t/chainlint/here-doc-multi-line-command-subst.test      | 2 ++\n t/chainlint/here-doc-multi-line-string.test             | 2 ++\n t/chainlint/here-doc.test                               | 2 ++\n t/chainlint/if-condition-split.test                     | 2 ++\n t/chainlint/if-in-loop.test                             | 2 ++\n t/chainlint/if-then-else.test                           | 2 ++\n t/chainlint/incomplete-line.test                        | 2 ++\n t/chainlint/inline-comment.test                         | 2 ++\n t/chainlint/loop-detect-failure.test                    | 2 ++\n t/chainlint/loop-detect-status.test                     | 2 ++\n t/chainlint/loop-in-if.test                             | 2 ++\n t/chainlint/loop-upstream-pipe.test                     | 2 ++\n t/chainlint/multi-line-nested-command-substitution.test | 2 ++\n t/chainlint/multi-line-string.test                      | 2 ++\n t/chainlint/negated-one-liner.test                      | 2 ++\n t/chainlint/nested-cuddled-subshell.test                | 2 ++\n t/chainlint/nested-here-doc.test                        | 2 ++\n t/chainlint/nested-loop-detect-failure.test             | 2 ++\n t/chainlint/nested-subshell-comment.test                | 2 ++\n t/chainlint/nested-subshell.test                        | 2 ++\n t/chainlint/not-heredoc.test                            | 2 ++\n t/chainlint/one-liner-for-loop.test                     | 2 ++\n t/chainlint/one-liner.test                              | 2 ++\n t/chainlint/p4-filespec.test                            | 2 ++\n t/chainlint/pipe.test                                   | 2 ++\n t/chainlint/return-loop.test                            | 2 ++\n t/chainlint/semicolon.test                              | 2 ++\n t/chainlint/sqstring-in-sqstring.test                   | 2 ++\n t/chainlint/subshell-here-doc.test                      | 2 ++\n t/chainlint/subshell-one-liner.test                     | 2 ++\n t/chainlint/t7900-subtree.test                          | 2 ++\n t/chainlint/token-pasting.test                          | 2 ++\n t/chainlint/unclosed-here-doc-indent.test               | 2 ++\n t/chainlint/unclosed-here-doc.test                      | 2 ++\n t/chainlint/while-loop.test                             | 2 ++\n 73 files changed, 145 insertions(+), 3 deletions(-)\n\ndiff --git a/t/Makefile b/t/Makefile\nindex b2eb9f770b..e7a476966e 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -109,9 +109,7 @@ clean-chainlint:\n check-chainlint:\n \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n-\t\techo \"test_expect_success '$$i' '\" && \\\n-\t\tsed -e '/^# LINT: /d' chainlint/$$i.test && \\\n-\t\techo \"'\"; \\\n+\t\tsed -e '/^# LINT: /d' chainlint/$$i.test; \\\n \tdone >'$(CHAINLINTTMP_SQ)'/tests && \\\n \t{ \\\n \t\techo \"# chainlint: $(CHAINLINTTMP_SQ)/tests\" && \\\ndiff --git a/t/chainlint/arithmetic-expansion.test b/t/chainlint/arithmetic-expansion.test\nindex 16206960d8..7b4c5c9a41 100644\n--- a/t/chainlint/arithmetic-expansion.test\n+++ b/t/chainlint/arithmetic-expansion.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'arithmetic-expansion' '\n (\n \tfoo &&\n # LINT: closing \")\" of $((...)) not misinterpreted as subshell-closing \")\"\n@@ -9,3 +10,4 @@\n \tbar=$((42 + 1))\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/bash-array.test b/t/chainlint/bash-array.test\nindex 92bbb777b8..4ca977d299 100644\n--- a/t/chainlint/bash-array.test\n+++ b/t/chainlint/bash-array.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'bash-array' '\n (\n \tfoo &&\n # LINT: \")\" in Bash array assignment not misinterpreted as subshell-closing \")\"\n@@ -10,3 +11,4 @@\n \tbar=${#bar[@]} &&\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/blank-line-before-esac.test b/t/chainlint/blank-line-before-esac.test\nindex cecccad19f..51f02ea0c5 100644\n--- a/t/chainlint/blank-line-before-esac.test\n+++ b/t/chainlint/blank-line-before-esac.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'blank-line-before-esac' '\n # LINT: blank line before \"esac\"\n test_done () {\n \tcase \"$test_failure\" in\n@@ -17,3 +18,4 @@ test_done () {\n \n \tesac\n }\n+'\ndiff --git a/t/chainlint/blank-line.test b/t/chainlint/blank-line.test\nindex 0fdf15b3e1..6f29a491de 100644\n--- a/t/chainlint/blank-line.test\n+++ b/t/chainlint/blank-line.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'blank-line' '\n (\n \n \tnothing &&\n@@ -8,3 +9,4 @@\n \n \n )\n+'\ndiff --git a/t/chainlint/block-comment.test b/t/chainlint/block-comment.test\nindex df2beea888..934ef4113a 100644\n--- a/t/chainlint/block-comment.test\n+++ b/t/chainlint/block-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'block-comment' '\n (\n \t{\n \t\t# show a\n@@ -6,3 +7,4 @@\n \t\techo b\n \t}\n )\n+'\ndiff --git a/t/chainlint/block.test b/t/chainlint/block.test\nindex 4ab69a4afc..a1b6b4dd32 100644\n--- a/t/chainlint/block.test\n+++ b/t/chainlint/block.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'block' '\n (\n # LINT: missing \"&&\" after first \"echo\"\n \tfoo &&\n@@ -25,3 +26,4 @@\n \techo \"done\"\n } &&\n finis\n+'\ndiff --git a/t/chainlint/broken-chain.test b/t/chainlint/broken-chain.test\nindex 2a44aa73b7..1966499ef9 100644\n--- a/t/chainlint/broken-chain.test\n+++ b/t/chainlint/broken-chain.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'broken-chain' '\n (\n \tfoo &&\n # LINT: missing \"&&\" from \"bar\"\n@@ -6,3 +7,4 @@\n # LINT: final statement before closing \")\" legitimately lacks \"&&\"\n \twop\n )\n+'\ndiff --git a/t/chainlint/case-comment.test b/t/chainlint/case-comment.test\nindex 641c157b98..3f31ae9010 100644\n--- a/t/chainlint/case-comment.test\n+++ b/t/chainlint/case-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'case-comment' '\n (\n \tcase \"$x\" in\n \t# found foo\n@@ -9,3 +10,4 @@\n \t\t;;\n \tesac\n )\n+'\ndiff --git a/t/chainlint/case.test b/t/chainlint/case.test\nindex 4cb086bf87..bea21fee4f 100644\n--- a/t/chainlint/case.test\n+++ b/t/chainlint/case.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'case' '\n (\n # LINT: \"...)\" arms in \"case\" not misinterpreted as subshell-closing \")\"\n \tcase \"$x\" in\n@@ -21,3 +22,4 @@\n \tcase \"$y\" in 2) false;; esac\n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/chain-break-background.test b/t/chainlint/chain-break-background.test\nindex e10f656b05..c68e1b04d5 100644\n--- a/t/chainlint/chain-break-background.test\n+++ b/t/chainlint/chain-break-background.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-background' '\n JGIT_DAEMON_PID= &&\n git init --bare empty.git &&\n >empty.git/git-daemon-export-ok &&\n@@ -8,3 +9,4 @@ mkfifo jgit_daemon_output &&\n \tJGIT_DAEMON_PID=$!\n } &&\n test_expect_code 2 git ls-remote --exit-code git://localhost:$JGIT_DAEMON_PORT/empty.git\n+'\ndiff --git a/t/chainlint/chain-break-continue.test b/t/chainlint/chain-break-continue.test\nindex f0af71d8bd..de8119b204 100644\n--- a/t/chainlint/chain-break-continue.test\n+++ b/t/chainlint/chain-break-continue.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-continue' '\n git ls-tree --name-only -r refs/notes/many_notes |\n while read path\n do\n@@ -11,3 +12,4 @@ do\n \t\treturn 1\n \tfi\n done\n+'\ndiff --git a/t/chainlint/chain-break-false.test b/t/chainlint/chain-break-false.test\nindex a5aaff8c8a..f78ad911fc 100644\n--- a/t/chainlint/chain-break-false.test\n+++ b/t/chainlint/chain-break-false.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-false' '\n # LINT: broken &&-chain okay if explicit \"false\" signals failure\n if condition not satisified\n then\n@@ -8,3 +9,4 @@ else\n \techo it went okay\n \tcongratulate user\n fi\n+'\ndiff --git a/t/chainlint/chain-break-return-exit.test b/t/chainlint/chain-break-return-exit.test\nindex 46542edf88..b6f519bb4d 100644\n--- a/t/chainlint/chain-break-return-exit.test\n+++ b/t/chainlint/chain-break-return-exit.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-return-exit' '\n case \"$(git ls-files)\" in\n one) echo pass one ;;\n # LINT: broken &&-chain okay if explicit \"return 1\" signals failuire\n@@ -21,3 +22,4 @@ for i in 1 2 3 4 ; do\n \tgit checkout main -b $i || return $?\n \ttest_commit $i $i $i tag$i || return $?\n done\n+'\ndiff --git a/t/chainlint/chain-break-status.test b/t/chainlint/chain-break-status.test\nindex a6602a7b99..d9fee190d9 100644\n--- a/t/chainlint/chain-break-status.test\n+++ b/t/chainlint/chain-break-status.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-status' '\n # LINT: broken &&-chain okay if next command handles \"$?\" explicitly\n OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 ) &&\n test_match_signal 13 \"$OUT\" &&\n@@ -9,3 +10,4 @@ test_match_signal 13 \"$OUT\" &&\n \ttest \"$ret\" = 3\n } &&\n test_cmp expect actual\n+'\ndiff --git a/t/chainlint/chained-block.test b/t/chainlint/chained-block.test\nindex 86f81ece63..71ef1d0b7f 100644\n--- a/t/chainlint/chained-block.test\n+++ b/t/chainlint/chained-block.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chained-block' '\n # LINT: start of block chained to preceding command\n echo nobody home && {\n \ttest the doohicky\n@@ -9,3 +10,4 @@ GIT_EXTERNAL_DIFF=echo git diff | {\n \tread path oldfile oldhex oldmode newfile newhex newmode &&\n \ttest \"z$oh\" = \"z$oldhex\"\n }\n+'\ndiff --git a/t/chainlint/chained-subshell.test b/t/chainlint/chained-subshell.test\nindex 4ff6ddd8cb..1f11f65398 100644\n--- a/t/chainlint/chained-subshell.test\n+++ b/t/chainlint/chained-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chained-subshell' '\n # LINT: start of subshell chained to preceding command\n mkdir sub && (\n \tcd sub &&\n@@ -11,3 +12,4 @@ test -f $s1\n test $(cat $s2) = tree2path1 &&\n # LINT: closing subshell \")\" correctly detected on same line as \"$(...)\"\n test $(cat $s3) = tree3path1)\n+'\ndiff --git a/t/chainlint/close-nested-and-parent-together.test b/t/chainlint/close-nested-and-parent-together.test\nindex 72d482f76d..56b28b186b 100644\n--- a/t/chainlint/close-nested-and-parent-together.test\n+++ b/t/chainlint/close-nested-and-parent-together.test\n@@ -1,3 +1,5 @@\n+test_expect_success 'close-nested-and-parent-together' '\n (cd foo &&\n \t(bar &&\n \t\tbaz))\n+'\ndiff --git a/t/chainlint/close-subshell.test b/t/chainlint/close-subshell.test\nindex 508ca447fd..b99f80569d 100644\n--- a/t/chainlint/close-subshell.test\n+++ b/t/chainlint/close-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'close-subshell' '\n # LINT: closing \")\" with various decorations (\"&&\", \">\", \"|\", etc.)\n (\n \tfoo\n@@ -25,3 +26,4 @@ fuzzle &&\n (\n \tyop\n )\n+'\ndiff --git a/t/chainlint/command-substitution-subsubshell.test b/t/chainlint/command-substitution-subsubshell.test\nindex 321de2951c..4ea772d60a 100644\n--- a/t/chainlint/command-substitution-subsubshell.test\n+++ b/t/chainlint/command-substitution-subsubshell.test\n@@ -1,3 +1,5 @@\n+test_expect_success 'command-substitution-subsubshell' '\n # LINT: subshell nested in subshell nested in command substitution\n OUT=$( ((large_git 1>&3) | :) 3>&1 ) &&\n test_match_signal 13 \"$OUT\"\n+'\ndiff --git a/t/chainlint/command-substitution.test b/t/chainlint/command-substitution.test\nindex 3bbb002a4c..494d671e80 100644\n--- a/t/chainlint/command-substitution.test\n+++ b/t/chainlint/command-substitution.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'command-substitution' '\n (\n \tfoo &&\n # LINT: closing \")\" of $(...) not misinterpreted as subshell-closing \")\"\n@@ -9,3 +10,4 @@\n \tbar=$(gobble blocks)\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/comment.test b/t/chainlint/comment.test\nindex 113c0c466f..c488beac0d 100644\n--- a/t/chainlint/comment.test\n+++ b/t/chainlint/comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'comment' '\n (\n # LINT: swallow comment lines\n \t# comment 1\n@@ -9,3 +10,4 @@\n \t# comment 3\n \t# comment 4\n )\n+'\ndiff --git a/t/chainlint/complex-if-in-cuddled-loop.test b/t/chainlint/complex-if-in-cuddled-loop.test\nindex 5efeda58b2..f98ae4c42d 100644\n--- a/t/chainlint/complex-if-in-cuddled-loop.test\n+++ b/t/chainlint/complex-if-in-cuddled-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'complex-if-in-cuddled-loop' '\n # LINT: \"for\" loop cuddled with \"(\" and \")\" and nested \"if\" with complex\n # LINT: multi-line condition; indented with spaces, not tabs\n (for i in a b c; do\n@@ -9,3 +10,4 @@\n    fi\n  done) &&\n test ! -f file\n+'\ndiff --git a/t/chainlint/cuddled-if-then-else.test b/t/chainlint/cuddled-if-then-else.test\nindex 7c53f4efe3..b1b42e1aac 100644\n--- a/t/chainlint/cuddled-if-then-else.test\n+++ b/t/chainlint/cuddled-if-then-else.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'cuddled-if-then-else' '\n # LINT: \"if\" cuddled with \"(\" and \")\"; indented with spaces, not tabs\n (if test -z \"\"; then\n     echo empty\n  else\n     echo bizzy\n  fi) &&\n echo foobar\n+'\ndiff --git a/t/chainlint/cuddled-loop.test b/t/chainlint/cuddled-loop.test\nindex 3c2a62f751..6fccb6ac22 100644\n--- a/t/chainlint/cuddled-loop.test\n+++ b/t/chainlint/cuddled-loop.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'cuddled-loop' '\n # LINT: \"while\" loop cuddled with \"(\" and \")\", with embedded (allowed)\n # LINT: \"|| exit {n}\" to exit loop early, and using redirection \"<\" to feed\n # LINT: loop; indented with spaces, not tabs\n ( while read x\n   do foobar bop || exit 1\n   done <file ) &&\n outside subshell\n+'\ndiff --git a/t/chainlint/cuddled.test b/t/chainlint/cuddled.test\nindex 257b5b5eed..5a6ef7a4a6 100644\n--- a/t/chainlint/cuddled.test\n+++ b/t/chainlint/cuddled.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'cuddled' '\n # LINT: first subshell statement cuddled with opening \"(\"\n (cd foo &&\n \tbar\n@@ -20,3 +21,4 @@\n # LINT: same with missing \"&&\"\n (cd foo\n \tbar)\n+'\ndiff --git a/t/chainlint/double-here-doc.test b/t/chainlint/double-here-doc.test\nindex cd584a4357..1b69b7a651 100644\n--- a/t/chainlint/double-here-doc.test\n+++ b/t/chainlint/double-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'double-here-doc' '\n run_sub_test_lib_test_err run-inv-range-start \\\n \t\"--run invalid range start\" \\\n \t--run=\"a-5\" <<-\\EOF &&\n@@ -10,3 +11,4 @@ check_sub_test_lib_test_err run-inv-range-start \\\n EOF_OUT\n > error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}\n EOF_ERR\n+'\ndiff --git a/t/chainlint/dqstring-line-splice.test b/t/chainlint/dqstring-line-splice.test\nindex b40714439f..f6aa637be8 100644\n--- a/t/chainlint/dqstring-line-splice.test\n+++ b/t/chainlint/dqstring-line-splice.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'dqstring-line-splice' '\n # LINT: line-splice within DQ-string\n '\"\n echo 'fatal: reword option of --fixup is mutually exclusive with'\\\n \t'--patch/--interactive/--all/--include/--only' >expect &&\n test_must_fail git commit --fixup=reword:HEAD~ $1 2>actual &&\n test_cmp expect actual\n \"'\n+'\ndiff --git a/t/chainlint/dqstring-no-interpolate.test b/t/chainlint/dqstring-no-interpolate.test\nindex d2f4219cbb..7ae079b558 100644\n--- a/t/chainlint/dqstring-no-interpolate.test\n+++ b/t/chainlint/dqstring-no-interpolate.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'dqstring-no-interpolate' '\n # LINT: regex dollar-sign eol anchor in double-quoted string not special\n grep \"^ ! \\[rejected\\][ ]*$BRANCH -> $BRANCH (non-fast-forward)$\" out &&\n \n@@ -13,3 +14,4 @@ grep \"^\\\\.git\\$\" output.txt &&\n \tcut -d ' ' -f 2 <output | sort >actual &&\n \ttest_cmp expect actual\n \"'\n+'\ndiff --git a/t/chainlint/empty-here-doc.test b/t/chainlint/empty-here-doc.test\nindex 24fc165de3..8b7ab6eb5f 100644\n--- a/t/chainlint/empty-here-doc.test\n+++ b/t/chainlint/empty-here-doc.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'empty-here-doc' '\n git ls-tree $tree path >current &&\n # LINT: empty here-doc\n cat >expected <<\\EOF &&\n EOF\n test_output\n+'\ndiff --git a/t/chainlint/exclamation.test b/t/chainlint/exclamation.test\nindex 323595b5bd..796de21b7c 100644\n--- a/t/chainlint/exclamation.test\n+++ b/t/chainlint/exclamation.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'exclamation' '\n # LINT: \"! word\" is two tokens\n if ! condition; then echo nope; else yep; fi &&\n # LINT: \"!word\" is single token, not two tokens \"!\" and \"word\"\n@@ -6,3 +7,4 @@ test_prerequisite !MINGW &&\n mail uucp!address &&\n # LINT: \"!word!\" is single token, not three tokens \"!\", \"word\", and \"!\"\n echo !whatever!\n+'\ndiff --git a/t/chainlint/exit-loop.test b/t/chainlint/exit-loop.test\nindex 2f038207e1..7e8b68b465 100644\n--- a/t/chainlint/exit-loop.test\n+++ b/t/chainlint/exit-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'exit-loop' '\n (\n \tfor i in a b c\n \tdo\n@@ -25,3 +26,4 @@\n \t\ti=$(($i + 1))\n \tdone\n )\n+'\ndiff --git a/t/chainlint/exit-subshell.test b/t/chainlint/exit-subshell.test\nindex 4e6ab69b88..05dff55cd7 100644\n--- a/t/chainlint/exit-subshell.test\n+++ b/t/chainlint/exit-subshell.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'exit-subshell' '\n (\n # LINT: \"|| exit {n}\" valid subshell escape without hurting &&-chain\n \tfoo || exit 1\n \tbar &&\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/for-loop-abbreviated.test b/t/chainlint/for-loop-abbreviated.test\nindex 1084eccb89..1dd14f2a44 100644\n--- a/t/chainlint/for-loop-abbreviated.test\n+++ b/t/chainlint/for-loop-abbreviated.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'for-loop-abbreviated' '\n # LINT: for-loop lacking optional \"in [word...]\" before \"do\"\n for it\n do\n \tpath=$(expr \"$it\" : '\\([^:]*\\)') &&\n \tgit update-index --add \"$path\" || exit\n done\n+'\ndiff --git a/t/chainlint/for-loop.test b/t/chainlint/for-loop.test\nindex 6cb3428158..6f2489eb19 100644\n--- a/t/chainlint/for-loop.test\n+++ b/t/chainlint/for-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'for-loop' '\n (\n # LINT: \"for\", \"do\", \"done\" do not need \"&&\"\n \tfor i in a b c\n@@ -17,3 +18,4 @@\n \t\tcat $i\n \tdone\n )\n+'\ndiff --git a/t/chainlint/function.test b/t/chainlint/function.test\nindex 5ee59562c9..763fcf3f87 100644\n--- a/t/chainlint/function.test\n+++ b/t/chainlint/function.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'function' '\n # LINT: \"()\" in function definition not mistaken for subshell\n sha1_file() {\n \techo \"$*\" | sed \"s#..#.git/objects/&/#\"\n@@ -11,3 +12,4 @@ remove_object() {\n }\n \n sha1_file arg && remove_object arg\n+'\ndiff --git a/t/chainlint/here-doc-close-subshell.test b/t/chainlint/here-doc-close-subshell.test\nindex b857ff5467..2458f3323b 100644\n--- a/t/chainlint/here-doc-close-subshell.test\n+++ b/t/chainlint/here-doc-close-subshell.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'here-doc-close-subshell' '\n (\n # LINT: line contains here-doc and closes nested subshell\n \tcat <<-\\INPUT)\n \tfizz\n \tINPUT\n+'\ndiff --git a/t/chainlint/here-doc-indent-operator.test b/t/chainlint/here-doc-indent-operator.test\nindex c8a6f18eb4..a2656f47c1 100644\n--- a/t/chainlint/here-doc-indent-operator.test\n+++ b/t/chainlint/here-doc-indent-operator.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-indent-operator' '\n # LINT: whitespace between operator \"<<-\" and tag legal\n cat >expect <<- EOF &&\n header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0\n@@ -11,3 +12,4 @@ this is not indented\n -EOF\n \n cleanup\n+'\ndiff --git a/t/chainlint/here-doc-multi-line-command-subst.test b/t/chainlint/here-doc-multi-line-command-subst.test\nindex 899bc5de8b..8710a8c483 100644\n--- a/t/chainlint/here-doc-multi-line-command-subst.test\n+++ b/t/chainlint/here-doc-multi-line-command-subst.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-multi-line-command-subst' '\n (\n # LINT: line contains here-doc and opens multi-line $(...)\n \tx=$(bobble <<-\\END &&\n@@ -7,3 +8,4 @@\n \t\twiffle)\n \techo $x\n )\n+'\ndiff --git a/t/chainlint/here-doc-multi-line-string.test b/t/chainlint/here-doc-multi-line-string.test\nindex a53edbcc8d..2f496002fd 100644\n--- a/t/chainlint/here-doc-multi-line-string.test\n+++ b/t/chainlint/here-doc-multi-line-string.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-multi-line-string' '\n (\n # LINT: line contains here-doc and opens multi-line string\n \tcat <<-\\TXT && echo \"multi-line\n@@ -6,3 +7,4 @@\n \tTXT\n \tbap\n )\n+'\ndiff --git a/t/chainlint/here-doc.test b/t/chainlint/here-doc.test\nindex 3f5f92cad3..c91b695319 100644\n--- a/t/chainlint/here-doc.test\n+++ b/t/chainlint/here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc' '\n # LINT: stitch together incomplete \\-ending lines\n # LINT: swallow here-doc to avoid false positives in content\n boodle wobba \\\n@@ -28,3 +29,4 @@ morticia\n wednesday\n pugsly\n EOF\n+'\ndiff --git a/t/chainlint/if-condition-split.test b/t/chainlint/if-condition-split.test\nindex 240daa9fd5..9a3b3ed04a 100644\n--- a/t/chainlint/if-condition-split.test\n+++ b/t/chainlint/if-condition-split.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-condition-split' '\n # LINT: \"if\" condition split across multiple lines at \"&&\" or \"||\"\n if bob &&\n    marcia ||\n@@ -6,3 +7,4 @@ then\n \techo \"nomads\"\n \techo \"for sure\"\n fi\n+'\ndiff --git a/t/chainlint/if-in-loop.test b/t/chainlint/if-in-loop.test\nindex 90c23976fe..5be9d1cfa5 100644\n--- a/t/chainlint/if-in-loop.test\n+++ b/t/chainlint/if-in-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-in-loop' '\n (\n \tfor i in a b c\n \tdo\n@@ -13,3 +14,4 @@\n \tdone\n \tbar\n )\n+'\ndiff --git a/t/chainlint/if-then-else.test b/t/chainlint/if-then-else.test\nindex 2055336c2b..6582a7f440 100644\n--- a/t/chainlint/if-then-else.test\n+++ b/t/chainlint/if-then-else.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-then-else' '\n (\n # LINT: \"if\", \"then\", \"elif\", \"else\", \"fi\" do not need \"&&\"\n \tif test -n \"\"\n@@ -27,3 +28,4 @@\n \t\techo empty\n \tfi\n )\n+'\ndiff --git a/t/chainlint/incomplete-line.test b/t/chainlint/incomplete-line.test\nindex d856658083..74a93021eb 100644\n--- a/t/chainlint/incomplete-line.test\n+++ b/t/chainlint/incomplete-line.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'incomplete-line' '\n # LINT: stitch together all incomplete \\-ending lines\n line 1 \\\n line 2 \\\n@@ -10,3 +11,4 @@ line 4 &&\n \tline 7 \\\n \tline 8\n )\n+'\ndiff --git a/t/chainlint/inline-comment.test b/t/chainlint/inline-comment.test\nindex 8f26856e77..4fbbf1058a 100644\n--- a/t/chainlint/inline-comment.test\n+++ b/t/chainlint/inline-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'inline-comment' '\n (\n # LINT: swallow inline comment (leaving command intact)\n \tfoobar && # comment 1\n@@ -10,3 +11,4 @@\n # LINT: \"#\" in string in cuddled subshell not misinterpreted as comment\n (cd foo &&\n \tflibble \"not a # comment\")\n+'\ndiff --git a/t/chainlint/loop-detect-failure.test b/t/chainlint/loop-detect-failure.test\nindex b9791cc802..44673aa394 100644\n--- a/t/chainlint/loop-detect-failure.test\n+++ b/t/chainlint/loop-detect-failure.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-detect-failure' '\n git init r1 &&\n # LINT: loop handles failure explicitly with \"|| return 1\"\n for n in 1 2 3 4 5\n@@ -15,3 +16,4 @@ do\n \tgit -C r2 add large.$n &&\n \tgit -C r2 commit -m \"$n\"\n done\n+'\ndiff --git a/t/chainlint/loop-detect-status.test b/t/chainlint/loop-detect-status.test\nindex 1c6c23cfc9..8b639be073 100644\n--- a/t/chainlint/loop-detect-status.test\n+++ b/t/chainlint/loop-detect-status.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-detect-status' '\n # LINT: \"$?\" handled explicitly within loop body\n (while test $i -le $blobcount\n  do\n@@ -17,3 +18,4 @@\n  cat commit) |\n git fast-import --big-file-threshold=2 &&\n test ! -f exit-status\n+'\ndiff --git a/t/chainlint/loop-in-if.test b/t/chainlint/loop-in-if.test\nindex dfcc3f98fb..b0d0d393cf 100644\n--- a/t/chainlint/loop-in-if.test\n+++ b/t/chainlint/loop-in-if.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-in-if' '\n (\n \tif true\n \tthen\n@@ -13,3 +14,4 @@\n \tfi\n \tbar\n )\n+'\ndiff --git a/t/chainlint/loop-upstream-pipe.test b/t/chainlint/loop-upstream-pipe.test\nindex efb77da897..8415a4db27 100644\n--- a/t/chainlint/loop-upstream-pipe.test\n+++ b/t/chainlint/loop-upstream-pipe.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-upstream-pipe' '\n (\n \tgit rev-list --objects --no-object-names base..loose |\n \twhile read oid\n@@ -9,3 +10,4 @@\n \tdone |\n \tsort -k1\n ) >expect &&\n+'\ndiff --git a/t/chainlint/multi-line-nested-command-substitution.test b/t/chainlint/multi-line-nested-command-substitution.test\nindex 300058341b..e811c63f2b 100644\n--- a/t/chainlint/multi-line-nested-command-substitution.test\n+++ b/t/chainlint/multi-line-nested-command-substitution.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'multi-line-nested-command-substitution' '\n (\n \tfoo &&\n \tx=$(\n@@ -16,3 +17,4 @@ sort &&\n \t\tfip) &&\n \techo fail\n )\n+'\ndiff --git a/t/chainlint/multi-line-string.test b/t/chainlint/multi-line-string.test\nindex 4a0af2107d..7b5048d2ea 100644\n--- a/t/chainlint/multi-line-string.test\n+++ b/t/chainlint/multi-line-string.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'multi-line-string' '\n (\n \tx=\"line 1\n \t\tline 2\n@@ -13,3 +14,4 @@\n \t\tghi\" &&\n \tbarfoo\n )\n+'\ndiff --git a/t/chainlint/negated-one-liner.test b/t/chainlint/negated-one-liner.test\nindex c9598e9153..30f4cc5a9b 100644\n--- a/t/chainlint/negated-one-liner.test\n+++ b/t/chainlint/negated-one-liner.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'negated-one-liner' '\n # LINT: top-level one-liner subshell\n ! (foo && bar) &&\n ! (foo && bar) >baz &&\n \n # LINT: top-level one-liner subshell missing internal \"&&\"\n ! (foo; bar) &&\n ! (foo; bar) >baz\n+'\ndiff --git a/t/chainlint/nested-cuddled-subshell.test b/t/chainlint/nested-cuddled-subshell.test\nindex 8fd656c7b5..31e92d3be4 100644\n--- a/t/chainlint/nested-cuddled-subshell.test\n+++ b/t/chainlint/nested-cuddled-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-cuddled-subshell' '\n (\n # LINT: opening \"(\" cuddled with first nested subshell statement\n \t(cd foo &&\n@@ -29,3 +30,4 @@\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/nested-here-doc.test b/t/chainlint/nested-here-doc.test\nindex f35404bf0f..9505c47a34 100644\n--- a/t/chainlint/nested-here-doc.test\n+++ b/t/chainlint/nested-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-here-doc' '\n # LINT: inner \"EOF\" not misintrepreted as closing ARBITRARY here-doc\n cat <<ARBITRARY >foop &&\n naddle\n@@ -31,3 +32,4 @@ ARBITRARY\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/nested-loop-detect-failure.test b/t/chainlint/nested-loop-detect-failure.test\nindex e6f0c1acfb..3d4b657412 100644\n--- a/t/chainlint/nested-loop-detect-failure.test\n+++ b/t/chainlint/nested-loop-detect-failure.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-loop-detect-failure' '\n # LINT: neither loop handles failure explicitly with \"|| return 1\"\n for i in 0 1 2 3 4 5 6 7 8 9;\n do\n@@ -33,3 +34,4 @@ do\n \t\techo \"$i$j\" >\"path$i$j\" || return 1\n \tdone || return 1\n done\n+'\ndiff --git a/t/chainlint/nested-subshell-comment.test b/t/chainlint/nested-subshell-comment.test\nindex 0215cdb192..b430580ce0 100644\n--- a/t/chainlint/nested-subshell-comment.test\n+++ b/t/chainlint/nested-subshell-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-subshell-comment' '\n (\n \tfoo &&\n \t(\n@@ -11,3 +12,4 @@\n \t)\n \tfuzzy\n )\n+'\ndiff --git a/t/chainlint/nested-subshell.test b/t/chainlint/nested-subshell.test\nindex 440ee9992d..c31da34b73 100644\n--- a/t/chainlint/nested-subshell.test\n+++ b/t/chainlint/nested-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-subshell' '\n (\n \tcd foo &&\n \t(\n@@ -11,3 +12,4 @@\n \t\techo b\n \t) >file\n )\n+'\ndiff --git a/t/chainlint/not-heredoc.test b/t/chainlint/not-heredoc.test\nindex 9aa57346cd..09711e45e0 100644\n--- a/t/chainlint/not-heredoc.test\n+++ b/t/chainlint/not-heredoc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'not-heredoc' '\n # LINT: \"<< ours\" inside string is not here-doc\n echo \"<<<<<<< ours\" &&\n echo ourside &&\n@@ -14,3 +15,4 @@ echo \">>>>>>> theirs\" &&\n \techo \">>>>>>> theirs\"\n \tpoodle\n ) >merged\n+'\ndiff --git a/t/chainlint/one-liner-for-loop.test b/t/chainlint/one-liner-for-loop.test\nindex 4bd8c066c7..00afd7ef76 100644\n--- a/t/chainlint/one-liner-for-loop.test\n+++ b/t/chainlint/one-liner-for-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'one-liner-for-loop' '\n git init dir-rename-and-content &&\n (\n \tcd dir-rename-and-content &&\n@@ -8,3 +9,4 @@ git init dir-rename-and-content &&\n \tgit add foo olddir &&\n \tgit commit -m \"original\" &&\n )\n+'\ndiff --git a/t/chainlint/one-liner.test b/t/chainlint/one-liner.test\nindex be9858fa29..6e42ee1b5e 100644\n--- a/t/chainlint/one-liner.test\n+++ b/t/chainlint/one-liner.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'one-liner' '\n # LINT: top-level one-liner subshell\n (foo && bar) &&\n (foo && bar) |\n@@ -10,3 +11,4 @@\n \n # LINT: \";\" in string not misinterpreted as broken &&-chain\n (foo \"bar; baz\")\n+'\ndiff --git a/t/chainlint/p4-filespec.test b/t/chainlint/p4-filespec.test\nindex 4fd2d6e2b8..8ba6b911dc 100644\n--- a/t/chainlint/p4-filespec.test\n+++ b/t/chainlint/p4-filespec.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'p4-filespec' '\n (\n # LINT: Perforce revspec in filespec not misinterpreted as in-line comment\n \tp4 print -1 //depot/fiddle#42 >file &&\n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/pipe.test b/t/chainlint/pipe.test\nindex dd82534c66..1af81c243b 100644\n--- a/t/chainlint/pipe.test\n+++ b/t/chainlint/pipe.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'pipe' '\n (\n # LINT: no \"&&\" needed on line ending with \"|\"\n \tfoo |\n@@ -10,3 +11,4 @@\n \n \tsunder\n )\n+'\ndiff --git a/t/chainlint/return-loop.test b/t/chainlint/return-loop.test\nindex f90b171300..ea76c3593a 100644\n--- a/t/chainlint/return-loop.test\n+++ b/t/chainlint/return-loop.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'return-loop' '\n while test $i -lt $((num - 5))\n do\n # LINT: \"|| return {n}\" valid loop escape outside subshell; no \"&&\" needed\n \tgit notes add -m \"notes for commit$i\" HEAD~$i || return 1\n \ti=$((i + 1))\n done\n+'\ndiff --git a/t/chainlint/semicolon.test b/t/chainlint/semicolon.test\nindex 67e1192c50..fc0ba1b539 100644\n--- a/t/chainlint/semicolon.test\n+++ b/t/chainlint/semicolon.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'semicolon' '\n (\n # LINT: missing internal \"&&\" and ending \"&&\"\n \tcat foo ; echo bar\n@@ -23,3 +24,4 @@\n # LINT: semicolon unnecessary but legitimate\n \t\techo;\n \tdone)\n+'\ndiff --git a/t/chainlint/sqstring-in-sqstring.test b/t/chainlint/sqstring-in-sqstring.test\nindex 77a425e0c7..24169724a5 100644\n--- a/t/chainlint/sqstring-in-sqstring.test\n+++ b/t/chainlint/sqstring-in-sqstring.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'sqstring-in-sqstring' '\n # LINT: SQ-string Perl code fragment within SQ-string\n perl -e '\\''\n \tdefined($_ = -s $_) or die for @ARGV;\n \texit 1 if $ARGV[0] <= $ARGV[1];\n '\\'' test-2-$packname_2.pack test-3-$packname_3.pack\n+'\ndiff --git a/t/chainlint/subshell-here-doc.test b/t/chainlint/subshell-here-doc.test\nindex d40eb65583..4a38f47f01 100644\n--- a/t/chainlint/subshell-here-doc.test\n+++ b/t/chainlint/subshell-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'subshell-here-doc' '\n (\n # LINT: stitch together incomplete \\-ending lines\n # LINT: swallow here-doc to avoid false positives in content\n@@ -33,3 +34,4 @@ EOF\n \tARBITRARY3\n \tmeep\n )\n+'\ndiff --git a/t/chainlint/subshell-one-liner.test b/t/chainlint/subshell-one-liner.test\nindex 37fa643c20..dac536afcc 100644\n--- a/t/chainlint/subshell-one-liner.test\n+++ b/t/chainlint/subshell-one-liner.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'subshell-one-liner' '\n (\n # LINT: nested one-liner subshell\n \t(foo && bar) &&\n@@ -22,3 +23,4 @@\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/t7900-subtree.test b/t/chainlint/t7900-subtree.test\nindex 02f3129232..1f4f03300f 100644\n--- a/t/chainlint/t7900-subtree.test\n+++ b/t/chainlint/t7900-subtree.test\n@@ -1,3 +1,4 @@\n+test_expect_success 't7900-subtree' '\n (\n \tchks=\"sub1\n sub2\n@@ -20,3 +21,4 @@ TXT\n \tcheck_equal \"$subfiles\" \"$chkms\n $chks\"\n )\n+'\ndiff --git a/t/chainlint/token-pasting.test b/t/chainlint/token-pasting.test\nindex b4610ce815..590914b733 100644\n--- a/t/chainlint/token-pasting.test\n+++ b/t/chainlint/token-pasting.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'token-pasting' '\n # LINT: single token; composite of multiple strings\n git config filter.rot13.smudge ./rot13.sh &&\n git config filter.rot13.clean ./rot13.sh &&\n@@ -30,3 +31,4 @@ downstream_url_for_sed=$(\n # LINT: exit/enter string context; \"&\" inside string not command terminator\n \tsed -e '\\''s/\\\\/\\\\\\\\/g'\\'' -e '\\''s/[[/.*^$]/\\\\&/g'\\''\n )\n+'\ndiff --git a/t/chainlint/unclosed-here-doc-indent.test b/t/chainlint/unclosed-here-doc-indent.test\nindex 5c841a9dfd..7ac9d0f7d7 100644\n--- a/t/chainlint/unclosed-here-doc-indent.test\n+++ b/t/chainlint/unclosed-here-doc-indent.test\n@@ -1,4 +1,6 @@\n+test_expect_success 'unclosed-here-doc-indent' '\n command_which_is_run &&\n cat >expect <<-\\EOF &&\n we forget to end the here-doc\n command_which_is_gobbled\n+'\ndiff --git a/t/chainlint/unclosed-here-doc.test b/t/chainlint/unclosed-here-doc.test\nindex 69d3786c34..68e78f06f3 100644\n--- a/t/chainlint/unclosed-here-doc.test\n+++ b/t/chainlint/unclosed-here-doc.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'unclosed-here-doc' '\n command_which_is_run &&\n cat >expect <<\\EOF &&\n \twe try to end the here-doc below,\n \tbut the indentation throws us off\n \tsince the operator is not \"<<-\".\n \tEOF\n command_which_is_gobbled\n+'\ndiff --git a/t/chainlint/while-loop.test b/t/chainlint/while-loop.test\nindex d09fb016e4..33a201906a 100644\n--- a/t/chainlint/while-loop.test\n+++ b/t/chainlint/while-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'while-loop' '\n (\n # LINT: \"while\", \"do\", \"done\" do not need \"&&\"\n \twhile true\n@@ -17,3 +18,4 @@\n \t\tcat bar\n \tdone\n )\n+'\n-- \n2.45.2.1178.gaaad15bb7b\n\n"},{"id":"498143","messageId":"20240706060754.GC700151@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240706060143.GD698153@coredump.intra.peff.net","subject":"[PATCH 3/3] t/chainlint: add tests for test body in heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:07:54Z","receivedAt":"2024-07-06T06:07:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The chainlint.pl script recently learned about our new:\n\n  test_expect_success 'some test' - <<\\EOT\n\tTEST_BODY\n  EOT\n\nsyntax, where TEST_BODY should be checked in the usual way. Let's make\nsure this works by adding a few tests. The \"here-doc-body\" file tests\nthe basic syntax, including an embedded here-doc which we should still\nbe able to recognize.\n\nLikewise the \"here-doc-body-indent\" checks the same thing, but using the\n\"<<-\" operator. We wouldn't expect this to be used normally, but we\nwould not want to accidentally miss a body that uses it.\n\nThe \"here-doc-double\" tests the handling of two here-doc tags on the\nsame line. This is not something we'd expect anybody to do in practice,\nbut the code was written defensively to handle this, so let's make sure\nit works.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThese could also be squashed into Eric's patch which introduces the new\nfunctionality.\n\n t/chainlint/here-doc-body-indent.expect |  2 ++\n t/chainlint/here-doc-body-indent.test   |  4 ++++\n t/chainlint/here-doc-body.expect        |  7 +++++++\n t/chainlint/here-doc-body.test          |  9 +++++++++\n t/chainlint/here-doc-double.expect      |  2 ++\n t/chainlint/here-doc-double.test        | 10 ++++++++++\n 6 files changed, 34 insertions(+)\n create mode 100644 t/chainlint/here-doc-body-indent.expect\n create mode 100644 t/chainlint/here-doc-body-indent.test\n create mode 100644 t/chainlint/here-doc-body.expect\n create mode 100644 t/chainlint/here-doc-body.test\n create mode 100644 t/chainlint/here-doc-double.expect\n create mode 100644 t/chainlint/here-doc-double.test\n\ndiff --git a/t/chainlint/here-doc-body-indent.expect b/t/chainlint/here-doc-body-indent.expect\nnew file mode 100644\nindex 0000000000..ba280af56e\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-indent.expect\n@@ -0,0 +1,2 @@\n+\techo \"we should find this\" ?!AMP?!\n+\techo \"even though our heredoc has its indent stripped\"\ndiff --git a/t/chainlint/here-doc-body-indent.test b/t/chainlint/here-doc-body-indent.test\nnew file mode 100644\nindex 0000000000..39ff970ef3\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-indent.test\n@@ -0,0 +1,4 @@\n+test_expect_success 'here-doc-body-indent' - <<-\\EOT\n+\techo \"we should find this\"\n+\techo \"even though our heredoc has its indent stripped\"\n+EOT\ndiff --git a/t/chainlint/here-doc-body.expect b/t/chainlint/here-doc-body.expect\nnew file mode 100644\nindex 0000000000..3d21ad2fd6\n--- /dev/null\n+++ b/t/chainlint/here-doc-body.expect\n@@ -0,0 +1,7 @@\n+\techo \"missing chain before\" ?!AMP?!\n+\tcat >file <<-\\EOF &&\n+\tinside inner here-doc\n+\tthese are not shell commands\n+\tEOF\n+\techo \"missing chain after\" ?!AMP?!\n+\techo \"but this line is OK because it's the end\"\ndiff --git a/t/chainlint/here-doc-body.test b/t/chainlint/here-doc-body.test\nnew file mode 100644\nindex 0000000000..989ac2f4e1\n--- /dev/null\n+++ b/t/chainlint/here-doc-body.test\n@@ -0,0 +1,9 @@\n+test_expect_success 'here-doc-body' - <<\\EOT\n+\techo \"missing chain before\"\n+\tcat >file <<-\\EOF &&\n+\tinside inner here-doc\n+\tthese are not shell commands\n+\tEOF\n+\techo \"missing chain after\"\n+\techo \"but this line is OK because it's the end\"\n+EOT\ndiff --git a/t/chainlint/here-doc-double.expect b/t/chainlint/here-doc-double.expect\nnew file mode 100644\nindex 0000000000..e164050d06\n--- /dev/null\n+++ b/t/chainlint/here-doc-double.expect\n@@ -0,0 +1,2 @@\n+\techo \"actual test commands\" ?!AMP?!\n+\techo \"that should be checked\"\ndiff --git a/t/chainlint/here-doc-double.test b/t/chainlint/here-doc-double.test\nnew file mode 100644\nindex 0000000000..777389f0d9\n--- /dev/null\n+++ b/t/chainlint/here-doc-double.test\n@@ -0,0 +1,10 @@\n+# This is obviously a ridiculous thing to do, but we should be able\n+# to handle two here-docs on the same line, and attribute them\n+# correctly.\n+test_expect_success \"$(cat <<END_OF_PREREQS)\" 'here-doc-double' - <<\\EOT\n+SOME\n+PREREQS\n+END_OF_PREREQS\n+\techo \"actual test commands\"\n+\techo \"that should be checked\"\n+EOT\n-- \n2.45.2.1178.gaaad15bb7b\n"},{"id":"498145","messageId":"20240706060951.GA700433@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240706060639.GB700151@coredump.intra.peff.net","subject":"Re: [PATCH 2/3] t/chainlint: add test_expect_success call to test snippets","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:09:51Z","receivedAt":"2024-07-06T06:09:52Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jul 06, 2024 at 02:06:39AM -0400, Jeff King wrote:\n\n> --- a/t/Makefile\n> +++ b/t/Makefile\n> @@ -109,9 +109,7 @@ clean-chainlint:\n>  check-chainlint:\n>  \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n>  \tfor i in $(CHAINLINTTESTS); do \\\n> -\t\techo \"test_expect_success '$$i' '\" && \\\n> -\t\tsed -e '/^# LINT: /d' chainlint/$$i.test && \\\n> -\t\techo \"'\"; \\\n> +\t\tsed -e '/^# LINT: /d' chainlint/$$i.test; \\\n>  \tdone >'$(CHAINLINTTMP_SQ)'/tests && \\\n>  \t{ \\\n>  \t\techo \"# chainlint: $(CHAINLINTTMP_SQ)/tests\" && \\\n\nBTW, not new in my patch, but I found it ironic that the shell snippet\nhere itself violates &&-chain rules. It should \"|| exit 1\" inside the\nloop if a sed call fails.\n\n-Peff\n"},{"id":"498147","messageId":"CAPig+cQnZjMBPooBqMJjPY78EiCEXQOSSyHBm4GtLcbsSqZKrw@mail.gmail.com","threadId":"61713","inReplyTo":"20240706053105.GB698153@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-06T06:11:13Z","receivedAt":"2024-07-06T06:11:26Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 1:31 AM Jeff King <peff@peff.net> wrote:\n> On Tue, Jul 02, 2024 at 05:25:48PM -0400, Eric Sunshine wrote:\n> > My implementation, however, takes a more formal and paranoid stance.\n> > Rather than squirreling away only the most-recently-seen heredoc body,\n> > it stores each heredoc body along with the tag which introduced it.\n> > This makes it robust against cases when multiple heredocs are\n> > initiated on the same line (even within different parse contexts):\n> >\n> >     cat <<EOFA && x=$(cat <<EOFB &&\n> >     A body\n> >     EOFA\n> >     B body\n> >     EOFB\n> >\n> > Of course, that's not likely to come up in the context of\n> > test_expect_* calls, but I prefer the added robustness over the more\n> > lax approach.\n>\n> Yes, that's so much better than what I wrote. I didn't engage my brain\n> very much when I read the in-code comments about multiple tags on the\n> same line, and I thought you meant:\n>\n>   cat <<FOO <<BAR\n>   this is foo\n>   FOO\n>   this is bar\n>   BAR\n>\n> which is...weird. It does \"work\" in the sense that \"FOO\" is a here-doc\n> that should be skipped past. But it is not doing anything useful; cat\n> sees only \"this is bar\" on stdin. So even for this case, the appending\n> behavior that my patch does would not make sense.\n>\n> And of course for the actual useful thing, which you wrote above,\n> appending is just nonsense. Recording and accessing by tag is the right\n> thing.\n\nIn retrospect, I think my claim is bogus in the context of\nScriptParser::parse_cmd(). Specifically, ScriptParser::parse_cmd()\ncalls its parent ShellParser::parse_cmd() to latch one command.\nShellParser::parse_cmd() stops parsing as soon as it encounters a\ncommand terminator (i.e. `;`, `&&`, `||`, `|`, '&', '\\n') and returns\nthe command. Moreover, by definition, given the language\nspecification, the lexer only consumes the heredocs upon encountering\n`\\n`. Thus, if someone writes:\n\n    test_expect_success title - <<\\EOT && whatever &&\n    ...test body...\n    EOT\n\nthen ScriptParser::parse_cmd() will receive the command\n`test_expect_success title -` from ShellParser::parse_cmd() but the\nheredoc will not yet have been consumed by the lexer since it hasn't\nyet encountered the newline[1].\n\nSo, the above example simply can't work correctly given the way\nScriptParser::parse_cmd() calls ScriptParser::check_test() as soon as\nit encounters a `test_expect_success/failure` invocation since it\ndoesn't know if the heredocs have been latched at that point. To make\nit properly robust, rather than immediately calling check_test(), it\nwould have to continue consuming commands, and saving the ones which\nmatch `test_expect_success/failure` invocation, until it finally hits\na `\\n`, and only then call check_test() with each command it saved.\nBut that's probably overkill at this point considering that we never\nwrite code like the above, so the submitted patch[2] is probably good\nenough for now.\n\nFOOTNOTES\n\n[1] One might rightly ask that if ShellParser::parse_cmd() returns\nimmediately upon seeing a command terminator (i.e. `;`, `&&`, etc.),\nthen how is it that even a simple:\n\n    test_expect_success title - <<\\EOT &&\n    ...test body...\n    EOT\n\ncan work correctly since the `\\n` comes after the `&&`. The answer is\nthat, as a special case, the very last thing ShellParser::parse_cmd()\ndoes is peek ahead to see if a `\\n` follows the command terminator\n(assuming the terminator is not itself a `\\n`). When the next token is\nindeed a `\\n`, that peek operation causes the lexer to consume the\nheredocs.\n\n[2]: https://lore.kernel.org/git/20240702235034.88219-1-ericsunshine@charter.net/\n"},{"id":"498154","messageId":"CAPig+cTSAaZmoKHRPiVkV=Jc9z_JZWmGn_JZj+Atsc4DdC7XeQ@mail.gmail.com","threadId":"61713","inReplyTo":"CAPig+cQnZjMBPooBqMJjPY78EiCEXQOSSyHBm4GtLcbsSqZKrw@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-06T06:47:57Z","receivedAt":"2024-07-06T06:48:10Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 2:11 AM Eric Sunshine <sunshine@sunshineco.com> wrote:\n> So, the above example simply can't work correctly given the way\n> ScriptParser::parse_cmd() calls ScriptParser::check_test() as soon as\n> it encounters a `test_expect_success/failure` invocation since it\n> doesn't know if the heredocs have been latched at that point. To make\n> it properly robust, rather than immediately calling check_test(), it\n> would have to continue consuming commands, and saving the ones which\n> match `test_expect_success/failure` invocation, until it finally hits\n> a `\\n`, and only then call check_test() with each command it saved.\n> But that's probably overkill at this point considering that we never\n> write code like the above, so the submitted patch[2] is probably good\n> enough for now.\n\nOf course, the more I think about it, the more I dislike relying upon\nwhat is effectively an accident of implementation; i.e. that in the\ntypical case, the heredoc will already have been latched by the time\nScriptParser::parse_cmd() has identified a `test_expect_success`\ncommand, due to the fact that ShellParser::parse_cmd() has that\nspecial case which peeks for `\\n` immediately following some other\ncommand terminator. As such, fixing ScriptParser::parse_cmd() to only\ncall check_test() once it is sure that a '\\n' has been encountered is\nbecoming more appealing, though it is of course a more invasive and\nfundamental change than the posted patch.\n"},{"id":"498155","messageId":"20240706065434.GA702300@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cQnZjMBPooBqMJjPY78EiCEXQOSSyHBm4GtLcbsSqZKrw@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:54:34Z","receivedAt":"2024-07-06T06:54:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jul 06, 2024 at 02:11:13AM -0400, Eric Sunshine wrote:\n\n> > >     cat <<EOFA && x=$(cat <<EOFB &&\n> > >     A body\n> > >     EOFA\n> > >     B body\n> > >     EOFB\n> [...]\n> In retrospect, I think my claim is bogus in the context of\n> ScriptParser::parse_cmd(). Specifically, ScriptParser::parse_cmd()\n> calls its parent ShellParser::parse_cmd() to latch one command.\n> ShellParser::parse_cmd() stops parsing as soon as it encounters a\n> command terminator (i.e. `;`, `&&`, `||`, `|`, '&', '\\n') and returns\n> the command. Moreover, by definition, given the language\n> specification, the lexer only consumes the heredocs upon encountering\n> `\\n`. Thus, if someone writes:\n> \n>     test_expect_success title - <<\\EOT && whatever &&\n>     ...test body...\n>     EOT\n> \n> then ScriptParser::parse_cmd() will receive the command\n> `test_expect_success title -` from ShellParser::parse_cmd() but the\n> heredoc will not yet have been consumed by the lexer since it hasn't\n> yet encountered the newline[1].\n> \n> So, the above example simply can't work correctly given the way\n> ScriptParser::parse_cmd() calls ScriptParser::check_test() as soon as\n> it encounters a `test_expect_success/failure` invocation since it\n> doesn't know if the heredocs have been latched at that point.\n\nAh, yeah, I think you're right. I had parsed your example in my mind as:\n\n  cat <<EOFA $(cat <<EOFB)\n\nwithout an intervening \"&&\" (taking the second here-doc as an argument\nto the original command). Which _does_ work with your patch.\n\n> To make it properly robust, rather than immediately calling\n> check_test(), it would have to continue consuming commands, and saving\n> the ones which match `test_expect_success/failure` invocation, until\n> it finally hits a `\\n`, and only then call check_test() with each\n> command it saved.  But that's probably overkill at this point\n> considering that we never write code like the above, so the submitted\n> patch[2] is probably good enough for now.\n\nYep, I'd agree with all of that.\n\n-Peff\n"},{"id":"498156","messageId":"20240706065534.GB702300@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cTSAaZmoKHRPiVkV=Jc9z_JZWmGn_JZj+Atsc4DdC7XeQ@mail.gmail.com","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T06:55:34Z","receivedAt":"2024-07-06T06:55:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jul 06, 2024 at 02:47:57AM -0400, Eric Sunshine wrote:\n\n> On Sat, Jul 6, 2024 at 2:11 AM Eric Sunshine <sunshine@sunshineco.com> wrote:\n> > So, the above example simply can't work correctly given the way\n> > ScriptParser::parse_cmd() calls ScriptParser::check_test() as soon as\n> > it encounters a `test_expect_success/failure` invocation since it\n> > doesn't know if the heredocs have been latched at that point. To make\n> > it properly robust, rather than immediately calling check_test(), it\n> > would have to continue consuming commands, and saving the ones which\n> > match `test_expect_success/failure` invocation, until it finally hits\n> > a `\\n`, and only then call check_test() with each command it saved.\n> > But that's probably overkill at this point considering that we never\n> > write code like the above, so the submitted patch[2] is probably good\n> > enough for now.\n> \n> Of course, the more I think about it, the more I dislike relying upon\n> what is effectively an accident of implementation; i.e. that in the\n> typical case, the heredoc will already have been latched by the time\n> ScriptParser::parse_cmd() has identified a `test_expect_success`\n> command, due to the fact that ShellParser::parse_cmd() has that\n> special case which peeks for `\\n` immediately following some other\n> command terminator. As such, fixing ScriptParser::parse_cmd() to only\n> call check_test() once it is sure that a '\\n' has been encountered is\n> becoming more appealing, though it is of course a more invasive and\n> fundamental change than the posted patch.\n\nRats, I just agreed with your earlier email. ;) I am OK with the\nslightly hacky version we've posted (modulo the fixes I discussed\nelsewhere). But if you want to take a little time to explore the more\nrobust fix, I am happy to review it.\n\n-Peff\n"},{"id":"498158","messageId":"CAPig+cQNVtJJ5esayT=zoahtvhLipDaCBbsqLdQPecpxLbhT7A@mail.gmail.com","threadId":"61713","inReplyTo":"20240706065534.GB702300@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] test-lib: allow test snippets as here-docs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-06T07:06:26Z","receivedAt":"2024-07-06T07:06:38Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 2:55 AM Jeff King <peff@peff.net> wrote:\n> On Sat, Jul 06, 2024 at 02:47:57AM -0400, Eric Sunshine wrote:\n> > Of course, the more I think about it, the more I dislike relying upon\n> > what is effectively an accident of implementation; i.e. that in the\n> > typical case, the heredoc will already have been latched by the time\n> > ScriptParser::parse_cmd() has identified a `test_expect_success`\n> > command, due to the fact that ShellParser::parse_cmd() has that\n> > special case which peeks for `\\n` immediately following some other\n> > command terminator. As such, fixing ScriptParser::parse_cmd() to only\n> > call check_test() once it is sure that a '\\n' has been encountered is\n> > becoming more appealing, though it is of course a more invasive and\n> > fundamental change than the posted patch.\n>\n> Rats, I just agreed with your earlier email. ;) I am OK with the\n> slightly hacky version we've posted (modulo the fixes I discussed\n> elsewhere). But if you want to take a little time to explore the more\n> robust fix, I am happy to review it.\n\nThe primary reason I said \"the more I dislike relying upon ... an\naccident of implementation\" is that this limitation is not documented\nanywhere other than in this email thread. That said, I don't mind the\nposted version of the patch being picked up. The \"correct\" approach\ncan always be implemented atop it at a later time.\n"},{"id":"498178","messageId":"xmqqr0c6makb.fsf@gitster.g","threadId":"61713","inReplyTo":"20240706060143.GD698153@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-06T22:15:32Z","receivedAt":"2024-07-06T22:15:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> I'll post some patches in a moment:\n>\n>   [1/3]: chainlint.pl: fix line number reporting\n>   [2/3]: t/chainlint: add test_expect_success call to test snippets\n>   [3/3]: t/chainlint: add tests for test body in heredoc\n>\n> with the idea that we'd apply your patch here on top of what Junio has\n> queued in jk/test-body-in-here-doc, and then these three on top.\n\nWould the final form be to have Eric's preparatory enhancement to\nchainlint and then these three first, and finally the \"here-docs\"\nconversion I queued from you earlier?\n\n"},{"id":"498180","messageId":"20240706231128.GA746087@coredump.intra.peff.net","threadId":"61713","inReplyTo":"xmqqr0c6makb.fsf@gitster.g","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-06T23:11:28Z","receivedAt":"2024-07-06T23:11:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Jul 06, 2024 at 03:15:32PM -0700, Junio C Hamano wrote:\n\n> Jeff King <peff@peff.net> writes:\n> \n> > I'll post some patches in a moment:\n> >\n> >   [1/3]: chainlint.pl: fix line number reporting\n> >   [2/3]: t/chainlint: add test_expect_success call to test snippets\n> >   [3/3]: t/chainlint: add tests for test body in heredoc\n> >\n> > with the idea that we'd apply your patch here on top of what Junio has\n> > queued in jk/test-body-in-here-doc, and then these three on top.\n> \n> Would the final form be to have Eric's preparatory enhancement to\n> chainlint and then these three first, and finally the \"here-docs\"\n> conversion I queued from you earlier?\n\nI had planned on top (leaving a brief moment where chainlint would\nignore the new format), but either way is fine.\n\nMy biggest question is around my patch 1 above:\n\n  - is it worth squashing in to Eric's patch? I didn't want to do that\n    without getting his OK on the approach.\n\n  - instead of bumping the line number in the caller, should the lexer\n    record the line number of the here-doc to be used later?\n\n  - the test harness in the Makefile strips the line numbers from the\n    chainlint output, so it's hard to verify those fixes. I saw them\n    only because the combination of the two bugs meant that the here-doc\n    had a \"line 0\" in it, which was enough to confuse the \"sed\"\n    invocation in the Makefile.\n\n    I did manually verify that it is OK after my fix, but do we want\n    that to be part of the chainlint tests? Just leaving the line\n    numbers in is a maintenance nightmare, since it depends on the order\n    of concatenating all of the tests together (so our \"expect\" files\n    would depend on all of the previous tests). But if we wanted to get\n    fancy, we could perhaps store relative offsets in the expect file. I\n    think it gets pretty complicated, though, since we print only\n    problematic lines.\n\nI was going to give it a few days for Eric to chime in on those points,\nand then assemble a final version for you to apply (but I certainly\ndon't mind if you want to pick up what's here in the meantime).\n\n-Peff\n"},{"id":"498208","messageId":"CAPig+cRzoyxQpNf15nGeHDdDQK64KRJXLdgKN2rkXaN+2SjTZQ@mail.gmail.com","threadId":"61713","inReplyTo":"20240706060754.GC700151@coredump.intra.peff.net","subject":"Re: [PATCH 3/3] t/chainlint: add tests for test body in heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T02:43:45Z","receivedAt":"2024-07-08T02:43:56Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 2:07 AM Jeff King <peff@peff.net> wrote:\n> The chainlint.pl script recently learned about our new:\n>\n>   test_expect_success 'some test' - <<\\EOT\n>         TEST_BODY\n>   EOT\n>\n> syntax, where TEST_BODY should be checked in the usual way. Let's make\n> sure this works by adding a few tests. [...]\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n> These could also be squashed into Eric's patch which introduces the new\n> functionality.\n\nAs the author of these tests, you should get credit, so I'd proposed\nnot squashing this into my patch.\n\n> diff --git a/t/chainlint/here-doc-body.test b/t/chainlint/here-doc-body.test\n> @@ -0,0 +1,9 @@\n> +test_expect_success 'here-doc-body' - <<\\EOT\n> +       echo \"missing chain before\"\n> +       cat >file <<-\\EOF &&\n> +       inside inner here-doc\n> +       these are not shell commands\n> +       EOF\n> +       echo \"missing chain after\"\n> +       echo \"but this line is OK because it's the end\"\n> +EOT\n\nThis one made me think of an additional pathological case, though I'm\nnot sure it's worth having a test:\n\n    test_expect_success 'pathological-here-doc-body' - <<\\EOF\n        echo \"missing chain before\"\n        cat >file <<-\\EOF &&\n        inside inner here-doc\n        these are not shell commands\n        EOF\n        echo \"missing chain after\"\n        echo \"but this line is OK because it's the end\"\n   EOF\n\nIt's exactly the same as your test except that the same tag (\"EOF\") is\nused for both outer and inner heredocs. It works because the outer\nheredoc is introduced with `<<` whereas the inner with `<<-`. The\nopposite case, in which outer is introduced with `<<-` and inner with\n`<<`, obviously would be bogus.\n"},{"id":"498209","messageId":"CAPig+cRUOSHuNezZfVUT43JGT5BWA3LTbQvKCkD37DxXn=L6Jg@mail.gmail.com","threadId":"61713","inReplyTo":"20240706060143.GD698153@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T03:40:19Z","receivedAt":"2024-07-08T03:40:32Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 2:01 AM Jeff King <peff@peff.net> wrote:\n> On Tue, Jul 02, 2024 at 07:50:34PM -0400, Eric Sunshine wrote:\n> I'll post some patches in a moment:\n>\n>   [1/3]: chainlint.pl: fix line number reporting\n>   [2/3]: t/chainlint: add test_expect_success call to test snippets\n>   [3/3]: t/chainlint: add tests for test body in heredoc\n>\n> with the idea that we'd apply your patch here on top of what Junio has\n> queued in jk/test-body-in-here-doc, and then these three on top. For\n> Junio's sanity, I'll roll it all up into one series. But I wanted to\n> show it to you incrementally first, especially because I think the fixes\n> from patch 1/3 above should probably just get squashed in (or even\n> rewritten). I'll discuss the bugs they fix below.\n\nConsidering the excellent explanation you crafted in your patch, I'd\nlike to say that it should remain separate from mine. However, since\nyou caught the problems in review, it would be irresponsible of us to\nlet my patch into the permanent history as-is. So, feel free to squash\nyour fixes into my patch. Perhaps add a Co-authored-by:? The bit from\nyour [1/3] commit message about incrementing $lineno for the\nheredoc-body case might be worth squashing in too?\n\nI wrote one minor (perhaps non-actionable) comment in response to\npatch [3/3]. The patches all looked fine to me, so:\n\n    Acked-by: Eric Sunshine <sunshine@sunshineco.com>\n\n> > @@ -232,7 +234,8 @@ sub new {\n> >       my $self = bless {\n> >               buff => [],\n> >               stop => [],\n> > -             output => []\n> > +             output => [],\n> > +             heredocs => {},\n> >       } => $class;\n> >       $self->{lexer} = Lexer->new($self, $s);\n> >       return $self;\n>\n> I think initializing is not strictly necessary here, since we'd only try\n> to read tags if we saw a here-doc. But there might be some invalid cases\n> where we could convince higher-level code to look for tags even though\n> there were none (and generate a perl warning about trying to dereference\n> undef as a hashref).\n\nYou're right, it's not necessary to initialize here, but it feels more\nconsistent to do so. That said, I don't feel strongly either way.\n\n> On the flip side, what about cleaning up? The \"heretags\" array is\n> emptied as we parse the heredocs in swallow_heredocs(). But I think once\n> a ShellParser's $self->{heredocs}->{FOO} is written, it will hang around\n> forever (even though it's valid only for that one command). Probably not\n> a big deal, but there's probably some correct spot to reset it.\n\nThere are a few reasons I wasn't overly concerned about cleaning up in\nthis case:\n\n(1) The parsers are all short-lived, so the collected heredoc bodies\nwon't stick around long anyhow. For each test checked, a TestParser is\ncreated and destroyed. For each script mentioned on the command-line,\na ScriptParser is created and destroyed. None of these parsers stick\naround for long, though, a ScriptParser outlives a TestParser.\n\n(2) The heredoc bodies in question tend to be pretty small, so it's\nnot consuming an inordinate amount of memory even if a single parser\nlatches bodies of multiple heredocs.\n\n(3) We tend to be quite consistent about naming our heredoc tag (i.e.\n\"EOF\", \"EOT\"), so a latched body in the parser's %heredocs hash is\nvery likely to get overwritten, thus the hash is probably not going to\neat up a lot of memory. Given the entire test suite, I'd be quite\nsurprised if any one parser ever latches more than three heredoc\nbodies at a time, and the vast majority of parsers are likely latching\nzero or one heredoc body.\n\n(4) I couldn't really think of a correct spot to reset %heredocs.\n\nThat said, after reading your message, I did try implementing an\napproach in which the heredoc body gets attached to the `<<` or `<<-`\ntoken. That way, a heredoc body would be cleaned along with its\nassociated lexer token. However, the implementation got too ugly and\nincreased cognitive load too much for my liking, so I abandoned it.\n\n> >  sub check_test {\n> >       my $self = shift @_;\n> > -     my ($title, $body) = map(unwrap, @_);\n> > +     my $title = unwrap(shift @_);\n> > +     my $body = unwrap(shift @_);\n> > +     $body = shift @_ if $body eq '-';\n> >       $self->{ntests}++;\n> >       my $parser = TestParser->new(\\$body);\n> >       my @tokens = $parser->parse();\n>\n> This has two problems related to line numbers. You can't see it in the\n> context, but we later do:\n>\n>   my $lineno = $_[1]->[3];\n>\n> Now that we're shifting @_, that array item is gone.\n\nUgh, this is embarrassing. I did run chainlint.pl on t0600 in which I\nhad intentionally broken some &&-chains, so I saw the output, but\nsomehow I overlooked that it broke the line numbering entirely.\n\n> The second is that the line number for the here-doc is actually one past\n> the initial line number of the test_expect_success. That works\n> automatically for hanging single-quotes, since the newline from that\n> line is inside the quoted area. But for a here-doc, we have to account\n> for it manually. In my original patch I prepended \"\\n\", but you can also\n> just increment $lineno (which is what I did in the fix I'm about to\n> send).\n\nNicely spotted. Simply incrementing $lineno does feel a bit hacky, but\nI agree that it is probably good enough for now; it doesn't seem\nlikely that it will break any time soon. But I also agree with the\ncommentary you wrote in patch [1/3] that it probably would be easy\nenough to latch the line number of the beginning of the heredoc body\nand employ that value. That would certainly be more robust.\n\n> > @@ -649,8 +654,13 @@ sub parse_cmd {\n> >       return @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n> >       my $n = $#tokens;\n> >       $n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n> > -     $self->check_test($tokens[1], $tokens[2]) if $n == 2; # title body\n> > -     $self->check_test($tokens[2], $tokens[3]) if $n > 2;  # prereq title body\n> > +     my $herebody;\n> > +     if ($n >= 2 && $tokens[$n-1]->[0] eq '-' && $tokens[$n]->[0] =~ /^<<-?(.+)$/) {\n> > +             $herebody = $self->{heredocs}->{$1};\n> > +             $n--;\n> > +     }\n> > +     $self->check_test($tokens[1], $tokens[2], $herebody) if $n == 2; # title body\n> > +     $self->check_test($tokens[2], $tokens[3], $herebody) if $n > 2;  # prereq title body\n> >       return @tokens;\n> >  }\n>\n> OK, mostly as expected. I think the check for \"-\" here is redundant with\n> what's in check_test(). We could just feed the heredoc body either way,\n> and in the nonsense case of:\n>\n>   test_expect_success 'title' 'test body' <<EOT\n>   nobody reads this!\n>   EOT\n>\n> the heredoc data would just be ignored.\n\nRight. I went back and forth with this, never sure if this code was\noverkill. On the other hand, we could make this more paranoid and\ncomplain if we see either of these cases:\n\n(1) \"-\" but no heredoc\n(2) heredoc present but something other than \"-\"\n\n> Requiring \"<<\" at the end is somewhat limiting. E.g. this is valid:\n>\n>   test_expect_success <<EOT 'title' -\n>   the test body\n>   EOT\n\nTrue, I didn't even think about that.\n\n> I don't expect anybody to do that, but it would be nice to be more\n> robust if we can. I think the tokens are still wrapped at this point, so\n> we could read through all of them looking for \"<<\" anywhere, without\n> getting confused by \"$(cat <<INNER_HEREDOC)\". I think, anyway (I didn't\n> test).\n\nCorrect. The stuff inside \"$(...)\" does get parsed and linted, but by\nthe time ScriptParser::parse_cmd() sees it, `$(cat <<INNER_HEREDOC)`\nis just a single (string) token.\n"},{"id":"498210","messageId":"CAPig+cTv-DaGRmwWWCk8b33MKzV25vfP2zPKd2VOAEOtz4FZ2A@mail.gmail.com","threadId":"61713","inReplyTo":"20240706231128.GA746087@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T03:51:15Z","receivedAt":"2024-07-08T03:51:27Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 7:11 PM Jeff King <peff@peff.net> wrote:\n> My biggest question is around my patch 1 above:\n>\n>   - is it worth squashing in to Eric's patch? I didn't want to do that\n>     without getting his OK on the approach.\n\nGiven the effort you put into the commit message and diagnosing my\nbugs, my knee-jerk response is that it would be nice to keep your\npatch separate so you retain authorship. But it also would be\nirresponsible for us to let my buggy patch into the project history\nas-is since you caught the problems at review time. So, squashing your\nfixes in seems like the correct approach.\n\n>   - instead of bumping the line number in the caller, should the lexer\n>     record the line number of the here-doc to be used later?\n\nIt would be more robust to do so, but I suspect things will be fine\nfor a long time even without such an enhancement. But I also agree\nwith your commentary in patch [1/3] that it probably would be easy to\nlatch the line number at the point at which the heredoc body is\nlatched.\n\n>   - the test harness in the Makefile strips the line numbers from the\n>     chainlint output, so it's hard to verify those fixes. I saw them\n>     only because the combination of the two bugs meant that the here-doc\n>     had a \"line 0\" in it, which was enough to confuse the \"sed\"\n>     invocation in the Makefile.\n>\n>     I did manually verify that it is OK after my fix, but do we want\n>     that to be part of the chainlint tests? Just leaving the line\n>     numbers in is a maintenance nightmare, since it depends on the order\n>     of concatenating all of the tests together (so our \"expect\" files\n>     would depend on all of the previous tests). But if we wanted to get\n>     fancy, we could perhaps store relative offsets in the expect file. I\n>     think it gets pretty complicated, though, since we print only\n>     problematic lines.\n\nGiven the way the Makefile currently concatenates all the self-tests,\nit would indeed be a nightmare to retain the line numbers. In the long\nrun, we probably ought someday to adopt Ævar's idea of checking the\nself-test files individually[*] rather than en masse. With that\napproach, it may make sense to revisit whether or not line numbers\nshould be present in the \"expected\" files.\n\n[*] https://lore.kernel.org/git/CAPig+cSBjsosRqoAafYN94Cco8+7SdUt0ND_jHS+jVPoM4K0JA@mail.gmail.com/\n"},{"id":"498211","messageId":"CAPig+cTfMf_t93arsTCq-7fSooiBKRw1L=DVmSX8Pp7xMJM9Sw@mail.gmail.com","threadId":"61713","inReplyTo":"20240706060951.GA700433@coredump.intra.peff.net","subject":"Re: [PATCH 2/3] t/chainlint: add test_expect_success call to test snippets","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T03:59:27Z","receivedAt":"2024-07-08T03:59:39Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Jul 6, 2024 at 2:09 AM Jeff King <peff@peff.net> wrote:\n> >       for i in $(CHAINLINTTESTS); do \\\n> > -             echo \"test_expect_success '$$i' '\" && \\\n> > -             sed -e '/^# LINT: /d' chainlint/$$i.test && \\\n> > -             echo \"'\"; \\\n> > +             sed -e '/^# LINT: /d' chainlint/$$i.test; \\\n> >       done >'$(CHAINLINTTMP_SQ)'/tests && \\\n>\n> BTW, not new in my patch, but I found it ironic that the shell snippet\n> here itself violates &&-chain rules. It should \"|| exit 1\" inside the\n> loop if a sed call fails.\n\nIndeed, what an embarrassing oversight, especially since this loop was\nadded in preparation for chainlint.pl which diagnoses missing `||\nexit` (or `|| return`) in loops in tests (unlike chainlint.sed which\ndidn't).\n"},{"id":"498212","messageId":"CAPig+cRxnpG9Yfix09EnJAbnzwN=yoUtSeYxt7S-Od+xBgfaYQ@mail.gmail.com","threadId":"61713","inReplyTo":"20240706060515.GA700151@coredump.intra.peff.net","subject":"Re: [PATCH 1/3] chainlint.pl: fix line number reporting","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T05:08:02Z","receivedAt":"2024-07-08T05:08:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On 7/6/24 2:05 AM, Jeff King wrote:\n> The previous commit taught chainlint.pl to handle test bodies in\n> heredocs, but there are two small bugs related to line numbers:\n>\n> 2. For an invocation like the one above, if the test_expect_success\n> line is X, then \"test body\" would correctly start at X+1, since the\n> hanging newline at the start of the single-quoted test body\n> increments the count. But for a here-doc, there is an implicit\n> newline at the end of the token stream before the here-doc starts.\n> We have to increment \"lineno\" to account for this.\n>\n> Actually, this is not _quite_ correct, as there could be multiple\n> here-docs, like:\n>\n> test_expect_success \"$(cat <<END_OF_TITLE)\" - <<END_OF_TEST\n> this is the title\n> END_OF_TITLE\n> this is the test\n> END_OF_TEST\n>\n> in which case we'd need to skip past END_OF_TITLE. Given how\n> unlikely it is for anybody to do this, and since it would only\n> affect line numbers, it's probably not worth caring about too much.\n> The solution would probably be to record the starting line number\n> of each here-doc section in the lexer/shellparser stage.\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n> I actually suspect the \"record the heredoc line number\" thing would not\n> be too hard. I.e., turn ShellParser's \"heredoc\" hash to point to\n> hashrefs like: \"{ content => ..., lineno => ... }\". And that would give\n> us a good spot to stick an \"interpolate\" boolean later if we want.\n\nIt turned out to be quite easy. See below for an implementation atop\nyour patch [1/3] (modulo Gmail whitespace damage). Given how simple\nthis ended up being, it probably makes sense to squash this change in,\nas well.\n\n--- >8 ---\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex c9ab79b6b0..b31cb263f8 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -174,8 +174,10 @@ sub swallow_heredocs {\n     $$b =~ /(?:\\G|\\n)$indent\\Q$$tag[0]\\E(?:\\n|\\z)/gc;\n     if (pos($$b) > $start) {\n       my $body = substr($$b, $start, pos($$b) - $start);\n-      $self->{parser}->{heredocs}->{$$tag[0]} =\n-          substr($body, 0, length($body) - length($&));\n+      $self->{parser}->{heredocs}->{$$tag[0]} = {\n+        content => substr($body, 0, length($body) - length($&)),\n+        start_line => $self->{lineno},\n+      };\n       $self->{lineno} += () = $body =~ /\\n/sg;\n       next;\n     }\n@@ -624,8 +626,9 @@ sub check_test {\n   my $lineno = $body->[3];\n   $body = unwrap($body);\n   if ($body eq '-') {\n-    $body = shift @_;\n-    $lineno++;\n+    my $herebody = shift @_;\n+    $body = $herebody->{content};\n+    $lineno = $herebody->{start_line};\n   }\n   $self->{ntests}++;\n   my $parser = TestParser->new(\\$body);\n--\n"},{"id":"498216","messageId":"20240708085909.GB819809@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cRzoyxQpNf15nGeHDdDQK64KRJXLdgKN2rkXaN+2SjTZQ@mail.gmail.com","subject":"Re: [PATCH 3/3] t/chainlint: add tests for test body in heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-08T08:59:09Z","receivedAt":"2024-07-08T08:59:10Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Jul 07, 2024 at 10:43:45PM -0400, Eric Sunshine wrote:\n\n> This one made me think of an additional pathological case, though I'm\n> not sure it's worth having a test:\n> \n>     test_expect_success 'pathological-here-doc-body' - <<\\EOF\n>         echo \"missing chain before\"\n>         cat >file <<-\\EOF &&\n>         inside inner here-doc\n>         these are not shell commands\n>         EOF\n>         echo \"missing chain after\"\n>         echo \"but this line is OK because it's the end\"\n>    EOF\n> \n> It's exactly the same as your test except that the same tag (\"EOF\") is\n> used for both outer and inner heredocs. It works because the outer\n> heredoc is introduced with `<<` whereas the inner with `<<-`. The\n> opposite case, in which outer is introduced with `<<-` and inner with\n> `<<`, obviously would be bogus.\n\nOoh, that's devious. I'll add it.\n\n-Peff\n"},{"id":"498217","messageId":"20240708090530.GC819809@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cRUOSHuNezZfVUT43JGT5BWA3LTbQvKCkD37DxXn=L6Jg@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-08T09:05:30Z","receivedAt":"2024-07-08T09:05:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Jul 07, 2024 at 11:40:19PM -0400, Eric Sunshine wrote:\n\n> > On the flip side, what about cleaning up? The \"heretags\" array is\n> > emptied as we parse the heredocs in swallow_heredocs(). But I think once\n> > a ShellParser's $self->{heredocs}->{FOO} is written, it will hang around\n> > forever (even though it's valid only for that one command). Probably not\n> > a big deal, but there's probably some correct spot to reset it.\n> \n> There are a few reasons I wasn't overly concerned about cleaning up in\n> this case:\n> \n> (1) The parsers are all short-lived, so the collected heredoc bodies\n> won't stick around long anyhow. For each test checked, a TestParser is\n> created and destroyed. For each script mentioned on the command-line,\n> a ScriptParser is created and destroyed. None of these parsers stick\n> around for long, though, a ScriptParser outlives a TestParser.\n> \n> (2) The heredoc bodies in question tend to be pretty small, so it's\n> not consuming an inordinate amount of memory even if a single parser\n> latches bodies of multiple heredocs.\n> \n> (3) We tend to be quite consistent about naming our heredoc tag (i.e.\n> \"EOF\", \"EOT\"), so a latched body in the parser's %heredocs hash is\n> very likely to get overwritten, thus the hash is probably not going to\n> eat up a lot of memory. Given the entire test suite, I'd be quite\n> surprised if any one parser ever latches more than three heredoc\n> bodies at a time, and the vast majority of parsers are likely latching\n> zero or one heredoc body.\n> \n> (4) I couldn't really think of a correct spot to reset %heredocs.\n\nAll of that makes sense to me, especially (4). :)\n\n> That said, after reading your message, I did try implementing an\n> approach in which the heredoc body gets attached to the `<<` or `<<-`\n> token. That way, a heredoc body would be cleaned along with its\n> associated lexer token. However, the implementation got too ugly and\n> increased cognitive load too much for my liking, so I abandoned it.\n\nOK, thanks for trying. I do think sticking it into the token stream\nwould make sense, but if the implementation got tricky, it is probably\nnot worth the effort. We can always revisit it later if we find some\nreason that it would be useful to do it that way.\n\n> > OK, mostly as expected. I think the check for \"-\" here is redundant with\n> > what's in check_test(). We could just feed the heredoc body either way,\n> > and in the nonsense case of:\n> >\n> >   test_expect_success 'title' 'test body' <<EOT\n> >   nobody reads this!\n> >   EOT\n> >\n> > the heredoc data would just be ignored.\n> \n> Right. I went back and forth with this, never sure if this code was\n> overkill. On the other hand, we could make this more paranoid and\n> complain if we see either of these cases:\n> \n> (1) \"-\" but no heredoc\n> (2) heredoc present but something other than \"-\"\n\nThose seem like good things to check for, and not too hard to add. I'll\nsee if I can work up some tests.\n\n> > Requiring \"<<\" at the end is somewhat limiting. E.g. this is valid:\n> >\n> >   test_expect_success <<EOT 'title' -\n> >   the test body\n> >   EOT\n> \n> True, I didn't even think about that.\n> \n> > I don't expect anybody to do that, but it would be nice to be more\n> > robust if we can. I think the tokens are still wrapped at this point, so\n> > we could read through all of them looking for \"<<\" anywhere, without\n> > getting confused by \"$(cat <<INNER_HEREDOC)\". I think, anyway (I didn't\n> > test).\n> \n> Correct. The stuff inside \"$(...)\" does get parsed and linted, but by\n> the time ScriptParser::parse_cmd() sees it, `$(cat <<INNER_HEREDOC)`\n> is just a single (string) token.\n\nOK, I'll see if I can generalize it a bit (and add a test).\n\n-Peff\n"},{"id":"498218","messageId":"20240708090837.GD819809@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cTv-DaGRmwWWCk8b33MKzV25vfP2zPKd2VOAEOtz4FZ2A@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-08T09:08:37Z","receivedAt":"2024-07-08T09:08:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Jul 07, 2024 at 11:51:15PM -0400, Eric Sunshine wrote:\n\n> >     I did manually verify that it is OK after my fix, but do we want\n> >     that to be part of the chainlint tests? Just leaving the line\n> >     numbers in is a maintenance nightmare, since it depends on the order\n> >     of concatenating all of the tests together (so our \"expect\" files\n> >     would depend on all of the previous tests). But if we wanted to get\n> >     fancy, we could perhaps store relative offsets in the expect file. I\n> >     think it gets pretty complicated, though, since we print only\n> >     problematic lines.\n> \n> Given the way the Makefile currently concatenates all the self-tests,\n> it would indeed be a nightmare to retain the line numbers. In the long\n> run, we probably ought someday to adopt Ævar's idea of checking the\n> self-test files individually[*] rather than en masse. With that\n> approach, it may make sense to revisit whether or not line numbers\n> should be present in the \"expected\" files.\n> \n> [*] https://lore.kernel.org/git/CAPig+cSBjsosRqoAafYN94Cco8+7SdUt0ND_jHS+jVPoM4K0JA@mail.gmail.com/\n\nI took a look at running each test individually. It's surprisingly quite\na bit slower! About 4s instead of 200ms. There's a bit of low-hanging\nfruit to get it down to ~1.7s (which I'll include in my series). But in\nthe end I punted on that for now, but did add line-number checks. Each\nexpect file just knows its own numbers, and I use a bit of perl to\nhandle the running offset.\n\n-Peff\n"},{"id":"498219","messageId":"20240708091026.GE819809@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cRxnpG9Yfix09EnJAbnzwN=yoUtSeYxt7S-Od+xBgfaYQ@mail.gmail.com","subject":"Re: [PATCH 1/3] chainlint.pl: fix line number reporting","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-08T09:10:26Z","receivedAt":"2024-07-08T09:10:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 08, 2024 at 01:08:02AM -0400, Eric Sunshine wrote:\n\n> > I actually suspect the \"record the heredoc line number\" thing would not\n> > be too hard. I.e., turn ShellParser's \"heredoc\" hash to point to\n> > hashrefs like: \"{ content => ..., lineno => ... }\". And that would give\n> > us a good spot to stick an \"interpolate\" boolean later if we want.\n> \n> It turned out to be quite easy. See below for an implementation atop\n> your patch [1/3] (modulo Gmail whitespace damage). Given how simple\n> this ended up being, it probably makes sense to squash this change in,\n> as well.\n\nVery nice! I was hoping it would be something like this. I've squashed\nthis in, and confirmed that it fixes the line numbers in my \"double\"\ncase:\n\n  test_expect_success \"$(cat <<END_OF_PREREQS)\" 'here-doc-double' - <<\\EOT\n  SOME\n  PREREQS\n  END_OF_PREREQS\n  \techo \"actual test commands\"\n  \techo \"that should be checked\"\n  EOT\n\nThe bogus line was incorrectly reported as line 2, because we did not\naccount for the first here-doc.\n\n-Peff\n"},{"id":"498311","messageId":"CAPig+cRzJXPa07wBXibCjfZ_uYtbAnRnOsXFrbMbCqRfwSxszg@mail.gmail.com","threadId":"61713","inReplyTo":"20240708090837.GD819809@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T19:46:51Z","receivedAt":"2024-07-08T19:47:03Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 8, 2024 at 5:08 AM Jeff King <peff@peff.net> wrote:\n> On Sun, Jul 07, 2024 at 11:51:15PM -0400, Eric Sunshine wrote:\n> > Given the way the Makefile currently concatenates all the self-tests,\n> > it would indeed be a nightmare to retain the line numbers. In the long\n> > run, we probably ought someday to adopt Ævar's idea of checking the\n> > self-test files individually[*] rather than en masse. With that\n> > approach, it may make sense to revisit whether or not line numbers\n> > should be present in the \"expected\" files.\n> >\n> > [*] https://lore.kernel.org/git/CAPig+cSBjsosRqoAafYN94Cco8+7SdUt0ND_jHS+jVPoM4K0JA@mail.gmail.com/\n>\n> I took a look at running each test individually. It's surprisingly quite\n> a bit slower! About 4s instead of 200ms.\n\nI'm not surprised. As currently implemented, `make test` chainlints\nthe self-tests and the Git test scripts unconditionally, even if none\nof them have changed. As I understand it, Ævar idea was that the\ncostly initial `make test` would be offset by subsequent `make test`\ninvocations since `make` will only recheck the self-test files and Git\ntest scripts if they have been changed. His particular use-case, as I\nrecall, was when running the full `make test` repeatedly, such as with\n`git rebase --exec 'make test' HEAD~n` to ensure that the entire test\nsuite passes for each patch of a multi-patch series prior to\nsubmitting the series; the repeated cost of linting unchanged files\nadds up, especially when the series is long.\n\n> There's a bit of low-hanging\n> fruit to get it down to ~1.7s (which I'll include in my series). But in\n> the end I punted on that for now, but did add line-number checks. Each\n> expect file just knows its own numbers, and I use a bit of perl to\n> handle the running offset.\n\nOkay.\n"},{"id":"498314","messageId":"CAPig+cTFZuU7zM7poqk4HeK09zn8bFrO37eUZiaGmeJ0yecpiw@mail.gmail.com","threadId":"61713","inReplyTo":"20240708090530.GC819809@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T20:06:47Z","receivedAt":"2024-07-08T20:06:59Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 8, 2024 at 5:05 AM Jeff King <peff@peff.net> wrote:\n> On Sun, Jul 07, 2024 at 11:40:19PM -0400, Eric Sunshine wrote:\n> > (3) We tend to be quite consistent about naming our heredoc tag (i.e.\n> > \"EOF\", \"EOT\"), so a latched body in the parser's %heredocs hash is\n> > very likely to get overwritten, thus the hash is probably not going to\n> > eat up a lot of memory. Given the entire test suite, I'd be quite\n> > surprised if any one parser ever latches more than three heredoc\n> > bodies at a time, and the vast majority of parsers are likely latching\n> > zero or one heredoc body.\n\nOne thing we may want to measure is how much extra time we're wasting\nfor the (very) common case of latching heredoc bodies only to then\nignore them. In particular, we may want to add a flag to ShellParser\ntelling it whether or not to latch heredoc bodies, and enable that\nflag in subclass ScriptParser, but leave it disabled in subclass\nTestParser since only ScriptParser currently cares about the heredoc\nbody.\n\n> > (4) I couldn't really think of a correct spot to reset %heredocs.\n>\n> All of that makes sense to me, especially (4). :)\n>\n> > That said, after reading your message, I did try implementing an\n> > approach in which the heredoc body gets attached to the `<<` or `<<-`\n> > token. That way, a heredoc body would be cleaned along with its\n> > associated lexer token. However, the implementation got too ugly and\n> > increased cognitive load too much for my liking, so I abandoned it.\n>\n> OK, thanks for trying. I do think sticking it into the token stream\n> would make sense, but if the implementation got tricky, it is probably\n> not worth the effort. We can always revisit it later if we find some\n> reason that it would be useful to do it that way.\n\nIn the long run, I think we probably want to build a full parse tree,\nattach relevant information (such as a heredoc body) to each node, and\nthen walk the tree, rather than trying to perform on-the-fly lints and\nother operations on the token stream as is currently the case.\n\nThis encapsulation would not only solve the problem of releasing\nrelated resources (such as releasing the heredoc body when the `<<` or\n`<<-` node is released), but it would also make it possible to perform\nother lints I've had in mind. For instance, a while ago, I added (but\ndid not submit) a lint to check for `cd` outside of a subshell. After\nimplementing that, I realized that the cd-outside-subshell lint would\nbe useful, not just within test bodies, but also at the script level\nitself. However, because actual linting functionality resides entirely\nin TestParser, I wasn't able to reuse the code for detecting\ncd-outside-subshell at the script level, and ended up having to write\nduplicate linting code in ScriptParser. If, on the other hand, the\nlinting code was just handed a parse tree, then it wouldn't matter if\nthat parse tree came from parsing a test body or parsing a script.\n\nAll (or most) of the checks in t/check-non-portable-shell.pl could\nalso be incorporated into chainlint.pl (though that makes the name\n\"chainlint.pl\" even more of an anachronism than it already is since it\noutgrew \"chain linting\" when it starting checking for missing `||\nreturn`, if not before then.)\n"},{"id":"498315","messageId":"CAPig+cTUsLHnCKcVMGTGUFXX85bDgMOAuvb6uXmDqpB3XswHcQ@mail.gmail.com","threadId":"61713","inReplyTo":"CAPig+cRzJXPa07wBXibCjfZ_uYtbAnRnOsXFrbMbCqRfwSxszg@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-08T20:17:46Z","receivedAt":"2024-07-08T20:17:57Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jul 8, 2024 at 3:46 PM Eric Sunshine <sunshine@sunshineco.com> wrote:\n> On Mon, Jul 8, 2024 at 5:08 AM Jeff King <peff@peff.net> wrote:\n> > I took a look at running each test individually. It's surprisingly quite\n> > a bit slower! About 4s instead of 200ms.\n>\n> I'm not surprised. As currently implemented, `make test` chainlints\n> the self-tests and the Git test scripts unconditionally, even if none\n> of them have changed. As I understand it, Ævar idea was that the\n> costly initial `make test` would be offset by subsequent `make test`\n> invocations since `make` will only recheck the self-test files and Git\n> test scripts if they have been changed. His particular use-case, as I\n> recall, was when running the full `make test` repeatedly, such as with\n> `git rebase --exec 'make test' HEAD~n` to ensure that the entire test\n> suite passes for each patch of a multi-patch series prior to\n> submitting the series; the repeated cost of linting unchanged files\n> adds up, especially when the series is long.\n\nBy the way, regarding your 4s instead of 200ms result, I don't think\nthat is necessarily reflective of what can be achieved. In particular,\nto properly measure the effect, you also need to remove all the\nthreading support from chainlint.pl since using \"ithreads\" adds a\nnot-insignificant amount of time to script startup, especially on\nWindows, but even on Unix it is quite noticeable.\n\nTo test this, I think you can just replace this block:\n\n    unless ($Config{useithreads} && eval {\n        require threads; threads->import();\n        require Thread::Queue; Thread::Queue->import();\n        1;\n        }) {\n        push(@stats, check_script(1, sub { shift(@scripts); }, sub {\nprint(@_); }));\n        show_stats($start_time, \\@stats) if $show_stats;\n        exit(exit_code(\\@stats));\n    }\n\nwith:\n\n    if (1) {\n        push(@stats, check_script(1, sub { shift(@scripts); }, sub {\nprint(@_); }));\n        show_stats($start_time, \\@stats) if $show_stats;\n        exit(exit_code(\\@stats));\n    }\n"},{"id":"498401","messageId":"20240710003737.GA1526498@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cTUsLHnCKcVMGTGUFXX85bDgMOAuvb6uXmDqpB3XswHcQ@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T00:37:37Z","receivedAt":"2024-07-10T00:37:39Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 08, 2024 at 04:17:46PM -0400, Eric Sunshine wrote:\n\n> By the way, regarding your 4s instead of 200ms result, I don't think\n> that is necessarily reflective of what can be achieved. In particular,\n> to properly measure the effect, you also need to remove all the\n> threading support from chainlint.pl since using \"ithreads\" adds a\n> not-insignificant amount of time to script startup, especially on\n> Windows, but even on Unix it is quite noticeable.\n\nYes, that is the low-hanging fruit I found. ;) Just adding:\n\n  $jobs = @scripts if @scripts < $jobs;\n\ncuts the time to run all scripts individually from ~4s to ~1.7s.\n\nRemoving the threading entirely goes to ~1.1s. I hadn't tried that until\nnow, but it is probably worth doing for the case of $jobs == 1.\n\n-Peff\n"},{"id":"498402","messageId":"20240710004817.GB1526498@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cTFZuU7zM7poqk4HeK09zn8bFrO37eUZiaGmeJ0yecpiw@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T00:48:17Z","receivedAt":"2024-07-10T00:48:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 08, 2024 at 04:06:47PM -0400, Eric Sunshine wrote:\n\n> One thing we may want to measure is how much extra time we're wasting\n> for the (very) common case of latching heredoc bodies only to then\n> ignore them. In particular, we may want to add a flag to ShellParser\n> telling it whether or not to latch heredoc bodies, and enable that\n> flag in subclass ScriptParser, but leave it disabled in subclass\n> TestParser since only ScriptParser currently cares about the heredoc\n> body.\n\nI doubt it's much to hold on to a few extra small buffers. But it should\nbe easy to measure. Here are hyperfine results for checking all of our\ntest scripts both before (old.pl) and after (new.pl) your chainlint.pl,\nwith threading disabled:\n\n  Benchmark 1: perl old.pl t[0-9]*.sh\n    Time (mean ± σ):      4.215 s ±  0.052 s    [User: 4.187 s, System: 0.028 s]\n    Range (min … max):    4.124 s …  4.288 s    10 runs\n  \n  Benchmark 2: perl new.pl t[0-9]*.sh\n    Time (mean ± σ):      4.295 s ±  0.060 s    [User: 4.264 s, System: 0.031 s]\n    Range (min … max):    4.229 s …  4.419 s    10 runs\n  \n  Summary\n    perl old.pl t[0-9]*.sh ran\n      1.02 ± 0.02 times faster than perl new.pl t[0-9]*.sh\n\nSo it does seem to make a small difference, but we're within the noise.\n\n> In the long run, I think we probably want to build a full parse tree,\n> attach relevant information (such as a heredoc body) to each node, and\n> then walk the tree, rather than trying to perform on-the-fly lints and\n> other operations on the token stream as is currently the case.\n> [...]\n\nYeah, all of that sounds very sensible long term, but probably not worth\nworrying about for this topic.\n\n-Peff\n"},{"id":"498406","messageId":"20240710010927.GA1529709@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240708090837.GD819809@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T01:09:27Z","receivedAt":"2024-07-10T01:09:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 08, 2024 at 05:08:37AM -0400, Jeff King wrote:\n\n> I took a look at running each test individually. It's surprisingly quite\n> a bit slower! About 4s instead of 200ms. There's a bit of low-hanging\n> fruit to get it down to ~1.7s (which I'll include in my series). But in\n> the end I punted on that for now, but did add line-number checks. Each\n> expect file just knows its own numbers, and I use a bit of perl to\n> handle the running offset.\n\nBy the way, in case you are wondering why I haven't sent out the next\niteration of the series: I am stuck trying to figure out some Windows\nline-ending nonsense.\n\nThe chainlint.pl parser chokes on CRLF line endings. So Windows CI\nproduces:\n\n  runneradmin@fv-az1390-742 MINGW64 /d/a/git/git/t\n  # perl chainlint.pl chainlint/for-loop.test\n  'nternal error scanning character '\n\n(the funny overwrite is because the invalid char is a CR). I tried a few\nsimple things to skip past this error, but the problem is pervasive. We\nreally just want to have perl handle the line endings on read. And doing\nthis works:\n\n  # PERLIO=:crlf perl chainlint.pl chainlint/for-loop.test\n  # chainlint: chainlint/for-loop.test\n  # chainlint: for-loop\n  [...etc, normal output...]\n\nWhich gives me all sorts of questions:\n\n  - isn't crlf handling usually the default for perl builds on Windows?\n    I guess this is probably getting into weird mingw vs native Windows\n    distinctions that generally leave me perplexed.\n\n  - why wasn't this a problem before? I'm guessing again in the \"weird\n    mingw stuff\" hand-waving way that when we used \"sed\" to assemble\n    everything, it stripped the CR's in the \"chainlinttmp/tests\" file.\n    And in my series, that \"cat\" is replaced with a perl script (that\n    writes the \"tests\" and \"expect\" files together).\n\n  - why doesn't \"PERLIO=:crlf make check-chainlint\" work? It seems that\n    perl spawned from \"make\" behaves differently. More mingw weirdness?\n\nI'm tempted to just do this:\n\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -779,7 +779,7 @@ sub check_script {\n        while (my $path = $next_script->()) {\n                $nscripts++;\n                my $fh;\n-               unless (open($fh, \"<\", $path)) {\n+               unless (open($fh, \"<:unix:crlf\", $path)) {\n                        $emit->(\"?!ERR?! $path: $!\\n\");\n                        next;\n                }\n\nIt feels like a hack, but it makes the parser's assumptions explicit,\nand it should just work everywhere.\n\n-Peff\n"},{"id":"498409","messageId":"CAPig+cQ5FgEADF+v+TAY4aqg-rC1OAx_MzuqK=a7MJDYDLLEPA@mail.gmail.com","threadId":"61713","inReplyTo":"20240710004817.GB1526498@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-10T02:38:46Z","receivedAt":"2024-07-10T02:38:58Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jul 9, 2024 at 8:48 PM Jeff King <peff@peff.net> wrote:\n> On Mon, Jul 08, 2024 at 04:06:47PM -0400, Eric Sunshine wrote:\n> > One thing we may want to measure is how much extra time we're wasting\n> > for the (very) common case of latching heredoc bodies only to then\n> > ignore them. In particular, we may want to add a flag to ShellParser\n> > telling it whether or not to latch heredoc bodies, and enable that\n> > flag in subclass ScriptParser, but leave it disabled in subclass\n> > TestParser since only ScriptParser currently cares about the heredoc\n> > body.\n>\n> I doubt it's much to hold on to a few extra small buffers.\n\nI was more concerned about the extra substr() consuming additional CPU\ntime and inflating wall-clock time.\n\n> So it does seem to make a small difference, but we're within the noise.\n\nOkay. Thanks for measuring.\n\n> > In the long run, I think we probably want to build a full parse tree,\n> > attach relevant information (such as a heredoc body) to each node, and\n> > then walk the tree, rather than trying to perform on-the-fly lints and\n> > other operations on the token stream as is currently the case.\n> > [...]\n>\n> Yeah, all of that sounds very sensible long term, but probably not worth\n> worrying about for this topic.\n\nAgreed.\n"},{"id":"498410","messageId":"CAPig+cRXkOesS_ctvxY2X=rwesTzgrBB0=5fvQLQsG3hZVY9TQ@mail.gmail.com","threadId":"61713","inReplyTo":"20240710010927.GA1529709@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-10T03:02:01Z","receivedAt":"2024-07-10T03:02:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Jul 9, 2024 at 9:09 PM Jeff King <peff@peff.net> wrote:\n> The chainlint.pl parser chokes on CRLF line endings. So Windows CI\n> produces:\n>\n>   runneradmin@fv-az1390-742 MINGW64 /d/a/git/git/t\n>   # perl chainlint.pl chainlint/for-loop.test\n>   'nternal error scanning character '\n\nAs far as I understand, chainlint is disabled in the Windows CI. Did\nyou manually re-enable it for testing? Or are you just running it\nmanually in the Windows CI?\n\n> We really just want to have perl handle the line endings on read. And doing\n> this works:\n>\n>   # PERLIO=:crlf perl chainlint.pl chainlint/for-loop.test\n>   # chainlint: chainlint/for-loop.test\n>   # chainlint: for-loop\n>   [...etc, normal output...]\n>\n> Which gives me all sorts of questions:\n>\n>   - isn't crlf handling usually the default for perl builds on Windows?\n>     I guess this is probably getting into weird mingw vs native Windows\n>     distinctions that generally leave me perplexed.\n\nCould be. I'm not sure how the Windows CI is provisioned, whether with\nsome native-compiled Perl or with msys2/mingw Perl.\n\n>   - why wasn't this a problem before? I'm guessing again in the \"weird\n>     mingw stuff\" hand-waving way that when we used \"sed\" to assemble\n>     everything, it stripped the CR's in the \"chainlinttmp/tests\" file.\n>     And in my series, that \"cat\" is replaced with a perl script (that\n>     writes the \"tests\" and \"expect\" files together).\n\nAssuming you manually re-enabled chaintlint in the Windows CI for this\ntesting or are running it manually, it may be the case that\nchainlint.pl has never been run in the Windows CI. Specifically,\nchainlint in Windows CI was disabled by a87e427e35 (ci: speed up\nWindows phase, 2019-01-29) which predates the switchover from\nchainlint.sed to chainlint.pl by d00113ec34 (t/Makefile: apply\nchainlint.pl to existing self-tests, 2022-09-01). So, it's quite\npossible that chainlint.pl has never run in Windows CI. But, perhaps\nI'm misunderstanding or missing some piece of information.\n\nThat said, I did thoroughly test chainlint.pl on Windows using Git For\nWindows, and it did run in that environment. (But if the Windows CI\nenvironment is somehow different, then that might explain the\nproblem?)\n\n>   - why doesn't \"PERLIO=:crlf make check-chainlint\" work? It seems that\n>     perl spawned from \"make\" behaves differently. More mingw weirdness?\n\nThat could indeed be an msys2 issue. It will automatically convert\ncolon \":\" to semicolon \";\" in environment variables since the PATH\nseparator on Windows is \";\", not \":\" as it is on Unix. Moreover, the\n\":\" to \";\" switcheroo logic is not restricted only to PATH since there\nare other PATH-like variables in common use, so it's applied to all\nenvironment variables.\n\n> I'm tempted to just do this:\n>\n>         while (my $path = $next_script->()) {\n>                 $nscripts++;\n>                 my $fh;\n> -               unless (open($fh, \"<\", $path)) {\n> +               unless (open($fh, \"<:unix:crlf\", $path)) {\n>\n> It feels like a hack, but it makes the parser's assumptions explicit,\n> and it should just work everywhere.\n\nYep, if this makes it work, then it seems like a good way forward,\nespecially since I don't think there's any obvious way to work around\nthe \":\" to \";\" switcheroo performed by msys2.\n"},{"id":"498414","messageId":"20240710070647.GA2048777@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cRXkOesS_ctvxY2X=rwesTzgrBB0=5fvQLQsG3hZVY9TQ@mail.gmail.com","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T07:06:47Z","receivedAt":"2024-07-10T07:06:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jul 09, 2024 at 11:02:01PM -0400, Eric Sunshine wrote:\n\n> On Tue, Jul 9, 2024 at 9:09 PM Jeff King <peff@peff.net> wrote:\n> > The chainlint.pl parser chokes on CRLF line endings. So Windows CI\n> > produces:\n> >\n> >   runneradmin@fv-az1390-742 MINGW64 /d/a/git/git/t\n> >   # perl chainlint.pl chainlint/for-loop.test\n> >   'nternal error scanning character '\n> \n> As far as I understand, chainlint is disabled in the Windows CI. Did\n> you manually re-enable it for testing? Or are you just running it\n> manually in the Windows CI?\n\nNeither. As far as I can tell, we still run the \"check-chainlint\" target\nas part of \"make test\", and that's what I saw fail. For instance:\n\n  https://github.com/peff/git/actions/runs/9856301557/job/27213352807\n\nEvery one of the \"win test\" jobs failed, with the same outcome: running\ncheck-chainlint triggered the \"internal scanning error\".\n\n> Assuming you manually re-enabled chaintlint in the Windows CI for this\n> testing or are running it manually, it may be the case that\n> chainlint.pl has never been run in the Windows CI. Specifically,\n> chainlint in Windows CI was disabled by a87e427e35 (ci: speed up\n> Windows phase, 2019-01-29) which predates the switchover from\n> chainlint.sed to chainlint.pl by d00113ec34 (t/Makefile: apply\n> chainlint.pl to existing self-tests, 2022-09-01). So, it's quite\n> possible that chainlint.pl has never run in Windows CI. But, perhaps\n> I'm misunderstanding or missing some piece of information.\n\nI think that commit would prevent it from running as part of the actual\ntest scripts. But we'd still do check-chainlint to run the chainlint\nself-tests. And because it only sets \"--no-chain-lint\" in GIT_TEST_OPTS\nand not GIT_TEST_CHAIN_LINT=0, I think that the bulk run of chainlint.pl\nby t/Makefile is still run (and then ironically, when that is run the\nMakefile manually suppresses the per-script runs, so that\n--no-chain-lint option is truly doing nothing).\n\nAnd I think is true even with the ci/run-test-slice.sh approach that the\nWindows tests use. They still drive it through \"make\", and just override\nthe $(T) variable.\n\n> >   - why doesn't \"PERLIO=:crlf make check-chainlint\" work? It seems that\n> >     perl spawned from \"make\" behaves differently. More mingw weirdness?\n> \n> That could indeed be an msys2 issue. It will automatically convert\n> colon \":\" to semicolon \";\" in environment variables since the PATH\n> separator on Windows is \";\", not \":\" as it is on Unix. Moreover, the\n> \":\" to \";\" switcheroo logic is not restricted only to PATH since there\n> are other PATH-like variables in common use, so it's applied to all\n> environment variables.\n\nAh, good thinking. I'm not sure if that's it, though. Just PERLIO=crlf\nshould behave the same way (the \":\" is technically a separator, and it\nis only a style suggestion that you prepend one). Likewise a space is\nsupposed to be OK, too, so PERLIO=\"unix crlf\" should work. But neither\nseems to work for me. So I'm still puzzled.\n\n> > I'm tempted to just do this:\n> >\n> >         while (my $path = $next_script->()) {\n> >                 $nscripts++;\n> >                 my $fh;\n> > -               unless (open($fh, \"<\", $path)) {\n> > +               unless (open($fh, \"<:unix:crlf\", $path)) {\n> >\n> > It feels like a hack, but it makes the parser's assumptions explicit,\n> > and it should just work everywhere.\n> \n> Yep, if this makes it work, then it seems like a good way forward,\n> especially since I don't think there's any obvious way to work around\n> the \":\" to \";\" switcheroo performed by msys2.\n\nOK, I'll add that to my series, then. The fact that we weren't really\n_intending_ to run chainlint there makes me tempted to just punt and\ndisable it. But AFAICT we have been running it for a while, and it could\nbenefit people on Windows (though it is a bit funky that we do a full\ncheck-chainlint in each slice). And I suspect disabling it reliably\nmight be a trickier change than what I wrote above anyway. ;)\n\n-Peff\n"},{"id":"498416","messageId":"CAPig+cR_yN6viMPgFDZQW90GF-=_q_RhrH6DMG7YAp7LRRYKWg@mail.gmail.com","threadId":"61713","inReplyTo":"20240710070647.GA2048777@coredump.intra.peff.net","subject":"Re: [PATCH] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-07-10T07:29:57Z","receivedAt":"2024-07-10T07:30:10Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Jul 10, 2024 at 3:06 AM Jeff King <peff@peff.net> wrote:\n> On Tue, Jul 09, 2024 at 11:02:01PM -0400, Eric Sunshine wrote:\n> > That could indeed be an msys2 issue. It will automatically convert\n> > colon \":\" to semicolon \";\" in environment variables since the PATH\n> > separator on Windows is \";\", not \":\" as it is on Unix. Moreover, the\n> > \":\" to \";\" switcheroo logic is not restricted only to PATH since there\n> > are other PATH-like variables in common use, so it's applied to all\n> > environment variables.\n>\n> Ah, good thinking. I'm not sure if that's it, though. Just PERLIO=crlf\n> should behave the same way (the \":\" is technically a separator, and it\n> is only a style suggestion that you prepend one). Likewise a space is\n> supposed to be OK, too, so PERLIO=\"unix crlf\" should work. But neither\n> seems to work for me. So I'm still puzzled.\n\nMe too. Perhaps Dscho would have more insight(?).\n"},{"id":"498418","messageId":"20240710083416.GA2060328@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240701220815.GA20293@coredump.intra.peff.net","subject":"[PATCH v2 0/9] here-doc test bodies (now with 100% more chainlinting)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:34:16Z","receivedAt":"2024-07-10T08:34:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 01, 2024 at 06:08:15PM -0400, Jeff King wrote:\n\n> This is a re-post of an idea from 2021:\n> \n>   https://lore.kernel.org/git/YHDUg6ZR5vu93kGm@coredump.intra.peff.net/\n> \n> that people seemed mostly positive on, and I just never got around to\n> following up. Mostly because it's not life-changing, but I think it is a\n> small quality of life improvement, and it came up again recently in:\n> \n>   https://lore.kernel.org/git/20240701032047.GA610406@coredump.intra.peff.net/\n> \n> So I thought it was worth considering again.\n\nAnd here's a v2 that addresses the chainlint issue mentioned by Eric.\nThere were a lot of patches flying around, but this is only the second\nposting of the whole topic. This can replace all of what's in\njk/test-body-in-here-doc.\n\nI bailed on trying to notice:\n\n  test_expect_success 'oops, forgot the dash' <<\\EOT\n\nor:\n\n  test_expect_success 'oops, forgot the here doc' -\n\nor:\n\n  test_expect_success <<\\EOT 'here-doc tag comes first' -\n\nAs those all require some big refactoring ScriptParser::check_test(),\netc, and this topic has already grown quite a lot.\n\nI won't bother with a range diff; patches 8 and 9 are just the original\nv1 patches verbatim, and everything else is new.\n\n  [1/9]: chainlint.pl: add test_expect_success call to test snippets\n\n    Test refactoring for chainlint before we change it.\n\n  [2/9]: chainlint.pl: only start threads if jobs > 1\n  [3/9]: chainlint.pl: do not spawn more threads than we have scripts\n\n    These two aren't strictly necessary, but some easy speedups I hit\n    along the way (they depend on patch 1 for showing the speedup).\n\n  [4/9]: chainlint.pl: force CRLF conversion when opening input files\n  [5/9]: chainlint.pl: check line numbers in expected output\n\n    These two make the chainlint tests more robust against the\n    line-number bugs we hit while developing patch 6.\n\n  [6/9]: chainlint.pl: recognize test bodies defined via heredoc\n\n    This is Eric's fix (thanks!) for chainlint to recognize the new\n    format, including the line-number fixes that we discussed.\n\n  [7/9]: chainlint.pl: add tests for test body in heredoc\n\n    And then I kept my tests of the new feature split into their own\n    commit.\n\n  [8/9]: test-lib: allow test snippets as here-docs\n  [9/9]: t: convert some here-doc test bodies\n\n    And then this is the actual purpose of the series. ;)\n\n t/Makefile                                    |  16 +-\n t/README                                      |   8 +\n t/chainlint-cat.pl                            |  29 +++\n t/chainlint.pl                                |  33 ++-\n t/chainlint/arithmetic-expansion.expect       |  18 +-\n t/chainlint/arithmetic-expansion.test         |   2 +\n t/chainlint/bash-array.expect                 |  20 +-\n t/chainlint/bash-array.test                   |   2 +\n t/chainlint/blank-line-before-esac.expect     |  36 ++--\n t/chainlint/blank-line-before-esac.test       |   2 +\n t/chainlint/blank-line.expect                 |  16 +-\n t/chainlint/blank-line.test                   |   2 +\n t/chainlint/block-comment.expect              |  16 +-\n t/chainlint/block-comment.test                |   2 +\n t/chainlint/block.expect                      |  46 ++--\n t/chainlint/block.test                        |   2 +\n t/chainlint/broken-chain.expect               |  12 +-\n t/chainlint/broken-chain.test                 |   2 +\n t/chainlint/case-comment.expect               |  22 +-\n t/chainlint/case-comment.test                 |   2 +\n t/chainlint/case.expect                       |  38 ++--\n t/chainlint/case.test                         |   2 +\n t/chainlint/chain-break-background.expect     |  18 +-\n t/chainlint/chain-break-background.test       |   2 +\n t/chainlint/chain-break-continue.expect       |  24 +--\n t/chainlint/chain-break-continue.test         |   2 +\n t/chainlint/chain-break-false.expect          |  18 +-\n t/chainlint/chain-break-false.test            |   2 +\n t/chainlint/chain-break-return-exit.expect    |  38 ++--\n t/chainlint/chain-break-return-exit.test      |   2 +\n t/chainlint/chain-break-status.expect         |  18 +-\n t/chainlint/chain-break-status.test           |   2 +\n t/chainlint/chained-block.expect              |  18 +-\n t/chainlint/chained-block.test                |   2 +\n t/chainlint/chained-subshell.expect           |  20 +-\n t/chainlint/chained-subshell.test             |   2 +\n .../close-nested-and-parent-together.expect   |   6 +-\n .../close-nested-and-parent-together.test     |   2 +\n t/chainlint/close-subshell.expect             |  52 ++---\n t/chainlint/close-subshell.test               |   2 +\n .../command-substitution-subsubshell.expect   |   4 +-\n .../command-substitution-subsubshell.test     |   2 +\n t/chainlint/command-substitution.expect       |  18 +-\n t/chainlint/command-substitution.test         |   2 +\n t/chainlint/comment.expect                    |  16 +-\n t/chainlint/comment.test                      |   2 +\n t/chainlint/complex-if-in-cuddled-loop.expect |  18 +-\n t/chainlint/complex-if-in-cuddled-loop.test   |   2 +\n t/chainlint/cuddled-if-then-else.expect       |  12 +-\n t/chainlint/cuddled-if-then-else.test         |   2 +\n t/chainlint/cuddled-loop.expect               |   8 +-\n t/chainlint/cuddled-loop.test                 |   2 +\n t/chainlint/cuddled.expect                    |  34 +--\n t/chainlint/cuddled.test                      |   2 +\n t/chainlint/double-here-doc.expect            |  24 +--\n t/chainlint/double-here-doc.test              |   2 +\n t/chainlint/dqstring-line-splice.expect       |  10 +-\n t/chainlint/dqstring-line-splice.test         |   2 +\n t/chainlint/dqstring-no-interpolate.expect    |  24 +--\n t/chainlint/dqstring-no-interpolate.test      |   2 +\n t/chainlint/empty-here-doc.expect             |   8 +-\n t/chainlint/empty-here-doc.test               |   2 +\n t/chainlint/exclamation.expect                |   8 +-\n t/chainlint/exclamation.test                  |   2 +\n t/chainlint/exit-loop.expect                  |  48 ++---\n t/chainlint/exit-loop.test                    |   2 +\n t/chainlint/exit-subshell.expect              |  10 +-\n t/chainlint/exit-subshell.test                |   2 +\n t/chainlint/for-loop-abbreviated.expect       |  10 +-\n t/chainlint/for-loop-abbreviated.test         |   2 +\n t/chainlint/for-loop.expect                   |  28 +--\n t/chainlint/for-loop.test                     |   2 +\n t/chainlint/function.expect                   |  22 +-\n t/chainlint/function.test                     |   2 +\n t/chainlint/here-doc-body-indent.expect       |   2 +\n t/chainlint/here-doc-body-indent.test         |   4 +\n t/chainlint/here-doc-body-pathological.expect |   7 +\n t/chainlint/here-doc-body-pathological.test   |   9 +\n t/chainlint/here-doc-body.expect              |   7 +\n t/chainlint/here-doc-body.test                |   9 +\n t/chainlint/here-doc-close-subshell.expect    |   8 +-\n t/chainlint/here-doc-close-subshell.test      |   2 +\n t/chainlint/here-doc-double.expect            |   2 +\n t/chainlint/here-doc-double.test              |  10 +\n t/chainlint/here-doc-indent-operator.expect   |  22 +-\n t/chainlint/here-doc-indent-operator.test     |   2 +\n .../here-doc-multi-line-command-subst.expect  |  16 +-\n .../here-doc-multi-line-command-subst.test    |   2 +\n t/chainlint/here-doc-multi-line-string.expect |  14 +-\n t/chainlint/here-doc-multi-line-string.test   |   2 +\n t/chainlint/here-doc.expect                   |  50 ++---\n t/chainlint/here-doc.test                     |   2 +\n t/chainlint/if-condition-split.expect         |  14 +-\n t/chainlint/if-condition-split.test           |   2 +\n t/chainlint/if-in-loop.expect                 |  24 +--\n t/chainlint/if-in-loop.test                   |   2 +\n t/chainlint/if-then-else.expect               |  44 ++--\n t/chainlint/if-then-else.test                 |   2 +\n t/chainlint/incomplete-line.expect            |  20 +-\n t/chainlint/incomplete-line.test              |   2 +\n t/chainlint/inline-comment.expect             |  16 +-\n t/chainlint/inline-comment.test               |   2 +\n t/chainlint/loop-detect-failure.expect        |  30 +--\n t/chainlint/loop-detect-failure.test          |   2 +\n t/chainlint/loop-detect-status.expect         |  36 ++--\n t/chainlint/loop-detect-status.test           |   2 +\n t/chainlint/loop-in-if.expect                 |  24 +--\n t/chainlint/loop-in-if.test                   |   2 +\n t/chainlint/loop-upstream-pipe.expect         |  20 +-\n t/chainlint/loop-upstream-pipe.test           |   2 +\n ...ti-line-nested-command-substitution.expect |  36 ++--\n ...ulti-line-nested-command-substitution.test |   2 +\n t/chainlint/multi-line-string.expect          |  28 +--\n t/chainlint/multi-line-string.test            |   2 +\n t/chainlint/negated-one-liner.expect          |  10 +-\n t/chainlint/negated-one-liner.test            |   2 +\n t/chainlint/nested-cuddled-subshell.expect    |  50 ++---\n t/chainlint/nested-cuddled-subshell.test      |   2 +\n t/chainlint/nested-here-doc.expect            |  60 +++---\n t/chainlint/nested-here-doc.test              |   2 +\n t/chainlint/nested-loop-detect-failure.expect |  62 +++---\n t/chainlint/nested-loop-detect-failure.test   |   2 +\n t/chainlint/nested-subshell-comment.expect    |  22 +-\n t/chainlint/nested-subshell-comment.test      |   2 +\n t/chainlint/nested-subshell.expect            |  26 +--\n t/chainlint/nested-subshell.test              |   2 +\n t/chainlint/not-heredoc.expect                |  28 +--\n t/chainlint/not-heredoc.test                  |   2 +\n t/chainlint/one-liner-for-loop.expect         |  18 +-\n t/chainlint/one-liner-for-loop.test           |   2 +\n t/chainlint/one-liner.expect                  |  18 +-\n t/chainlint/one-liner.test                    |   2 +\n t/chainlint/p4-filespec.expect                |   8 +-\n t/chainlint/p4-filespec.test                  |   2 +\n t/chainlint/pipe.expect                       |  20 +-\n t/chainlint/pipe.test                         |   2 +\n t/chainlint/return-loop.expect                |  10 +-\n t/chainlint/return-loop.test                  |   2 +\n t/chainlint/semicolon.expect                  |  38 ++--\n t/chainlint/semicolon.test                    |   2 +\n t/chainlint/sqstring-in-sqstring.expect       |   8 +-\n t/chainlint/sqstring-in-sqstring.test         |   2 +\n t/chainlint/subshell-here-doc.expect          |  60 +++---\n t/chainlint/subshell-here-doc.test            |   2 +\n t/chainlint/subshell-one-liner.expect         |  38 ++--\n t/chainlint/subshell-one-liner.test           |   2 +\n t/chainlint/t7900-subtree.expect              |  44 ++--\n t/chainlint/t7900-subtree.test                |   2 +\n t/chainlint/token-pasting.expect              |  54 ++---\n t/chainlint/token-pasting.test                |   2 +\n t/chainlint/unclosed-here-doc-indent.expect   |   8 +-\n t/chainlint/unclosed-here-doc-indent.test     |   2 +\n t/chainlint/unclosed-here-doc.expect          |  14 +-\n t/chainlint/unclosed-here-doc.test            |   2 +\n t/chainlint/while-loop.expect                 |  28 +--\n t/chainlint/while-loop.test                   |   2 +\n t/t0600-reffiles-backend.sh                   |  38 ++--\n t/t1404-update-ref-errors.sh                  | 196 +++++++++---------\n t/test-lib-functions.sh                       |  32 ++-\n 159 files changed, 1285 insertions(+), 1025 deletions(-)\n create mode 100644 t/chainlint-cat.pl\n create mode 100644 t/chainlint/here-doc-body-indent.expect\n create mode 100644 t/chainlint/here-doc-body-indent.test\n create mode 100644 t/chainlint/here-doc-body-pathological.expect\n create mode 100644 t/chainlint/here-doc-body-pathological.test\n create mode 100644 t/chainlint/here-doc-body.expect\n create mode 100644 t/chainlint/here-doc-body.test\n create mode 100644 t/chainlint/here-doc-double.expect\n create mode 100644 t/chainlint/here-doc-double.test\n\n"},{"id":"498419","messageId":"20240710083448.GA2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 1/9] chainlint.pl: add test_expect_success call to test snippets","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:34:48Z","receivedAt":"2024-07-10T08:34:49Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The chainlint tests are a series of individual files, each holding a\ntest body. The \"make check-chainlint\" target assembles them into a\nsingle file, adding a \"test_expect_success\" function call around each.\nLet's instead include that function call in the files themselves. This\nis a little more boilerplate, but has several advantages:\n\n  1. You can now run chainlint manually on snippets with just \"perl\n     chainlint.perl chainlint/foo.test\". This can make developing and\n     debugging a little easier.\n\n  2. Many of the tests implicitly relied on the syntax of the lines\n     added by the Makefile (in particular the use of single-quotes).\n     This assumption is much easier to see when the single-quotes are\n     alongside the test body.\n\n  3. We had no way to test how the chainlint program handled\n     various test_expect_success lines themselves. Now we'll be able to\n     check variations.\n\nThe change to the .test files was done mechanically, using the same\ntest names they would have been assigned by the Makefile (this is\nimportant to match the expected output). The Makefile has the minimal\nchange to drop the extra lines; there are more cleanups possible but a\nfuture patch in this series will rewrite this substantially anyway.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/Makefile                                              | 4 +---\n t/chainlint/arithmetic-expansion.test                   | 2 ++\n t/chainlint/bash-array.test                             | 2 ++\n t/chainlint/blank-line-before-esac.test                 | 2 ++\n t/chainlint/blank-line.test                             | 2 ++\n t/chainlint/block-comment.test                          | 2 ++\n t/chainlint/block.test                                  | 2 ++\n t/chainlint/broken-chain.test                           | 2 ++\n t/chainlint/case-comment.test                           | 2 ++\n t/chainlint/case.test                                   | 2 ++\n t/chainlint/chain-break-background.test                 | 2 ++\n t/chainlint/chain-break-continue.test                   | 2 ++\n t/chainlint/chain-break-false.test                      | 2 ++\n t/chainlint/chain-break-return-exit.test                | 2 ++\n t/chainlint/chain-break-status.test                     | 2 ++\n t/chainlint/chained-block.test                          | 2 ++\n t/chainlint/chained-subshell.test                       | 2 ++\n t/chainlint/close-nested-and-parent-together.test       | 2 ++\n t/chainlint/close-subshell.test                         | 2 ++\n t/chainlint/command-substitution-subsubshell.test       | 2 ++\n t/chainlint/command-substitution.test                   | 2 ++\n t/chainlint/comment.test                                | 2 ++\n t/chainlint/complex-if-in-cuddled-loop.test             | 2 ++\n t/chainlint/cuddled-if-then-else.test                   | 2 ++\n t/chainlint/cuddled-loop.test                           | 2 ++\n t/chainlint/cuddled.test                                | 2 ++\n t/chainlint/double-here-doc.test                        | 2 ++\n t/chainlint/dqstring-line-splice.test                   | 2 ++\n t/chainlint/dqstring-no-interpolate.test                | 2 ++\n t/chainlint/empty-here-doc.test                         | 2 ++\n t/chainlint/exclamation.test                            | 2 ++\n t/chainlint/exit-loop.test                              | 2 ++\n t/chainlint/exit-subshell.test                          | 2 ++\n t/chainlint/for-loop-abbreviated.test                   | 2 ++\n t/chainlint/for-loop.test                               | 2 ++\n t/chainlint/function.test                               | 2 ++\n t/chainlint/here-doc-close-subshell.test                | 2 ++\n t/chainlint/here-doc-indent-operator.test               | 2 ++\n t/chainlint/here-doc-multi-line-command-subst.test      | 2 ++\n t/chainlint/here-doc-multi-line-string.test             | 2 ++\n t/chainlint/here-doc.test                               | 2 ++\n t/chainlint/if-condition-split.test                     | 2 ++\n t/chainlint/if-in-loop.test                             | 2 ++\n t/chainlint/if-then-else.test                           | 2 ++\n t/chainlint/incomplete-line.test                        | 2 ++\n t/chainlint/inline-comment.test                         | 2 ++\n t/chainlint/loop-detect-failure.test                    | 2 ++\n t/chainlint/loop-detect-status.test                     | 2 ++\n t/chainlint/loop-in-if.test                             | 2 ++\n t/chainlint/loop-upstream-pipe.test                     | 2 ++\n t/chainlint/multi-line-nested-command-substitution.test | 2 ++\n t/chainlint/multi-line-string.test                      | 2 ++\n t/chainlint/negated-one-liner.test                      | 2 ++\n t/chainlint/nested-cuddled-subshell.test                | 2 ++\n t/chainlint/nested-here-doc.test                        | 2 ++\n t/chainlint/nested-loop-detect-failure.test             | 2 ++\n t/chainlint/nested-subshell-comment.test                | 2 ++\n t/chainlint/nested-subshell.test                        | 2 ++\n t/chainlint/not-heredoc.test                            | 2 ++\n t/chainlint/one-liner-for-loop.test                     | 2 ++\n t/chainlint/one-liner.test                              | 2 ++\n t/chainlint/p4-filespec.test                            | 2 ++\n t/chainlint/pipe.test                                   | 2 ++\n t/chainlint/return-loop.test                            | 2 ++\n t/chainlint/semicolon.test                              | 2 ++\n t/chainlint/sqstring-in-sqstring.test                   | 2 ++\n t/chainlint/subshell-here-doc.test                      | 2 ++\n t/chainlint/subshell-one-liner.test                     | 2 ++\n t/chainlint/t7900-subtree.test                          | 2 ++\n t/chainlint/token-pasting.test                          | 2 ++\n t/chainlint/unclosed-here-doc-indent.test               | 2 ++\n t/chainlint/unclosed-here-doc.test                      | 2 ++\n t/chainlint/while-loop.test                             | 2 ++\n 73 files changed, 145 insertions(+), 3 deletions(-)\n\ndiff --git a/t/Makefile b/t/Makefile\nindex b2eb9f770b..e7a476966e 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -109,9 +109,7 @@ clean-chainlint:\n check-chainlint:\n \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n-\t\techo \"test_expect_success '$$i' '\" && \\\n-\t\tsed -e '/^# LINT: /d' chainlint/$$i.test && \\\n-\t\techo \"'\"; \\\n+\t\tsed -e '/^# LINT: /d' chainlint/$$i.test; \\\n \tdone >'$(CHAINLINTTMP_SQ)'/tests && \\\n \t{ \\\n \t\techo \"# chainlint: $(CHAINLINTTMP_SQ)/tests\" && \\\ndiff --git a/t/chainlint/arithmetic-expansion.test b/t/chainlint/arithmetic-expansion.test\nindex 16206960d8..7b4c5c9a41 100644\n--- a/t/chainlint/arithmetic-expansion.test\n+++ b/t/chainlint/arithmetic-expansion.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'arithmetic-expansion' '\n (\n \tfoo &&\n # LINT: closing \")\" of $((...)) not misinterpreted as subshell-closing \")\"\n@@ -9,3 +10,4 @@\n \tbar=$((42 + 1))\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/bash-array.test b/t/chainlint/bash-array.test\nindex 92bbb777b8..4ca977d299 100644\n--- a/t/chainlint/bash-array.test\n+++ b/t/chainlint/bash-array.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'bash-array' '\n (\n \tfoo &&\n # LINT: \")\" in Bash array assignment not misinterpreted as subshell-closing \")\"\n@@ -10,3 +11,4 @@\n \tbar=${#bar[@]} &&\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/blank-line-before-esac.test b/t/chainlint/blank-line-before-esac.test\nindex cecccad19f..51f02ea0c5 100644\n--- a/t/chainlint/blank-line-before-esac.test\n+++ b/t/chainlint/blank-line-before-esac.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'blank-line-before-esac' '\n # LINT: blank line before \"esac\"\n test_done () {\n \tcase \"$test_failure\" in\n@@ -17,3 +18,4 @@ test_done () {\n \n \tesac\n }\n+'\ndiff --git a/t/chainlint/blank-line.test b/t/chainlint/blank-line.test\nindex 0fdf15b3e1..6f29a491de 100644\n--- a/t/chainlint/blank-line.test\n+++ b/t/chainlint/blank-line.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'blank-line' '\n (\n \n \tnothing &&\n@@ -8,3 +9,4 @@\n \n \n )\n+'\ndiff --git a/t/chainlint/block-comment.test b/t/chainlint/block-comment.test\nindex df2beea888..934ef4113a 100644\n--- a/t/chainlint/block-comment.test\n+++ b/t/chainlint/block-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'block-comment' '\n (\n \t{\n \t\t# show a\n@@ -6,3 +7,4 @@\n \t\techo b\n \t}\n )\n+'\ndiff --git a/t/chainlint/block.test b/t/chainlint/block.test\nindex 4ab69a4afc..a1b6b4dd32 100644\n--- a/t/chainlint/block.test\n+++ b/t/chainlint/block.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'block' '\n (\n # LINT: missing \"&&\" after first \"echo\"\n \tfoo &&\n@@ -25,3 +26,4 @@\n \techo \"done\"\n } &&\n finis\n+'\ndiff --git a/t/chainlint/broken-chain.test b/t/chainlint/broken-chain.test\nindex 2a44aa73b7..1966499ef9 100644\n--- a/t/chainlint/broken-chain.test\n+++ b/t/chainlint/broken-chain.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'broken-chain' '\n (\n \tfoo &&\n # LINT: missing \"&&\" from \"bar\"\n@@ -6,3 +7,4 @@\n # LINT: final statement before closing \")\" legitimately lacks \"&&\"\n \twop\n )\n+'\ndiff --git a/t/chainlint/case-comment.test b/t/chainlint/case-comment.test\nindex 641c157b98..3f31ae9010 100644\n--- a/t/chainlint/case-comment.test\n+++ b/t/chainlint/case-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'case-comment' '\n (\n \tcase \"$x\" in\n \t# found foo\n@@ -9,3 +10,4 @@\n \t\t;;\n \tesac\n )\n+'\ndiff --git a/t/chainlint/case.test b/t/chainlint/case.test\nindex 4cb086bf87..bea21fee4f 100644\n--- a/t/chainlint/case.test\n+++ b/t/chainlint/case.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'case' '\n (\n # LINT: \"...)\" arms in \"case\" not misinterpreted as subshell-closing \")\"\n \tcase \"$x\" in\n@@ -21,3 +22,4 @@\n \tcase \"$y\" in 2) false;; esac\n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/chain-break-background.test b/t/chainlint/chain-break-background.test\nindex e10f656b05..c68e1b04d5 100644\n--- a/t/chainlint/chain-break-background.test\n+++ b/t/chainlint/chain-break-background.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-background' '\n JGIT_DAEMON_PID= &&\n git init --bare empty.git &&\n >empty.git/git-daemon-export-ok &&\n@@ -8,3 +9,4 @@ mkfifo jgit_daemon_output &&\n \tJGIT_DAEMON_PID=$!\n } &&\n test_expect_code 2 git ls-remote --exit-code git://localhost:$JGIT_DAEMON_PORT/empty.git\n+'\ndiff --git a/t/chainlint/chain-break-continue.test b/t/chainlint/chain-break-continue.test\nindex f0af71d8bd..de8119b204 100644\n--- a/t/chainlint/chain-break-continue.test\n+++ b/t/chainlint/chain-break-continue.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-continue' '\n git ls-tree --name-only -r refs/notes/many_notes |\n while read path\n do\n@@ -11,3 +12,4 @@ do\n \t\treturn 1\n \tfi\n done\n+'\ndiff --git a/t/chainlint/chain-break-false.test b/t/chainlint/chain-break-false.test\nindex a5aaff8c8a..f78ad911fc 100644\n--- a/t/chainlint/chain-break-false.test\n+++ b/t/chainlint/chain-break-false.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-false' '\n # LINT: broken &&-chain okay if explicit \"false\" signals failure\n if condition not satisified\n then\n@@ -8,3 +9,4 @@ else\n \techo it went okay\n \tcongratulate user\n fi\n+'\ndiff --git a/t/chainlint/chain-break-return-exit.test b/t/chainlint/chain-break-return-exit.test\nindex 46542edf88..b6f519bb4d 100644\n--- a/t/chainlint/chain-break-return-exit.test\n+++ b/t/chainlint/chain-break-return-exit.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-return-exit' '\n case \"$(git ls-files)\" in\n one) echo pass one ;;\n # LINT: broken &&-chain okay if explicit \"return 1\" signals failuire\n@@ -21,3 +22,4 @@ for i in 1 2 3 4 ; do\n \tgit checkout main -b $i || return $?\n \ttest_commit $i $i $i tag$i || return $?\n done\n+'\ndiff --git a/t/chainlint/chain-break-status.test b/t/chainlint/chain-break-status.test\nindex a6602a7b99..d9fee190d9 100644\n--- a/t/chainlint/chain-break-status.test\n+++ b/t/chainlint/chain-break-status.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chain-break-status' '\n # LINT: broken &&-chain okay if next command handles \"$?\" explicitly\n OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 ) &&\n test_match_signal 13 \"$OUT\" &&\n@@ -9,3 +10,4 @@ test_match_signal 13 \"$OUT\" &&\n \ttest \"$ret\" = 3\n } &&\n test_cmp expect actual\n+'\ndiff --git a/t/chainlint/chained-block.test b/t/chainlint/chained-block.test\nindex 86f81ece63..71ef1d0b7f 100644\n--- a/t/chainlint/chained-block.test\n+++ b/t/chainlint/chained-block.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chained-block' '\n # LINT: start of block chained to preceding command\n echo nobody home && {\n \ttest the doohicky\n@@ -9,3 +10,4 @@ GIT_EXTERNAL_DIFF=echo git diff | {\n \tread path oldfile oldhex oldmode newfile newhex newmode &&\n \ttest \"z$oh\" = \"z$oldhex\"\n }\n+'\ndiff --git a/t/chainlint/chained-subshell.test b/t/chainlint/chained-subshell.test\nindex 4ff6ddd8cb..1f11f65398 100644\n--- a/t/chainlint/chained-subshell.test\n+++ b/t/chainlint/chained-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'chained-subshell' '\n # LINT: start of subshell chained to preceding command\n mkdir sub && (\n \tcd sub &&\n@@ -11,3 +12,4 @@ test -f $s1\n test $(cat $s2) = tree2path1 &&\n # LINT: closing subshell \")\" correctly detected on same line as \"$(...)\"\n test $(cat $s3) = tree3path1)\n+'\ndiff --git a/t/chainlint/close-nested-and-parent-together.test b/t/chainlint/close-nested-and-parent-together.test\nindex 72d482f76d..56b28b186b 100644\n--- a/t/chainlint/close-nested-and-parent-together.test\n+++ b/t/chainlint/close-nested-and-parent-together.test\n@@ -1,3 +1,5 @@\n+test_expect_success 'close-nested-and-parent-together' '\n (cd foo &&\n \t(bar &&\n \t\tbaz))\n+'\ndiff --git a/t/chainlint/close-subshell.test b/t/chainlint/close-subshell.test\nindex 508ca447fd..b99f80569d 100644\n--- a/t/chainlint/close-subshell.test\n+++ b/t/chainlint/close-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'close-subshell' '\n # LINT: closing \")\" with various decorations (\"&&\", \">\", \"|\", etc.)\n (\n \tfoo\n@@ -25,3 +26,4 @@ fuzzle &&\n (\n \tyop\n )\n+'\ndiff --git a/t/chainlint/command-substitution-subsubshell.test b/t/chainlint/command-substitution-subsubshell.test\nindex 321de2951c..4ea772d60a 100644\n--- a/t/chainlint/command-substitution-subsubshell.test\n+++ b/t/chainlint/command-substitution-subsubshell.test\n@@ -1,3 +1,5 @@\n+test_expect_success 'command-substitution-subsubshell' '\n # LINT: subshell nested in subshell nested in command substitution\n OUT=$( ((large_git 1>&3) | :) 3>&1 ) &&\n test_match_signal 13 \"$OUT\"\n+'\ndiff --git a/t/chainlint/command-substitution.test b/t/chainlint/command-substitution.test\nindex 3bbb002a4c..494d671e80 100644\n--- a/t/chainlint/command-substitution.test\n+++ b/t/chainlint/command-substitution.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'command-substitution' '\n (\n \tfoo &&\n # LINT: closing \")\" of $(...) not misinterpreted as subshell-closing \")\"\n@@ -9,3 +10,4 @@\n \tbar=$(gobble blocks)\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/comment.test b/t/chainlint/comment.test\nindex 113c0c466f..c488beac0d 100644\n--- a/t/chainlint/comment.test\n+++ b/t/chainlint/comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'comment' '\n (\n # LINT: swallow comment lines\n \t# comment 1\n@@ -9,3 +10,4 @@\n \t# comment 3\n \t# comment 4\n )\n+'\ndiff --git a/t/chainlint/complex-if-in-cuddled-loop.test b/t/chainlint/complex-if-in-cuddled-loop.test\nindex 5efeda58b2..f98ae4c42d 100644\n--- a/t/chainlint/complex-if-in-cuddled-loop.test\n+++ b/t/chainlint/complex-if-in-cuddled-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'complex-if-in-cuddled-loop' '\n # LINT: \"for\" loop cuddled with \"(\" and \")\" and nested \"if\" with complex\n # LINT: multi-line condition; indented with spaces, not tabs\n (for i in a b c; do\n@@ -9,3 +10,4 @@\n    fi\n  done) &&\n test ! -f file\n+'\ndiff --git a/t/chainlint/cuddled-if-then-else.test b/t/chainlint/cuddled-if-then-else.test\nindex 7c53f4efe3..b1b42e1aac 100644\n--- a/t/chainlint/cuddled-if-then-else.test\n+++ b/t/chainlint/cuddled-if-then-else.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'cuddled-if-then-else' '\n # LINT: \"if\" cuddled with \"(\" and \")\"; indented with spaces, not tabs\n (if test -z \"\"; then\n     echo empty\n  else\n     echo bizzy\n  fi) &&\n echo foobar\n+'\ndiff --git a/t/chainlint/cuddled-loop.test b/t/chainlint/cuddled-loop.test\nindex 3c2a62f751..6fccb6ac22 100644\n--- a/t/chainlint/cuddled-loop.test\n+++ b/t/chainlint/cuddled-loop.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'cuddled-loop' '\n # LINT: \"while\" loop cuddled with \"(\" and \")\", with embedded (allowed)\n # LINT: \"|| exit {n}\" to exit loop early, and using redirection \"<\" to feed\n # LINT: loop; indented with spaces, not tabs\n ( while read x\n   do foobar bop || exit 1\n   done <file ) &&\n outside subshell\n+'\ndiff --git a/t/chainlint/cuddled.test b/t/chainlint/cuddled.test\nindex 257b5b5eed..5a6ef7a4a6 100644\n--- a/t/chainlint/cuddled.test\n+++ b/t/chainlint/cuddled.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'cuddled' '\n # LINT: first subshell statement cuddled with opening \"(\"\n (cd foo &&\n \tbar\n@@ -20,3 +21,4 @@\n # LINT: same with missing \"&&\"\n (cd foo\n \tbar)\n+'\ndiff --git a/t/chainlint/double-here-doc.test b/t/chainlint/double-here-doc.test\nindex cd584a4357..1b69b7a651 100644\n--- a/t/chainlint/double-here-doc.test\n+++ b/t/chainlint/double-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'double-here-doc' '\n run_sub_test_lib_test_err run-inv-range-start \\\n \t\"--run invalid range start\" \\\n \t--run=\"a-5\" <<-\\EOF &&\n@@ -10,3 +11,4 @@ check_sub_test_lib_test_err run-inv-range-start \\\n EOF_OUT\n > error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}\n EOF_ERR\n+'\ndiff --git a/t/chainlint/dqstring-line-splice.test b/t/chainlint/dqstring-line-splice.test\nindex b40714439f..f6aa637be8 100644\n--- a/t/chainlint/dqstring-line-splice.test\n+++ b/t/chainlint/dqstring-line-splice.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'dqstring-line-splice' '\n # LINT: line-splice within DQ-string\n '\"\n echo 'fatal: reword option of --fixup is mutually exclusive with'\\\n \t'--patch/--interactive/--all/--include/--only' >expect &&\n test_must_fail git commit --fixup=reword:HEAD~ $1 2>actual &&\n test_cmp expect actual\n \"'\n+'\ndiff --git a/t/chainlint/dqstring-no-interpolate.test b/t/chainlint/dqstring-no-interpolate.test\nindex d2f4219cbb..7ae079b558 100644\n--- a/t/chainlint/dqstring-no-interpolate.test\n+++ b/t/chainlint/dqstring-no-interpolate.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'dqstring-no-interpolate' '\n # LINT: regex dollar-sign eol anchor in double-quoted string not special\n grep \"^ ! \\[rejected\\][ ]*$BRANCH -> $BRANCH (non-fast-forward)$\" out &&\n \n@@ -13,3 +14,4 @@ grep \"^\\\\.git\\$\" output.txt &&\n \tcut -d ' ' -f 2 <output | sort >actual &&\n \ttest_cmp expect actual\n \"'\n+'\ndiff --git a/t/chainlint/empty-here-doc.test b/t/chainlint/empty-here-doc.test\nindex 24fc165de3..8b7ab6eb5f 100644\n--- a/t/chainlint/empty-here-doc.test\n+++ b/t/chainlint/empty-here-doc.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'empty-here-doc' '\n git ls-tree $tree path >current &&\n # LINT: empty here-doc\n cat >expected <<\\EOF &&\n EOF\n test_output\n+'\ndiff --git a/t/chainlint/exclamation.test b/t/chainlint/exclamation.test\nindex 323595b5bd..796de21b7c 100644\n--- a/t/chainlint/exclamation.test\n+++ b/t/chainlint/exclamation.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'exclamation' '\n # LINT: \"! word\" is two tokens\n if ! condition; then echo nope; else yep; fi &&\n # LINT: \"!word\" is single token, not two tokens \"!\" and \"word\"\n@@ -6,3 +7,4 @@ test_prerequisite !MINGW &&\n mail uucp!address &&\n # LINT: \"!word!\" is single token, not three tokens \"!\", \"word\", and \"!\"\n echo !whatever!\n+'\ndiff --git a/t/chainlint/exit-loop.test b/t/chainlint/exit-loop.test\nindex 2f038207e1..7e8b68b465 100644\n--- a/t/chainlint/exit-loop.test\n+++ b/t/chainlint/exit-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'exit-loop' '\n (\n \tfor i in a b c\n \tdo\n@@ -25,3 +26,4 @@\n \t\ti=$(($i + 1))\n \tdone\n )\n+'\ndiff --git a/t/chainlint/exit-subshell.test b/t/chainlint/exit-subshell.test\nindex 4e6ab69b88..05dff55cd7 100644\n--- a/t/chainlint/exit-subshell.test\n+++ b/t/chainlint/exit-subshell.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'exit-subshell' '\n (\n # LINT: \"|| exit {n}\" valid subshell escape without hurting &&-chain\n \tfoo || exit 1\n \tbar &&\n \tbaz\n )\n+'\ndiff --git a/t/chainlint/for-loop-abbreviated.test b/t/chainlint/for-loop-abbreviated.test\nindex 1084eccb89..1dd14f2a44 100644\n--- a/t/chainlint/for-loop-abbreviated.test\n+++ b/t/chainlint/for-loop-abbreviated.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'for-loop-abbreviated' '\n # LINT: for-loop lacking optional \"in [word...]\" before \"do\"\n for it\n do\n \tpath=$(expr \"$it\" : '\\([^:]*\\)') &&\n \tgit update-index --add \"$path\" || exit\n done\n+'\ndiff --git a/t/chainlint/for-loop.test b/t/chainlint/for-loop.test\nindex 6cb3428158..6f2489eb19 100644\n--- a/t/chainlint/for-loop.test\n+++ b/t/chainlint/for-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'for-loop' '\n (\n # LINT: \"for\", \"do\", \"done\" do not need \"&&\"\n \tfor i in a b c\n@@ -17,3 +18,4 @@\n \t\tcat $i\n \tdone\n )\n+'\ndiff --git a/t/chainlint/function.test b/t/chainlint/function.test\nindex 5ee59562c9..763fcf3f87 100644\n--- a/t/chainlint/function.test\n+++ b/t/chainlint/function.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'function' '\n # LINT: \"()\" in function definition not mistaken for subshell\n sha1_file() {\n \techo \"$*\" | sed \"s#..#.git/objects/&/#\"\n@@ -11,3 +12,4 @@ remove_object() {\n }\n \n sha1_file arg && remove_object arg\n+'\ndiff --git a/t/chainlint/here-doc-close-subshell.test b/t/chainlint/here-doc-close-subshell.test\nindex b857ff5467..2458f3323b 100644\n--- a/t/chainlint/here-doc-close-subshell.test\n+++ b/t/chainlint/here-doc-close-subshell.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'here-doc-close-subshell' '\n (\n # LINT: line contains here-doc and closes nested subshell\n \tcat <<-\\INPUT)\n \tfizz\n \tINPUT\n+'\ndiff --git a/t/chainlint/here-doc-indent-operator.test b/t/chainlint/here-doc-indent-operator.test\nindex c8a6f18eb4..a2656f47c1 100644\n--- a/t/chainlint/here-doc-indent-operator.test\n+++ b/t/chainlint/here-doc-indent-operator.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-indent-operator' '\n # LINT: whitespace between operator \"<<-\" and tag legal\n cat >expect <<- EOF &&\n header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0\n@@ -11,3 +12,4 @@ this is not indented\n -EOF\n \n cleanup\n+'\ndiff --git a/t/chainlint/here-doc-multi-line-command-subst.test b/t/chainlint/here-doc-multi-line-command-subst.test\nindex 899bc5de8b..8710a8c483 100644\n--- a/t/chainlint/here-doc-multi-line-command-subst.test\n+++ b/t/chainlint/here-doc-multi-line-command-subst.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-multi-line-command-subst' '\n (\n # LINT: line contains here-doc and opens multi-line $(...)\n \tx=$(bobble <<-\\END &&\n@@ -7,3 +8,4 @@\n \t\twiffle)\n \techo $x\n )\n+'\ndiff --git a/t/chainlint/here-doc-multi-line-string.test b/t/chainlint/here-doc-multi-line-string.test\nindex a53edbcc8d..2f496002fd 100644\n--- a/t/chainlint/here-doc-multi-line-string.test\n+++ b/t/chainlint/here-doc-multi-line-string.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc-multi-line-string' '\n (\n # LINT: line contains here-doc and opens multi-line string\n \tcat <<-\\TXT && echo \"multi-line\n@@ -6,3 +7,4 @@\n \tTXT\n \tbap\n )\n+'\ndiff --git a/t/chainlint/here-doc.test b/t/chainlint/here-doc.test\nindex 3f5f92cad3..c91b695319 100644\n--- a/t/chainlint/here-doc.test\n+++ b/t/chainlint/here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'here-doc' '\n # LINT: stitch together incomplete \\-ending lines\n # LINT: swallow here-doc to avoid false positives in content\n boodle wobba \\\n@@ -28,3 +29,4 @@ morticia\n wednesday\n pugsly\n EOF\n+'\ndiff --git a/t/chainlint/if-condition-split.test b/t/chainlint/if-condition-split.test\nindex 240daa9fd5..9a3b3ed04a 100644\n--- a/t/chainlint/if-condition-split.test\n+++ b/t/chainlint/if-condition-split.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-condition-split' '\n # LINT: \"if\" condition split across multiple lines at \"&&\" or \"||\"\n if bob &&\n    marcia ||\n@@ -6,3 +7,4 @@ then\n \techo \"nomads\"\n \techo \"for sure\"\n fi\n+'\ndiff --git a/t/chainlint/if-in-loop.test b/t/chainlint/if-in-loop.test\nindex 90c23976fe..5be9d1cfa5 100644\n--- a/t/chainlint/if-in-loop.test\n+++ b/t/chainlint/if-in-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-in-loop' '\n (\n \tfor i in a b c\n \tdo\n@@ -13,3 +14,4 @@\n \tdone\n \tbar\n )\n+'\ndiff --git a/t/chainlint/if-then-else.test b/t/chainlint/if-then-else.test\nindex 2055336c2b..6582a7f440 100644\n--- a/t/chainlint/if-then-else.test\n+++ b/t/chainlint/if-then-else.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'if-then-else' '\n (\n # LINT: \"if\", \"then\", \"elif\", \"else\", \"fi\" do not need \"&&\"\n \tif test -n \"\"\n@@ -27,3 +28,4 @@\n \t\techo empty\n \tfi\n )\n+'\ndiff --git a/t/chainlint/incomplete-line.test b/t/chainlint/incomplete-line.test\nindex d856658083..74a93021eb 100644\n--- a/t/chainlint/incomplete-line.test\n+++ b/t/chainlint/incomplete-line.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'incomplete-line' '\n # LINT: stitch together all incomplete \\-ending lines\n line 1 \\\n line 2 \\\n@@ -10,3 +11,4 @@ line 4 &&\n \tline 7 \\\n \tline 8\n )\n+'\ndiff --git a/t/chainlint/inline-comment.test b/t/chainlint/inline-comment.test\nindex 8f26856e77..4fbbf1058a 100644\n--- a/t/chainlint/inline-comment.test\n+++ b/t/chainlint/inline-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'inline-comment' '\n (\n # LINT: swallow inline comment (leaving command intact)\n \tfoobar && # comment 1\n@@ -10,3 +11,4 @@\n # LINT: \"#\" in string in cuddled subshell not misinterpreted as comment\n (cd foo &&\n \tflibble \"not a # comment\")\n+'\ndiff --git a/t/chainlint/loop-detect-failure.test b/t/chainlint/loop-detect-failure.test\nindex b9791cc802..44673aa394 100644\n--- a/t/chainlint/loop-detect-failure.test\n+++ b/t/chainlint/loop-detect-failure.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-detect-failure' '\n git init r1 &&\n # LINT: loop handles failure explicitly with \"|| return 1\"\n for n in 1 2 3 4 5\n@@ -15,3 +16,4 @@ do\n \tgit -C r2 add large.$n &&\n \tgit -C r2 commit -m \"$n\"\n done\n+'\ndiff --git a/t/chainlint/loop-detect-status.test b/t/chainlint/loop-detect-status.test\nindex 1c6c23cfc9..8b639be073 100644\n--- a/t/chainlint/loop-detect-status.test\n+++ b/t/chainlint/loop-detect-status.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-detect-status' '\n # LINT: \"$?\" handled explicitly within loop body\n (while test $i -le $blobcount\n  do\n@@ -17,3 +18,4 @@\n  cat commit) |\n git fast-import --big-file-threshold=2 &&\n test ! -f exit-status\n+'\ndiff --git a/t/chainlint/loop-in-if.test b/t/chainlint/loop-in-if.test\nindex dfcc3f98fb..b0d0d393cf 100644\n--- a/t/chainlint/loop-in-if.test\n+++ b/t/chainlint/loop-in-if.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-in-if' '\n (\n \tif true\n \tthen\n@@ -13,3 +14,4 @@\n \tfi\n \tbar\n )\n+'\ndiff --git a/t/chainlint/loop-upstream-pipe.test b/t/chainlint/loop-upstream-pipe.test\nindex efb77da897..8415a4db27 100644\n--- a/t/chainlint/loop-upstream-pipe.test\n+++ b/t/chainlint/loop-upstream-pipe.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'loop-upstream-pipe' '\n (\n \tgit rev-list --objects --no-object-names base..loose |\n \twhile read oid\n@@ -9,3 +10,4 @@\n \tdone |\n \tsort -k1\n ) >expect &&\n+'\ndiff --git a/t/chainlint/multi-line-nested-command-substitution.test b/t/chainlint/multi-line-nested-command-substitution.test\nindex 300058341b..e811c63f2b 100644\n--- a/t/chainlint/multi-line-nested-command-substitution.test\n+++ b/t/chainlint/multi-line-nested-command-substitution.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'multi-line-nested-command-substitution' '\n (\n \tfoo &&\n \tx=$(\n@@ -16,3 +17,4 @@ sort &&\n \t\tfip) &&\n \techo fail\n )\n+'\ndiff --git a/t/chainlint/multi-line-string.test b/t/chainlint/multi-line-string.test\nindex 4a0af2107d..7b5048d2ea 100644\n--- a/t/chainlint/multi-line-string.test\n+++ b/t/chainlint/multi-line-string.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'multi-line-string' '\n (\n \tx=\"line 1\n \t\tline 2\n@@ -13,3 +14,4 @@\n \t\tghi\" &&\n \tbarfoo\n )\n+'\ndiff --git a/t/chainlint/negated-one-liner.test b/t/chainlint/negated-one-liner.test\nindex c9598e9153..30f4cc5a9b 100644\n--- a/t/chainlint/negated-one-liner.test\n+++ b/t/chainlint/negated-one-liner.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'negated-one-liner' '\n # LINT: top-level one-liner subshell\n ! (foo && bar) &&\n ! (foo && bar) >baz &&\n \n # LINT: top-level one-liner subshell missing internal \"&&\"\n ! (foo; bar) &&\n ! (foo; bar) >baz\n+'\ndiff --git a/t/chainlint/nested-cuddled-subshell.test b/t/chainlint/nested-cuddled-subshell.test\nindex 8fd656c7b5..31e92d3be4 100644\n--- a/t/chainlint/nested-cuddled-subshell.test\n+++ b/t/chainlint/nested-cuddled-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-cuddled-subshell' '\n (\n # LINT: opening \"(\" cuddled with first nested subshell statement\n \t(cd foo &&\n@@ -29,3 +30,4 @@\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/nested-here-doc.test b/t/chainlint/nested-here-doc.test\nindex f35404bf0f..9505c47a34 100644\n--- a/t/chainlint/nested-here-doc.test\n+++ b/t/chainlint/nested-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-here-doc' '\n # LINT: inner \"EOF\" not misintrepreted as closing ARBITRARY here-doc\n cat <<ARBITRARY >foop &&\n naddle\n@@ -31,3 +32,4 @@ ARBITRARY\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/nested-loop-detect-failure.test b/t/chainlint/nested-loop-detect-failure.test\nindex e6f0c1acfb..3d4b657412 100644\n--- a/t/chainlint/nested-loop-detect-failure.test\n+++ b/t/chainlint/nested-loop-detect-failure.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-loop-detect-failure' '\n # LINT: neither loop handles failure explicitly with \"|| return 1\"\n for i in 0 1 2 3 4 5 6 7 8 9;\n do\n@@ -33,3 +34,4 @@ do\n \t\techo \"$i$j\" >\"path$i$j\" || return 1\n \tdone || return 1\n done\n+'\ndiff --git a/t/chainlint/nested-subshell-comment.test b/t/chainlint/nested-subshell-comment.test\nindex 0215cdb192..b430580ce0 100644\n--- a/t/chainlint/nested-subshell-comment.test\n+++ b/t/chainlint/nested-subshell-comment.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-subshell-comment' '\n (\n \tfoo &&\n \t(\n@@ -11,3 +12,4 @@\n \t)\n \tfuzzy\n )\n+'\ndiff --git a/t/chainlint/nested-subshell.test b/t/chainlint/nested-subshell.test\nindex 440ee9992d..c31da34b73 100644\n--- a/t/chainlint/nested-subshell.test\n+++ b/t/chainlint/nested-subshell.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'nested-subshell' '\n (\n \tcd foo &&\n \t(\n@@ -11,3 +12,4 @@\n \t\techo b\n \t) >file\n )\n+'\ndiff --git a/t/chainlint/not-heredoc.test b/t/chainlint/not-heredoc.test\nindex 9aa57346cd..09711e45e0 100644\n--- a/t/chainlint/not-heredoc.test\n+++ b/t/chainlint/not-heredoc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'not-heredoc' '\n # LINT: \"<< ours\" inside string is not here-doc\n echo \"<<<<<<< ours\" &&\n echo ourside &&\n@@ -14,3 +15,4 @@ echo \">>>>>>> theirs\" &&\n \techo \">>>>>>> theirs\"\n \tpoodle\n ) >merged\n+'\ndiff --git a/t/chainlint/one-liner-for-loop.test b/t/chainlint/one-liner-for-loop.test\nindex 4bd8c066c7..00afd7ef76 100644\n--- a/t/chainlint/one-liner-for-loop.test\n+++ b/t/chainlint/one-liner-for-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'one-liner-for-loop' '\n git init dir-rename-and-content &&\n (\n \tcd dir-rename-and-content &&\n@@ -8,3 +9,4 @@ git init dir-rename-and-content &&\n \tgit add foo olddir &&\n \tgit commit -m \"original\" &&\n )\n+'\ndiff --git a/t/chainlint/one-liner.test b/t/chainlint/one-liner.test\nindex be9858fa29..6e42ee1b5e 100644\n--- a/t/chainlint/one-liner.test\n+++ b/t/chainlint/one-liner.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'one-liner' '\n # LINT: top-level one-liner subshell\n (foo && bar) &&\n (foo && bar) |\n@@ -10,3 +11,4 @@\n \n # LINT: \";\" in string not misinterpreted as broken &&-chain\n (foo \"bar; baz\")\n+'\ndiff --git a/t/chainlint/p4-filespec.test b/t/chainlint/p4-filespec.test\nindex 4fd2d6e2b8..8ba6b911dc 100644\n--- a/t/chainlint/p4-filespec.test\n+++ b/t/chainlint/p4-filespec.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'p4-filespec' '\n (\n # LINT: Perforce revspec in filespec not misinterpreted as in-line comment\n \tp4 print -1 //depot/fiddle#42 >file &&\n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/pipe.test b/t/chainlint/pipe.test\nindex dd82534c66..1af81c243b 100644\n--- a/t/chainlint/pipe.test\n+++ b/t/chainlint/pipe.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'pipe' '\n (\n # LINT: no \"&&\" needed on line ending with \"|\"\n \tfoo |\n@@ -10,3 +11,4 @@\n \n \tsunder\n )\n+'\ndiff --git a/t/chainlint/return-loop.test b/t/chainlint/return-loop.test\nindex f90b171300..ea76c3593a 100644\n--- a/t/chainlint/return-loop.test\n+++ b/t/chainlint/return-loop.test\n@@ -1,6 +1,8 @@\n+test_expect_success 'return-loop' '\n while test $i -lt $((num - 5))\n do\n # LINT: \"|| return {n}\" valid loop escape outside subshell; no \"&&\" needed\n \tgit notes add -m \"notes for commit$i\" HEAD~$i || return 1\n \ti=$((i + 1))\n done\n+'\ndiff --git a/t/chainlint/semicolon.test b/t/chainlint/semicolon.test\nindex 67e1192c50..fc0ba1b539 100644\n--- a/t/chainlint/semicolon.test\n+++ b/t/chainlint/semicolon.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'semicolon' '\n (\n # LINT: missing internal \"&&\" and ending \"&&\"\n \tcat foo ; echo bar\n@@ -23,3 +24,4 @@\n # LINT: semicolon unnecessary but legitimate\n \t\techo;\n \tdone)\n+'\ndiff --git a/t/chainlint/sqstring-in-sqstring.test b/t/chainlint/sqstring-in-sqstring.test\nindex 77a425e0c7..24169724a5 100644\n--- a/t/chainlint/sqstring-in-sqstring.test\n+++ b/t/chainlint/sqstring-in-sqstring.test\n@@ -1,5 +1,7 @@\n+test_expect_success 'sqstring-in-sqstring' '\n # LINT: SQ-string Perl code fragment within SQ-string\n perl -e '\\''\n \tdefined($_ = -s $_) or die for @ARGV;\n \texit 1 if $ARGV[0] <= $ARGV[1];\n '\\'' test-2-$packname_2.pack test-3-$packname_3.pack\n+'\ndiff --git a/t/chainlint/subshell-here-doc.test b/t/chainlint/subshell-here-doc.test\nindex d40eb65583..4a38f47f01 100644\n--- a/t/chainlint/subshell-here-doc.test\n+++ b/t/chainlint/subshell-here-doc.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'subshell-here-doc' '\n (\n # LINT: stitch together incomplete \\-ending lines\n # LINT: swallow here-doc to avoid false positives in content\n@@ -33,3 +34,4 @@ EOF\n \tARBITRARY3\n \tmeep\n )\n+'\ndiff --git a/t/chainlint/subshell-one-liner.test b/t/chainlint/subshell-one-liner.test\nindex 37fa643c20..dac536afcc 100644\n--- a/t/chainlint/subshell-one-liner.test\n+++ b/t/chainlint/subshell-one-liner.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'subshell-one-liner' '\n (\n # LINT: nested one-liner subshell\n \t(foo && bar) &&\n@@ -22,3 +23,4 @@\n \n \tfoobar\n )\n+'\ndiff --git a/t/chainlint/t7900-subtree.test b/t/chainlint/t7900-subtree.test\nindex 02f3129232..1f4f03300f 100644\n--- a/t/chainlint/t7900-subtree.test\n+++ b/t/chainlint/t7900-subtree.test\n@@ -1,3 +1,4 @@\n+test_expect_success 't7900-subtree' '\n (\n \tchks=\"sub1\n sub2\n@@ -20,3 +21,4 @@ TXT\n \tcheck_equal \"$subfiles\" \"$chkms\n $chks\"\n )\n+'\ndiff --git a/t/chainlint/token-pasting.test b/t/chainlint/token-pasting.test\nindex b4610ce815..590914b733 100644\n--- a/t/chainlint/token-pasting.test\n+++ b/t/chainlint/token-pasting.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'token-pasting' '\n # LINT: single token; composite of multiple strings\n git config filter.rot13.smudge ./rot13.sh &&\n git config filter.rot13.clean ./rot13.sh &&\n@@ -30,3 +31,4 @@ downstream_url_for_sed=$(\n # LINT: exit/enter string context; \"&\" inside string not command terminator\n \tsed -e '\\''s/\\\\/\\\\\\\\/g'\\'' -e '\\''s/[[/.*^$]/\\\\&/g'\\''\n )\n+'\ndiff --git a/t/chainlint/unclosed-here-doc-indent.test b/t/chainlint/unclosed-here-doc-indent.test\nindex 5c841a9dfd..7ac9d0f7d7 100644\n--- a/t/chainlint/unclosed-here-doc-indent.test\n+++ b/t/chainlint/unclosed-here-doc-indent.test\n@@ -1,4 +1,6 @@\n+test_expect_success 'unclosed-here-doc-indent' '\n command_which_is_run &&\n cat >expect <<-\\EOF &&\n we forget to end the here-doc\n command_which_is_gobbled\n+'\ndiff --git a/t/chainlint/unclosed-here-doc.test b/t/chainlint/unclosed-here-doc.test\nindex 69d3786c34..68e78f06f3 100644\n--- a/t/chainlint/unclosed-here-doc.test\n+++ b/t/chainlint/unclosed-here-doc.test\n@@ -1,7 +1,9 @@\n+test_expect_success 'unclosed-here-doc' '\n command_which_is_run &&\n cat >expect <<\\EOF &&\n \twe try to end the here-doc below,\n \tbut the indentation throws us off\n \tsince the operator is not \"<<-\".\n \tEOF\n command_which_is_gobbled\n+'\ndiff --git a/t/chainlint/while-loop.test b/t/chainlint/while-loop.test\nindex d09fb016e4..33a201906a 100644\n--- a/t/chainlint/while-loop.test\n+++ b/t/chainlint/while-loop.test\n@@ -1,3 +1,4 @@\n+test_expect_success 'while-loop' '\n (\n # LINT: \"while\", \"do\", \"done\" do not need \"&&\"\n \twhile true\n@@ -17,3 +18,4 @@\n \t\tcat bar\n \tdone\n )\n+'\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498420","messageId":"20240710083513.GB2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 2/9] chainlint.pl: only start threads if jobs > 1","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:35:13Z","receivedAt":"2024-07-10T08:35:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"If the system supports threads, chainlint.pl will always spawn worker\nthreads to do the real work. But when --jobs=1, this is pointless, since\nwe could just do the work in the main thread. And spawning even a single\nthread has a high overhead. For example, on my Linux system, running:\n\n  for i in chainlint/*.test; do\n\tperl chainlint.pl --jobs=1 $i\n  done >/dev/null\n\ntakes ~1.7s without this patch, and ~1.1s after. We don't usually spawn\na bunch of individual chainlint.pl processes (instead we feed several\nscripts at once, and the parallelism outweighs the setup cost). But it's\nsomething we've considered doing, and since we already have fallback\ncode for systems without thread support, it's pretty easy to make this\nwork.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/chainlint.pl | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex 1bbd985b78..1864d048ae 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -807,7 +807,8 @@ sub exit_code {\n \texit;\n }\n \n-unless ($Config{useithreads} && eval {\n+unless ($jobs > 1 &&\n+\t$Config{useithreads} && eval {\n \trequire threads; threads->import();\n \trequire Thread::Queue; Thread::Queue->import();\n \t1;\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498421","messageId":"20240710083557.GC2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 3/9] chainlint.pl: do not spawn more threads than we have scripts","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:35:57Z","receivedAt":"2024-07-10T08:35:58Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The chainlint.pl script spawns worker threads to check many scripts in\nparallel. This is good if you feed it a lot of scripts. But if you give\nit few (or one), then the overhead of spawning the threads dominates. We\ncan easily notice that we have fewer scripts than threads and scale back\nas appropriate.\n\nThis patch reduces the time to run:\n\n  time for i in chainlint/*.test; do\n\tperl chainlint.pl $i\n  done >/dev/null\n\non my system from ~4.1s to ~1.1s, where I have 8+8 cores.\n\nAs with the previous patch, this isn't the usual way we run chainlint\n(we feed many scripts at once, which is why it supports threading in the\nfirst place). So this won't make a big difference in the real world, but\nit may help us out in the future, and it makes experimenting with and\ndebugging the chainlint tests a bit more pleasant.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/chainlint.pl | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex 1864d048ae..118a229a96 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -806,6 +806,7 @@ sub exit_code {\n \tshow_stats($start_time, \\@stats) if $show_stats;\n \texit;\n }\n+$jobs = @scripts if @scripts < $jobs;\n \n unless ($jobs > 1 &&\n \t$Config{useithreads} && eval {\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498422","messageId":"20240710083730.GD2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 4/9] chainlint.pl: force CRLF conversion when opening input files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:37:30Z","receivedAt":"2024-07-10T08:37:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The lexer in chainlint.pl can't handle CRLF line endings; it complains\nabout an internal error in scan_token() if we see one. For example, in\nour Windows CI environment:\n\n  $ perl chainlint.pl chainlint/for-loop.test | cat -v\n  Thread 2 terminated abnormally: internal error scanning character '^M'\n\nThis doesn't break \"make check-chainlint\" (yet), because we assemble a\nconcatenated input by passing the contents of each file through \"sed\".\nAnd the \"sed\" we use will strip out the CRLFs. But the next patch is\ngoing to rework this a bit, which does break check-chainlint on Windows.\nPlus it's probably nicer to folks on Windows who might work on chainlint\nitself and write new tests.\n\nIn theory we could fix the parser to handle this, but it's not really\nworth the trouble. We should be able to ask the input layer to translate\nthe line endings for us. In fact, I'd expect this to happen by default,\nas perl's documentation claims Win32 uses the \":unix:crlf\" PERLIO layer\nby default (\"unix\" here just refers to using read/write syscalls, and\nthen \"crlf\" layers the translation on top). However, this doesn't seem\nto be the case in our Windows CI environment. I didn't dig into the\nexact reason, but it is perhaps because we are using an msys build of\nperl rather than a \"true\" Win32 build.\n\nAt any rate, it is easy-ish to just ask explicitly for the conversion.\nIn the above example, setting PERLIO=crlf in the environment is enough\nto make it work. Curiously, though, this doesn't work when invoking\nchainlint via \"make\". Again, I didn't dig into it, but it may have to do\nwith msys programs calling Windows programs or vice versa.\n\nWe can make it work consistently by just explicitly asking for CRLF\ntranslation when we open the files. This will even work on non-Windows\nplatforms, though we wouldn't really expect to find CRLF files there.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/chainlint.pl | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex 118a229a96..fb749d3d5c 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -762,7 +762,7 @@ sub check_script {\n \twhile (my $path = $next_script->()) {\n \t\t$nscripts++;\n \t\tmy $fh;\n-\t\tunless (open($fh, \"<\", $path)) {\n+\t\tunless (open($fh, \"<:unix:crlf\", $path)) {\n \t\t\t$emit->(\"?!ERR?! $path: $!\\n\");\n \t\t\tnext;\n \t\t}\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498423","messageId":"20240710083755.GE2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 5/9] chainlint.pl: check line numbers in expected output","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:37:55Z","receivedAt":"2024-07-10T08:37:57Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"While working on chainlint.pl recently, we introduced some bugs that\nshowed incorrect line numbers in the output. But it was hard to notice,\nsince we sanitize the output by removing all of the line numbers! It\nwould be nice to retain these so we can catch any regressions.\n\nThe main reason we sanitize is for maintainability: we concatenate all\nof the test snippets into a single file, so it's hard for each \".expect\"\nfile to know at which offset its test input will be found. We can handle\nthat by storing the per-test line numbers in the \".expect\" files, and\nthen dynamically offsetting them as we build the concatenated test and\nexpect files together.\n\nThe changes to the \".expect\" files look like tedious boilerplate, but it\nactually makes adding new tests easier. You can now just run:\n\n  perl chainlint.pl chainlint/foo.test |\n  tail -n +2 >chainlint/foo.expect\n\nto save the output of the script minus the comment headers (after\nchecking that it is correct, of course). Whereas before you had to strip\nthe line numbers. The conversions here were done mechanically using\nsomething like the script above, and then spot-checked manually.\n\nIt would be possible to do all of this in shell via the Makefile, but it\ngets a bit complicated (and requires a lot of extra processes). Instead,\nI've written a short perl script that generates the concatenated files\n(we already depend on perl, since chainlint.pl uses it). Incidentally,\nthis improves a few other things:\n\n  - we incorrectly used $(CHAINLINTTMP_SQ) inside a double-quoted\n    string. So if your test directory required quoting, like:\n\n       make \"TEST_OUTPUT_DIRECTORY=/tmp/h'orrible\"\n\n    we'd fail the chainlint tests.\n\n  - the shell in the Makefile didn't handle &&-chaining correctly in its\n    loops (though in practice the \"sed\" and \"cat\" invocations are not\n    likely to fail).\n\n  - likewise, the sed invocation to strip numbers was hiding the exit\n    code of chainlint.pl itself. In practice this isn't a big deal;\n    since there are linter violations in the test files, we expect it to\n    exit non-zero. But we could later use exit codes to distinguish\n    serious errors from expected ones.\n\n  - we now use a constant number of processes, instead of scaling with\n    the number of test scripts. So it should be a little faster (on my\n    machine, \"make check-chainlint\" goes from 133ms to 73ms).\n\nThere are some alternatives to this approach, but I think this is still\na good intermediate step:\n\n  1. We could invoke chainlint.pl individually on each test file, and\n     compare it to the expected output (and possibly using \"make\" to\n     avoid repeating already-done checks). This is a much bigger change\n     (and we'd have to figure out what to do with the \"# LINT\" lines in\n     the inputs). But in this case we'd still want the \"expect\" files to\n     be annotated with line numbers. So most of what's in this patch\n     would be needed anyway.\n\n  2. Likewise, we could run a single chainlint.pl and feed it all of the\n     scripts (with \"--jobs=1\" to get deterministic output). But we'd\n     still need to annotate the scripts as we did here, and we'd still\n     need to either assemble the \"expect\" file, or break apart the\n     script output to compare to each individual \".expect\" file.\n\nSo we may pursue those in the long run, but this patch gives us more\nrobust tests without too much extra work or moving in a useless\ndirection.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/Makefile                                    | 14 +----\n t/chainlint-cat.pl                            | 29 +++++++++\n t/chainlint/arithmetic-expansion.expect       | 18 +++---\n t/chainlint/bash-array.expect                 | 20 +++---\n t/chainlint/blank-line-before-esac.expect     | 36 +++++------\n t/chainlint/blank-line.expect                 | 16 ++---\n t/chainlint/block-comment.expect              | 16 ++---\n t/chainlint/block.expect                      | 46 +++++++-------\n t/chainlint/broken-chain.expect               | 12 ++--\n t/chainlint/case-comment.expect               | 22 +++----\n t/chainlint/case.expect                       | 38 ++++++------\n t/chainlint/chain-break-background.expect     | 18 +++---\n t/chainlint/chain-break-continue.expect       | 24 +++----\n t/chainlint/chain-break-false.expect          | 18 +++---\n t/chainlint/chain-break-return-exit.expect    | 38 ++++++------\n t/chainlint/chain-break-status.expect         | 18 +++---\n t/chainlint/chained-block.expect              | 18 +++---\n t/chainlint/chained-subshell.expect           | 20 +++---\n .../close-nested-and-parent-together.expect   |  6 +-\n t/chainlint/close-subshell.expect             | 52 ++++++++--------\n .../command-substitution-subsubshell.expect   |  4 +-\n t/chainlint/command-substitution.expect       | 18 +++---\n t/chainlint/comment.expect                    | 16 ++---\n t/chainlint/complex-if-in-cuddled-loop.expect | 18 +++---\n t/chainlint/cuddled-if-then-else.expect       | 12 ++--\n t/chainlint/cuddled-loop.expect               |  8 +--\n t/chainlint/cuddled.expect                    | 34 +++++-----\n t/chainlint/double-here-doc.expect            | 24 +++----\n t/chainlint/dqstring-line-splice.expect       | 10 +--\n t/chainlint/dqstring-no-interpolate.expect    | 24 +++----\n t/chainlint/empty-here-doc.expect             |  8 +--\n t/chainlint/exclamation.expect                |  8 +--\n t/chainlint/exit-loop.expect                  | 48 +++++++-------\n t/chainlint/exit-subshell.expect              | 10 +--\n t/chainlint/for-loop-abbreviated.expect       | 10 +--\n t/chainlint/for-loop.expect                   | 28 ++++-----\n t/chainlint/function.expect                   | 22 +++----\n t/chainlint/here-doc-close-subshell.expect    |  8 +--\n t/chainlint/here-doc-indent-operator.expect   | 22 +++----\n .../here-doc-multi-line-command-subst.expect  | 16 ++---\n t/chainlint/here-doc-multi-line-string.expect | 14 ++---\n t/chainlint/here-doc.expect                   | 50 +++++++--------\n t/chainlint/if-condition-split.expect         | 14 ++---\n t/chainlint/if-in-loop.expect                 | 24 +++----\n t/chainlint/if-then-else.expect               | 44 ++++++-------\n t/chainlint/incomplete-line.expect            | 20 +++---\n t/chainlint/inline-comment.expect             | 16 ++---\n t/chainlint/loop-detect-failure.expect        | 30 ++++-----\n t/chainlint/loop-detect-status.expect         | 36 +++++------\n t/chainlint/loop-in-if.expect                 | 24 +++----\n t/chainlint/loop-upstream-pipe.expect         | 20 +++---\n ...ti-line-nested-command-substitution.expect | 36 +++++------\n t/chainlint/multi-line-string.expect          | 28 ++++-----\n t/chainlint/negated-one-liner.expect          | 10 +--\n t/chainlint/nested-cuddled-subshell.expect    | 50 +++++++--------\n t/chainlint/nested-here-doc.expect            | 60 +++++++++---------\n t/chainlint/nested-loop-detect-failure.expect | 62 +++++++++----------\n t/chainlint/nested-subshell-comment.expect    | 22 +++----\n t/chainlint/nested-subshell.expect            | 26 ++++----\n t/chainlint/not-heredoc.expect                | 28 ++++-----\n t/chainlint/one-liner-for-loop.expect         | 18 +++---\n t/chainlint/one-liner.expect                  | 18 +++---\n t/chainlint/p4-filespec.expect                |  8 +--\n t/chainlint/pipe.expect                       | 20 +++---\n t/chainlint/return-loop.expect                | 10 +--\n t/chainlint/semicolon.expect                  | 38 ++++++------\n t/chainlint/sqstring-in-sqstring.expect       |  8 +--\n t/chainlint/subshell-here-doc.expect          | 60 +++++++++---------\n t/chainlint/subshell-one-liner.expect         | 38 ++++++------\n t/chainlint/t7900-subtree.expect              | 44 ++++++-------\n t/chainlint/token-pasting.expect              | 54 ++++++++--------\n t/chainlint/unclosed-here-doc-indent.expect   |  8 +--\n t/chainlint/unclosed-here-doc.expect          | 14 ++---\n t/chainlint/while-loop.expect                 | 28 ++++-----\n 74 files changed, 913 insertions(+), 894 deletions(-)\n create mode 100644 t/chainlint-cat.pl\n\ndiff --git a/t/Makefile b/t/Makefile\nindex e7a476966e..4c30e7c06f 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -108,18 +108,8 @@ clean-chainlint:\n \n check-chainlint:\n \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n-\tfor i in $(CHAINLINTTESTS); do \\\n-\t\tsed -e '/^# LINT: /d' chainlint/$$i.test; \\\n-\tdone >'$(CHAINLINTTMP_SQ)'/tests && \\\n-\t{ \\\n-\t\techo \"# chainlint: $(CHAINLINTTMP_SQ)/tests\" && \\\n-\t\tfor i in $(CHAINLINTTESTS); do \\\n-\t\t\techo \"# chainlint: $$i\" && \\\n-\t\t\tcat chainlint/$$i.expect; \\\n-\t\tdone \\\n-\t} >'$(CHAINLINTTMP_SQ)'/expect && \\\n-\t$(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \\\n-\t\tsed -e 's/^[1-9][0-9]* //' >'$(CHAINLINTTMP_SQ)'/actual && \\\n+\t'$(PERL_PATH_SQ)' chainlint-cat.pl '$(CHAINLINTTMP_SQ)' $(CHAINLINTTESTS) && \\\n+\t{ $(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests >'$(CHAINLINTTMP_SQ)'/actual || true; } && \\\n \tdiff -u '$(CHAINLINTTMP_SQ)'/expect '$(CHAINLINTTMP_SQ)'/actual\n \n test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax \\\ndiff --git a/t/chainlint-cat.pl b/t/chainlint-cat.pl\nnew file mode 100644\nindex 0000000000..388f6e1e41\n--- /dev/null\n+++ b/t/chainlint-cat.pl\n@@ -0,0 +1,29 @@\n+#!/usr/bin/env perl\n+\n+my $outdir = shift;\n+open(my $tests, '>', \"$outdir/tests\")\n+\tor die \"unable to open $outdir/tests: $!\";\n+open(my $expect, '>', \"$outdir/expect\")\n+\tor die \"unable to open $outdir/expect: $!\";\n+\n+print $expect \"# chainlint: $outdir/tests\\n\";\n+\n+my $offset = 0;\n+for my $script (@ARGV) {\n+\tprint $expect \"# chainlint: $script\\n\";\n+\n+\topen(my $expect_in, '<', \"chainlint/$script.expect\")\n+\t\tor die \"unable to open chainlint/$script.expect: $!\";\n+\twhile (<$expect_in>) {\n+\t\ts/^\\d+/$& + $offset/e;\n+\t\tprint $expect $_;\n+\t}\n+\n+\topen(my $test_in, '<', \"chainlint/$script.test\")\n+\t\tor die \"unable to open chainlint/$script.test: $!\";\n+\twhile (<$test_in>) {\n+\t\t/^# LINT: / and next;\n+\t\tprint $tests $_;\n+\t\t$offset++;\n+\t}\n+}\ndiff --git a/t/chainlint/arithmetic-expansion.expect b/t/chainlint/arithmetic-expansion.expect\nindex 46ee1046af..338ecd5861 100644\n--- a/t/chainlint/arithmetic-expansion.expect\n+++ b/t/chainlint/arithmetic-expansion.expect\n@@ -1,9 +1,9 @@\n-(\n-\tfoo &&\n-\tbar=$((42 + 1)) &&\n-\tbaz\n-) &&\n-(\n-\tbar=$((42 + 1)) ?!AMP?!\n-\tbaz\n-)\n+2 (\n+3 \tfoo &&\n+4 \tbar=$((42 + 1)) &&\n+5 \tbaz\n+6 ) &&\n+7 (\n+8 \tbar=$((42 + 1)) ?!AMP?!\n+9 \tbaz\n+10 )\ndiff --git a/t/chainlint/bash-array.expect b/t/chainlint/bash-array.expect\nindex 4c34eaee45..435dc8bdc8 100644\n--- a/t/chainlint/bash-array.expect\n+++ b/t/chainlint/bash-array.expect\n@@ -1,10 +1,10 @@\n-(\n-\tfoo &&\n-\tbar=(gumbo stumbo wumbo) &&\n-\tbaz\n-) &&\n-(\n-\tfoo &&\n-\tbar=${#bar[@]} &&\n-\tbaz\n-)\n+2 (\n+3 \tfoo &&\n+4 \tbar=(gumbo stumbo wumbo) &&\n+5 \tbaz\n+6 ) &&\n+7 (\n+8 \tfoo &&\n+9 \tbar=${#bar[@]} &&\n+10 \tbaz\n+11 )\ndiff --git a/t/chainlint/blank-line-before-esac.expect b/t/chainlint/blank-line-before-esac.expect\nindex 056e03003d..b88ba919eb 100644\n--- a/t/chainlint/blank-line-before-esac.expect\n+++ b/t/chainlint/blank-line-before-esac.expect\n@@ -1,18 +1,18 @@\n-test_done () {\n-\tcase \"$test_failure\" in\n-\t0)\n-\t\ttest_at_end_hook_\n-\n-\t\texit 0 ;;\n-\n-\t*)\n-\t\tif test $test_external_has_tap -eq 0\n-\t\tthen\n-\t\t\tsay_color error \"# failed $test_failure among $msg\"\n-\t\t\tsay \"1..$test_count\"\n-\t\tfi\n-\n-\t\texit 1 ;;\n-\n-\tesac\n-}\n+2 test_done () {\n+3 \tcase \"$test_failure\" in\n+4 \t0)\n+5 \t\ttest_at_end_hook_\n+6 \n+7 \t\texit 0 ;;\n+8 \n+9 \t*)\n+10 \t\tif test $test_external_has_tap -eq 0\n+11 \t\tthen\n+12 \t\t\tsay_color error \"# failed $test_failure among $msg\"\n+13 \t\t\tsay \"1..$test_count\"\n+14 \t\tfi\n+15 \n+16 \t\texit 1 ;;\n+17 \n+18 \tesac\n+19 }\ndiff --git a/t/chainlint/blank-line.expect b/t/chainlint/blank-line.expect\nindex b47827d749..6ae39dd174 100644\n--- a/t/chainlint/blank-line.expect\n+++ b/t/chainlint/blank-line.expect\n@@ -1,8 +1,8 @@\n-(\n-\n-\tnothing &&\n-\n-\tsomething\n-\n-\n-)\n+2 (\n+3 \n+4 \tnothing &&\n+5 \n+6 \tsomething\n+7 \n+8 \n+9 )\ndiff --git a/t/chainlint/block-comment.expect b/t/chainlint/block-comment.expect\nindex df2beea888..7926936c18 100644\n--- a/t/chainlint/block-comment.expect\n+++ b/t/chainlint/block-comment.expect\n@@ -1,8 +1,8 @@\n-(\n-\t{\n-\t\t# show a\n-\t\techo a &&\n-\t\t# show b\n-\t\techo b\n-\t}\n-)\n+2 (\n+3 \t{\n+4 \t\t# show a\n+5 \t\techo a &&\n+6 \t\t# show b\n+7 \t\techo b\n+8 \t}\n+9 )\ndiff --git a/t/chainlint/block.expect b/t/chainlint/block.expect\nindex 1c87326364..b62e3d58c3 100644\n--- a/t/chainlint/block.expect\n+++ b/t/chainlint/block.expect\n@@ -1,23 +1,23 @@\n-(\n-\tfoo &&\n-\t{\n-\t\techo a ?!AMP?!\n-\t\techo b\n-\t} &&\n-\tbar &&\n-\t{\n-\t\techo c\n-\t} ?!AMP?!\n-\tbaz\n-) &&\n-\n-{\n-\techo a; ?!AMP?! echo b\n-} &&\n-{ echo a; ?!AMP?! echo b; } &&\n-\n-{\n-\techo \"${var}9\" &&\n-\techo \"done\"\n-} &&\n-finis\n+2 (\n+3 \tfoo &&\n+4 \t{\n+5 \t\techo a ?!AMP?!\n+6 \t\techo b\n+7 \t} &&\n+8 \tbar &&\n+9 \t{\n+10 \t\techo c\n+11 \t} ?!AMP?!\n+12 \tbaz\n+13 ) &&\n+14 \n+15 {\n+16 \techo a; ?!AMP?! echo b\n+17 } &&\n+18 { echo a; ?!AMP?! echo b; } &&\n+19 \n+20 {\n+21 \techo \"${var}9\" &&\n+22 \techo \"done\"\n+23 } &&\n+24 finis\ndiff --git a/t/chainlint/broken-chain.expect b/t/chainlint/broken-chain.expect\nindex cfb58fb6b9..9a1838736f 100644\n--- a/t/chainlint/broken-chain.expect\n+++ b/t/chainlint/broken-chain.expect\n@@ -1,6 +1,6 @@\n-(\n-\tfoo &&\n-\tbar ?!AMP?!\n-\tbaz &&\n-\twop\n-)\n+2 (\n+3 \tfoo &&\n+4 \tbar ?!AMP?!\n+5 \tbaz &&\n+6 \twop\n+7 )\ndiff --git a/t/chainlint/case-comment.expect b/t/chainlint/case-comment.expect\nindex 641c157b98..2442dd5f25 100644\n--- a/t/chainlint/case-comment.expect\n+++ b/t/chainlint/case-comment.expect\n@@ -1,11 +1,11 @@\n-(\n-\tcase \"$x\" in\n-\t# found foo\n-\tx) foo ;;\n-\t# found other\n-\t*)\n-\t\t# treat it as bar\n-\t\tbar\n-\t\t;;\n-\tesac\n-)\n+2 (\n+3 \tcase \"$x\" in\n+4 \t# found foo\n+5 \tx) foo ;;\n+6 \t# found other\n+7 \t*)\n+8 \t\t# treat it as bar\n+9 \t\tbar\n+10 \t\t;;\n+11 \tesac\n+12 )\ndiff --git a/t/chainlint/case.expect b/t/chainlint/case.expect\nindex 31f280d8ce..c04c61ff36 100644\n--- a/t/chainlint/case.expect\n+++ b/t/chainlint/case.expect\n@@ -1,19 +1,19 @@\n-(\n-\tcase \"$x\" in\n-\tx) foo ;;\n-\t*) bar ;;\n-\tesac &&\n-\tfoobar\n-) &&\n-(\n-\tcase \"$x\" in\n-\tx) foo ;;\n-\t*) bar ;;\n-\tesac ?!AMP?!\n-\tfoobar\n-) &&\n-(\n-\tcase \"$x\" in 1) true;; esac &&\n-\tcase \"$y\" in 2) false;; esac ?!AMP?!\n-\tfoobar\n-)\n+2 (\n+3 \tcase \"$x\" in\n+4 \tx) foo ;;\n+5 \t*) bar ;;\n+6 \tesac &&\n+7 \tfoobar\n+8 ) &&\n+9 (\n+10 \tcase \"$x\" in\n+11 \tx) foo ;;\n+12 \t*) bar ;;\n+13 \tesac ?!AMP?!\n+14 \tfoobar\n+15 ) &&\n+16 (\n+17 \tcase \"$x\" in 1) true;; esac &&\n+18 \tcase \"$y\" in 2) false;; esac ?!AMP?!\n+19 \tfoobar\n+20 )\ndiff --git a/t/chainlint/chain-break-background.expect b/t/chainlint/chain-break-background.expect\nindex 20d0bb5333..d06deadae7 100644\n--- a/t/chainlint/chain-break-background.expect\n+++ b/t/chainlint/chain-break-background.expect\n@@ -1,9 +1,9 @@\n-JGIT_DAEMON_PID= &&\n-git init --bare empty.git &&\n->empty.git/git-daemon-export-ok &&\n-mkfifo jgit_daemon_output &&\n-{\n-\tjgit daemon --port=\"$JGIT_DAEMON_PORT\" . >jgit_daemon_output &\n-\tJGIT_DAEMON_PID=$!\n-} &&\n-test_expect_code 2 git ls-remote --exit-code git://localhost:$JGIT_DAEMON_PORT/empty.git\n+2 JGIT_DAEMON_PID= &&\n+3 git init --bare empty.git &&\n+4 >empty.git/git-daemon-export-ok &&\n+5 mkfifo jgit_daemon_output &&\n+6 {\n+7 \tjgit daemon --port=\"$JGIT_DAEMON_PORT\" . >jgit_daemon_output &\n+8 \tJGIT_DAEMON_PID=$!\n+9 } &&\n+10 test_expect_code 2 git ls-remote --exit-code git://localhost:$JGIT_DAEMON_PORT/empty.git\ndiff --git a/t/chainlint/chain-break-continue.expect b/t/chainlint/chain-break-continue.expect\nindex 47a3457710..4bb60aae25 100644\n--- a/t/chainlint/chain-break-continue.expect\n+++ b/t/chainlint/chain-break-continue.expect\n@@ -1,12 +1,12 @@\n-git ls-tree --name-only -r refs/notes/many_notes |\n-while read path\n-do\n-\ttest \"$path\" = \"foobar/non-note.txt\" && continue\n-\ttest \"$path\" = \"deadbeef\" && continue\n-\ttest \"$path\" = \"de/adbeef\" && continue\n-\n-\tif test $(expr length \"$path\") -ne $hexsz\n-\tthen\n-\t\treturn 1\n-\tfi\n-done\n+2 git ls-tree --name-only -r refs/notes/many_notes |\n+3 while read path\n+4 do\n+5 \ttest \"$path\" = \"foobar/non-note.txt\" && continue\n+6 \ttest \"$path\" = \"deadbeef\" && continue\n+7 \ttest \"$path\" = \"de/adbeef\" && continue\n+8 \n+9 \tif test $(expr length \"$path\") -ne $hexsz\n+10 \tthen\n+11 \t\treturn 1\n+12 \tfi\n+13 done\ndiff --git a/t/chainlint/chain-break-false.expect b/t/chainlint/chain-break-false.expect\nindex 989766fb85..4f815f8e14 100644\n--- a/t/chainlint/chain-break-false.expect\n+++ b/t/chainlint/chain-break-false.expect\n@@ -1,9 +1,9 @@\n-if condition not satisified\n-then\n-\techo it did not work...\n-\techo failed!\n-\tfalse\n-else\n-\techo it went okay ?!AMP?!\n-\tcongratulate user\n-fi\n+2 if condition not satisified\n+3 then\n+4 \techo it did not work...\n+5 \techo failed!\n+6 \tfalse\n+7 else\n+8 \techo it went okay ?!AMP?!\n+9 \tcongratulate user\n+10 fi\ndiff --git a/t/chainlint/chain-break-return-exit.expect b/t/chainlint/chain-break-return-exit.expect\nindex 4cd18e2edf..ba0ec51aa0 100644\n--- a/t/chainlint/chain-break-return-exit.expect\n+++ b/t/chainlint/chain-break-return-exit.expect\n@@ -1,19 +1,19 @@\n-case \"$(git ls-files)\" in\n-one) echo pass one ;;\n-*) echo bad one; return 1 ;;\n-esac &&\n-(\n-\tcase \"$(git ls-files)\" in\n-\ttwo) echo pass two ;;\n-\t*) echo bad two; exit 1 ;;\n-\tesac\n-) &&\n-case \"$(git ls-files)\" in\n-dir/two\"$LF\"one) echo pass both ;;\n-*) echo bad; return 1 ;;\n-esac &&\n-\n-for i in 1 2 3 4 ; do\n-\tgit checkout main -b $i || return $?\n-\ttest_commit $i $i $i tag$i || return $?\n-done\n+2 case \"$(git ls-files)\" in\n+3 one) echo pass one ;;\n+4 *) echo bad one; return 1 ;;\n+5 esac &&\n+6 (\n+7 \tcase \"$(git ls-files)\" in\n+8 \ttwo) echo pass two ;;\n+9 \t*) echo bad two; exit 1 ;;\n+10 \tesac\n+11 ) &&\n+12 case \"$(git ls-files)\" in\n+13 dir/two\"$LF\"one) echo pass both ;;\n+14 *) echo bad; return 1 ;;\n+15 esac &&\n+16 \n+17 for i in 1 2 3 4 ; do\n+18 \tgit checkout main -b $i || return $?\n+19 \ttest_commit $i $i $i tag$i || return $?\n+20 done\ndiff --git a/t/chainlint/chain-break-status.expect b/t/chainlint/chain-break-status.expect\nindex e6b3b2193e..23c0caa7d8 100644\n--- a/t/chainlint/chain-break-status.expect\n+++ b/t/chainlint/chain-break-status.expect\n@@ -1,9 +1,9 @@\n-OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 ) &&\n-test_match_signal 13 \"$OUT\" &&\n-\n-{ test-tool sigchain >actual; ret=$?; } &&\n-{\n-\ttest_match_signal 15 \"$ret\" ||\n-\ttest \"$ret\" = 3\n-} &&\n-test_cmp expect actual\n+2 OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 ) &&\n+3 test_match_signal 13 \"$OUT\" &&\n+4 \n+5 { test-tool sigchain >actual; ret=$?; } &&\n+6 {\n+7 \ttest_match_signal 15 \"$ret\" ||\n+8 \ttest \"$ret\" = 3\n+9 } &&\n+10 test_cmp expect actual\ndiff --git a/t/chainlint/chained-block.expect b/t/chainlint/chained-block.expect\nindex 574cdceb07..a546b714a6 100644\n--- a/t/chainlint/chained-block.expect\n+++ b/t/chainlint/chained-block.expect\n@@ -1,9 +1,9 @@\n-echo nobody home && {\n-\ttest the doohicky ?!AMP?!\n-\tright now\n-} &&\n-\n-GIT_EXTERNAL_DIFF=echo git diff | {\n-\tread path oldfile oldhex oldmode newfile newhex newmode &&\n-\ttest \"z$oh\" = \"z$oldhex\"\n-}\n+2 echo nobody home && {\n+3 \ttest the doohicky ?!AMP?!\n+4 \tright now\n+5 } &&\n+6 \n+7 GIT_EXTERNAL_DIFF=echo git diff | {\n+8 \tread path oldfile oldhex oldmode newfile newhex newmode &&\n+9 \ttest \"z$oh\" = \"z$oldhex\"\n+10 }\ndiff --git a/t/chainlint/chained-subshell.expect b/t/chainlint/chained-subshell.expect\nindex 83810ea7ec..f78b268291 100644\n--- a/t/chainlint/chained-subshell.expect\n+++ b/t/chainlint/chained-subshell.expect\n@@ -1,10 +1,10 @@\n-mkdir sub && (\n-\tcd sub &&\n-\tfoo the bar ?!AMP?!\n-\tnuff said\n-) &&\n-\n-cut \"-d \" -f actual | (read s1 s2 s3 &&\n-test -f $s1 ?!AMP?!\n-test $(cat $s2) = tree2path1 &&\n-test $(cat $s3) = tree3path1)\n+2 mkdir sub && (\n+3 \tcd sub &&\n+4 \tfoo the bar ?!AMP?!\n+5 \tnuff said\n+6 ) &&\n+7 \n+8 cut \"-d \" -f actual | (read s1 s2 s3 &&\n+9 test -f $s1 ?!AMP?!\n+10 test $(cat $s2) = tree2path1 &&\n+11 test $(cat $s3) = tree3path1)\ndiff --git a/t/chainlint/close-nested-and-parent-together.expect b/t/chainlint/close-nested-and-parent-together.expect\nindex 72d482f76d..4167e54a59 100644\n--- a/t/chainlint/close-nested-and-parent-together.expect\n+++ b/t/chainlint/close-nested-and-parent-together.expect\n@@ -1,3 +1,3 @@\n-(cd foo &&\n-\t(bar &&\n-\t\tbaz))\n+2 (cd foo &&\n+3 \t(bar &&\n+4 \t\tbaz))\ndiff --git a/t/chainlint/close-subshell.expect b/t/chainlint/close-subshell.expect\nindex 2192a2870a..a272cfe72e 100644\n--- a/t/chainlint/close-subshell.expect\n+++ b/t/chainlint/close-subshell.expect\n@@ -1,26 +1,26 @@\n-(\n-\tfoo\n-) &&\n-(\n-\tbar\n-) >out &&\n-(\n-\tbaz\n-) 2>err &&\n-(\n-\tboo\n-) <input &&\n-(\n-\tbip\n-) | wuzzle &&\n-(\n-\tbop\n-) | fazz \\\n-\tfozz &&\n-(\n-\tbup\n-) |\n-fuzzle &&\n-(\n-\tyop\n-)\n+2 (\n+3 \tfoo\n+4 ) &&\n+5 (\n+6 \tbar\n+7 ) >out &&\n+8 (\n+9 \tbaz\n+10 ) 2>err &&\n+11 (\n+12 \tboo\n+13 ) <input &&\n+14 (\n+15 \tbip\n+16 ) | wuzzle &&\n+17 (\n+18 \tbop\n+19 ) | fazz \\\n+20 \tfozz &&\n+21 (\n+22 \tbup\n+23 ) |\n+24 fuzzle &&\n+25 (\n+26 \tyop\n+27 )\ndiff --git a/t/chainlint/command-substitution-subsubshell.expect b/t/chainlint/command-substitution-subsubshell.expect\nindex ec42f2c30c..f2a9312dc8 100644\n--- a/t/chainlint/command-substitution-subsubshell.expect\n+++ b/t/chainlint/command-substitution-subsubshell.expect\n@@ -1,2 +1,2 @@\n-OUT=$( ((large_git 1>&3) | :) 3>&1 ) &&\n-test_match_signal 13 \"$OUT\"\n+2 OUT=$( ((large_git 1>&3) | :) 3>&1 ) &&\n+3 test_match_signal 13 \"$OUT\"\ndiff --git a/t/chainlint/command-substitution.expect b/t/chainlint/command-substitution.expect\nindex c72e4df9e7..5e31b36db6 100644\n--- a/t/chainlint/command-substitution.expect\n+++ b/t/chainlint/command-substitution.expect\n@@ -1,9 +1,9 @@\n-(\n-\tfoo &&\n-\tbar=$(gobble) &&\n-\tbaz\n-) &&\n-(\n-\tbar=$(gobble blocks) ?!AMP?!\n-\tbaz\n-)\n+2 (\n+3 \tfoo &&\n+4 \tbar=$(gobble) &&\n+5 \tbaz\n+6 ) &&\n+7 (\n+8 \tbar=$(gobble blocks) ?!AMP?!\n+9 \tbaz\n+10 )\ndiff --git a/t/chainlint/comment.expect b/t/chainlint/comment.expect\nindex a68f1f9d7c..584098d6ba 100644\n--- a/t/chainlint/comment.expect\n+++ b/t/chainlint/comment.expect\n@@ -1,8 +1,8 @@\n-(\n-\t# comment 1\n-\tnothing &&\n-\t# comment 2\n-\tsomething\n-\t# comment 3\n-\t# comment 4\n-)\n+2 (\n+3 \t# comment 1\n+4 \tnothing &&\n+5 \t# comment 2\n+6 \tsomething\n+7 \t# comment 3\n+8 \t# comment 4\n+9 )\ndiff --git a/t/chainlint/complex-if-in-cuddled-loop.expect b/t/chainlint/complex-if-in-cuddled-loop.expect\nindex dac2d0fd1d..3a740103db 100644\n--- a/t/chainlint/complex-if-in-cuddled-loop.expect\n+++ b/t/chainlint/complex-if-in-cuddled-loop.expect\n@@ -1,9 +1,9 @@\n-(for i in a b c; do\n-   if test \"$(echo $(waffle bat))\" = \"eleventeen\" &&\n-     test \"$x\" = \"$y\"; then\n-     :\n-   else\n-     echo >file\n-   fi ?!LOOP?!\n- done) &&\n-test ! -f file\n+2 (for i in a b c; do\n+3    if test \"$(echo $(waffle bat))\" = \"eleventeen\" &&\n+4      test \"$x\" = \"$y\"; then\n+5      :\n+6    else\n+7      echo >file\n+8    fi ?!LOOP?!\n+9  done) &&\n+10 test ! -f file\ndiff --git a/t/chainlint/cuddled-if-then-else.expect b/t/chainlint/cuddled-if-then-else.expect\nindex 1d8ed58c49..72da8794cb 100644\n--- a/t/chainlint/cuddled-if-then-else.expect\n+++ b/t/chainlint/cuddled-if-then-else.expect\n@@ -1,6 +1,6 @@\n-(if test -z \"\"; then\n-    echo empty\n- else\n-    echo bizzy\n- fi) &&\n-echo foobar\n+2 (if test -z \"\"; then\n+3     echo empty\n+4  else\n+5     echo bizzy\n+6  fi) &&\n+7 echo foobar\ndiff --git a/t/chainlint/cuddled-loop.expect b/t/chainlint/cuddled-loop.expect\nindex 9cf260708e..c38585c756 100644\n--- a/t/chainlint/cuddled-loop.expect\n+++ b/t/chainlint/cuddled-loop.expect\n@@ -1,4 +1,4 @@\n-( while read x\n-  do foobar bop || exit 1\n-  done <file ) &&\n-outside subshell\n+2 ( while read x\n+3   do foobar bop || exit 1\n+4   done <file ) &&\n+5 outside subshell\ndiff --git a/t/chainlint/cuddled.expect b/t/chainlint/cuddled.expect\nindex c3e0be4047..b06d638311 100644\n--- a/t/chainlint/cuddled.expect\n+++ b/t/chainlint/cuddled.expect\n@@ -1,17 +1,17 @@\n-(cd foo &&\n-\tbar\n-) &&\n-\n-(cd foo ?!AMP?!\n-\tbar\n-) &&\n-\n-(\n-\tcd foo &&\n-\tbar) &&\n-\n-(cd foo &&\n-\tbar) &&\n-\n-(cd foo ?!AMP?!\n-\tbar)\n+2 (cd foo &&\n+3 \tbar\n+4 ) &&\n+5 \n+6 (cd foo ?!AMP?!\n+7 \tbar\n+8 ) &&\n+9 \n+10 (\n+11 \tcd foo &&\n+12 \tbar) &&\n+13 \n+14 (cd foo &&\n+15 \tbar) &&\n+16 \n+17 (cd foo ?!AMP?!\n+18 \tbar)\ndiff --git a/t/chainlint/double-here-doc.expect b/t/chainlint/double-here-doc.expect\nindex cd584a4357..48c04ecd58 100644\n--- a/t/chainlint/double-here-doc.expect\n+++ b/t/chainlint/double-here-doc.expect\n@@ -1,12 +1,12 @@\n-run_sub_test_lib_test_err run-inv-range-start \\\n-\t\"--run invalid range start\" \\\n-\t--run=\"a-5\" <<-\\EOF &&\n-test_expect_success \"passing test #1\" \"true\"\n-test_done\n-EOF\n-check_sub_test_lib_test_err run-inv-range-start \\\n-\t<<-\\EOF_OUT 3<<-EOF_ERR\n-> FATAL: Unexpected exit with code 1\n-EOF_OUT\n-> error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}\n-EOF_ERR\n+2 run_sub_test_lib_test_err run-inv-range-start \\\n+3 \t\"--run invalid range start\" \\\n+4 \t--run=\"a-5\" <<-\\EOF &&\n+5 test_expect_success \"passing test #1\" \"true\"\n+6 test_done\n+7 EOF\n+8 check_sub_test_lib_test_err run-inv-range-start \\\n+9 \t<<-\\EOF_OUT 3<<-EOF_ERR\n+10 > FATAL: Unexpected exit with code 1\n+11 EOF_OUT\n+12 > error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}\n+13 EOF_ERR\ndiff --git a/t/chainlint/dqstring-line-splice.expect b/t/chainlint/dqstring-line-splice.expect\nindex 37eab80738..2ca1c92cd6 100644\n--- a/t/chainlint/dqstring-line-splice.expect\n+++ b/t/chainlint/dqstring-line-splice.expect\n@@ -1,5 +1,5 @@\n-\n-echo 'fatal: reword option of --fixup is mutually exclusive with'\t'--patch/--interactive/--all/--include/--only' >expect &&\n-test_must_fail git commit --fixup=reword:HEAD~ $1 2>actual &&\n-test_cmp expect actual\n-\n+2 \n+3 echo 'fatal: reword option of --fixup is mutually exclusive with'\t'--patch/--interactive/--all/--include/--only' >expect &&\n+4 test_must_fail git commit --fixup=reword:HEAD~ $1 2>actual &&\n+5 test_cmp expect actual\n+6 \ndiff --git a/t/chainlint/dqstring-no-interpolate.expect b/t/chainlint/dqstring-no-interpolate.expect\nindex 087eda15e4..c9f75849c5 100644\n--- a/t/chainlint/dqstring-no-interpolate.expect\n+++ b/t/chainlint/dqstring-no-interpolate.expect\n@@ -1,12 +1,12 @@\n-grep \"^ ! [rejected][ ]*$BRANCH -> $BRANCH (non-fast-forward)$\" out &&\n-\n-grep \"^\\.git$\" output.txt &&\n-\n-\n-(\n-\tcd client$version &&\n-\tGIT_TEST_PROTOCOL_VERSION=$version git fetch-pack --no-progress .. $(cat ../input)\n-) >output &&\n-\tcut -d ' ' -f 2 <output | sort >actual &&\n-\ttest_cmp expect actual\n-\n+2 grep \"^ ! [rejected][ ]*$BRANCH -> $BRANCH (non-fast-forward)$\" out &&\n+3 \n+4 grep \"^\\.git$\" output.txt &&\n+5 \n+6 \n+7 (\n+8 \tcd client$version &&\n+9 \tGIT_TEST_PROTOCOL_VERSION=$version git fetch-pack --no-progress .. $(cat ../input)\n+10 ) >output &&\n+11 \tcut -d ' ' -f 2 <output | sort >actual &&\n+12 \ttest_cmp expect actual\n+13 \ndiff --git a/t/chainlint/empty-here-doc.expect b/t/chainlint/empty-here-doc.expect\nindex 8507721192..54b33f823a 100644\n--- a/t/chainlint/empty-here-doc.expect\n+++ b/t/chainlint/empty-here-doc.expect\n@@ -1,4 +1,4 @@\n-git ls-tree $tree path >current &&\n-cat >expected <<\\EOF &&\n-EOF\n-test_output\n+2 git ls-tree $tree path >current &&\n+3 cat >expected <<\\EOF &&\n+4 EOF\n+5 test_output\ndiff --git a/t/chainlint/exclamation.expect b/t/chainlint/exclamation.expect\nindex 765a35bb4c..078744b61b 100644\n--- a/t/chainlint/exclamation.expect\n+++ b/t/chainlint/exclamation.expect\n@@ -1,4 +1,4 @@\n-if ! condition; then echo nope; else yep; fi &&\n-test_prerequisite !MINGW &&\n-mail uucp!address &&\n-echo !whatever!\n+2 if ! condition; then echo nope; else yep; fi &&\n+3 test_prerequisite !MINGW &&\n+4 mail uucp!address &&\n+5 echo !whatever!\ndiff --git a/t/chainlint/exit-loop.expect b/t/chainlint/exit-loop.expect\nindex f76aa60466..407278094c 100644\n--- a/t/chainlint/exit-loop.expect\n+++ b/t/chainlint/exit-loop.expect\n@@ -1,24 +1,24 @@\n-(\n-\tfor i in a b c\n-\tdo\n-\t\tfoo || exit 1\n-\t\tbar &&\n-\t\tbaz\n-\tdone\n-) &&\n-(\n-\twhile true\n-\tdo\n-\t\tfoo || exit 1\n-\t\tbar &&\n-\t\tbaz\n-\tdone\n-) &&\n-(\n-\ti=0 &&\n-\twhile test $i -lt 10\n-\tdo\n-\t\techo $i || exit\n-\t\ti=$(($i + 1))\n-\tdone\n-)\n+2 (\n+3 \tfor i in a b c\n+4 \tdo\n+5 \t\tfoo || exit 1\n+6 \t\tbar &&\n+7 \t\tbaz\n+8 \tdone\n+9 ) &&\n+10 (\n+11 \twhile true\n+12 \tdo\n+13 \t\tfoo || exit 1\n+14 \t\tbar &&\n+15 \t\tbaz\n+16 \tdone\n+17 ) &&\n+18 (\n+19 \ti=0 &&\n+20 \twhile test $i -lt 10\n+21 \tdo\n+22 \t\techo $i || exit\n+23 \t\ti=$(($i + 1))\n+24 \tdone\n+25 )\ndiff --git a/t/chainlint/exit-subshell.expect b/t/chainlint/exit-subshell.expect\nindex da80339f78..793db12453 100644\n--- a/t/chainlint/exit-subshell.expect\n+++ b/t/chainlint/exit-subshell.expect\n@@ -1,5 +1,5 @@\n-(\n-\tfoo || exit 1\n-\tbar &&\n-\tbaz\n-)\n+2 (\n+3 \tfoo || exit 1\n+4 \tbar &&\n+5 \tbaz\n+6 )\ndiff --git a/t/chainlint/for-loop-abbreviated.expect b/t/chainlint/for-loop-abbreviated.expect\nindex 02c0d15cca..5574831976 100644\n--- a/t/chainlint/for-loop-abbreviated.expect\n+++ b/t/chainlint/for-loop-abbreviated.expect\n@@ -1,5 +1,5 @@\n-for it\n-do\n-\tpath=$(expr \"$it\" : ([^:]*)) &&\n-\tgit update-index --add \"$path\" || exit\n-done\n+2 for it\n+3 do\n+4 \tpath=$(expr \"$it\" : ([^:]*)) &&\n+5 \tgit update-index --add \"$path\" || exit\n+6 done\ndiff --git a/t/chainlint/for-loop.expect b/t/chainlint/for-loop.expect\nindex d2237f1e38..908aeedf96 100644\n--- a/t/chainlint/for-loop.expect\n+++ b/t/chainlint/for-loop.expect\n@@ -1,14 +1,14 @@\n-(\n-\tfor i in a b c\n-\tdo\n-\t\techo $i ?!AMP?!\n-\t\tcat <<-\\EOF ?!LOOP?!\n-\t\tbar\n-\t\tEOF\n-\tdone ?!AMP?!\n-\n-\tfor i in a b c; do\n-\t\techo $i &&\n-\t\tcat $i ?!LOOP?!\n-\tdone\n-)\n+2 (\n+3 \tfor i in a b c\n+4 \tdo\n+5 \t\techo $i ?!AMP?!\n+6 \t\tcat <<-\\EOF ?!LOOP?!\n+7 \t\tbar\n+8 \t\tEOF\n+9 \tdone ?!AMP?!\n+10 \n+11 \tfor i in a b c; do\n+12 \t\techo $i &&\n+13 \t\tcat $i ?!LOOP?!\n+14 \tdone\n+15 )\ndiff --git a/t/chainlint/function.expect b/t/chainlint/function.expect\nindex dd7c997a3c..c226246b25 100644\n--- a/t/chainlint/function.expect\n+++ b/t/chainlint/function.expect\n@@ -1,11 +1,11 @@\n-sha1_file() {\n-\techo \"$*\" | sed \"s#..#.git/objects/&/#\"\n-} &&\n-\n-remove_object() {\n-\tfile=$(sha1_file \"$*\") &&\n-\ttest -e \"$file\" ?!AMP?!\n-\trm -f \"$file\"\n-} ?!AMP?!\n-\n-sha1_file arg && remove_object arg\n+2 sha1_file() {\n+3 \techo \"$*\" | sed \"s#..#.git/objects/&/#\"\n+4 } &&\n+5 \n+6 remove_object() {\n+7 \tfile=$(sha1_file \"$*\") &&\n+8 \ttest -e \"$file\" ?!AMP?!\n+9 \trm -f \"$file\"\n+10 } ?!AMP?!\n+11 \n+12 sha1_file arg && remove_object arg\ndiff --git a/t/chainlint/here-doc-close-subshell.expect b/t/chainlint/here-doc-close-subshell.expect\nindex 7d9c2b5607..965813f463 100644\n--- a/t/chainlint/here-doc-close-subshell.expect\n+++ b/t/chainlint/here-doc-close-subshell.expect\n@@ -1,4 +1,4 @@\n-(\n-\tcat <<-\\INPUT)\n-\tfizz\n-\tINPUT\n+2 (\n+3 \tcat <<-\\INPUT)\n+4 \tfizz\n+5 \tINPUT\ndiff --git a/t/chainlint/here-doc-indent-operator.expect b/t/chainlint/here-doc-indent-operator.expect\nindex f92a7ce999..277a11202d 100644\n--- a/t/chainlint/here-doc-indent-operator.expect\n+++ b/t/chainlint/here-doc-indent-operator.expect\n@@ -1,11 +1,11 @@\n-cat >expect <<- EOF &&\n-header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0\n-num_commits: $1\n-chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data\n-EOF\n-\n-cat >expect << -EOF ?!AMP?!\n-this is not indented\n--EOF\n-\n-cleanup\n+2 cat >expect <<- EOF &&\n+3 header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0\n+4 num_commits: $1\n+5 chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data\n+6 EOF\n+7 \n+8 cat >expect << -EOF ?!AMP?!\n+9 this is not indented\n+10 -EOF\n+11 \n+12 cleanup\ndiff --git a/t/chainlint/here-doc-multi-line-command-subst.expect b/t/chainlint/here-doc-multi-line-command-subst.expect\nindex b7364c82c8..41b55f6437 100644\n--- a/t/chainlint/here-doc-multi-line-command-subst.expect\n+++ b/t/chainlint/here-doc-multi-line-command-subst.expect\n@@ -1,8 +1,8 @@\n-(\n-\tx=$(bobble <<-\\END &&\n-\t\tfossil\n-\t\tvegetable\n-\t\tEND\n-\t\twiffle) ?!AMP?!\n-\techo $x\n-)\n+2 (\n+3 \tx=$(bobble <<-\\END &&\n+4 \t\tfossil\n+5 \t\tvegetable\n+6 \t\tEND\n+7 \t\twiffle) ?!AMP?!\n+8 \techo $x\n+9 )\ndiff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect\nindex 6c13bdcbfb..c71828589e 100644\n--- a/t/chainlint/here-doc-multi-line-string.expect\n+++ b/t/chainlint/here-doc-multi-line-string.expect\n@@ -1,7 +1,7 @@\n-(\n-\tcat <<-\\TXT && echo \"multi-line\n-\tstring\" ?!AMP?!\n-\tfizzle\n-\tTXT\n-\tbap\n-)\n+2 (\n+3 \tcat <<-\\TXT && echo \"multi-line\n+4 \tstring\" ?!AMP?!\n+5 \tfizzle\n+6 \tTXT\n+7 \tbap\n+8 )\ndiff --git a/t/chainlint/here-doc.expect b/t/chainlint/here-doc.expect\nindex 91b961242a..2c382dd8eb 100644\n--- a/t/chainlint/here-doc.expect\n+++ b/t/chainlint/here-doc.expect\n@@ -1,25 +1,25 @@\n-boodle wobba \\\n-       gorgo snoot \\\n-       wafta snurb <<EOF &&\n-quoth the raven,\n-nevermore...\n-EOF\n-\n-cat <<-Arbitrary_Tag_42 >foo &&\n-snoz\n-boz\n-woz\n-Arbitrary_Tag_42\n-\n-cat <<\"zump\" >boo &&\n-snoz\n-boz\n-woz\n-zump\n-\n-horticulture <<\\EOF\n-gomez\n-morticia\n-wednesday\n-pugsly\n-EOF\n+2 boodle wobba \\\n+3        gorgo snoot \\\n+4        wafta snurb <<EOF &&\n+5 quoth the raven,\n+6 nevermore...\n+7 EOF\n+8 \n+9 cat <<-Arbitrary_Tag_42 >foo &&\n+10 snoz\n+11 boz\n+12 woz\n+13 Arbitrary_Tag_42\n+14 \n+15 cat <<\"zump\" >boo &&\n+16 snoz\n+17 boz\n+18 woz\n+19 zump\n+20 \n+21 horticulture <<\\EOF\n+22 gomez\n+23 morticia\n+24 wednesday\n+25 pugsly\n+26 EOF\ndiff --git a/t/chainlint/if-condition-split.expect b/t/chainlint/if-condition-split.expect\nindex ee745ef8d7..9daf3d294a 100644\n--- a/t/chainlint/if-condition-split.expect\n+++ b/t/chainlint/if-condition-split.expect\n@@ -1,7 +1,7 @@\n-if bob &&\n-   marcia ||\n-   kevin\n-then\n-\techo \"nomads\" ?!AMP?!\n-\techo \"for sure\"\n-fi\n+2 if bob &&\n+3    marcia ||\n+4    kevin\n+5 then\n+6 \techo \"nomads\" ?!AMP?!\n+7 \techo \"for sure\"\n+8 fi\ndiff --git a/t/chainlint/if-in-loop.expect b/t/chainlint/if-in-loop.expect\nindex d6514ae749..ff8c60dbdb 100644\n--- a/t/chainlint/if-in-loop.expect\n+++ b/t/chainlint/if-in-loop.expect\n@@ -1,12 +1,12 @@\n-(\n-\tfor i in a b c\n-\tdo\n-\t\tif false\n-\t\tthen\n-\t\t\techo \"err\"\n-\t\t\texit 1\n-\t\tfi ?!AMP?!\n-\t\tfoo\n-\tdone ?!AMP?!\n-\tbar\n-)\n+2 (\n+3 \tfor i in a b c\n+4 \tdo\n+5 \t\tif false\n+6 \t\tthen\n+7 \t\t\techo \"err\"\n+8 \t\t\texit 1\n+9 \t\tfi ?!AMP?!\n+10 \t\tfoo\n+11 \tdone ?!AMP?!\n+12 \tbar\n+13 )\ndiff --git a/t/chainlint/if-then-else.expect b/t/chainlint/if-then-else.expect\nindex cbaaf857d4..965d7e41a2 100644\n--- a/t/chainlint/if-then-else.expect\n+++ b/t/chainlint/if-then-else.expect\n@@ -1,22 +1,22 @@\n-(\n-\tif test -n \"\"\n-\tthen\n-\t\techo very ?!AMP?!\n-\t\techo empty\n-\telif test -z \"\"\n-\tthen\n-\t\techo foo\n-\telse\n-\t\techo foo &&\n-\t\tcat <<-\\EOF\n-\t\tbar\n-\t\tEOF\n-\tfi ?!AMP?!\n-\techo poodle\n-) &&\n-(\n-\tif test -n \"\"; then\n-\t\techo very &&\n-\t\techo empty\n-\tfi\n-)\n+2 (\n+3 \tif test -n \"\"\n+4 \tthen\n+5 \t\techo very ?!AMP?!\n+6 \t\techo empty\n+7 \telif test -z \"\"\n+8 \tthen\n+9 \t\techo foo\n+10 \telse\n+11 \t\techo foo &&\n+12 \t\tcat <<-\\EOF\n+13 \t\tbar\n+14 \t\tEOF\n+15 \tfi ?!AMP?!\n+16 \techo poodle\n+17 ) &&\n+18 (\n+19 \tif test -n \"\"; then\n+20 \t\techo very &&\n+21 \t\techo empty\n+22 \tfi\n+23 )\ndiff --git a/t/chainlint/incomplete-line.expect b/t/chainlint/incomplete-line.expect\nindex 134d3a14f5..b15e00b901 100644\n--- a/t/chainlint/incomplete-line.expect\n+++ b/t/chainlint/incomplete-line.expect\n@@ -1,10 +1,10 @@\n-line 1 \\\n-line 2 \\\n-line 3 \\\n-line 4 &&\n-(\n-\tline 5 \\\n-\tline 6 \\\n-\tline 7 \\\n-\tline 8\n-)\n+2 line 1 \\\n+3 line 2 \\\n+4 line 3 \\\n+5 line 4 &&\n+6 (\n+7 \tline 5 \\\n+8 \tline 6 \\\n+9 \tline 7 \\\n+10 \tline 8\n+11 )\ndiff --git a/t/chainlint/inline-comment.expect b/t/chainlint/inline-comment.expect\nindex 6bad218530..0285c0b22c 100644\n--- a/t/chainlint/inline-comment.expect\n+++ b/t/chainlint/inline-comment.expect\n@@ -1,8 +1,8 @@\n-(\n-\tfoobar && # comment 1\n-\tbarfoo ?!AMP?! # wrong position for &&\n-\tflibble \"not a # comment\"\n-) &&\n-\n-(cd foo &&\n-\tflibble \"not a # comment\")\n+2 (\n+3 \tfoobar && # comment 1\n+4 \tbarfoo ?!AMP?! # wrong position for &&\n+5 \tflibble \"not a # comment\"\n+6 ) &&\n+7 \n+8 (cd foo &&\n+9 \tflibble \"not a # comment\")\ndiff --git a/t/chainlint/loop-detect-failure.expect b/t/chainlint/loop-detect-failure.expect\nindex a66025c39d..40c06f0d53 100644\n--- a/t/chainlint/loop-detect-failure.expect\n+++ b/t/chainlint/loop-detect-failure.expect\n@@ -1,15 +1,15 @@\n-git init r1 &&\n-for n in 1 2 3 4 5\n-do\n-\techo \"This is file: $n\" > r1/file.$n &&\n-\tgit -C r1 add file.$n &&\n-\tgit -C r1 commit -m \"$n\" || return 1\n-done &&\n-\n-git init r2 &&\n-for n in 1000 10000\n-do\n-\tprintf \"%\"$n\"s\" X > r2/large.$n &&\n-\tgit -C r2 add large.$n &&\n-\tgit -C r2 commit -m \"$n\" ?!LOOP?!\n-done\n+2 git init r1 &&\n+3 for n in 1 2 3 4 5\n+4 do\n+5 \techo \"This is file: $n\" > r1/file.$n &&\n+6 \tgit -C r1 add file.$n &&\n+7 \tgit -C r1 commit -m \"$n\" || return 1\n+8 done &&\n+9 \n+10 git init r2 &&\n+11 for n in 1000 10000\n+12 do\n+13 \tprintf \"%\"$n\"s\" X > r2/large.$n &&\n+14 \tgit -C r2 add large.$n &&\n+15 \tgit -C r2 commit -m \"$n\" ?!LOOP?!\n+16 done\ndiff --git a/t/chainlint/loop-detect-status.expect b/t/chainlint/loop-detect-status.expect\nindex 7ce3a34806..0f180b08de 100644\n--- a/t/chainlint/loop-detect-status.expect\n+++ b/t/chainlint/loop-detect-status.expect\n@@ -1,18 +1,18 @@\n-(while test $i -le $blobcount\n- do\n-\tprintf \"Generating blob $i/$blobcount\\r\" >&2 &&\n-\tprintf \"blob\\nmark :$i\\ndata $blobsize\\n\" &&\n-\t#test-tool genrandom $i $blobsize &&\n-\tprintf \"%-${blobsize}s\" $i &&\n-\techo \"M 100644 :$i $i\" >> commit &&\n-\ti=$(($i+1)) ||\n-\techo $? > exit-status\n- done &&\n- echo \"commit refs/heads/main\" &&\n- echo \"author A U Thor <author@email.com> 123456789 +0000\" &&\n- echo \"committer C O Mitter <committer@email.com> 123456789 +0000\" &&\n- echo \"data 5\" &&\n- echo \">2gb\" &&\n- cat commit) |\n-git fast-import --big-file-threshold=2 &&\n-test ! -f exit-status\n+2 (while test $i -le $blobcount\n+3  do\n+4 \tprintf \"Generating blob $i/$blobcount\\r\" >&2 &&\n+5 \tprintf \"blob\\nmark :$i\\ndata $blobsize\\n\" &&\n+6 \t#test-tool genrandom $i $blobsize &&\n+7 \tprintf \"%-${blobsize}s\" $i &&\n+8 \techo \"M 100644 :$i $i\" >> commit &&\n+9 \ti=$(($i+1)) ||\n+10 \techo $? > exit-status\n+11  done &&\n+12  echo \"commit refs/heads/main\" &&\n+13  echo \"author A U Thor <author@email.com> 123456789 +0000\" &&\n+14  echo \"committer C O Mitter <committer@email.com> 123456789 +0000\" &&\n+15  echo \"data 5\" &&\n+16  echo \">2gb\" &&\n+17  cat commit) |\n+18 git fast-import --big-file-threshold=2 &&\n+19 test ! -f exit-status\ndiff --git a/t/chainlint/loop-in-if.expect b/t/chainlint/loop-in-if.expect\nindex 6c5d6e5b24..4e8c67c914 100644\n--- a/t/chainlint/loop-in-if.expect\n+++ b/t/chainlint/loop-in-if.expect\n@@ -1,12 +1,12 @@\n-(\n-\tif true\n-\tthen\n-\t\twhile true\n-\t\tdo\n-\t\t\techo \"pop\" ?!AMP?!\n-\t\t\techo \"glup\" ?!LOOP?!\n-\t\tdone ?!AMP?!\n-\t\tfoo\n-\tfi ?!AMP?!\n-\tbar\n-)\n+2 (\n+3 \tif true\n+4 \tthen\n+5 \t\twhile true\n+6 \t\tdo\n+7 \t\t\techo \"pop\" ?!AMP?!\n+8 \t\t\techo \"glup\" ?!LOOP?!\n+9 \t\tdone ?!AMP?!\n+10 \t\tfoo\n+11 \tfi ?!AMP?!\n+12 \tbar\n+13 )\ndiff --git a/t/chainlint/loop-upstream-pipe.expect b/t/chainlint/loop-upstream-pipe.expect\nindex 0b82ecc4b9..bef82479ca 100644\n--- a/t/chainlint/loop-upstream-pipe.expect\n+++ b/t/chainlint/loop-upstream-pipe.expect\n@@ -1,10 +1,10 @@\n-(\n-\tgit rev-list --objects --no-object-names base..loose |\n-\twhile read oid\n-\tdo\n-\t\tpath=\"$objdir/$(test_oid_to_path \"$oid\")\" &&\n-\t\tprintf \"%s %d\\n\" \"$oid\" \"$(test-tool chmtime --get \"$path\")\" ||\n-\t\techo \"object list generation failed for $oid\"\n-\tdone |\n-\tsort -k1\n-) >expect &&\n+2 (\n+3 \tgit rev-list --objects --no-object-names base..loose |\n+4 \twhile read oid\n+5 \tdo\n+6 \t\tpath=\"$objdir/$(test_oid_to_path \"$oid\")\" &&\n+7 \t\tprintf \"%s %d\\n\" \"$oid\" \"$(test-tool chmtime --get \"$path\")\" ||\n+8 \t\techo \"object list generation failed for $oid\"\n+9 \tdone |\n+10 \tsort -k1\n+11 ) >expect &&\ndiff --git a/t/chainlint/multi-line-nested-command-substitution.expect b/t/chainlint/multi-line-nested-command-substitution.expect\nindex 300058341b..ad27e43e05 100644\n--- a/t/chainlint/multi-line-nested-command-substitution.expect\n+++ b/t/chainlint/multi-line-nested-command-substitution.expect\n@@ -1,18 +1,18 @@\n-(\n-\tfoo &&\n-\tx=$(\n-\t\techo bar |\n-\t\tcat\n-\t) &&\n-\techo ok\n-) |\n-sort &&\n-(\n-\tbar &&\n-\tx=$(echo bar |\n-\t\tcat\n-\t) &&\n-\ty=$(echo baz |\n-\t\tfip) &&\n-\techo fail\n-)\n+2 (\n+3 \tfoo &&\n+4 \tx=$(\n+5 \t\techo bar |\n+6 \t\tcat\n+7 \t) &&\n+8 \techo ok\n+9 ) |\n+10 sort &&\n+11 (\n+12 \tbar &&\n+13 \tx=$(echo bar |\n+14 \t\tcat\n+15 \t) &&\n+16 \ty=$(echo baz |\n+17 \t\tfip) &&\n+18 \techo fail\n+19 )\ndiff --git a/t/chainlint/multi-line-string.expect b/t/chainlint/multi-line-string.expect\nindex 27ff95218e..62c54e3a5e 100644\n--- a/t/chainlint/multi-line-string.expect\n+++ b/t/chainlint/multi-line-string.expect\n@@ -1,14 +1,14 @@\n-(\n-\tx=\"line 1\n-\t\tline 2\n-\t\tline 3\" &&\n-\ty=\"line 1\n-\t\tline2\" ?!AMP?!\n-\tfoobar\n-) &&\n-(\n-\techo \"xyz\" \"abc\n-\t\tdef\n-\t\tghi\" &&\n-\tbarfoo\n-)\n+2 (\n+3 \tx=\"line 1\n+4 \t\tline 2\n+5 \t\tline 3\" &&\n+6 \ty=\"line 1\n+7 \t\tline2\" ?!AMP?!\n+8 \tfoobar\n+9 ) &&\n+10 (\n+11 \techo \"xyz\" \"abc\n+12 \t\tdef\n+13 \t\tghi\" &&\n+14 \tbarfoo\n+15 )\ndiff --git a/t/chainlint/negated-one-liner.expect b/t/chainlint/negated-one-liner.expect\nindex ad4c2d949e..a6ce52a1da 100644\n--- a/t/chainlint/negated-one-liner.expect\n+++ b/t/chainlint/negated-one-liner.expect\n@@ -1,5 +1,5 @@\n-! (foo && bar) &&\n-! (foo && bar) >baz &&\n-\n-! (foo; ?!AMP?! bar) &&\n-! (foo; ?!AMP?! bar) >baz\n+2 ! (foo && bar) &&\n+3 ! (foo && bar) >baz &&\n+4 \n+5 ! (foo; ?!AMP?! bar) &&\n+6 ! (foo; ?!AMP?! bar) >baz\ndiff --git a/t/chainlint/nested-cuddled-subshell.expect b/t/chainlint/nested-cuddled-subshell.expect\nindex 3836049cc4..0191c9c294 100644\n--- a/t/chainlint/nested-cuddled-subshell.expect\n+++ b/t/chainlint/nested-cuddled-subshell.expect\n@@ -1,25 +1,25 @@\n-(\n-\t(cd foo &&\n-\t\tbar\n-\t) &&\n-\n-\t(cd foo &&\n-\t\tbar\n-\t) ?!AMP?!\n-\n-\t(\n-\t\tcd foo &&\n-\t\tbar) &&\n-\n-\t(\n-\t\tcd foo &&\n-\t\tbar) ?!AMP?!\n-\n-\t(cd foo &&\n-\t\tbar) &&\n-\n-\t(cd foo &&\n-\t\tbar) ?!AMP?!\n-\n-\tfoobar\n-)\n+2 (\n+3 \t(cd foo &&\n+4 \t\tbar\n+5 \t) &&\n+6 \n+7 \t(cd foo &&\n+8 \t\tbar\n+9 \t) ?!AMP?!\n+10 \n+11 \t(\n+12 \t\tcd foo &&\n+13 \t\tbar) &&\n+14 \n+15 \t(\n+16 \t\tcd foo &&\n+17 \t\tbar) ?!AMP?!\n+18 \n+19 \t(cd foo &&\n+20 \t\tbar) &&\n+21 \n+22 \t(cd foo &&\n+23 \t\tbar) ?!AMP?!\n+24 \n+25 \tfoobar\n+26 )\ndiff --git a/t/chainlint/nested-here-doc.expect b/t/chainlint/nested-here-doc.expect\nindex 29b3832a98..70d9b68dc9 100644\n--- a/t/chainlint/nested-here-doc.expect\n+++ b/t/chainlint/nested-here-doc.expect\n@@ -1,30 +1,30 @@\n-cat <<ARBITRARY >foop &&\n-naddle\n-fub <<EOF\n-\tnozzle\n-\tnoodle\n-EOF\n-formp\n-ARBITRARY\n-\n-(\n-\tcat <<-\\INPUT_END &&\n-\tfish are mice\n-\tbut geese go slow\n-\tdata <<EOF\n-\t\tperl is lerp\n-\t\tand nothing else\n-\tEOF\n-\ttoink\n-\tINPUT_END\n-\n-\tcat <<-\\EOT ?!AMP?!\n-\ttext goes here\n-\tdata <<EOF\n-\t\tdata goes here\n-\tEOF\n-\tmore test here\n-\tEOT\n-\n-\tfoobar\n-)\n+2 cat <<ARBITRARY >foop &&\n+3 naddle\n+4 fub <<EOF\n+5 \tnozzle\n+6 \tnoodle\n+7 EOF\n+8 formp\n+9 ARBITRARY\n+10 \n+11 (\n+12 \tcat <<-\\INPUT_END &&\n+13 \tfish are mice\n+14 \tbut geese go slow\n+15 \tdata <<EOF\n+16 \t\tperl is lerp\n+17 \t\tand nothing else\n+18 \tEOF\n+19 \ttoink\n+20 \tINPUT_END\n+21 \n+22 \tcat <<-\\EOT ?!AMP?!\n+23 \ttext goes here\n+24 \tdata <<EOF\n+25 \t\tdata goes here\n+26 \tEOF\n+27 \tmore test here\n+28 \tEOT\n+29 \n+30 \tfoobar\n+31 )\ndiff --git a/t/chainlint/nested-loop-detect-failure.expect b/t/chainlint/nested-loop-detect-failure.expect\nindex 3461df40e5..c13c4d2f90 100644\n--- a/t/chainlint/nested-loop-detect-failure.expect\n+++ b/t/chainlint/nested-loop-detect-failure.expect\n@@ -1,31 +1,31 @@\n-for i in 0 1 2 3 4 5 6 7 8 9;\n-do\n-\tfor j in 0 1 2 3 4 5 6 7 8 9;\n-\tdo\n-\t\techo \"$i$j\" >\"path$i$j\" ?!LOOP?!\n-\tdone ?!LOOP?!\n-done &&\n-\n-for i in 0 1 2 3 4 5 6 7 8 9;\n-do\n-\tfor j in 0 1 2 3 4 5 6 7 8 9;\n-\tdo\n-\t\techo \"$i$j\" >\"path$i$j\" || return 1\n-\tdone\n-done &&\n-\n-for i in 0 1 2 3 4 5 6 7 8 9;\n-do\n-\tfor j in 0 1 2 3 4 5 6 7 8 9;\n-\tdo\n-\t\techo \"$i$j\" >\"path$i$j\" ?!LOOP?!\n-\tdone || return 1\n-done &&\n-\n-for i in 0 1 2 3 4 5 6 7 8 9;\n-do\n-\tfor j in 0 1 2 3 4 5 6 7 8 9;\n-\tdo\n-\t\techo \"$i$j\" >\"path$i$j\" || return 1\n-\tdone || return 1\n-done\n+2 for i in 0 1 2 3 4 5 6 7 8 9;\n+3 do\n+4 \tfor j in 0 1 2 3 4 5 6 7 8 9;\n+5 \tdo\n+6 \t\techo \"$i$j\" >\"path$i$j\" ?!LOOP?!\n+7 \tdone ?!LOOP?!\n+8 done &&\n+9 \n+10 for i in 0 1 2 3 4 5 6 7 8 9;\n+11 do\n+12 \tfor j in 0 1 2 3 4 5 6 7 8 9;\n+13 \tdo\n+14 \t\techo \"$i$j\" >\"path$i$j\" || return 1\n+15 \tdone\n+16 done &&\n+17 \n+18 for i in 0 1 2 3 4 5 6 7 8 9;\n+19 do\n+20 \tfor j in 0 1 2 3 4 5 6 7 8 9;\n+21 \tdo\n+22 \t\techo \"$i$j\" >\"path$i$j\" ?!LOOP?!\n+23 \tdone || return 1\n+24 done &&\n+25 \n+26 for i in 0 1 2 3 4 5 6 7 8 9;\n+27 do\n+28 \tfor j in 0 1 2 3 4 5 6 7 8 9;\n+29 \tdo\n+30 \t\techo \"$i$j\" >\"path$i$j\" || return 1\n+31 \tdone || return 1\n+32 done\ndiff --git a/t/chainlint/nested-subshell-comment.expect b/t/chainlint/nested-subshell-comment.expect\nindex 9138cf386d..f89a8d03a8 100644\n--- a/t/chainlint/nested-subshell-comment.expect\n+++ b/t/chainlint/nested-subshell-comment.expect\n@@ -1,11 +1,11 @@\n-(\n-\tfoo &&\n-\t(\n-\t\tbar &&\n-\t\t# bottles wobble while fiddles gobble\n-\t\t# minor numbers of cows (or do they?)\n-\t\tbaz &&\n-\t\tsnaff\n-\t) ?!AMP?!\n-\tfuzzy\n-)\n+2 (\n+3 \tfoo &&\n+4 \t(\n+5 \t\tbar &&\n+6 \t\t# bottles wobble while fiddles gobble\n+7 \t\t# minor numbers of cows (or do they?)\n+8 \t\tbaz &&\n+9 \t\tsnaff\n+10 \t) ?!AMP?!\n+11 \tfuzzy\n+12 )\ndiff --git a/t/chainlint/nested-subshell.expect b/t/chainlint/nested-subshell.expect\nindex 73ff28546a..811e8a7912 100644\n--- a/t/chainlint/nested-subshell.expect\n+++ b/t/chainlint/nested-subshell.expect\n@@ -1,13 +1,13 @@\n-(\n-\tcd foo &&\n-\t(\n-\t\techo a &&\n-\t\techo b\n-\t) >file &&\n-\n-\tcd foo &&\n-\t(\n-\t\techo a ?!AMP?!\n-\t\techo b\n-\t) >file\n-)\n+2 (\n+3 \tcd foo &&\n+4 \t(\n+5 \t\techo a &&\n+6 \t\techo b\n+7 \t) >file &&\n+8 \n+9 \tcd foo &&\n+10 \t(\n+11 \t\techo a ?!AMP?!\n+12 \t\techo b\n+13 \t) >file\n+14 )\ndiff --git a/t/chainlint/not-heredoc.expect b/t/chainlint/not-heredoc.expect\nindex 2e9bb135fe..611b7b75cb 100644\n--- a/t/chainlint/not-heredoc.expect\n+++ b/t/chainlint/not-heredoc.expect\n@@ -1,14 +1,14 @@\n-echo \"<<<<<<< ours\" &&\n-echo ourside &&\n-echo \"=======\" &&\n-echo theirside &&\n-echo \">>>>>>> theirs\" &&\n-\n-(\n-\techo \"<<<<<<< ours\" &&\n-\techo ourside &&\n-\techo \"=======\" &&\n-\techo theirside &&\n-\techo \">>>>>>> theirs\" ?!AMP?!\n-\tpoodle\n-) >merged\n+2 echo \"<<<<<<< ours\" &&\n+3 echo ourside &&\n+4 echo \"=======\" &&\n+5 echo theirside &&\n+6 echo \">>>>>>> theirs\" &&\n+7 \n+8 (\n+9 \techo \"<<<<<<< ours\" &&\n+10 \techo ourside &&\n+11 \techo \"=======\" &&\n+12 \techo theirside &&\n+13 \techo \">>>>>>> theirs\" ?!AMP?!\n+14 \tpoodle\n+15 ) >merged\ndiff --git a/t/chainlint/one-liner-for-loop.expect b/t/chainlint/one-liner-for-loop.expect\nindex 51a3dc7c54..49dcf065ef 100644\n--- a/t/chainlint/one-liner-for-loop.expect\n+++ b/t/chainlint/one-liner-for-loop.expect\n@@ -1,9 +1,9 @@\n-git init dir-rename-and-content &&\n-(\n-\tcd dir-rename-and-content &&\n-\ttest_write_lines 1 2 3 4 5 >foo &&\n-\tmkdir olddir &&\n-\tfor i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!\n-\tgit add foo olddir &&\n-\tgit commit -m \"original\" &&\n-)\n+2 git init dir-rename-and-content &&\n+3 (\n+4 \tcd dir-rename-and-content &&\n+5 \ttest_write_lines 1 2 3 4 5 >foo &&\n+6 \tmkdir olddir &&\n+7 \tfor i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!\n+8 \tgit add foo olddir &&\n+9 \tgit commit -m \"original\" &&\n+10 )\ndiff --git a/t/chainlint/one-liner.expect b/t/chainlint/one-liner.expect\nindex 57a7a444c1..9861811283 100644\n--- a/t/chainlint/one-liner.expect\n+++ b/t/chainlint/one-liner.expect\n@@ -1,9 +1,9 @@\n-(foo && bar) &&\n-(foo && bar) |\n-(foo && bar) >baz &&\n-\n-(foo; ?!AMP?! bar) &&\n-(foo; ?!AMP?! bar) |\n-(foo; ?!AMP?! bar) >baz &&\n-\n-(foo \"bar; baz\")\n+2 (foo && bar) &&\n+3 (foo && bar) |\n+4 (foo && bar) >baz &&\n+5 \n+6 (foo; ?!AMP?! bar) &&\n+7 (foo; ?!AMP?! bar) |\n+8 (foo; ?!AMP?! bar) >baz &&\n+9 \n+10 (foo \"bar; baz\")\ndiff --git a/t/chainlint/p4-filespec.expect b/t/chainlint/p4-filespec.expect\nindex 1290fd1ff2..cff3e4e3d1 100644\n--- a/t/chainlint/p4-filespec.expect\n+++ b/t/chainlint/p4-filespec.expect\n@@ -1,4 +1,4 @@\n-(\n-\tp4 print -1 //depot/fiddle#42 >file &&\n-\tfoobar\n-)\n+2 (\n+3 \tp4 print -1 //depot/fiddle#42 >file &&\n+4 \tfoobar\n+5 )\ndiff --git a/t/chainlint/pipe.expect b/t/chainlint/pipe.expect\nindex 811971b1a3..1bbe5a2ce1 100644\n--- a/t/chainlint/pipe.expect\n+++ b/t/chainlint/pipe.expect\n@@ -1,10 +1,10 @@\n-(\n-\tfoo |\n-\tbar |\n-\tbaz &&\n-\n-\tfish |\n-\tcow ?!AMP?!\n-\n-\tsunder\n-)\n+2 (\n+3 \tfoo |\n+4 \tbar |\n+5 \tbaz &&\n+6 \n+7 \tfish |\n+8 \tcow ?!AMP?!\n+9 \n+10 \tsunder\n+11 )\ndiff --git a/t/chainlint/return-loop.expect b/t/chainlint/return-loop.expect\nindex cfc0549bef..da8f9abea3 100644\n--- a/t/chainlint/return-loop.expect\n+++ b/t/chainlint/return-loop.expect\n@@ -1,5 +1,5 @@\n-while test $i -lt $((num - 5))\n-do\n-\tgit notes add -m \"notes for commit$i\" HEAD~$i || return 1\n-\ti=$((i + 1))\n-done\n+2 while test $i -lt $((num - 5))\n+3 do\n+4 \tgit notes add -m \"notes for commit$i\" HEAD~$i || return 1\n+5 \ti=$((i + 1))\n+6 done\ndiff --git a/t/chainlint/semicolon.expect b/t/chainlint/semicolon.expect\nindex 3aa2259f36..866438310c 100644\n--- a/t/chainlint/semicolon.expect\n+++ b/t/chainlint/semicolon.expect\n@@ -1,19 +1,19 @@\n-(\n-\tcat foo ; ?!AMP?! echo bar ?!AMP?!\n-\tcat foo ; ?!AMP?! echo bar\n-) &&\n-(\n-\tcat foo ; ?!AMP?! echo bar &&\n-\tcat foo ; ?!AMP?! echo bar\n-) &&\n-(\n-\techo \"foo; bar\" &&\n-\tcat foo; ?!AMP?! echo bar\n-) &&\n-(\n-\tfoo;\n-) &&\n-(cd foo &&\n-\tfor i in a b c; do\n-\t\techo; ?!LOOP?!\n-\tdone)\n+2 (\n+3 \tcat foo ; ?!AMP?! echo bar ?!AMP?!\n+4 \tcat foo ; ?!AMP?! echo bar\n+5 ) &&\n+6 (\n+7 \tcat foo ; ?!AMP?! echo bar &&\n+8 \tcat foo ; ?!AMP?! echo bar\n+9 ) &&\n+10 (\n+11 \techo \"foo; bar\" &&\n+12 \tcat foo; ?!AMP?! echo bar\n+13 ) &&\n+14 (\n+15 \tfoo;\n+16 ) &&\n+17 (cd foo &&\n+18 \tfor i in a b c; do\n+19 \t\techo; ?!LOOP?!\n+20 \tdone)\ndiff --git a/t/chainlint/sqstring-in-sqstring.expect b/t/chainlint/sqstring-in-sqstring.expect\nindex cf0b591cf7..ba5d3c3a6d 100644\n--- a/t/chainlint/sqstring-in-sqstring.expect\n+++ b/t/chainlint/sqstring-in-sqstring.expect\n@@ -1,4 +1,4 @@\n-perl -e '\n-\tdefined($_ = -s $_) or die for @ARGV;\n-\texit 1 if $ARGV[0] <= $ARGV[1];\n-' test-2-$packname_2.pack test-3-$packname_3.pack\n+2 perl -e '\n+3 \tdefined($_ = -s $_) or die for @ARGV;\n+4 \texit 1 if $ARGV[0] <= $ARGV[1];\n+5 ' test-2-$packname_2.pack test-3-$packname_3.pack\ndiff --git a/t/chainlint/subshell-here-doc.expect b/t/chainlint/subshell-here-doc.expect\nindex 75d6f607e2..5647500c82 100644\n--- a/t/chainlint/subshell-here-doc.expect\n+++ b/t/chainlint/subshell-here-doc.expect\n@@ -1,30 +1,30 @@\n-(\n-\techo wobba \\\n-\t       gorgo snoot \\\n-\t       wafta snurb <<-EOF &&\n-\tquoth the raven,\n-\tnevermore...\n-\tEOF\n-\n-\tcat <<EOF >bip ?!AMP?!\n-\tfish fly high\n-EOF\n-\n-\techo <<-\\EOF >bop\n-\tgomez\n-\tmorticia\n-\twednesday\n-\tpugsly\n-\tEOF\n-) &&\n-(\n-\tcat <<-\\ARBITRARY >bup &&\n-\tglink\n-\tFIZZ\n-\tARBITRARY\n-\tcat <<-\"ARBITRARY3\" >bup3 &&\n-\tglink\n-\tFIZZ\n-\tARBITRARY3\n-\tmeep\n-)\n+2 (\n+3 \techo wobba \\\n+4 \t       gorgo snoot \\\n+5 \t       wafta snurb <<-EOF &&\n+6 \tquoth the raven,\n+7 \tnevermore...\n+8 \tEOF\n+9 \n+10 \tcat <<EOF >bip ?!AMP?!\n+11 \tfish fly high\n+12 EOF\n+13 \n+14 \techo <<-\\EOF >bop\n+15 \tgomez\n+16 \tmorticia\n+17 \twednesday\n+18 \tpugsly\n+19 \tEOF\n+20 ) &&\n+21 (\n+22 \tcat <<-\\ARBITRARY >bup &&\n+23 \tglink\n+24 \tFIZZ\n+25 \tARBITRARY\n+26 \tcat <<-\"ARBITRARY3\" >bup3 &&\n+27 \tglink\n+28 \tFIZZ\n+29 \tARBITRARY3\n+30 \tmeep\n+31 )\ndiff --git a/t/chainlint/subshell-one-liner.expect b/t/chainlint/subshell-one-liner.expect\nindex 8f694990e8..214316c6a0 100644\n--- a/t/chainlint/subshell-one-liner.expect\n+++ b/t/chainlint/subshell-one-liner.expect\n@@ -1,19 +1,19 @@\n-(\n-\t(foo && bar) &&\n-\t(foo && bar) |\n-\t(foo && bar) >baz &&\n-\n-\t(foo; ?!AMP?! bar) &&\n-\t(foo; ?!AMP?! bar) |\n-\t(foo; ?!AMP?! bar) >baz &&\n-\n-\t(foo || exit 1) &&\n-\t(foo || exit 1) |\n-\t(foo || exit 1) >baz &&\n-\n-\t(foo && bar) ?!AMP?!\n-\n-\t(foo && bar; ?!AMP?! baz) ?!AMP?!\n-\n-\tfoobar\n-)\n+2 (\n+3 \t(foo && bar) &&\n+4 \t(foo && bar) |\n+5 \t(foo && bar) >baz &&\n+6 \n+7 \t(foo; ?!AMP?! bar) &&\n+8 \t(foo; ?!AMP?! bar) |\n+9 \t(foo; ?!AMP?! bar) >baz &&\n+10 \n+11 \t(foo || exit 1) &&\n+12 \t(foo || exit 1) |\n+13 \t(foo || exit 1) >baz &&\n+14 \n+15 \t(foo && bar) ?!AMP?!\n+16 \n+17 \t(foo && bar; ?!AMP?! baz) ?!AMP?!\n+18 \n+19 \tfoobar\n+20 )\ndiff --git a/t/chainlint/t7900-subtree.expect b/t/chainlint/t7900-subtree.expect\nindex 02f3129232..9e60338bcf 100644\n--- a/t/chainlint/t7900-subtree.expect\n+++ b/t/chainlint/t7900-subtree.expect\n@@ -1,22 +1,22 @@\n-(\n-\tchks=\"sub1\n-sub2\n-sub3\n-sub4\" &&\n-\tchks_sub=$(cat <<TXT | sed \"s,^,sub dir/,\"\n-$chks\n-TXT\n-) &&\n-\tchkms=\"main-sub1\n-main-sub2\n-main-sub3\n-main-sub4\" &&\n-\tchkms_sub=$(cat <<TXT | sed \"s,^,sub dir/,\"\n-$chkms\n-TXT\n-) &&\n-\n-\tsubfiles=$(git ls-files) &&\n-\tcheck_equal \"$subfiles\" \"$chkms\n-$chks\"\n-)\n+2 (\n+3 \tchks=\"sub1\n+4 sub2\n+5 sub3\n+6 sub4\" &&\n+7 \tchks_sub=$(cat <<TXT | sed \"s,^,sub dir/,\"\n+8 $chks\n+9 TXT\n+10 ) &&\n+11 \tchkms=\"main-sub1\n+12 main-sub2\n+13 main-sub3\n+14 main-sub4\" &&\n+15 \tchkms_sub=$(cat <<TXT | sed \"s,^,sub dir/,\"\n+16 $chkms\n+17 TXT\n+18 ) &&\n+19 \n+20 \tsubfiles=$(git ls-files) &&\n+21 \tcheck_equal \"$subfiles\" \"$chkms\n+22 $chks\"\n+23 )\ndiff --git a/t/chainlint/token-pasting.expect b/t/chainlint/token-pasting.expect\nindex 6a387917a7..64f3235d26 100644\n--- a/t/chainlint/token-pasting.expect\n+++ b/t/chainlint/token-pasting.expect\n@@ -1,27 +1,27 @@\n-git config filter.rot13.smudge ./rot13.sh &&\n-git config filter.rot13.clean ./rot13.sh &&\n-\n-{\n-    echo \"*.t filter=rot13\" ?!AMP?!\n-    echo \"*.i ident\"\n-} >.gitattributes &&\n-\n-{\n-    echo a b c d e f g h i j k l m ?!AMP?!\n-    echo n o p q r s t u v w x y z ?!AMP?!\n-    echo '$Id$'\n-} >test &&\n-cat test >test.t &&\n-cat test >test.o &&\n-cat test >test.i &&\n-git add test test.t test.i &&\n-rm -f test test.t test.i &&\n-git checkout -- test test.t test.i &&\n-\n-echo \"content-test2\" >test2.o &&\n-echo \"content-test3 - filename with special characters\" >\"test3 'sq',$x=.o\" ?!AMP?!\n-\n-downstream_url_for_sed=$(\n-\tprintf \"%sn\" \"$downstream_url\" |\n-\tsed -e 's/\\/\\\\/g' -e 's/[[/.*^$]/\\&/g'\n-)\n+2 git config filter.rot13.smudge ./rot13.sh &&\n+3 git config filter.rot13.clean ./rot13.sh &&\n+4 \n+5 {\n+6     echo \"*.t filter=rot13\" ?!AMP?!\n+7     echo \"*.i ident\"\n+8 } >.gitattributes &&\n+9 \n+10 {\n+11     echo a b c d e f g h i j k l m ?!AMP?!\n+12     echo n o p q r s t u v w x y z ?!AMP?!\n+13     echo '$Id$'\n+14 } >test &&\n+15 cat test >test.t &&\n+16 cat test >test.o &&\n+17 cat test >test.i &&\n+18 git add test test.t test.i &&\n+19 rm -f test test.t test.i &&\n+20 git checkout -- test test.t test.i &&\n+21 \n+22 echo \"content-test2\" >test2.o &&\n+23 echo \"content-test3 - filename with special characters\" >\"test3 'sq',$x=.o\" ?!AMP?!\n+24 \n+25 downstream_url_for_sed=$(\n+26 \tprintf \"%sn\" \"$downstream_url\" |\n+27 \tsed -e 's/\\/\\\\/g' -e 's/[[/.*^$]/\\&/g'\n+28 )\ndiff --git a/t/chainlint/unclosed-here-doc-indent.expect b/t/chainlint/unclosed-here-doc-indent.expect\nindex 7c30a1a024..f78e23cb63 100644\n--- a/t/chainlint/unclosed-here-doc-indent.expect\n+++ b/t/chainlint/unclosed-here-doc-indent.expect\n@@ -1,4 +1,4 @@\n-command_which_is_run &&\n-cat >expect <<-\\EOF ?!UNCLOSED-HEREDOC?! &&\n-we forget to end the here-doc\n-command_which_is_gobbled\n+2 command_which_is_run &&\n+3 cat >expect <<-\\EOF ?!UNCLOSED-HEREDOC?! &&\n+4 we forget to end the here-doc\n+5 command_which_is_gobbled\ndiff --git a/t/chainlint/unclosed-here-doc.expect b/t/chainlint/unclosed-here-doc.expect\nindex d65e50f78d..51304672cf 100644\n--- a/t/chainlint/unclosed-here-doc.expect\n+++ b/t/chainlint/unclosed-here-doc.expect\n@@ -1,7 +1,7 @@\n-command_which_is_run &&\n-cat >expect <<\\EOF ?!UNCLOSED-HEREDOC?! &&\n-\twe try to end the here-doc below,\n-\tbut the indentation throws us off\n-\tsince the operator is not \"<<-\".\n-\tEOF\n-command_which_is_gobbled\n+2 command_which_is_run &&\n+3 cat >expect <<\\EOF ?!UNCLOSED-HEREDOC?! &&\n+4 \twe try to end the here-doc below,\n+5 \tbut the indentation throws us off\n+6 \tsince the operator is not \"<<-\".\n+7 \tEOF\n+8 command_which_is_gobbled\ndiff --git a/t/chainlint/while-loop.expect b/t/chainlint/while-loop.expect\nindex 06c1567f48..5ffabd5a93 100644\n--- a/t/chainlint/while-loop.expect\n+++ b/t/chainlint/while-loop.expect\n@@ -1,14 +1,14 @@\n-(\n-\twhile true\n-\tdo\n-\t\techo foo ?!AMP?!\n-\t\tcat <<-\\EOF ?!LOOP?!\n-\t\tbar\n-\t\tEOF\n-\tdone ?!AMP?!\n-\n-\twhile true; do\n-\t\techo foo &&\n-\t\tcat bar ?!LOOP?!\n-\tdone\n-)\n+2 (\n+3 \twhile true\n+4 \tdo\n+5 \t\techo foo ?!AMP?!\n+6 \t\tcat <<-\\EOF ?!LOOP?!\n+7 \t\tbar\n+8 \t\tEOF\n+9 \tdone ?!AMP?!\n+10 \n+11 \twhile true; do\n+12 \t\techo foo &&\n+13 \t\tcat bar ?!LOOP?!\n+14 \tdone\n+15 )\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498424","messageId":"20240710083831.GF2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 6/9] chainlint.pl: recognize test bodies defined via heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:38:31Z","receivedAt":"2024-07-10T08:38:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"From: Eric Sunshine <sunshine@sunshineco.com>\n\nIn order to check tests for semantic problems, chainlint.pl scans test\nscripts, looking for tests defined as:\n\n    test_expect_success [prereq] title '\n        body\n    '\n\nwhere `body` is a single string which is then treated as a standalone\nchunk of code and \"linted\" to detect semantic issues. (The same happens\nfor `test_expect_failure` definitions.)\n\nThe introduction of test definitions in which the test body is instead\npresented via a heredoc rather than as a single string creates a blind\nspot in the linting process since such invocations are not recognized by\nchainlint.pl.\n\nPrepare for this new style by also recognizing tests defined as:\n\n    test_expect_success [prereq] title - <<\\EOT\n        body\n    EOT\n\nA minor complication is that chainlint.pl has never considered heredoc\nbodies significant since it doesn't scan them for semantic problems,\nthus it has always simply thrown them away. However, with the new\n`test_expect_success` calling sequence, heredoc bodies become\nmeaningful, thus need to be captured.\n\nSigned-off-by: Eric Sunshine <sunshine@sunshineco.com>\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/chainlint.pl | 27 ++++++++++++++++++++++-----\n 1 file changed, 22 insertions(+), 5 deletions(-)\n\ndiff --git a/t/chainlint.pl b/t/chainlint.pl\nindex fb749d3d5c..5361f23b1d 100755\n--- a/t/chainlint.pl\n+++ b/t/chainlint.pl\n@@ -174,6 +174,10 @@ sub swallow_heredocs {\n \t\t$$b =~ /(?:\\G|\\n)$indent\\Q$$tag[0]\\E(?:\\n|\\z)/gc;\n \t\tif (pos($$b) > $start) {\n \t\t\tmy $body = substr($$b, $start, pos($$b) - $start);\n+\t\t\t$self->{parser}->{heredocs}->{$$tag[0]} = {\n+\t\t\t\tcontent => substr($body, 0, length($body) - length($&)),\n+\t\t\t\tstart_line => $self->{lineno},\n+\t\t        };\n \t\t\t$self->{lineno} += () = $body =~ /\\n/sg;\n \t\t\tnext;\n \t\t}\n@@ -232,7 +236,8 @@ sub new {\n \tmy $self = bless {\n \t\tbuff => [],\n \t\tstop => [],\n-\t\toutput => []\n+\t\toutput => [],\n+\t\theredocs => {},\n \t} => $class;\n \t$self->{lexer} = Lexer->new($self, $s);\n \treturn $self;\n@@ -616,14 +621,21 @@ sub unwrap {\n \n sub check_test {\n \tmy $self = shift @_;\n-\tmy ($title, $body) = map(unwrap, @_);\n+\tmy $title = unwrap(shift @_);\n+\tmy $body = shift @_;\n+\tmy $lineno = $body->[3];\n+\t$body = unwrap($body);\n+\tif ($body eq '-') {\n+\t\tmy $herebody = shift @_;\n+\t\t$body = $herebody->{content};\n+\t\t$lineno = $herebody->{start_line};\n+\t}\n \t$self->{ntests}++;\n \tmy $parser = TestParser->new(\\$body);\n \tmy @tokens = $parser->parse();\n \tmy $problems = $parser->{problems};\n \treturn unless $emit_all || @$problems;\n \tmy $c = main::fd_colors(1);\n-\tmy $lineno = $_[1]->[3];\n \tmy $start = 0;\n \tmy $checked = '';\n \tfor (sort {$a->[1]->[2] <=> $b->[1]->[2]} @$problems) {\n@@ -649,8 +661,13 @@ sub parse_cmd {\n \treturn @tokens unless @tokens && $tokens[0]->[0] =~ /^test_expect_(?:success|failure)$/;\n \tmy $n = $#tokens;\n \t$n-- while $n >= 0 && $tokens[$n]->[0] =~ /^(?:[;&\\n|]|&&|\\|\\|)$/;\n-\t$self->check_test($tokens[1], $tokens[2]) if $n == 2; # title body\n-\t$self->check_test($tokens[2], $tokens[3]) if $n > 2;  # prereq title body\n+\tmy $herebody;\n+\tif ($n >= 2 && $tokens[$n-1]->[0] eq '-' && $tokens[$n]->[0] =~ /^<<-?(.+)$/) {\n+\t\t$herebody = $self->{heredocs}->{$1};\n+\t\t$n--;\n+\t}\n+\t$self->check_test($tokens[1], $tokens[2], $herebody) if $n == 2; # title body\n+\t$self->check_test($tokens[2], $tokens[3], $herebody) if $n > 2;  # prereq title body\n \treturn @tokens;\n }\n \n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498425","messageId":"20240710083916.GG2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 7/9] chainlint.pl: add tests for test body in heredoc","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:39:16Z","receivedAt":"2024-07-10T08:39:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The chainlint.pl script recently learned about the upcoming:\n\n  test_expect_success 'some test' - <<\\EOT\n\tTEST_BODY\n  EOT\n\nsyntax, where TEST_BODY should be checked in the usual way. Let's make\nsure this works by adding a few tests. The \"here-doc-body\" file tests\nthe basic syntax, including an embedded here-doc which we should still\nbe able to recognize.\n\nLikewise the \"here-doc-body-indent\" checks the same thing, but using the\n\"<<-\" operator. We wouldn't expect this to be used normally, but we\nwould not want to accidentally miss a body that uses it. The\n\"pathological\" variant checks the opposite: we don't get confused by an\nindented tag within the here-doc body.\n\nThe \"here-doc-double\" tests the handling of two here-doc tags on the\nsame line. This is not something we'd expect anybody to do in practice,\nbut the code was written defensively to handle this, so let's make sure\nit works.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/chainlint/here-doc-body-indent.expect       |  2 ++\n t/chainlint/here-doc-body-indent.test         |  4 ++++\n t/chainlint/here-doc-body-pathological.expect |  7 +++++++\n t/chainlint/here-doc-body-pathological.test   |  9 +++++++++\n t/chainlint/here-doc-body.expect              |  7 +++++++\n t/chainlint/here-doc-body.test                |  9 +++++++++\n t/chainlint/here-doc-double.expect            |  2 ++\n t/chainlint/here-doc-double.test              | 10 ++++++++++\n 8 files changed, 50 insertions(+)\n create mode 100644 t/chainlint/here-doc-body-indent.expect\n create mode 100644 t/chainlint/here-doc-body-indent.test\n create mode 100644 t/chainlint/here-doc-body-pathological.expect\n create mode 100644 t/chainlint/here-doc-body-pathological.test\n create mode 100644 t/chainlint/here-doc-body.expect\n create mode 100644 t/chainlint/here-doc-body.test\n create mode 100644 t/chainlint/here-doc-double.expect\n create mode 100644 t/chainlint/here-doc-double.test\n\ndiff --git a/t/chainlint/here-doc-body-indent.expect b/t/chainlint/here-doc-body-indent.expect\nnew file mode 100644\nindex 0000000000..4323acc93d\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-indent.expect\n@@ -0,0 +1,2 @@\n+2 \techo \"we should find this\" ?!AMP?!\n+3 \techo \"even though our heredoc has its indent stripped\"\ndiff --git a/t/chainlint/here-doc-body-indent.test b/t/chainlint/here-doc-body-indent.test\nnew file mode 100644\nindex 0000000000..39ff970ef3\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-indent.test\n@@ -0,0 +1,4 @@\n+test_expect_success 'here-doc-body-indent' - <<-\\EOT\n+\techo \"we should find this\"\n+\techo \"even though our heredoc has its indent stripped\"\n+EOT\ndiff --git a/t/chainlint/here-doc-body-pathological.expect b/t/chainlint/here-doc-body-pathological.expect\nnew file mode 100644\nindex 0000000000..a93a1fa3aa\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-pathological.expect\n@@ -0,0 +1,7 @@\n+2 \techo \"outer here-doc does not allow indented end-tag\" ?!AMP?!\n+3 \tcat >file <<-\\EOF &&\n+4 \tbut this inner here-doc\n+5 \tdoes allow indented EOF\n+6 \tEOF\n+7 \techo \"missing chain after\" ?!AMP?!\n+8 \techo \"but this line is OK because it's the end\"\ndiff --git a/t/chainlint/here-doc-body-pathological.test b/t/chainlint/here-doc-body-pathological.test\nnew file mode 100644\nindex 0000000000..7d2daa44f9\n--- /dev/null\n+++ b/t/chainlint/here-doc-body-pathological.test\n@@ -0,0 +1,9 @@\n+test_expect_success 'here-doc-body-pathological' - <<\\EOF\n+\techo \"outer here-doc does not allow indented end-tag\"\n+\tcat >file <<-\\EOF &&\n+\tbut this inner here-doc\n+\tdoes allow indented EOF\n+\tEOF\n+\techo \"missing chain after\"\n+\techo \"but this line is OK because it's the end\"\n+EOF\ndiff --git a/t/chainlint/here-doc-body.expect b/t/chainlint/here-doc-body.expect\nnew file mode 100644\nindex 0000000000..ddf1c412af\n--- /dev/null\n+++ b/t/chainlint/here-doc-body.expect\n@@ -0,0 +1,7 @@\n+2 \techo \"missing chain before\" ?!AMP?!\n+3 \tcat >file <<-\\EOF &&\n+4 \tinside inner here-doc\n+5 \tthese are not shell commands\n+6 \tEOF\n+7 \techo \"missing chain after\" ?!AMP?!\n+8 \techo \"but this line is OK because it's the end\"\ndiff --git a/t/chainlint/here-doc-body.test b/t/chainlint/here-doc-body.test\nnew file mode 100644\nindex 0000000000..989ac2f4e1\n--- /dev/null\n+++ b/t/chainlint/here-doc-body.test\n@@ -0,0 +1,9 @@\n+test_expect_success 'here-doc-body' - <<\\EOT\n+\techo \"missing chain before\"\n+\tcat >file <<-\\EOF &&\n+\tinside inner here-doc\n+\tthese are not shell commands\n+\tEOF\n+\techo \"missing chain after\"\n+\techo \"but this line is OK because it's the end\"\n+EOT\ndiff --git a/t/chainlint/here-doc-double.expect b/t/chainlint/here-doc-double.expect\nnew file mode 100644\nindex 0000000000..20dba4b452\n--- /dev/null\n+++ b/t/chainlint/here-doc-double.expect\n@@ -0,0 +1,2 @@\n+8 \techo \"actual test commands\" ?!AMP?!\n+9 \techo \"that should be checked\"\ndiff --git a/t/chainlint/here-doc-double.test b/t/chainlint/here-doc-double.test\nnew file mode 100644\nindex 0000000000..777389f0d9\n--- /dev/null\n+++ b/t/chainlint/here-doc-double.test\n@@ -0,0 +1,10 @@\n+# This is obviously a ridiculous thing to do, but we should be able\n+# to handle two here-docs on the same line, and attribute them\n+# correctly.\n+test_expect_success \"$(cat <<END_OF_PREREQS)\" 'here-doc-double' - <<\\EOT\n+SOME\n+PREREQS\n+END_OF_PREREQS\n+\techo \"actual test commands\"\n+\techo \"that should be checked\"\n+EOT\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498426","messageId":"20240710083929.GH2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 8/9] test-lib: allow test snippets as here-docs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:39:29Z","receivedAt":"2024-07-10T08:39:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Most test snippets are wrapped in single quotes, like:\n\n  test_expect_success 'some description' '\n          do_something\n  '\n\nThis sometimes makes the snippets awkward to write, because you can't\neasily use single quotes within them. We sometimes work around this with\n$SQ, or by loosening regexes to use \".\" instead of a literal quote, or\nby using double quotes when we'd prefer to use single-quotes (and just\nadding extra backslash-escapes to avoid interpolation).\n\nThis commit adds another option: feeding the snippet via the function's\nstdin. This doesn't conflict with anything the snippet would want to do,\nbecause we always redirect its stdin from /dev/null anyway (which we'll\ncontinue to do).\n\nA few notes on the implementation:\n\n  - it would be nice to push this down into test_run_, but we can't, as\n    test_expect_success and test_expect_failure want to see the actual\n    script content to report it for verbose-mode. A helper function\n    limits the amount of duplication in those callers here.\n\n  - The helper function is a little awkward to call, as you feed it the\n    name of the variable you want to set. The more natural thing in\n    shell would be command substitution like:\n\n      body=$(body_or_stdin \"$2\")\n\n    but that loses trailing whitespace. There are tricks around this,\n    like:\n\n      body=$(body_or_stdin \"$2\"; printf .)\n      body=${body%.}\n\n    but we'd prefer to keep such tricks in the helper, not in each\n    caller.\n\n  - I implemented the helper using a sequence of \"read\" calls. Together\n    with \"-r\" and unsetting the IFS, this preserves incoming whitespace.\n    An alternative is to use \"cat\" (which then requires the gross \".\"\n    trick above). But this saves us a process, which is probably a good\n    thing. The \"read\" builtin does use more read() syscalls than\n    necessary (one per byte), but that is almost certainly a win over a\n    separate process.\n\n    Both are probably slower than passing a single-quoted string, but\n    the difference is lost in the noise for a script that I converted as\n    an experiment.\n\n  - I handle test_expect_success and test_expect_failure here. If we\n    like this style, we could easily extend it to other spots (e.g.,\n    lazy_prereq bodies) on top of this patch.\n\n  - even though we are using \"local\", we have to be careful about our\n    variable names. Within test_expect_success, any variable we declare\n    with local will be seen as local by the test snippets themselves (so\n    it wouldn't persist between tests like normal variables would).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/README                |  8 ++++++++\n t/test-lib-functions.sh | 32 +++++++++++++++++++++++++++-----\n 2 files changed, 35 insertions(+), 5 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex d9e0e07506..dec644f997 100644\n--- a/t/README\n+++ b/t/README\n@@ -906,6 +906,14 @@ see test-lib-functions.sh for the full list and their options.\n \t    'git-write-tree should be able to write an empty tree.' \\\n \t    'tree=$(git-write-tree)'\n \n+   If <script> is `-` (a single dash), then the script to run is read\n+   from stdin. This lets you more easily use single quotes within the\n+   script by using a here-doc. For example:\n+\n+\ttest_expect_success 'output contains expected string' - <<\\EOT\n+\t\tgrep \"this string has 'quotes' in it\" output\n+\tEOT\n+\n    If you supply three parameters the first will be taken to be a\n    prerequisite; see the test_set_prereq and test_have_prereq\n    documentation below:\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 1ea9f31225..fde9bf54fc 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -872,6 +872,24 @@ test_verify_prereq () {\n \tBUG \"'$test_prereq' does not look like a prereq\"\n }\n \n+# assign the variable named by \"$1\" with the contents of \"$2\";\n+# if \"$2\" is \"-\", then read stdin into \"$1\" instead\n+test_body_or_stdin () {\n+\tif test \"$2\" != \"-\"\n+\tthen\n+\t\teval \"$1=\\$2\"\n+\t\treturn\n+\tfi\n+\n+\t# start with a newline, to match hanging newline from open-quote style\n+\teval \"$1=\\$LF\"\n+\tlocal test_line\n+\twhile IFS= read -r test_line\n+\tdo\n+\t\teval \"$1=\\${$1}\\${test_line}\\${LF}\"\n+\tdone\n+}\n+\n test_expect_failure () {\n \ttest_start_ \"$@\"\n \ttest \"$#\" = 3 && { test_prereq=$1; shift; } || test_prereq=\n@@ -881,9 +899,11 @@ test_expect_failure () {\n \texport test_prereq\n \tif ! test_skip \"$@\"\n \tthen\n+\t\tlocal test_body\n+\t\ttest_body_or_stdin test_body \"$2\"\n \t\ttest -n \"$test_skip_test_preamble\" ||\n-\t\tsay >&3 \"checking known breakage of $TEST_NUMBER.$test_count '$1': $2\"\n-\t\tif test_run_ \"$2\" expecting_failure\n+\t\tsay >&3 \"checking known breakage of $TEST_NUMBER.$test_count '$1': $test_body\"\n+\t\tif test_run_ \"$test_body\" expecting_failure\n \t\tthen\n \t\t\ttest_known_broken_ok_ \"$1\"\n \t\telse\n@@ -902,13 +922,15 @@ test_expect_success () {\n \texport test_prereq\n \tif ! test_skip \"$@\"\n \tthen\n+\t\tlocal test_body\n+\t\ttest_body_or_stdin test_body \"$2\"\n \t\ttest -n \"$test_skip_test_preamble\" ||\n-\t\tsay >&3 \"expecting success of $TEST_NUMBER.$test_count '$1': $2\"\n-\t\tif test_run_ \"$2\"\n+\t\tsay >&3 \"expecting success of $TEST_NUMBER.$test_count '$1': $test_body\"\n+\t\tif test_run_ \"$test_body\"\n \t\tthen\n \t\t\ttest_ok_ \"$1\"\n \t\telse\n-\t\t\ttest_failure_ \"$@\"\n+\t\t\ttest_failure_ \"$1\" \"$test_body\"\n \t\tfi\n \tfi\n \ttest_finish_\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498427","messageId":"20240710083931.GI2060601@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 9/9] t: convert some here-doc test bodies","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:39:31Z","receivedAt":"2024-07-10T08:39:33Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"The t1404 script checks a lot of output from Git which contains single\nquotes. Because the test snippets are themselves wrapped in the same\nsingle-quotes, we have to resort to using $SQ to match them.  This is\nerror-prone and makes the tests harder to read.\n\nInstead, let's use the new here-doc feature added in the previous\ncommit, which lets us write anything in the test body we want (except\nthe here-doc end marker on a line by itself, of course).\n\nNote that we do use \"\\\" in our marker to avoid interpolation (which is\nthe whole point). But we don't use \"<<-\", as we want to preserve\nwhitespace in the snippet (and running with \"-v\" before and after shows\nthat we produce the exact same output, except with the ugly $SQ\nreferences fixed).\n\nI just converted every test here, even though only some of them use\n$SQ. But it would be equally correct to mix-and-match styles if we don't\nmind the inconsistency.\n\nI've also converted a few tests in t0600 which were moved from t1404 (I\nhad written this patch before they were moved, but it seemed worth\nporting over the changes rather than losing them).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/t0600-reffiles-backend.sh  |  38 +++----\n t/t1404-update-ref-errors.sh | 196 +++++++++++++++++------------------\n 2 files changed, 117 insertions(+), 117 deletions(-)\n\ndiff --git a/t/t0600-reffiles-backend.sh b/t/t0600-reffiles-backend.sh\nindex b2a771ff2b..20df336cc3 100755\n--- a/t/t0600-reffiles-backend.sh\n+++ b/t/t0600-reffiles-backend.sh\n@@ -91,82 +91,82 @@ test_expect_success 'empty directory should not fool 1-arg delete' '\n \tgit update-ref --stdin\n '\n \n-test_expect_success 'non-empty directory blocks create' '\n+test_expect_success 'non-empty directory blocks create' - <<\\EOT\n \tprefix=refs/ne-create &&\n \tmkdir -p .git/$prefix/foo/bar &&\n \t: >.git/$prefix/foo/bar/baz.lock &&\n \ttest_when_finished \"rm -f .git/$prefix/foo/bar/baz.lock\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: there is a non-empty directory $SQ.git/$prefix/foo$SQ blocking reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': there is a non-empty directory '.git/$prefix/foo' blocking reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'broken reference blocks create' '\n+test_expect_success 'broken reference blocks create' - <<\\EOT\n \tprefix=refs/broken-create &&\n \tmkdir -p .git/$prefix &&\n \techo \"gobbledigook\" >.git/$prefix/foo &&\n \ttest_when_finished \"rm -f .git/$prefix/foo\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'non-empty directory blocks indirect create' '\n+test_expect_success 'non-empty directory blocks indirect create' - <<\\EOT\n \tprefix=refs/ne-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tmkdir -p .git/$prefix/foo/bar &&\n \t: >.git/$prefix/foo/bar/baz.lock &&\n \ttest_when_finished \"rm -f .git/$prefix/foo/bar/baz.lock\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: there is a non-empty directory $SQ.git/$prefix/foo$SQ blocking reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': there is a non-empty directory '.git/$prefix/foo' blocking reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'broken reference blocks indirect create' '\n+test_expect_success 'broken reference blocks indirect create' - <<\\EOT\n \tprefix=refs/broken-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \techo \"gobbledigook\" >.git/$prefix/foo &&\n \ttest_when_finished \"rm -f .git/$prefix/foo\" &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ: reference broken\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo': reference broken\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $D $C\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n test_expect_success 'no bogus intermediate values during delete' '\n \tprefix=refs/slow-transaction &&\n@@ -224,7 +224,7 @@ test_expect_success 'no bogus intermediate values during delete' '\n \ttest_must_fail git rev-parse --verify --quiet $prefix/foo\n '\n \n-test_expect_success 'delete fails cleanly if packed-refs file is locked' '\n+test_expect_success 'delete fails cleanly if packed-refs file is locked' - <<\\EOT\n \tprefix=refs/locked-packed-refs &&\n \t# Set up a reference with differing loose and packed versions:\n \tgit update-ref $prefix/foo $C &&\n@@ -236,9 +236,9 @@ test_expect_success 'delete fails cleanly if packed-refs file is locked' '\n \ttest_when_finished \"rm -f .git/packed-refs.lock\" &&\n \ttest_must_fail git update-ref -d $prefix/foo >out 2>err &&\n \tgit for-each-ref $prefix >actual &&\n-\ttest_grep \"Unable to create $SQ.*packed-refs.lock$SQ: \" err &&\n+\ttest_grep \"Unable to create '.*packed-refs.lock': \" err &&\n \ttest_cmp unchanged actual\n-'\n+EOT\n \n test_expect_success 'delete fails cleanly if packed-refs.new write fails' '\n \t# Setup and expectations are similar to the test above.\ndiff --git a/t/t1404-update-ref-errors.sh b/t/t1404-update-ref-errors.sh\nindex 67ebd81a4c..df90112618 100755\n--- a/t/t1404-update-ref-errors.sh\n+++ b/t/t1404-update-ref-errors.sh\n@@ -100,297 +100,297 @@ df_test() {\n \t\tprintf \"%s\\n\" \"delete $delname\" \"create $addname $D\"\n \tfi >commands &&\n \ttest_must_fail git update-ref --stdin <commands 2>output.err &&\n-\tgrep -E \"fatal:( cannot lock ref $SQ$addname$SQ:)? $SQ$delref$SQ exists; cannot create $SQ$addref$SQ\" output.err &&\n+\tgrep -E \"fatal:( cannot lock ref '$addname':)? '$delref' exists; cannot create '$addref'\" output.err &&\n \tprintf \"%s\\n\" \"$C $delref\" >expected-refs &&\n \tgit for-each-ref --format=\"%(objectname) %(refname)\" $prefix/r >actual-refs &&\n \ttest_cmp expected-refs actual-refs\n }\n \n-test_expect_success 'setup' '\n+test_expect_success 'setup' - <<\\EOT\n \n \tgit commit --allow-empty -m Initial &&\n \tC=$(git rev-parse HEAD) &&\n \tgit commit --allow-empty -m Second &&\n \tD=$(git rev-parse HEAD) &&\n \tgit commit --allow-empty -m Third &&\n \tE=$(git rev-parse HEAD)\n-'\n+EOT\n \n-test_expect_success 'existing loose ref is a simple prefix of new' '\n+test_expect_success 'existing loose ref is a simple prefix of new' - <<\\EOT\n \n \tprefix=refs/1l &&\n \ttest_update_rejected \"a c e\" false \"b c/x d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing packed ref is a simple prefix of new' '\n+test_expect_success 'existing packed ref is a simple prefix of new' - <<\\EOT\n \n \tprefix=refs/1p &&\n \ttest_update_rejected \"a c e\" true \"b c/x d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing loose ref is a deeper prefix of new' '\n+test_expect_success 'existing loose ref is a deeper prefix of new' - <<\\EOT\n \n \tprefix=refs/2l &&\n \ttest_update_rejected \"a c e\" false \"b c/x/y d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x/y$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x/y'\"\n \n-'\n+EOT\n \n-test_expect_success 'existing packed ref is a deeper prefix of new' '\n+test_expect_success 'existing packed ref is a deeper prefix of new' - <<\\EOT\n \n \tprefix=refs/2p &&\n \ttest_update_rejected \"a c e\" true \"b c/x/y d\" \\\n-\t\t\"$SQ$prefix/c$SQ exists; cannot create $SQ$prefix/c/x/y$SQ\"\n+\t\t\"'$prefix/c' exists; cannot create '$prefix/c/x/y'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a simple prefix of existing loose' '\n+test_expect_success 'new ref is a simple prefix of existing loose' - <<\\EOT\n \n \tprefix=refs/3l &&\n \ttest_update_rejected \"a c/x e\" false \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a simple prefix of existing packed' '\n+test_expect_success 'new ref is a simple prefix of existing packed' - <<\\EOT\n \n \tprefix=refs/3p &&\n \ttest_update_rejected \"a c/x e\" true \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a deeper prefix of existing loose' '\n+test_expect_success 'new ref is a deeper prefix of existing loose' - <<\\EOT\n \n \tprefix=refs/4l &&\n \ttest_update_rejected \"a c/x/y e\" false \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x/y$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x/y' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'new ref is a deeper prefix of existing packed' '\n+test_expect_success 'new ref is a deeper prefix of existing packed' - <<\\EOT\n \n \tprefix=refs/4p &&\n \ttest_update_rejected \"a c/x/y e\" true \"b c d\" \\\n-\t\t\"$SQ$prefix/c/x/y$SQ exists; cannot create $SQ$prefix/c$SQ\"\n+\t\t\"'$prefix/c/x/y' exists; cannot create '$prefix/c'\"\n \n-'\n+EOT\n \n-test_expect_success 'one new ref is a simple prefix of another' '\n+test_expect_success 'one new ref is a simple prefix of another' - <<\\EOT\n \n \tprefix=refs/5 &&\n \ttest_update_rejected \"a e\" false \"b c c/x d\" \\\n-\t\t\"cannot process $SQ$prefix/c$SQ and $SQ$prefix/c/x$SQ at the same time\"\n+\t\t\"cannot process '$prefix/c' and '$prefix/c/x' at the same time\"\n \n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + delete short' '\n+test_expect_success 'D/F conflict prevents add long + delete short' - <<\\EOT\n \tdf_test refs/df-al-ds --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add short + delete long' '\n+test_expect_success 'D/F conflict prevents add short + delete long' - <<\\EOT\n \tdf_test refs/df-as-dl --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete long + add short' '\n+test_expect_success 'D/F conflict prevents delete long + add short' - <<\\EOT\n \tdf_test refs/df-dl-as --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete short + add long' '\n+test_expect_success 'D/F conflict prevents delete short + add long' - <<\\EOT\n \tdf_test refs/df-ds-al --del-add foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + delete short packed' '\n+test_expect_success 'D/F conflict prevents add long + delete short packed' - <<\\EOT\n \tdf_test refs/df-al-dsp --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add short + delete long packed' '\n+test_expect_success 'D/F conflict prevents add short + delete long packed' - <<\\EOT\n \tdf_test refs/df-as-dlp --pack --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete long packed + add short' '\n+test_expect_success 'D/F conflict prevents delete long packed + add short' - <<\\EOT\n \tdf_test refs/df-dlp-as --pack --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents delete short packed + add long' '\n+test_expect_success 'D/F conflict prevents delete short packed + add long' - <<\\EOT\n \tdf_test refs/df-dsp-al --pack --del-add foo foo/bar\n-'\n+EOT\n \n # Try some combinations involving symbolic refs...\n \n-test_expect_success 'D/F conflict prevents indirect add long + delete short' '\n+test_expect_success 'D/F conflict prevents indirect add long + delete short' - <<\\EOT\n \tdf_test refs/df-ial-ds --sym-add --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + indirect delete short' '\n+test_expect_success 'D/F conflict prevents indirect add long + indirect delete short' - <<\\EOT\n \tdf_test refs/df-ial-ids --sym-add --sym-del --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add short + indirect delete long' '\n+test_expect_success 'D/F conflict prevents indirect add short + indirect delete long' - <<\\EOT\n \tdf_test refs/df-ias-idl --sym-add --sym-del --add-del foo foo/bar\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect delete long + indirect add short' '\n+test_expect_success 'D/F conflict prevents indirect delete long + indirect add short' - <<\\EOT\n \tdf_test refs/df-idl-ias --sym-add --sym-del --del-add foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + delete short packed' '\n+test_expect_success 'D/F conflict prevents indirect add long + delete short packed' - <<\\EOT\n \tdf_test refs/df-ial-dsp --sym-add --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect add long + indirect delete short packed' '\n+test_expect_success 'D/F conflict prevents indirect add long + indirect delete short packed' - <<\\EOT\n \tdf_test refs/df-ial-idsp --sym-add --sym-del --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents add long + indirect delete short packed' '\n+test_expect_success 'D/F conflict prevents add long + indirect delete short packed' - <<\\EOT\n \tdf_test refs/df-al-idsp --sym-del --pack --add-del foo/bar foo\n-'\n+EOT\n \n-test_expect_success 'D/F conflict prevents indirect delete long packed + indirect add short' '\n+test_expect_success 'D/F conflict prevents indirect delete long packed + indirect add short' - <<\\EOT\n \tdf_test refs/df-idlp-ias --sym-add --sym-del --pack --del-add foo/bar foo\n-'\n+EOT\n \n # Test various errors when reading the old values of references...\n \n-test_expect_success 'missing old value blocks update' '\n+test_expect_success 'missing old value blocks update' - <<\\EOT\n \tprefix=refs/missing-update &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/foo': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks update' '\n+test_expect_success 'incorrect old value blocks update' - <<\\EOT\n \tprefix=refs/incorrect-update &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/foo': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/foo $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks create' '\n+test_expect_success 'existing old value blocks create' - <<\\EOT\n \tprefix=refs/existing-create &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/foo': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"create $prefix/foo $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks delete' '\n+test_expect_success 'incorrect old value blocks delete' - <<\\EOT\n \tprefix=refs/incorrect-delete &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/foo$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/foo': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"delete $prefix/foo $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'missing old value blocks indirect update' '\n+test_expect_success 'missing old value blocks indirect update' - <<\\EOT\n \tprefix=refs/missing-indirect-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: unable to resolve reference $SQ$prefix/foo$SQ\n+\tfatal: cannot lock ref '$prefix/symref': unable to resolve reference '$prefix/foo'\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect update' '\n+test_expect_success 'incorrect old value blocks indirect update' - <<\\EOT\n \tprefix=refs/incorrect-indirect-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks indirect create' '\n+test_expect_success 'existing old value blocks indirect create' - <<\\EOT\n \tprefix=refs/existing-indirect-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/symref': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"create $prefix/symref $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect delete' '\n+test_expect_success 'incorrect old value blocks indirect delete' - <<\\EOT\n \tprefix=refs/incorrect-indirect-delete &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"delete $prefix/symref $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'missing old value blocks indirect no-deref update' '\n+test_expect_success 'missing old value blocks indirect no-deref update' - <<\\EOT\n \tprefix=refs/missing-noderef-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference is missing but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': reference is missing but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect no-deref update' '\n+test_expect_success 'incorrect old value blocks indirect no-deref update' - <<\\EOT\n \tprefix=refs/incorrect-noderef-update &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"update $prefix/symref $E $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'existing old value blocks indirect no-deref create' '\n+test_expect_success 'existing old value blocks indirect no-deref create' - <<\\EOT\n \tprefix=refs/existing-noderef-create &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: reference already exists\n+\tfatal: cannot lock ref '$prefix/symref': reference already exists\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"create $prefix/symref $E\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n-test_expect_success 'incorrect old value blocks indirect no-deref delete' '\n+test_expect_success 'incorrect old value blocks indirect no-deref delete' - <<\\EOT\n \tprefix=refs/incorrect-noderef-delete &&\n \tgit symbolic-ref $prefix/symref $prefix/foo &&\n \tgit update-ref $prefix/foo $C &&\n \tcat >expected <<-EOF &&\n-\tfatal: cannot lock ref $SQ$prefix/symref$SQ: is at $C but expected $D\n+\tfatal: cannot lock ref '$prefix/symref': is at $C but expected $D\n \tEOF\n \tprintf \"%s\\n\" \"option no-deref\" \"delete $prefix/symref $D\" |\n \ttest_must_fail git update-ref --stdin 2>output.err &&\n \ttest_cmp expected output.err\n-'\n+EOT\n \n test_done\n-- \n2.45.2.1249.gb036353db5\n"},{"id":"498428","messageId":"20240710084704.GA2061142@coredump.intra.peff.net","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"[PATCH v2 10/9] t/.gitattributes: ignore whitespace in chainlint expect files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-07-10T08:47:04Z","receivedAt":"2024-07-10T08:47:05Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jul 10, 2024 at 04:34:17AM -0400, Jeff King wrote:\n\n>   [5/9]: chainlint.pl: check line numbers in expected output\n\nI just noticed that this one throws off a bunch of whitespace errors\nwhen you apply it. We might want this on top of the series:\n\n-- >8 --\nSubject: t/.gitattributes: ignore whitespace in chainlint expect files\n\nThe \".expect\" files in t/chainlint/ are snippets of expected output from\nthe chainlint script, and do not necessarily conform to our usual code\nstyle. Especially with the recent change to retain line numbers, blank\nlines in the input script end up with trailing whitespace as we print\n\"3 \" for line 3, for example. The point of these files is to match the\noutput verbatim, so let's not complain about the trailing spaces.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/.gitattributes | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/.gitattributes b/t/.gitattributes\nindex b9cea1795d..7664c6e027 100644\n--- a/t/.gitattributes\n+++ b/t/.gitattributes\n@@ -1,5 +1,5 @@\n t[0-9][0-9][0-9][0-9]/* -whitespace\n-/chainlint/*.expect eol=lf\n+/chainlint/*.expect eol=lf -whitespace\n /t0110/url-* binary\n /t3206/* eol=lf\n /t3900/*.txt eol=lf\n-- \n2.45.2.1249.gb036353db5\n\n"},{"id":"498470","messageId":"xmqq5xtdxj5y.fsf@gitster.g","threadId":"61713","inReplyTo":"20240710084704.GA2061142@coredump.intra.peff.net","subject":"Re: [PATCH v2 10/9] t/.gitattributes: ignore whitespace in chainlint expect files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-10T17:15:37Z","receivedAt":"2024-07-10T17:15:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Wed, Jul 10, 2024 at 04:34:17AM -0400, Jeff King wrote:\n>\n>>   [5/9]: chainlint.pl: check line numbers in expected output\n>\n> I just noticed that this one throws off a bunch of whitespace errors\n> when you apply it. We might want this on top of the series:\n\nOr at the bottom ;-)  I do agree with the reasoning.\n\nThanks.\n\n> -- >8 --\n> Subject: t/.gitattributes: ignore whitespace in chainlint expect files\n>\n> The \".expect\" files in t/chainlint/ are snippets of expected output from\n> the chainlint script, and do not necessarily conform to our usual code\n> style. Especially with the recent change to retain line numbers, blank\n> lines in the input script end up with trailing whitespace as we print\n> \"3 \" for line 3, for example. The point of these files is to match the\n> output verbatim, so let's not complain about the trailing spaces.\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n>  t/.gitattributes | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/t/.gitattributes b/t/.gitattributes\n> index b9cea1795d..7664c6e027 100644\n> --- a/t/.gitattributes\n> +++ b/t/.gitattributes\n> @@ -1,5 +1,5 @@\n>  t[0-9][0-9][0-9][0-9]/* -whitespace\n> -/chainlint/*.expect eol=lf\n> +/chainlint/*.expect eol=lf -whitespace\n>  /t0110/url-* binary\n>  /t3206/* eol=lf\n>  /t3900/*.txt eol=lf\n"},{"id":"501404","messageId":"CAPig+cTACjostXvjJMnLEpgbnfat9cjM63pLXwNJm1=2P3gq8g@mail.gmail.com","threadId":"61713","inReplyTo":"20240710083755.GE2060601@coredump.intra.peff.net","subject":"Re: [PATCH v2 5/9] chainlint.pl: check line numbers in expected output","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-08-21T07:00:05Z","receivedAt":"2024-08-21T07:00:19Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Jul 10, 2024 at 4:37 AM Jeff King <peff@peff.net> wrote:\n> [...]\n> It would be possible to do all of this in shell via the Makefile, but it\n> gets a bit complicated (and requires a lot of extra processes). Instead,\n> I've written a short perl script that generates the concatenated files\n> (we already depend on perl, since chainlint.pl uses it). Incidentally,\n> this improves a few other things:\n> [...]\n> diff --git a/t/chainlint-cat.pl b/t/chainlint-cat.pl\n> @@ -0,0 +1,29 @@\n> +#!/usr/bin/env perl\n> +\n> +my $outdir = shift;\n> +open(my $tests, '>', \"$outdir/tests\")\n> +       or die \"unable to open $outdir/tests: $!\";\n> +open(my $expect, '>', \"$outdir/expect\")\n> +       or die \"unable to open $outdir/expect: $!\";\n> +\n> +print $expect \"# chainlint: $outdir/tests\\n\";\n> +\n> +my $offset = 0;\n> +for my $script (@ARGV) {\n> +       print $expect \"# chainlint: $script\\n\";\n> +\n> +       open(my $expect_in, '<', \"chainlint/$script.expect\")\n> +               or die \"unable to open chainlint/$script.expect: $!\";\n> +       while (<$expect_in>) {\n> +               s/^\\d+/$& + $offset/e;\n> +               print $expect $_;\n> +       }\n> +\n> +       open(my $test_in, '<', \"chainlint/$script.test\")\n> +               or die \"unable to open chainlint/$script.test: $!\";\n> +       while (<$test_in>) {\n> +               /^# LINT: / and next;\n> +               print $tests $_;\n> +               $offset++;\n> +       }\n> +}\n\nI'm surprised that we're not closing the two file handles opened on\neach iteration of this loop. Is that intentional? Or am I forgetting\nmy Perl and they are somehow getting closed anyhow (for instance, by\nthe <...> operator hitting EOF)?\n"},{"id":"501407","messageId":"CAPig+cRiVnVX=7y_ZnornQabOh8TaCCud5T0=ynY1+BZEYYNFA@mail.gmail.com","threadId":"61713","inReplyTo":"20240710083416.GA2060328@coredump.intra.peff.net","subject":"Re: [PATCH v2 0/9] here-doc test bodies (now with 100% more chainlinting)","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-08-21T07:31:02Z","receivedAt":"2024-08-21T07:31:14Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Jul 10, 2024 at 4:34 AM Jeff King <peff@peff.net> wrote:\n> And here's a v2 that addresses the chainlint issue mentioned by Eric.\n> There were a lot of patches flying around, but this is only the second\n> posting of the whole topic. This can replace all of what's in\n> jk/test-body-in-here-doc.\n>\n> I won't bother with a range diff; patches 8 and 9 are just the original\n> v1 patches verbatim, and everything else is new.\n\nSorry for the late response; I only just got around to reading this\nseries (which, I know, is already in \"master\").\n\nWith the possible exception of one question I asked[*] about patch\n5/9, the entire series looks good. Thanks.\n\n[*]: https://lore.kernel.org/git/CAPig+cTACjostXvjJMnLEpgbnfat9cjM63pLXwNJm1=2P3gq8g@mail.gmail.com/\n"},{"id":"501421","messageId":"20240821121454.GA498761@coredump.intra.peff.net","threadId":"61713","inReplyTo":"CAPig+cTACjostXvjJMnLEpgbnfat9cjM63pLXwNJm1=2P3gq8g@mail.gmail.com","subject":"Re: [PATCH v2 5/9] chainlint.pl: check line numbers in expected output","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2024-08-21T12:14:54Z","receivedAt":"2024-08-21T12:15:02Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Aug 21, 2024 at 03:00:05AM -0400, Eric Sunshine wrote:\n\n> > +for my $script (@ARGV) {\n> > +       print $expect \"# chainlint: $script\\n\";\n> > +\n> > +       open(my $expect_in, '<', \"chainlint/$script.expect\")\n> > +               or die \"unable to open chainlint/$script.expect: $!\";\n> > +       while (<$expect_in>) {\n> > +               s/^\\d+/$& + $offset/e;\n> > +               print $expect $_;\n> > +       }\n> > +\n> > +       open(my $test_in, '<', \"chainlint/$script.test\")\n> > +               or die \"unable to open chainlint/$script.test: $!\";\n> > +       while (<$test_in>) {\n> > +               /^# LINT: / and next;\n> > +               print $tests $_;\n> > +               $offset++;\n> > +       }\n> > +}\n> \n> I'm surprised that we're not closing the two file handles opened on\n> each iteration of this loop. Is that intentional? Or am I forgetting\n> my Perl and they are somehow getting closed anyhow (for instance, by\n> the <...> operator hitting EOF)?\n\nThey're scoped to the loop with \"my\", so they'll both be closed for each\niteration of the outer loop when the handles go out of scope.\n\nYou can verify with something like:\n\n  touch foo bar baz\n  strace -e openat,write,close \\\n  perl -e '\n\tfor my $script (@ARGV) {\n\t\tsyswrite(STDOUT, \"opening $script\");\n\t\topen(my $in, \"<\", $script);\n\t\tsyswrite(STDOUT, \"finished $script\");\n\t}\n  ' foo bar baz >/dev/null\n\nwhich should show:\n\n  write(1, \"opening foo\", 11)             = 11\n  openat(AT_FDCWD, \"foo\", O_RDONLY|O_CLOEXEC) = 3\n  write(1, \"finished foo\", 12)            = 12\n  close(3)                                = 0\n  write(1, \"opening bar\", 11)             = 11\n  [...etc...]\n\n-Peff\n"},{"id":"501445","messageId":"CAPig+cRDgzPnYw77VnyGfZbnM69+fu9tMtuR8_2BVK36=Vn2tw@mail.gmail.com","threadId":"61713","inReplyTo":"20240821121454.GA498761@coredump.intra.peff.net","subject":"Re: [PATCH v2 5/9] chainlint.pl: check line numbers in expected output","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2024-08-21T17:02:44Z","receivedAt":"2024-08-21T17:02:56Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Aug 21, 2024 at 8:14 AM Jeff King <peff@peff.net> wrote:\n> On Wed, Aug 21, 2024 at 03:00:05AM -0400, Eric Sunshine wrote:\n> > I'm surprised that we're not closing the two file handles opened on\n> > each iteration of this loop. Is that intentional? Or am I forgetting\n> > my Perl and they are somehow getting closed anyhow (for instance, by\n> > the <...> operator hitting EOF)?\n>\n> They're scoped to the loop with \"my\", so they'll both be closed for each\n> iteration of the outer loop when the handles go out of scope.\n\nMakes sense. I thought that might be the case but it's been years\nsince I read the \"camel book\" (thus may have forgotten it), and wasn't\nable to find any Perl documentation which stated so (which is not to\nsay such documentation doesn't exist, but rather that I couldn't find\nit).\n\n> You can verify with something like:\n>\n>   touch foo bar baz\n>   strace -e openat,write,close \\\n>   perl -e '\n>         for my $script (@ARGV) {\n>                 syswrite(STDOUT, \"opening $script\");\n>                 open(my $in, \"<\", $script);\n>                 syswrite(STDOUT, \"finished $script\");\n>         }\n>   ' foo bar baz >/dev/null\n>\n> which should show:\n>\n>   write(1, \"opening foo\", 11)             = 11\n>   openat(AT_FDCWD, \"foo\", O_RDONLY|O_CLOEXEC) = 3\n>   write(1, \"finished foo\", 12)            = 12\n>   close(3)                                = 0\n>   write(1, \"opening bar\", 11)             = 11\n\nThanks for illustrating. I did think of `strace` but I never use it\nbecause it doesn't exist on macOS and I was too lazy to spin up a\nLinux VM and read the `strace` documentation to figure out how to do\nwhat you did above.\n"}]}