{"thread":{"id":"54669","subject":"[PATCH 1/2] tests: make sure nested lazy prereqs work reliably","startedAt":"2020-11-18T19:04:33Z","lastAt":"2020-11-20T01:33:01Z","messageCount":13,"participants":["SZEDER Gábor","Junio C Hamano","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"410261","messageId":"20201118190414.32616-1-szeder.dev@gmail.com","threadId":"54669","inReplyTo":null,"subject":"[PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2020-11-18T19:04:13Z","receivedAt":"2020-11-18T19:04:33Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Some test prereqs depend on other prereqs, so in a couple of cases we\nhave nested prereqs that look something like this:\n\n  test_lazy_prereq FOO '\n      test_have_prereq BAR &&\n      check-foo\n  '\n\nThis can be problematic, because lazy prereqs are evaluated in the\n'$TRASH_DIRECTORY/prereq-test-dir' directory, which is the same for\nevery prereq, and which is automatically removed after the prereq has\nbeen evaluated.  So if the inner prereq (BAR above) is a lazy prereq\nthat hasn't been evaluated yet, then after its evaluation the\n'prereq-test-dir' shared with the outer prereq will be removed.\nConsequently, 'check-foo' will find itself in a non-existing\ndirectory, and won't be able to create/access any files in its cwd,\nwhich could result in an unfulfilled outer prereq.\n\nLuckily, this doesn't affect any of our current nested prereqs, either\nbecause the inner prereq is not a lazy prereq (e.g. MINGW, CYGWIN or\nPERL), or because the outer prereq happens to be checked without\ntouching any paths in its cwd (GPGSM and RFC1991 in 'lib-gpg.sh').\n\nSo to prevent nested prereqs from interfering with each other let's\nevaluate each prereq in its own dedicated directory by appending the\nprereq's name to the directory name, e.g. 'prereq-test-dir-SYMLINKS'.\nIn the test we check not only that the prereq test dir is still there,\nbut also that the inner prereq can't mess with the outer prereq's\nfiles.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n\nThis came up while looking into:\n\n  https://public-inbox.org/git/nycvar.QRO.7.76.6.2011152252520.18437@tvgsbejvaqbjf.bet/\n\nbecause this did become an issue when I made the JGIT prereq depend on\nSHA1.\n\n t/t0000-basic.sh        | 21 +++++++++++++++++++++\n t/test-lib-functions.sh |  6 +++---\n 2 files changed, 24 insertions(+), 3 deletions(-)\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex 923281af93..4031a0e3fc 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -831,6 +831,27 @@ then\n \texit 1\n fi\n \n+test_lazy_prereq NESTED_INNER '\n+\t>inner &&\n+\trm -f outer\n+'\n+test_lazy_prereq NESTED_PREREQ '\n+\t>outer &&\n+\ttest_have_prereq NESTED_INNER &&\n+\techo \"can create new file in cwd\" >file &&\n+\ttest -f outer &&\n+\ttest ! -f inner\n+'\n+test_expect_success NESTED_PREREQ 'evaluating nested lazy prereqs dont interfere with each other' '\n+\tnestedworks=yes\n+'\n+\n+if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" && test \"$nestedworks\" != yes\n+then\n+\tsay 'bug in test framework: nested lazy prerequisites do not work'\n+\texit 1\n+fi\n+\n test_expect_success 'lazy prereqs do not turn off tracing' \"\n \trun_sub_test_lib_test lazy-prereq-and-tracing \\\n \t\t'lazy prereqs and -x' -v -x <<-\\\\EOF &&\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 8d59b90348..94395b9807 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -474,15 +474,15 @@ test_lazy_prereq () {\n \n test_run_lazy_prereq_ () {\n \tscript='\n-mkdir -p \"$TRASH_DIRECTORY/prereq-test-dir\" &&\n+mkdir -p \"$TRASH_DIRECTORY/prereq-test-dir-'\"$1\"'\" &&\n (\n-\tcd \"$TRASH_DIRECTORY/prereq-test-dir\" &&'\"$2\"'\n+\tcd \"$TRASH_DIRECTORY/prereq-test-dir-'\"$1\"'\" &&'\"$2\"'\n )'\n \tsay >&3 \"checking prerequisite: $1\"\n \tsay >&3 \"$script\"\n \ttest_eval_ \"$script\"\n \teval_ret=$?\n-\trm -rf \"$TRASH_DIRECTORY/prereq-test-dir\"\n+\trm -rf \"$TRASH_DIRECTORY/prereq-test-dir-$1\"\n \tif test \"$eval_ret\" = 0; then\n \t\tsay >&3 \"prerequisite $1 ok\"\n \telse\n-- \n2.29.2.612.g41c40d3f73\n\n"},{"id":"410262","messageId":"20201118190414.32616-2-szeder.dev@gmail.com","threadId":"54669","inReplyTo":"20201118190414.32616-1-szeder.dev@gmail.com","subject":"[PATCH 2/2] tests: fix description of 'test_set_prereq'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2020-11-18T19:04:14Z","receivedAt":"2020-11-18T19:04:36Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"'test_set_prereq's description claims that prereqs can be specified to\n'test_expect_code', but that is not the case (it is not meant to run a\ntest _case_, but a git command), so remove it.\n\nOTOH that description doesn't mention 'test_external' and\n'test_external_without_stderr' that do accept prereqs, so mention\nthem.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n t/test-lib-functions.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 94395b9807..21b2ea55b7 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -423,7 +423,7 @@ write_script () {\n # - Explicitly using test_have_prereq.\n #\n # - Implicitly by specifying the prerequisite tag in the calls to\n-#   test_expect_{success,failure,code}.\n+#   test_expect_{success,failure} and test_external{,_without_stderr}.\n #\n # The single parameter is the prerequisite tag (a simple word, in all\n # capital letters by convention).\n-- \n2.29.2.612.g41c40d3f73\n\n"},{"id":"410269","messageId":"xmqqh7pm1llj.fsf@gitster.c.googlers.com","threadId":"54669","inReplyTo":"20201118190414.32616-1-szeder.dev@gmail.com","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-18T20:00:40Z","receivedAt":"2020-11-18T20:01:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"SZEDER Gábor <szeder.dev@gmail.com> writes:\n\n> This can be problematic, because lazy prereqs are evaluated in the\n> '$TRASH_DIRECTORY/prereq-test-dir' directory, which is the same for\n> every prereq, and which is automatically removed after the prereq has\n> been evaluated.  So if the inner prereq (BAR above) is a lazy prereq\n> that hasn't been evaluated yet, then after its evaluation the\n> 'prereq-test-dir' shared with the outer prereq will be removed.\n\nOK.\n\n> Consequently, 'check-foo' will find itself in a non-existing\n> directory, and won't be able to create/access any files in its cwd,\n> which could result in an unfulfilled outer prereq.\n\nI have a feeling that this would be filesystem dependent what\nhappens in an unlinked directory.  But in any case, if an outer lazy\nprereq creates a file in the test directory, tries to evaluate\nanother lazy prereq (which would use the same directory and then\nwould remove everything in it and the directory itself when done),\nand then expects that the file it created before evaluating the\ninner lazy prereq is still there, it would not work, so I think this\nis a good change.  I just think \"won't be able to\" is a bit too\nstrong here (\"may not be able to (depending on the filesystem)\", I\nwould buy).\n\n> +test_lazy_prereq NESTED_INNER '\n> +\t>inner &&\n> +\trm -f outer\n> +'\n> +test_lazy_prereq NESTED_PREREQ '\n> +\t>outer &&\n> +\ttest_have_prereq NESTED_INNER &&\n> +\techo \"can create new file in cwd\" >file &&\n> +\ttest -f outer &&\n> +\ttest ! -f inner\n> +'\n\nAnd the existence check for outer is exactly what I wrote above.\nThat would portably break without separte directory.  I do not know\nif \"can create new file\" step would fail portably.\n\n> diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\n> index 8d59b90348..94395b9807 100644\n> --- a/t/test-lib-functions.sh\n> +++ b/t/test-lib-functions.sh\n> @@ -474,15 +474,15 @@ test_lazy_prereq () {\n>  \n>  test_run_lazy_prereq_ () {\n>  \tscript='\n> -mkdir -p \"$TRASH_DIRECTORY/prereq-test-dir\" &&\n> +mkdir -p \"$TRASH_DIRECTORY/prereq-test-dir-'\"$1\"'\" &&\n>  (\n> -\tcd \"$TRASH_DIRECTORY/prereq-test-dir\" &&'\"$2\"'\n> +\tcd \"$TRASH_DIRECTORY/prereq-test-dir-'\"$1\"'\" &&'\"$2\"'\n>  )'\n\nQoting rules are a bit tricky here, but I think it does not matter\ntoo much, as $1 (name or prereq) won't have $IFS or \"'\" or anything\nfunny in it.\n\n>  \tsay >&3 \"checking prerequisite: $1\"\n>  \tsay >&3 \"$script\"\n>  \ttest_eval_ \"$script\"\n>  \teval_ret=$?\n> -\trm -rf \"$TRASH_DIRECTORY/prereq-test-dir\"\n> +\trm -rf \"$TRASH_DIRECTORY/prereq-test-dir-$1\"\n\nAnd this is obviously safe no matter what is in $1 ;-)\n\n>  \tif test \"$eval_ret\" = 0; then\n>  \t\tsay >&3 \"prerequisite $1 ok\"\n>  \telse\n\nLooks good.\n"},{"id":"410369","messageId":"20201119155824.GB25426@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201118190414.32616-1-szeder.dev@gmail.com","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-19T15:58:24Z","receivedAt":"2020-11-19T15:58:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Nov 18, 2020 at 08:04:13PM +0100, SZEDER Gábor wrote:\n\n> So to prevent nested prereqs from interfering with each other let's\n> evaluate each prereq in its own dedicated directory by appending the\n> prereq's name to the directory name, e.g. 'prereq-test-dir-SYMLINKS'.\n> In the test we check not only that the prereq test dir is still there,\n> but also that the inner prereq can't mess with the outer prereq's\n> files.\n\nThat sounds reasonable. I do wonder, though, whether simply creating the\nprereq directory in the _current_ directory would be sufficient. Then\nyou'd get prereq-test-dir/prereq-test-dir for a nested invocation. But\nthe prereqs aren't supposed to care about which specific directory\nthey're in.\n\nI.e., your test passes equally well with just:\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 59bbf75e83..f5dc6801d9 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -474,15 +474,15 @@ test_lazy_prereq () {\n \n test_run_lazy_prereq_ () {\n \tscript='\n-mkdir -p \"$TRASH_DIRECTORY/prereq-test-dir\" &&\n+mkdir -p \"prereq-test-dir\" &&\n (\n-\tcd \"$TRASH_DIRECTORY/prereq-test-dir\" &&'\"$2\"'\n+\tcd \"prereq-test-dir\" &&'\"$2\"'\n )'\n \tsay >&3 \"checking prerequisite: $1\"\n \tsay >&3 \"$script\"\n \ttest_eval_ \"$script\"\n \teval_ret=$?\n-\trm -rf \"$TRASH_DIRECTORY/prereq-test-dir\"\n+\trm -rf \"prereq-test-dir\"\n \tif test \"$eval_ret\" = 0; then\n \t\tsay >&3 \"prerequisite $1 ok\"\n \telse\n\nthough I guess it is not really much simpler (it avoids the funky\nquoting around $1 in the embedded script, but we already have that for\n$2). And perhaps debugging is easier with a more predictable and\ndescriptive directory name.\n\n> +test_lazy_prereq NESTED_INNER '\n> +\t>inner &&\n> +\trm -f outer\n> +'\n> +test_lazy_prereq NESTED_PREREQ '\n> +\t>outer &&\n> +\ttest_have_prereq NESTED_INNER &&\n> +\techo \"can create new file in cwd\" >file &&\n> +\ttest -f outer &&\n> +\ttest ! -f inner\n> +'\n> +test_expect_success NESTED_PREREQ 'evaluating nested lazy prereqs dont interfere with each other' '\n> +\tnestedworks=yes\n> +'\n> +\n> +if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" && test \"$nestedworks\" != yes\n> +then\n> +\tsay 'bug in test framework: nested lazy prerequisites do not work'\n> +\texit 1\n> +fi\n\nI was surprised to see this bare exit, because I know we have some\nfunctions (run_sub_test_*) to help with testing the framework itself. It\nlooks like the other prereq tests don't use it either, though. I wonder\nif there is a technical reason, or if they were simply added at a\ndifferent time. (Either way, I am OK for your new test to match the\nsurrounding ones like you have here).\n\n-Peff\n"},{"id":"410372","messageId":"20201119175608.GA132922@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201119155824.GB25426@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-19T17:56:08Z","receivedAt":"2020-11-19T17:56:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 19, 2020 at 10:58:24AM -0500, Jeff King wrote:\n\n> > +if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" && test \"$nestedworks\" != yes\n> > +then\n> > +\tsay 'bug in test framework: nested lazy prerequisites do not work'\n> > +\texit 1\n> > +fi\n> \n> I was surprised to see this bare exit, because I know we have some\n> functions (run_sub_test_*) to help with testing the framework itself. It\n> looks like the other prereq tests don't use it either, though. I wonder\n> if there is a technical reason, or if they were simply added at a\n> different time. (Either way, I am OK for your new test to match the\n> surrounding ones like you have here).\n\nI took a look at converting some of the existing tests. This seems to\nwork. It's a bit longer to read, perhaps, but I kind of like that the\nexpected outcome is all laid out. It also pollutes the test output less\n(e.g., if you wanted to count up skipped tests in the whole suite, you'd\nget a bunch of noise from t0000 for these uninteresting skips).\n\nThoughts? I think this is something I'd do on top of your patch.\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex f4ba2e8c85..f369af76be 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -759,43 +759,50 @@ test_expect_success '--run invalid range end' \"\n \tEOF_ERR\n \"\n \n-\n-test_set_prereq HAVEIT\n-haveit=no\n-test_expect_success HAVEIT 'test runs if prerequisite is satisfied' '\n-\ttest_have_prereq HAVEIT &&\n-\thaveit=yes\n-'\n-donthaveit=yes\n-test_expect_success DONTHAVEIT 'unmet prerequisite causes test to be skipped' '\n-\tdonthaveit=no\n-'\n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $haveit$donthaveit != yesyes\n-then\n-\tsay \"bug in test framework: prerequisite tags do not work reliably\"\n-\texit 1\n-fi\n-\n-test_set_prereq HAVETHIS\n-haveit=no\n-test_expect_success HAVETHIS,HAVEIT 'test runs if prerequisites are satisfied' '\n-\ttest_have_prereq HAVEIT &&\n-\ttest_have_prereq HAVETHIS &&\n-\thaveit=yes\n-'\n-donthaveit=yes\n-test_expect_success HAVEIT,DONTHAVEIT 'unmet prerequisites causes test to be skipped' '\n-\tdonthaveit=no\n-'\n-donthaveiteither=yes\n-test_expect_success DONTHAVEIT,HAVEIT 'unmet prerequisites causes test to be skipped' '\n-\tdonthaveiteither=no\n+test_expect_success 'tests respect prerequisites' '\n+\trun_sub_test_lib_test prereqs \"tests respect prereqs\" <<-\\EOF &&\n+\n+\ttest_set_prereq HAVEIT\n+\thaveit=no\n+\ttest_expect_success HAVEIT \"prereq is satisfied\" \"\n+\t\ttest_have_prereq HAVEIT &&\n+\t\thaveit=yes\n+\t\"\n+\n+\tdonthaveit=yes\n+\ttest_expect_success DONTHAVEIT \"prereq not satisfied\" \"\n+\t\tdonthaveit=no\n+\t\"\n+\n+\ttest_set_prereq HAVETHIS\n+\thaveit=no\n+\ttest_expect_success HAVETHIS,HAVEIT \"multiple prereqs\" \"\n+\t\ttest_have_prereq HAVEIT &&\n+\t\ttest_have_prereq HAVETHIS &&\n+\t\thaveit=yes\n+\t\"\n+\n+\tdonthaveit=yes\n+\ttest_expect_success HAVEIT,DONTHAVEIT \"mixed prereqs (yes,no)\" \"\n+\t\tdonthaveit=no\n+\t\"\n+\n+\tdonthaveiteither=yes\n+\ttest_expect_success DONTHAVEIT,HAVEIT \"mixed prereqs (no,yes)\" \"\n+\t\tdonthaveiteither=no\n+\t\"\n+\ttest_done\n+\tEOF\n+\tcheck_sub_test_lib_test prereqs <<-\\EOF\n+\tok 1 - prereq is satisfied\n+\tok 2 # skip prereq not satisfied (missing DONTHAVEIT)\n+\tok 3 - multiple prereqs\n+\tok 4 # skip mixed prereqs (yes,no) (missing DONTHAVEIT of HAVEIT,DONTHAVEIT)\n+\tok 5 # skip mixed prereqs (no,yes) (missing DONTHAVEIT of DONTHAVEIT,HAVEIT)\n+\t# passed all 5 test(s)\n+\t1..5\n+\tEOF\n '\n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $haveit$donthaveit$donthaveiteither != yesyesyes\n-then\n-\tsay \"bug in test framework: multiple prerequisite tags do not work reliably\"\n-\texit 1\n-fi\n \n test_lazy_prereq LAZY_TRUE true\n havetrue=no\n"},{"id":"410377","messageId":"xmqqd009xh19.fsf@gitster.c.googlers.com","threadId":"54669","inReplyTo":"20201119175608.GA132922@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-19T19:50:26Z","receivedAt":"2020-11-19T19:50:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> I took a look at converting some of the existing tests. This seems to\n> work. It's a bit longer to read, perhaps, but I kind of like that the\n> expected outcome is all laid out. It also pollutes the test output less\n> (e.g., if you wanted to count up skipped tests in the whole suite, you'd\n> get a bunch of noise from t0000 for these uninteresting skips).\n>\n> Thoughts? I think this is something I'd do on top of your patch.\n\nYes, it looks nice as the expectation is expressed much clearly.\n"},{"id":"410395","messageId":"xmqqpn48vqkq.fsf@gitster.c.googlers.com","threadId":"54669","inReplyTo":"20201119155824.GB25426@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-20T00:07:17Z","receivedAt":"2020-11-20T00:07:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Wed, Nov 18, 2020 at 08:04:13PM +0100, SZEDER Gábor wrote:\n>\n>> So to prevent nested prereqs from interfering with each other let's\n>> evaluate each prereq in its own dedicated directory by appending the\n>> prereq's name to the directory name, e.g. 'prereq-test-dir-SYMLINKS'.\n>> In the test we check not only that the prereq test dir is still there,\n>> but also that the inner prereq can't mess with the outer prereq's\n>> files.\n>\n> That sounds reasonable. I do wonder, though, whether simply creating the\n> prereq directory in the _current_ directory would be sufficient. Then\n> you'd get prereq-test-dir/prereq-test-dir for a nested invocation. But\n> the prereqs aren't supposed to care about which specific directory\n> they're in.\n\nTrue.  That does sound conceptually simpler.  As we've already seen\nthe patch, I do not mind too deeply either way, though.\n\nThanks.\n"},{"id":"410396","messageId":"20201120001458.GA274082@coredump.intra.peff.net","threadId":"54669","inReplyTo":"xmqqd009xh19.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-20T00:14:58Z","receivedAt":"2020-11-20T00:15:04Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 19, 2020 at 11:50:26AM -0800, Junio C Hamano wrote:\n\n> Jeff King <peff@peff.net> writes:\n> \n> > I took a look at converting some of the existing tests. This seems to\n> > work. It's a bit longer to read, perhaps, but I kind of like that the\n> > expected outcome is all laid out. It also pollutes the test output less\n> > (e.g., if you wanted to count up skipped tests in the whole suite, you'd\n> > get a bunch of noise from t0000 for these uninteresting skips).\n> >\n> > Thoughts? I think this is something I'd do on top of your patch.\n> \n> Yes, it looks nice as the expectation is expressed much clearly.\n\nOK, then here's the whole thing. I ended up with a few more cleanups,\ntoo. This is all on top of Gábor's patches. It's conceptually\nindependent, but the textual wrangling was annoying enough it didn't\nmake any sense to require you to do it again during merging. ;) Plus I\ndo not think either topic is high-risk nor urgent enough to worry too\nmuch about one blocking the other.\n\nThe diffstat is scary, but it's mostly the final patch, which is pretty\nmechanical.\n\n  [1/4]: t0000: keep clean-up tests together\n  [2/4]: t0000: run prereq tests inside sub-test\n  [3/4]: t0000: run cleaning test inside sub-test\n  [4/4]: t0000: consistently use single quotes for outer tests\n\n t/t0000-basic.sh | 570 +++++++++++++++++++++++------------------------\n 1 file changed, 284 insertions(+), 286 deletions(-)\n\n-Peff\n"},{"id":"410397","messageId":"20201120001723.GA307112@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201120001458.GA274082@coredump.intra.peff.net","subject":"[PATCH 1/4] t0000: keep clean-up tests together","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-20T00:17:23Z","receivedAt":"2020-11-20T00:17:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We check that test_when_finished cleans up after a test, and that it\nruns even after a failure. Those two were originally adjacent, but got\nsplit apart by the new test added in 477dcaddb6 (tests: do not let lazy\nprereqs inside `test_expect_*` turn off tracing, 2020-03-26), and then\nfurther by more lazy-prereq tests. Let's move them back together.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/t0000-basic.sh | 22 +++++++++++-----------\n 1 file changed, 11 insertions(+), 11 deletions(-)\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex f4ba2e8c85..d49b5dd4ac 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -829,17 +829,6 @@ then\n \texit 1\n fi\n \n-clean=no\n-test_expect_success 'tests clean up after themselves' '\n-\ttest_when_finished clean=yes\n-'\n-\n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $clean != yes\n-then\n-\tsay \"bug in test framework: basic cleanup command does not work reliably\"\n-\texit 1\n-fi\n-\n test_lazy_prereq NESTED_INNER '\n \t>inner &&\n \trm -f outer\n@@ -874,6 +863,17 @@ test_expect_success 'lazy prereqs do not turn off tracing' \"\n \tgrep 'echo trace' lazy-prereq-and-tracing/err\n \"\n \n+clean=no\n+test_expect_success 'tests clean up after themselves' '\n+\ttest_when_finished clean=yes\n+'\n+\n+if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $clean != yes\n+then\n+\tsay \"bug in test framework: basic cleanup command does not work reliably\"\n+\texit 1\n+fi\n+\n test_expect_success 'tests clean up even on failures' \"\n \trun_sub_test_lib_test_err \\\n \t\tfailing-cleanup 'Failing tests with cleanup commands' <<-\\\\EOF &&\n-- \n2.29.2.730.g3e418f96ba\n\n"},{"id":"410398","messageId":"20201120002022.GB307112@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201120001458.GA274082@coredump.intra.peff.net","subject":"[PATCH 2/4] t0000: run prereq tests inside sub-test","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-20T00:20:22Z","receivedAt":"2020-11-20T00:20:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We test the behavior of prerequisites in t0000 by setting up fake ones\nin the main test script, trying to run some tests, and then seeing if\nthose tests impacted the environment correctly. If they didn't, then we\nwrite a message and manually call exit.\n\nInstead, let's push these down into a sub-test, like many of the other\ntests covering the framework itself. This has a few advantages:\n\n  - it does not pollute the test output with mention of skipped tests\n    (that we know are uninteresting -- the point of the test was to see\n    that these are skipped).\n\n  - when running in a TAP harness, we get a useful test failure message\n    (whereas when the script exits early, a tool like \"prove\" simply\n    says \"Dubious, test returned 1\").\n\n  - we do not have to worry about different test environments, such as\n    when GIT_TEST_FAIL_PREREQS_INTERNAL is set. Our sub-test helpers\n    already give us a known environment.\n\n  - the tests themselves are a bit easier to read, as we can just check\n    the test-framework output to see what happened (and get the usual\n    test_cmp diff if it failed)\n\nA few notes on the implementation:\n\n  - we could do one sub-test per each individual test_expect_success. I\n    broke it up here into a few logical groups, as I think this makes it\n    more readable\n\n  - the original tests modified environment variables inside the test\n    bodies. Instead, I've used \"true\" as the body of a test we expect to\n    run and \"false\" otherwise. Technically this does not confirm that\n    the body of the \"true\" test actually ran. We are trusting the\n    framework output to believe that it truly ran, which is sufficient\n    for these tests. And I think the end result is much simpler to\n    follow.\n\n  - the nested_prereq test uses a few bare \"test -f\" calls; I converted\n    these to our usual test_path_is_* helpers while moving the code\n    around.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThe diff is pretty gnarly. I recommend just reading the before/after\nsections (I wish there was a way to convince diff _not_ to generate the\nminimal diff; two big blocks would have been more readable).\n\n t/t0000-basic.sh | 149 ++++++++++++++++++++++-------------------------\n 1 file changed, 69 insertions(+), 80 deletions(-)\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex d49b5dd4ac..3d36a87610 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -759,96 +759,85 @@ test_expect_success '--run invalid range end' \"\n \tEOF_ERR\n \"\n \n+test_expect_success 'tests respect prerequisites' '\n+\trun_sub_test_lib_test prereqs \"tests respect prereqs\" <<-\\EOF &&\n \n-test_set_prereq HAVEIT\n-haveit=no\n-test_expect_success HAVEIT 'test runs if prerequisite is satisfied' '\n-\ttest_have_prereq HAVEIT &&\n-\thaveit=yes\n-'\n-donthaveit=yes\n-test_expect_success DONTHAVEIT 'unmet prerequisite causes test to be skipped' '\n-\tdonthaveit=no\n-'\n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $haveit$donthaveit != yesyes\n-then\n-\tsay \"bug in test framework: prerequisite tags do not work reliably\"\n-\texit 1\n-fi\n+\ttest_set_prereq HAVEIT\n+\ttest_expect_success HAVEIT \"prereq is satisfied\" \"true\"\n+\ttest_expect_success \"have_prereq works\" \"\n+\t\ttest_have_prereq HAVEIT\n+\t\"\n+\ttest_expect_success DONTHAVEIT \"prereq not satisfied\" \"false\"\n \n-test_set_prereq HAVETHIS\n-haveit=no\n-test_expect_success HAVETHIS,HAVEIT 'test runs if prerequisites are satisfied' '\n-\ttest_have_prereq HAVEIT &&\n-\ttest_have_prereq HAVETHIS &&\n-\thaveit=yes\n-'\n-donthaveit=yes\n-test_expect_success HAVEIT,DONTHAVEIT 'unmet prerequisites causes test to be skipped' '\n-\tdonthaveit=no\n-'\n-donthaveiteither=yes\n-test_expect_success DONTHAVEIT,HAVEIT 'unmet prerequisites causes test to be skipped' '\n-\tdonthaveiteither=no\n-'\n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $haveit$donthaveit$donthaveiteither != yesyesyes\n-then\n-\tsay \"bug in test framework: multiple prerequisite tags do not work reliably\"\n-\texit 1\n-fi\n+\ttest_set_prereq HAVETHIS\n+\ttest_expect_success HAVETHIS,HAVEIT \"multiple prereqs\" \"true\"\n+\ttest_expect_success HAVEIT,DONTHAVEIT \"mixed prereqs (yes,no)\" \"false\"\n+\ttest_expect_success DONTHAVEIT,HAVEIT \"mixed prereqs (no,yes)\" \"false\"\n \n-test_lazy_prereq LAZY_TRUE true\n-havetrue=no\n-test_expect_success LAZY_TRUE 'test runs if lazy prereq is satisfied' '\n-\thavetrue=yes\n-'\n-donthavetrue=yes\n-test_expect_success !LAZY_TRUE 'missing lazy prereqs skip tests' '\n-\tdonthavetrue=no\n+\ttest_done\n+\tEOF\n+\n+\tcheck_sub_test_lib_test prereqs <<-\\EOF\n+\tok 1 - prereq is satisfied\n+\tok 2 - have_prereq works\n+\tok 3 # skip prereq not satisfied (missing DONTHAVEIT)\n+\tok 4 - multiple prereqs\n+\tok 5 # skip mixed prereqs (yes,no) (missing DONTHAVEIT of HAVEIT,DONTHAVEIT)\n+\tok 6 # skip mixed prereqs (no,yes) (missing DONTHAVEIT of DONTHAVEIT,HAVEIT)\n+\t# passed all 6 test(s)\n+\t1..6\n+\tEOF\n '\n \n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a \"$havetrue$donthavetrue\" != yesyes\n-then\n-\tsay 'bug in test framework: lazy prerequisites do not work'\n-\texit 1\n-fi\n+test_expect_success 'tests respect lazy prerequisites' '\n+\trun_sub_test_lib_test lazy-prereqs \"respect lazy prereqs\" <<-\\EOF &&\n \n-test_lazy_prereq LAZY_FALSE false\n-nothavefalse=no\n-test_expect_success !LAZY_FALSE 'negative lazy prereqs checked' '\n-\tnothavefalse=yes\n-'\n-havefalse=yes\n-test_expect_success LAZY_FALSE 'missing negative lazy prereqs will skip' '\n-\thavefalse=no\n-'\n+\ttest_lazy_prereq LAZY_TRUE true\n+\ttest_expect_success LAZY_TRUE \"lazy prereq is satisifed\" \"true\"\n+\ttest_expect_success !LAZY_TRUE \"negative lazy prereq\" \"false\"\n \n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a \"$nothavefalse$havefalse\" != yesyes\n-then\n-\tsay 'bug in test framework: negative lazy prerequisites do not work'\n-\texit 1\n-fi\n+\ttest_lazy_prereq LAZY_FALSE false\n+\ttest_expect_success LAZY_FALSE \"lazy prereq not satisfied\" \"false\"\n+\ttest_expect_success !LAZY_FALSE \"negative false prereq\" \"true\"\n \n-test_lazy_prereq NESTED_INNER '\n-\t>inner &&\n-\trm -f outer\n-'\n-test_lazy_prereq NESTED_PREREQ '\n-\t>outer &&\n-\ttest_have_prereq NESTED_INNER &&\n-\techo \"can create new file in cwd\" >file &&\n-\ttest -f outer &&\n-\ttest ! -f inner\n-'\n-test_expect_success NESTED_PREREQ 'evaluating nested lazy prereqs dont interfere with each other' '\n-\tnestedworks=yes\n+\ttest_done\n+\tEOF\n+\n+\tcheck_sub_test_lib_test lazy-prereqs <<-\\EOF\n+\tok 1 - lazy prereq is satisifed\n+\tok 2 # skip negative lazy prereq (missing !LAZY_TRUE)\n+\tok 3 # skip lazy prereq not satisfied (missing LAZY_FALSE)\n+\tok 4 - negative false prereq\n+\t# passed all 4 test(s)\n+\t1..4\n+\tEOF\n '\n \n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" && test \"$nestedworks\" != yes\n-then\n-\tsay 'bug in test framework: nested lazy prerequisites do not work'\n-\texit 1\n-fi\n+test_expect_success 'nested lazy prerequisites' '\n+\trun_sub_test_lib_test nested-lazy \"nested lazy prereqs\" <<-\\EOF &&\n+\n+\ttest_lazy_prereq NESTED_INNER \"\n+\t\t>inner &&\n+\t\trm -f outer\n+\t\"\n+\ttest_lazy_prereq NESTED_PREREQ \"\n+\t\t>outer &&\n+\t\ttest_have_prereq NESTED_INNER &&\n+\t\techo can create new file in cwd >file &&\n+\t\ttest_path_is_file outer &&\n+\t\ttest_path_is_missing inner\n+\t\"\n+\ttest_expect_success NESTED_PREREQ \"evaluate nested prereq\" \"true\"\n+\n+\ttest_done\n+\tEOF\n+\n+\tcheck_sub_test_lib_test nested-lazy <<-\\EOF\n+\tok 1 - evaluate nested prereq\n+\t# passed all 1 test(s)\n+\t1..1\n+\tEOF\n+'\n \n test_expect_success 'lazy prereqs do not turn off tracing' \"\n \trun_sub_test_lib_test lazy-prereq-and-tracing \\\n-- \n2.29.2.730.g3e418f96ba\n\n"},{"id":"410399","messageId":"20201120002247.GC307112@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201120001458.GA274082@coredump.intra.peff.net","subject":"[PATCH 3/4] t0000: run cleaning test inside sub-test","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-20T00:22:47Z","receivedAt":"2020-11-20T00:23:10Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Our check of test_when_finished is done directly in the main script, and\nif we failed to clean, we complain and exit immediately. It's nicer to\nsignal a test failure here, for a few reasons:\n\n  - this gives better output to the user when run under a TAP harness\n    like \"prove\"\n\n  - constency; it's the only test left in the file that behaves this way\n\n  - half of its \"if\" conditional is nonsense anyway; it picked up a\n    reference to GIT_TEST_FAIL_PREREQS internal in dfe1a17df9 (tests:\n    add a special setup where prerequisites fail, 2019-05-13) along with\n    its neighbors, even though it has nothing to do with that flag\n\nWe could actually do this without a sub-test at all, and just put our\ntwo tests (one to do cleanup, and one to check that it happened) in the\nmain script. But doing it in a subtest is conceptually cleaner (from the\nperspective of the main test script, we are checking only one thing),\nand it remains consistent with the \"cleanup when failing\" test directly\nafter it, which has to happen in a sub-test (to avoid the main script\ncomplaining of the failed test).\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n t/t0000-basic.sh | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex 3d36a87610..502375bdf6 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -852,16 +852,25 @@ test_expect_success 'lazy prereqs do not turn off tracing' \"\n \tgrep 'echo trace' lazy-prereq-and-tracing/err\n \"\n \n-clean=no\n test_expect_success 'tests clean up after themselves' '\n-\ttest_when_finished clean=yes\n-'\n+\trun_sub_test_lib_test cleanup \"test with cleanup\" <<-\\EOF &&\n+\tclean=no\n+\ttest_expect_success \"do cleanup\" \"\n+\t\ttest_when_finished clean=yes\n+\t\"\n+\ttest_expect_success \"cleanup happened\" \"\n+\t\ttest $clean = yes\n+\t\"\n+\ttest_done\n+\tEOF\n \n-if test -z \"$GIT_TEST_FAIL_PREREQS_INTERNAL\" -a $clean != yes\n-then\n-\tsay \"bug in test framework: basic cleanup command does not work reliably\"\n-\texit 1\n-fi\n+\tcheck_sub_test_lib_test cleanup <<-\\EOF\n+\tok 1 - do cleanup\n+\tok 2 - cleanup happened\n+\t# passed all 2 test(s)\n+\t1..2\n+\tEOF\n+'\n \n test_expect_success 'tests clean up even on failures' \"\n \trun_sub_test_lib_test_err \\\n-- \n2.29.2.730.g3e418f96ba\n\n"},{"id":"410400","messageId":"20201120002703.GD307112@coredump.intra.peff.net","threadId":"54669","inReplyTo":"20201120001458.GA274082@coredump.intra.peff.net","subject":"[PATCH 4/4] t0000: consistently use single quotes for outer tests","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-11-20T00:27:03Z","receivedAt":"2020-11-20T00:27:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"When we use the sub-test helpers, we end defining one shell snippet\ninside another shell snippet. So if we use single-quotes for the outer\nsnippet, we have to use double-quotes within the inner snippet (it's\nincluded as here-doc within the outer snippet, but using a single quote\nwould end the outer snippet early). Or vice versa we can use double\nquotes for the outer snippet, but then single quotes in the inner.\n\nWe have some of each in the script, and neither is wrong. But it would\nbe nice to be consistent unless there is a good reason not to. Using\nsingle quotes for the outer script is preferable, because it requires\nless metacharacter quoting overall. For example, in:\n\n  test_expect_success 'outer' '\n\trun_sub_test_lib_test ...  <<-\\EOF\n\t\techo $foo &&\n\t\ttest_expect_success \"inner\" \"\n\t\t\techo \\$bar\n\t\t\"\n\tEOF\n  '\n\nwe need only quote inside \"inner\", but not inside \"outer\" or the\nhere-doc. Whereas if we flip them, we have to quote in both places:\n\n  test_expect_success 'outer' \"\n\trun_sub_test_lib_test ...  <<-\\EOF\n\t\techo \\$foo &&\n\t\ttest_expect_success 'inner' '\n\t\t\techo \\$bar\n\t\t'\n\tEOF\n  \"\n\nThe exception is when we need a literal single-quote in an expected\noutput here-doc. There we can either use outer double-quotes, or just\nuse ${SQ} within the doc. I chose the latter for consistency (within\nthis test, but also with other test scripts that face the same problem).\n\nThere is one other interesting case, which is some tests that do:\n\n  test_expect_success ... \"\n\tdo_something --run='\"'!3'\"'\n  \"\n\nThis is rather confusing to read, but is correct. The outer script sees\n'!3' in single-quotes, as does the eval'd snippet. This is perhaps being\noverly cautious. In many interactive shells, an exclamation triggers\nhistory expansion even inside double quotes, but that is not generally\ntrue in non-interactive shells.\n\nThere's some conflicting information here. Commit 784ce03d55 (t4216:\navoid unnecessary subshell in test_bloom_filters_not_used, 2020-05-19)\nreports it as a problem with OpenBSD 6.7's /bin/sh. However, we have\nmany instances in this script of prereqs like !LAZY_TRUE, which haven't\nbeen a problem. I left them un-escaped here to test out this theory.\nIt's much nicer if we can not worry about this as a portability issue,\nso it's worth knowing.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nI have often thought that test_expect_* should take the snippet on stdin\nvia a here-doc, which removes one layer of quoting and makes this all\neven simpler. We could obviously do that with a helper here, but I\ndidn't want to stray too far from usual practice within the rest of the\ntest suite.\n\n t/t0000-basic.sh | 380 +++++++++++++++++++++++------------------------\n 1 file changed, 190 insertions(+), 190 deletions(-)\n\ndiff --git a/t/t0000-basic.sh b/t/t0000-basic.sh\nindex 502375bdf6..a6e570d674 100755\n--- a/t/t0000-basic.sh\n+++ b/t/t0000-basic.sh\n@@ -135,77 +135,77 @@ check_sub_test_lib_test_err () {\n \t)\n }\n \n-test_expect_success 'pretend we have a fully passing test suite' \"\n-\trun_sub_test_lib_test full-pass '3 passing tests' <<-\\\\EOF &&\n+test_expect_success 'pretend we have a fully passing test suite' '\n+\trun_sub_test_lib_test full-pass \"3 passing tests\" <<-\\EOF &&\n \tfor i in 1 2 3\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test full-pass <<-\\\\EOF\n+\tcheck_sub_test_lib_test full-pass <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 - passing test #3\n \t> # passed all 3 test(s)\n \t> 1..3\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have a partially passing test suite' \"\n+test_expect_success 'pretend we have a partially passing test suite' '\n \trun_sub_test_lib_test_err \\\n-\t\tpartial-pass '2/3 tests passing' <<-\\\\EOF &&\n-\ttest_expect_success 'passing test #1' 'true'\n-\ttest_expect_success 'failing test #2' 'false'\n-\ttest_expect_success 'passing test #3' 'true'\n+\t\tpartial-pass \"2/3 tests passing\" <<-\\EOF &&\n+\ttest_expect_success \"passing test #1\" \"true\"\n+\ttest_expect_success \"failing test #2\" \"false\"\n+\ttest_expect_success \"passing test #3\" \"true\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test partial-pass <<-\\\\EOF\n+\tcheck_sub_test_lib_test partial-pass <<-\\EOF\n \t> ok 1 - passing test #1\n \t> not ok 2 - failing test #2\n \t#\tfalse\n \t> ok 3 - passing test #3\n \t> # failed 1 among 3 test(s)\n \t> 1..3\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have a known breakage' \"\n-\trun_sub_test_lib_test failing-todo 'A failing TODO test' <<-\\\\EOF &&\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_failure 'pretend we have a known breakage' 'false'\n+test_expect_success 'pretend we have a known breakage' '\n+\trun_sub_test_lib_test failing-todo \"A failing TODO test\" <<-\\EOF &&\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_failure \"pretend we have a known breakage\" \"false\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test failing-todo <<-\\\\EOF\n+\tcheck_sub_test_lib_test failing-todo <<-\\EOF\n \t> ok 1 - passing test\n \t> not ok 2 - pretend we have a known breakage # TODO known breakage\n \t> # still have 1 known breakage(s)\n \t> # passed all remaining 1 test(s)\n \t> 1..2\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have fixed a known breakage' \"\n-\trun_sub_test_lib_test passing-todo 'A passing TODO test' <<-\\\\EOF &&\n-\ttest_expect_failure 'pretend we have fixed a known breakage' 'true'\n+test_expect_success 'pretend we have fixed a known breakage' '\n+\trun_sub_test_lib_test passing-todo \"A passing TODO test\" <<-\\EOF &&\n+\ttest_expect_failure \"pretend we have fixed a known breakage\" \"true\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test passing-todo <<-\\\\EOF\n+\tcheck_sub_test_lib_test passing-todo <<-\\EOF\n \t> ok 1 - pretend we have fixed a known breakage # TODO known breakage vanished\n \t> # 1 known breakage(s) vanished; please update test(s)\n \t> 1..1\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' \"\n+test_expect_success 'pretend we have fixed one of two known breakages (run in sub test-lib)' '\n \trun_sub_test_lib_test partially-passing-todos \\\n-\t\t'2 TODO tests, one passing' <<-\\\\EOF &&\n-\ttest_expect_failure 'pretend we have a known breakage' 'false'\n-\ttest_expect_success 'pretend we have a passing test' 'true'\n-\ttest_expect_failure 'pretend we have fixed another known breakage' 'true'\n+\t\t\"2 TODO tests, one passing\" <<-\\EOF &&\n+\ttest_expect_failure \"pretend we have a known breakage\" \"false\"\n+\ttest_expect_success \"pretend we have a passing test\" \"true\"\n+\ttest_expect_failure \"pretend we have fixed another known breakage\" \"true\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test partially-passing-todos <<-\\\\EOF\n+\tcheck_sub_test_lib_test partially-passing-todos <<-\\EOF\n \t> not ok 1 - pretend we have a known breakage # TODO known breakage\n \t> ok 2 - pretend we have a passing test\n \t> ok 3 - pretend we have fixed another known breakage # TODO known breakage vanished\n@@ -214,17 +214,17 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su\n \t> # passed all remaining 1 test(s)\n \t> 1..3\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have a pass, fail, and known breakage' \"\n+test_expect_success 'pretend we have a pass, fail, and known breakage' '\n \trun_sub_test_lib_test_err \\\n-\t\tmixed-results1 'mixed results #1' <<-\\\\EOF &&\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_success 'failing test' 'false'\n-\ttest_expect_failure 'pretend we have a known breakage' 'false'\n+\t\tmixed-results1 \"mixed results #1\" <<-\\EOF &&\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_success \"failing test\" \"false\"\n+\ttest_expect_failure \"pretend we have a known breakage\" \"false\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test mixed-results1 <<-\\\\EOF\n+\tcheck_sub_test_lib_test mixed-results1 <<-\\EOF\n \t> ok 1 - passing test\n \t> not ok 2 - failing test\n \t> #\tfalse\n@@ -233,24 +233,24 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' \"\n \t> # failed 1 among remaining 2 test(s)\n \t> 1..3\n \tEOF\n-\"\n+'\n \n-test_expect_success 'pretend we have a mix of all possible results' \"\n+test_expect_success 'pretend we have a mix of all possible results' '\n \trun_sub_test_lib_test_err \\\n-\t\tmixed-results2 'mixed results #2' <<-\\\\EOF &&\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_success 'passing test' 'true'\n-\ttest_expect_success 'failing test' 'false'\n-\ttest_expect_success 'failing test' 'false'\n-\ttest_expect_success 'failing test' 'false'\n-\ttest_expect_failure 'pretend we have a known breakage' 'false'\n-\ttest_expect_failure 'pretend we have a known breakage' 'false'\n-\ttest_expect_failure 'pretend we have fixed a known breakage' 'true'\n+\t\tmixed-results2 \"mixed results #2\" <<-\\EOF &&\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_success \"passing test\" \"true\"\n+\ttest_expect_success \"failing test\" \"false\"\n+\ttest_expect_success \"failing test\" \"false\"\n+\ttest_expect_success \"failing test\" \"false\"\n+\ttest_expect_failure \"pretend we have a known breakage\" \"false\"\n+\ttest_expect_failure \"pretend we have a known breakage\" \"false\"\n+\ttest_expect_failure \"pretend we have fixed a known breakage\" \"true\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test mixed-results2 <<-\\\\EOF\n+\tcheck_sub_test_lib_test mixed-results2 <<-\\EOF\n \t> ok 1 - passing test\n \t> ok 2 - passing test\n \t> ok 3 - passing test\n@@ -269,7 +269,7 @@ test_expect_success 'pretend we have a mix of all possible results' \"\n \t> # failed 3 among remaining 7 test(s)\n \t> 1..10\n \tEOF\n-\"\n+'\n \n test_expect_success C_LOCALE_OUTPUT 'test --verbose' '\n \trun_sub_test_lib_test_err \\\n@@ -321,39 +321,39 @@ test_expect_success 'test --verbose-only' '\n \tEOF\n '\n \n-test_expect_success 'GIT_SKIP_TESTS' \"\n+test_expect_success 'GIT_SKIP_TESTS' '\n \t(\n-\t\tGIT_SKIP_TESTS='git.2' && export GIT_SKIP_TESTS &&\n+\t\tGIT_SKIP_TESTS=\"git.2\" && export GIT_SKIP_TESTS &&\n \t\trun_sub_test_lib_test git-skip-tests-basic \\\n-\t\t\t'GIT_SKIP_TESTS' <<-\\\\EOF &&\n+\t\t\t\"GIT_SKIP_TESTS\" <<-\\EOF &&\n \t\tfor i in 1 2 3\n \t\tdo\n-\t\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\t\ttest_expect_success \"passing test #$i\" \"true\"\n \t\tdone\n \t\ttest_done\n \t\tEOF\n-\t\tcheck_sub_test_lib_test git-skip-tests-basic <<-\\\\EOF\n+\t\tcheck_sub_test_lib_test git-skip-tests-basic <<-\\EOF\n \t\t> ok 1 - passing test #1\n \t\t> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)\n \t\t> ok 3 - passing test #3\n \t\t> # passed all 3 test(s)\n \t\t> 1..3\n \t\tEOF\n \t)\n-\"\n+'\n \n-test_expect_success 'GIT_SKIP_TESTS several tests' \"\n+test_expect_success 'GIT_SKIP_TESTS several tests' '\n \t(\n-\t\tGIT_SKIP_TESTS='git.2 git.5' && export GIT_SKIP_TESTS &&\n+\t\tGIT_SKIP_TESTS=\"git.2 git.5\" && export GIT_SKIP_TESTS &&\n \t\trun_sub_test_lib_test git-skip-tests-several \\\n-\t\t\t'GIT_SKIP_TESTS several tests' <<-\\\\EOF &&\n+\t\t\t\"GIT_SKIP_TESTS several tests\" <<-\\EOF &&\n \t\tfor i in 1 2 3 4 5 6\n \t\tdo\n-\t\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\t\ttest_expect_success \"passing test #$i\" \"true\"\n \t\tdone\n \t\ttest_done\n \t\tEOF\n-\t\tcheck_sub_test_lib_test git-skip-tests-several <<-\\\\EOF\n+\t\tcheck_sub_test_lib_test git-skip-tests-several <<-\\EOF\n \t\t> ok 1 - passing test #1\n \t\t> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)\n \t\t> ok 3 - passing test #3\n@@ -364,20 +364,20 @@ test_expect_success 'GIT_SKIP_TESTS several tests' \"\n \t\t> 1..6\n \t\tEOF\n \t)\n-\"\n+'\n \n-test_expect_success 'GIT_SKIP_TESTS sh pattern' \"\n+test_expect_success 'GIT_SKIP_TESTS sh pattern' '\n \t(\n-\t\tGIT_SKIP_TESTS='git.[2-5]' && export GIT_SKIP_TESTS &&\n+\t\tGIT_SKIP_TESTS=\"git.[2-5]\" && export GIT_SKIP_TESTS &&\n \t\trun_sub_test_lib_test git-skip-tests-sh-pattern \\\n-\t\t\t'GIT_SKIP_TESTS sh pattern' <<-\\\\EOF &&\n+\t\t\t\"GIT_SKIP_TESTS sh pattern\" <<-\\EOF &&\n \t\tfor i in 1 2 3 4 5 6\n \t\tdo\n-\t\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\t\ttest_expect_success \"passing test #$i\" \"true\"\n \t\tdone\n \t\ttest_done\n \t\tEOF\n-\t\tcheck_sub_test_lib_test git-skip-tests-sh-pattern <<-\\\\EOF\n+\t\tcheck_sub_test_lib_test git-skip-tests-sh-pattern <<-\\EOF\n \t\t> ok 1 - passing test #1\n \t\t> ok 2 # skip passing test #2 (GIT_SKIP_TESTS)\n \t\t> ok 3 # skip passing test #3 (GIT_SKIP_TESTS)\n@@ -388,56 +388,56 @@ test_expect_success 'GIT_SKIP_TESTS sh pattern' \"\n \t\t> 1..6\n \t\tEOF\n \t)\n-\"\n+'\n \n-test_expect_success 'GIT_SKIP_TESTS entire suite' \"\n+test_expect_success 'GIT_SKIP_TESTS entire suite' '\n \t(\n-\t\tGIT_SKIP_TESTS='git' && export GIT_SKIP_TESTS &&\n+\t\tGIT_SKIP_TESTS=\"git\" && export GIT_SKIP_TESTS &&\n \t\trun_sub_test_lib_test git-skip-tests-entire-suite \\\n-\t\t\t'GIT_SKIP_TESTS entire suite' <<-\\\\EOF &&\n+\t\t\t\"GIT_SKIP_TESTS entire suite\" <<-\\EOF &&\n \t\tfor i in 1 2 3\n \t\tdo\n-\t\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\t\ttest_expect_success \"passing test #$i\" \"true\"\n \t\tdone\n \t\ttest_done\n \t\tEOF\n-\t\tcheck_sub_test_lib_test git-skip-tests-entire-suite <<-\\\\EOF\n+\t\tcheck_sub_test_lib_test git-skip-tests-entire-suite <<-\\EOF\n \t\t> 1..0 # SKIP skip all tests in git\n \t\tEOF\n \t)\n-\"\n+'\n \n-test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' \"\n+test_expect_success 'GIT_SKIP_TESTS does not skip unmatched suite' '\n \t(\n-\t\tGIT_SKIP_TESTS='notgit' && export GIT_SKIP_TESTS &&\n+\t\tGIT_SKIP_TESTS=\"notgit\" && export GIT_SKIP_TESTS &&\n \t\trun_sub_test_lib_test git-skip-tests-unmatched-suite \\\n-\t\t\t'GIT_SKIP_TESTS does not skip unmatched suite' <<-\\\\EOF &&\n+\t\t\t\"GIT_SKIP_TESTS does not skip unmatched suite\" <<-\\EOF &&\n \t\tfor i in 1 2 3\n \t\tdo\n-\t\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\t\ttest_expect_success \"passing test #$i\" \"true\"\n \t\tdone\n \t\ttest_done\n \t\tEOF\n-\t\tcheck_sub_test_lib_test git-skip-tests-unmatched-suite <<-\\\\EOF\n+\t\tcheck_sub_test_lib_test git-skip-tests-unmatched-suite <<-\\EOF\n \t\t> ok 1 - passing test #1\n \t\t> ok 2 - passing test #2\n \t\t> ok 3 - passing test #3\n \t\t> # passed all 3 test(s)\n \t\t> 1..3\n \t\tEOF\n \t)\n-\"\n+'\n \n-test_expect_success '--run basic' \"\n+test_expect_success '--run basic' '\n \trun_sub_test_lib_test run-basic \\\n-\t\t'--run basic' --run='1,3,5' <<-\\\\EOF &&\n+\t\t\"--run basic\" --run=\"1,3,5\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-basic <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-basic <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 # skip passing test #2 (--run)\n \t> ok 3 - passing test #3\n@@ -447,18 +447,18 @@ test_expect_success '--run basic' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with a range' \"\n+test_expect_success '--run with a range' '\n \trun_sub_test_lib_test run-range \\\n-\t\t'--run with a range' --run='1-3' <<-\\\\EOF &&\n+\t\t\"--run with a range\" --run=\"1-3\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-range <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-range <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 - passing test #3\n@@ -468,18 +468,18 @@ test_expect_success '--run with a range' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with two ranges' \"\n+test_expect_success '--run with two ranges' '\n \trun_sub_test_lib_test run-two-ranges \\\n-\t\t'--run with two ranges' --run='1-2,5-6' <<-\\\\EOF &&\n+\t\t\"--run with two ranges\" --run=\"1-2,5-6\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-two-ranges <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-two-ranges <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -489,18 +489,18 @@ test_expect_success '--run with two ranges' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with a left open range' \"\n+test_expect_success '--run with a left open range' '\n \trun_sub_test_lib_test run-left-open-range \\\n-\t\t'--run with a left open range' --run='-3' <<-\\\\EOF &&\n+\t\t\"--run with a left open range\" --run=\"-3\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-left-open-range <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-left-open-range <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 - passing test #3\n@@ -510,18 +510,18 @@ test_expect_success '--run with a left open range' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with a right open range' \"\n+test_expect_success '--run with a right open range' '\n \trun_sub_test_lib_test run-right-open-range \\\n-\t\t'--run with a right open range' --run='4-' <<-\\\\EOF &&\n+\t\t\"--run with a right open range\" --run=\"4-\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-right-open-range <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-right-open-range <<-\\EOF\n \t> ok 1 # skip passing test #1 (--run)\n \t> ok 2 # skip passing test #2 (--run)\n \t> ok 3 # skip passing test #3 (--run)\n@@ -531,18 +531,18 @@ test_expect_success '--run with a right open range' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with basic negation' \"\n+test_expect_success '--run with basic negation' '\n \trun_sub_test_lib_test run-basic-neg \\\n-\t\t'--run with basic negation' --run='\"'!3'\"' <<-\\\\EOF &&\n+\t\t\"--run with basic negation\" --run=\"!3\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-basic-neg <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-basic-neg <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -552,18 +552,18 @@ test_expect_success '--run with basic negation' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run with two negations' \"\n+test_expect_success '--run with two negations' '\n \trun_sub_test_lib_test run-two-neg \\\n-\t\t'--run with two negations' --run='\"'!3,!6'\"' <<-\\\\EOF &&\n+\t\t\"--run with two negations\" --run=\"!3,!6\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-two-neg <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-two-neg <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -573,18 +573,18 @@ test_expect_success '--run with two negations' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run a range and negation' \"\n+test_expect_success '--run a range and negation' '\n \trun_sub_test_lib_test run-range-and-neg \\\n-\t\t'--run a range and negation' --run='\"'-4,!2'\"' <<-\\\\EOF &&\n+\t\t\"--run a range and negation\" --run=\"-4,!2\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-range-and-neg <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-range-and-neg <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 # skip passing test #2 (--run)\n \t> ok 3 - passing test #3\n@@ -594,18 +594,18 @@ test_expect_success '--run a range and negation' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run range negation' \"\n+test_expect_success '--run range negation' '\n \trun_sub_test_lib_test run-range-neg \\\n-\t\t'--run range negation' --run='\"'!1-3'\"' <<-\\\\EOF &&\n+\t\t\"--run range negation\" --run=\"!1-3\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-range-neg <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-range-neg <<-\\EOF\n \t> ok 1 # skip passing test #1 (--run)\n \t> ok 2 # skip passing test #2 (--run)\n \t> ok 3 # skip passing test #3 (--run)\n@@ -615,19 +615,19 @@ test_expect_success '--run range negation' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run include, exclude and include' \"\n+test_expect_success '--run include, exclude and include' '\n \trun_sub_test_lib_test run-inc-neg-inc \\\n-\t\t'--run include, exclude and include' \\\n-\t\t--run='\"'1-5,!1-3,2'\"' <<-\\\\EOF &&\n+\t\t\"--run include, exclude and include\" \\\n+\t\t--run=\"1-5,!1-3,2\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-inc-neg-inc <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-inc-neg-inc <<-\\EOF\n \t> ok 1 # skip passing test #1 (--run)\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -637,19 +637,19 @@ test_expect_success '--run include, exclude and include' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run include, exclude and include, comma separated' \"\n+test_expect_success '--run include, exclude and include, comma separated' '\n \trun_sub_test_lib_test run-inc-neg-inc-comma \\\n-\t\t'--run include, exclude and include, comma separated' \\\n-\t\t--run=1-5,\\!1-3,2 <<-\\\\EOF &&\n+\t\t\"--run include, exclude and include, comma separated\" \\\n+\t\t--run=1-5,!1-3,2 <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-inc-neg-inc-comma <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-inc-neg-inc-comma <<-\\EOF\n \t> ok 1 # skip passing test #1 (--run)\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -659,19 +659,19 @@ test_expect_success '--run include, exclude and include, comma separated' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run exclude and include' \"\n+test_expect_success '--run exclude and include' '\n \trun_sub_test_lib_test run-neg-inc \\\n-\t\t'--run exclude and include' \\\n-\t\t--run='\"'!3-,5'\"' <<-\\\\EOF &&\n+\t\t\"--run exclude and include\" \\\n+\t\t--run=\"!3-,5\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-neg-inc <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-neg-inc <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 - passing test #2\n \t> ok 3 # skip passing test #3 (--run)\n@@ -681,19 +681,19 @@ test_expect_success '--run exclude and include' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run empty selectors' \"\n+test_expect_success '--run empty selectors' '\n \trun_sub_test_lib_test run-empty-sel \\\n-\t\t'--run empty selectors' \\\n-\t\t--run='1,,3,,,5' <<-\\\\EOF &&\n+\t\t\"--run empty selectors\" \\\n+\t\t--run=\"1,,3,,,5\" <<-\\EOF &&\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"passing test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"passing test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-empty-sel <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-empty-sel <<-\\EOF\n \t> ok 1 - passing test #1\n \t> ok 2 # skip passing test #2 (--run)\n \t> ok 3 - passing test #3\n@@ -703,20 +703,20 @@ test_expect_success '--run empty selectors' \"\n \t> # passed all 6 test(s)\n \t> 1..6\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run substring selector' \"\n+test_expect_success '--run substring selector' '\n \trun_sub_test_lib_test run-substring-selector \\\n-\t\t'--run empty selectors' \\\n-\t\t--run='relevant' <<-\\\\EOF &&\n-\ttest_expect_success \\\"relevant test\\\" 'true'\n+\t\t\"--run empty selectors\" \\\n+\t\t--run=\"relevant\" <<-\\EOF &&\n+\ttest_expect_success \"relevant test\" \"true\"\n \tfor i in 1 2 3 4 5 6\n \tdo\n-\t\ttest_expect_success \\\"other test #\\$i\\\" 'true'\n+\t\ttest_expect_success \"other test #$i\" \"true\"\n \tdone\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test run-substring-selector <<-\\\\EOF\n+\tcheck_sub_test_lib_test run-substring-selector <<-\\EOF\n \t> ok 1 - relevant test\n \t> ok 2 # skip other test #1 (--run)\n \t> ok 3 # skip other test #2 (--run)\n@@ -727,37 +727,37 @@ test_expect_success '--run substring selector' \"\n \t> # passed all 7 test(s)\n \t> 1..7\n \tEOF\n-\"\n+'\n \n-test_expect_success '--run keyword selection' \"\n+test_expect_success '--run keyword selection' '\n \trun_sub_test_lib_test_err run-inv-range-start \\\n-\t\t'--run invalid range start' \\\n-\t\t--run='a-5' <<-\\\\EOF &&\n-\ttest_expect_success \\\"passing test #1\\\" 'true'\n+\t\t\"--run invalid range start\" \\\n+\t\t--run=\"a-5\" <<-\\EOF &&\n+\ttest_expect_success \"passing test #1\" \"true\"\n \ttest_done\n \tEOF\n \tcheck_sub_test_lib_test_err run-inv-range-start \\\n-\t\t<<-\\\\EOF_OUT 3<<-\\\\EOF_ERR\n+\t\t<<-\\EOF_OUT 3<<-EOF_ERR\n \t> FATAL: Unexpected exit with code 1\n \tEOF_OUT\n-\t> error: --run: invalid non-numeric in range start: 'a-5'\n+\t> error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}\n \tEOF_ERR\n-\"\n+'\n \n-test_expect_success '--run invalid range end' \"\n+test_expect_success '--run invalid range end' '\n \trun_sub_test_lib_test_err run-inv-range-end \\\n-\t\t'--run invalid range end' \\\n-\t\t--run='1-z' <<-\\\\EOF &&\n-\ttest_expect_success \\\"passing test #1\\\" 'true'\n+\t\t\"--run invalid range end\" \\\n+\t\t--run=\"1-z\" <<-\\EOF &&\n+\ttest_expect_success \"passing test #1\" \"true\"\n \ttest_done\n \tEOF\n \tcheck_sub_test_lib_test_err run-inv-range-end \\\n-\t\t<<-\\\\EOF_OUT 3<<-\\\\EOF_ERR\n+\t\t<<-\\EOF_OUT 3<<-EOF_ERR\n \t> FATAL: Unexpected exit with code 1\n \tEOF_OUT\n-\t> error: --run: invalid non-numeric in range end: '1-z'\n+\t> error: --run: invalid non-numeric in range end: ${SQ}1-z${SQ}\n \tEOF_ERR\n-\"\n+'\n \n test_expect_success 'tests respect prerequisites' '\n \trun_sub_test_lib_test prereqs \"tests respect prereqs\" <<-\\EOF &&\n@@ -839,18 +839,18 @@ test_expect_success 'nested lazy prerequisites' '\n \tEOF\n '\n \n-test_expect_success 'lazy prereqs do not turn off tracing' \"\n+test_expect_success 'lazy prereqs do not turn off tracing' '\n \trun_sub_test_lib_test lazy-prereq-and-tracing \\\n-\t\t'lazy prereqs and -x' -v -x <<-\\\\EOF &&\n+\t\t\"lazy prereqs and -x\" -v -x <<-\\EOF &&\n \ttest_lazy_prereq LAZY true\n \n-\ttest_expect_success lazy 'test_have_prereq LAZY && echo trace'\n+\ttest_expect_success lazy \"test_have_prereq LAZY && echo trace\"\n \n \ttest_done\n \tEOF\n \n-\tgrep 'echo trace' lazy-prereq-and-tracing/err\n-\"\n+\tgrep \"echo trace\" lazy-prereq-and-tracing/err\n+'\n \n test_expect_success 'tests clean up after themselves' '\n \trun_sub_test_lib_test cleanup \"test with cleanup\" <<-\\EOF &&\n@@ -872,20 +872,20 @@ test_expect_success 'tests clean up after themselves' '\n \tEOF\n '\n \n-test_expect_success 'tests clean up even on failures' \"\n+test_expect_success 'tests clean up even on failures' '\n \trun_sub_test_lib_test_err \\\n-\t\tfailing-cleanup 'Failing tests with cleanup commands' <<-\\\\EOF &&\n-\ttest_expect_success 'tests clean up even after a failure' '\n+\t\tfailing-cleanup \"Failing tests with cleanup commands\" <<-\\EOF &&\n+\ttest_expect_success \"tests clean up even after a failure\" \"\n \t\ttouch clean-after-failure &&\n \t\ttest_when_finished rm clean-after-failure &&\n \t\t(exit 1)\n-\t'\n-\ttest_expect_success 'failure to clean up causes the test to fail' '\n+\t\"\n+\ttest_expect_success \"failure to clean up causes the test to fail\" \"\n \t\ttest_when_finished \\\"(exit 2)\\\"\n-\t'\n+\t\"\n \ttest_done\n \tEOF\n-\tcheck_sub_test_lib_test failing-cleanup <<-\\\\EOF\n+\tcheck_sub_test_lib_test failing-cleanup <<-\\EOF\n \t> not ok 1 - tests clean up even after a failure\n \t> #\tZ\n \t> #\ttouch clean-after-failure &&\n@@ -894,30 +894,30 @@ test_expect_success 'tests clean up even on failures' \"\n \t> #\tZ\n \t> not ok 2 - failure to clean up causes the test to fail\n \t> #\tZ\n-\t> #\ttest_when_finished \\\"(exit 2)\\\"\n+\t> #\ttest_when_finished \"(exit 2)\"\n \t> #\tZ\n \t> # failed 2 among 2 test(s)\n \t> 1..2\n \tEOF\n-\"\n+'\n \n-test_expect_success 'test_atexit is run' \"\n+test_expect_success 'test_atexit is run' '\n \trun_sub_test_lib_test_err \\\n-\t\tatexit-cleanup 'Run atexit commands' -i <<-\\\\EOF &&\n-\ttest_expect_success 'tests clean up even after a failure' '\n+\t\tatexit-cleanup \"Run atexit commands\" -i <<-\\EOF &&\n+\ttest_expect_success \"tests clean up even after a failure\" \"\n \t\t> ../../clean-atexit &&\n \t\ttest_atexit rm ../../clean-atexit &&\n \t\t> ../../also-clean-atexit &&\n \t\ttest_atexit rm ../../also-clean-atexit &&\n \t\t> ../../dont-clean-atexit &&\n \t\t(exit 1)\n-\t'\n+\t\"\n \ttest_done\n \tEOF\n \ttest_path_is_file dont-clean-atexit &&\n \ttest_path_is_missing clean-atexit &&\n \ttest_path_is_missing also-clean-atexit\n-\"\n+'\n \n test_expect_success 'test_oid provides sane info by default' '\n \ttest_oid zero >actual &&\n-- \n2.29.2.730.g3e418f96ba\n"},{"id":"410402","messageId":"xmqqlfewvmmj.fsf@gitster.c.googlers.com","threadId":"54669","inReplyTo":"20201120001458.GA274082@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] tests: make sure nested lazy prereqs work reliably","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-11-20T01:32:36Z","receivedAt":"2020-11-20T01:33:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> OK, then here's the whole thing. I ended up with a few more cleanups,\n> too. This is all on top of Gábor's patches. It's conceptually\n> independent, but the textual wrangling was annoying enough it didn't\n> make any sense to require you to do it again during merging. ;) Plus I\n> do not think either topic is high-risk nor urgent enough to worry too\n> much about one blocking the other.\n\nI'll ask you to do the last step maybe in a few weeks; the range\nnotation tests seem to have changed since where I queued Gábor's\npatches and where [4/4] is based on (yours is based on newer\ncodebase).\n\n> The diffstat is scary, but it's mostly the final patch, which is pretty\n> mechanical.\n\nYup, and the result is much easier to read.\n\nThanks.\n\n>   [1/4]: t0000: keep clean-up tests together\n>   [2/4]: t0000: run prereq tests inside sub-test\n>   [3/4]: t0000: run cleaning test inside sub-test\n>   [4/4]: t0000: consistently use single quotes for outer tests\n>\n>  t/t0000-basic.sh | 570 +++++++++++++++++++++++------------------------\n>  1 file changed, 284 insertions(+), 286 deletions(-)\n>\n> -Peff\n"}]}