{"thread":{"id":"52238","subject":"[PATCH] test-lib: don't check prereqs of test cases that won't be run anyway","startedAt":"2019-11-12T12:24:45Z","lastAt":"2019-11-12T19:29:20Z","messageCount":3,"participants":["SZEDER Gábor","Jeff King","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"386011","messageId":"20191112122438.17057-1-szeder.dev@gmail.com","threadId":"52238","inReplyTo":null,"subject":"[PATCH] test-lib: don't check prereqs of test cases that won't be run anyway","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-11-12T12:24:38Z","receivedAt":"2019-11-12T12:24:45Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"With './t1234-foo.sh -r 5,6' we can run only specific test cases in a\ntest script, but our test framwork still evaluates all lazy prereqs\nthat the excluded test cases might depend on.  This is unnecessary and\nproduces verbose and trace output that can be distracting.  This has\nbeen an issue ever since the '-r|--run=' options were introduced in\n0445e6f0a1 (test-lib: '--run' to run only specific tests, 2014-04-30),\nbecause that commit added the check of the list of test cases\nspecified with '-r' after evaluating the prereqs.\n\nAvoid this unnecessary prereq evaluation by checking the list of test\ncases specified with '-r' before looking at the prereqs.\n\nNote that GIT_SKIP_TESTS has always been checked before the prereqs,\nso prereqs necessary for tests skipped that way were not evaluated.\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n\nFollowing up on:\n\n  https://public-inbox.org/git/20190923165828.GA27068@szeder.dev/\n\n t/test-lib.sh | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex e06fa02a0e..24b541f494 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1000,6 +1000,12 @@ test_skip () {\n \t\tto_skip=t\n \t\tskipped_reason=\"GIT_SKIP_TESTS\"\n \tfi\n+\tif test -z \"$to_skip\" && test -n \"$run_list\" &&\n+\t   ! match_test_selector_list '--run' $test_count \"$run_list\"\n+\tthen\n+\t\tto_skip=t\n+\t\tskipped_reason=\"--run\"\n+\tfi\n \tif test -z \"$to_skip\" && test -n \"$test_prereq\" &&\n \t   ! test_have_prereq \"$test_prereq\"\n \tthen\n@@ -1012,12 +1018,6 @@ test_skip () {\n \t\tfi\n \t\tskipped_reason=\"missing $missing_prereq${of_prereq}\"\n \tfi\n-\tif test -z \"$to_skip\" && test -n \"$run_list\" &&\n-\t\t! match_test_selector_list '--run' $test_count \"$run_list\"\n-\tthen\n-\t\tto_skip=t\n-\t\tskipped_reason=\"--run\"\n-\tfi\n \n \tcase \"$to_skip\" in\n \tt)\n-- \n2.24.0.388.gde53c094ea\n\n"},{"id":"386013","messageId":"20191112140554.GA13771@sigill.intra.peff.net","threadId":"52238","inReplyTo":"20191112122438.17057-1-szeder.dev@gmail.com","subject":"Re: [PATCH] test-lib: don't check prereqs of test cases that won't be run anyway","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-11-12T14:05:54Z","receivedAt":"2019-11-12T14:05:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Nov 12, 2019 at 01:24:38PM +0100, SZEDER Gábor wrote:\n\n> With './t1234-foo.sh -r 5,6' we can run only specific test cases in a\n> test script, but our test framwork still evaluates all lazy prereqs\n> that the excluded test cases might depend on.  This is unnecessary and\n> produces verbose and trace output that can be distracting.  This has\n> been an issue ever since the '-r|--run=' options were introduced in\n> 0445e6f0a1 (test-lib: '--run' to run only specific tests, 2014-04-30),\n> because that commit added the check of the list of test cases\n> specified with '-r' after evaluating the prereqs.\n> \n> Avoid this unnecessary prereq evaluation by checking the list of test\n> cases specified with '-r' before looking at the prereqs.\n> \n> Note that GIT_SKIP_TESTS has always been checked before the prereqs,\n> so prereqs necessary for tests skipped that way were not evaluated.\n\nThanks. This definitely seems like an improvement. I verified that it\nworks in practice, and the patch itself looks obviously correct.\n\nOne of the things I had to double check is how to_skip is handled, since\nwe check it in the moved block. It's fine, but I wondered if the whole\nthing would be easier as an if/else cascade with an early return. Like\nthe patch below (on top of your patch; try \"diff -w\" to quiet the\nindentation noise).\n\nBut maybe it's not worth the churn (and if we pursue it at all, it\nshould not hold up your patch).\n\n-Peff\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 24b541f494..a1784114cb 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -993,49 +993,38 @@ test_finish_ () {\n }\n \n test_skip () {\n-\tto_skip=\n \tskipped_reason=\n \tif match_pattern_list $this_test.$test_count $GIT_SKIP_TESTS\n \tthen\n-\t\tto_skip=t\n \t\tskipped_reason=\"GIT_SKIP_TESTS\"\n-\tfi\n-\tif test -z \"$to_skip\" && test -n \"$run_list\" &&\n+\telif test -n \"$run_list\" &&\n \t   ! match_test_selector_list '--run' $test_count \"$run_list\"\n \tthen\n-\t\tto_skip=t\n \t\tskipped_reason=\"--run\"\n-\tfi\n-\tif test -z \"$to_skip\" && test -n \"$test_prereq\" &&\n+\telif\n+\tif test -n \"$test_prereq\" &&\n \t   ! test_have_prereq \"$test_prereq\"\n \tthen\n-\t\tto_skip=t\n-\n \t\tof_prereq=\n \t\tif test \"$missing_prereq\" != \"$test_prereq\"\n \t\tthen\n \t\t\tof_prereq=\" of $test_prereq\"\n \t\tfi\n \t\tskipped_reason=\"missing $missing_prereq${of_prereq}\"\n+\telse\n+\t\treturn 1\n \tfi\n \n-\tcase \"$to_skip\" in\n-\tt)\n-\t\tif test -n \"$write_junit_xml\"\n-\t\tthen\n-\t\t\tmessage=\"$(xml_attr_encode \"$skipped_reason\")\"\n-\t\t\twrite_junit_xml_testcase \"$1\" \\\n-\t\t\t\t\"      <skipped message=\\\"$message\\\" />\"\n-\t\tfi\n+\tif test -n \"$write_junit_xml\"\n+\tthen\n+\t\tmessage=\"$(xml_attr_encode \"$skipped_reason\")\"\n+\t\twrite_junit_xml_testcase \"$1\" \\\n+\t\t\t\"      <skipped message=\\\"$message\\\" />\"\n+\tfi\n \n-\t\tsay_color skip >&3 \"skipping test: $@\"\n-\t\tsay_color skip \"ok $test_count # skip $1 ($skipped_reason)\"\n-\t\t: true\n-\t\t;;\n-\t*)\n-\t\tfalse\n-\t\t;;\n-\tesac\n+\tsay_color skip >&3 \"skipping test: $@\"\n+\tsay_color skip \"ok $test_count # skip $1 ($skipped_reason)\"\n+\treturn 0\n }\n \n # stub; perf-lib overrides it\n"},{"id":"386039","messageId":"nycvar.QRO.7.76.6.1911122028270.46@tvgsbejvaqbjf.bet","threadId":"52238","inReplyTo":"20191112122438.17057-1-szeder.dev@gmail.com","subject":"Re: [PATCH] test-lib: don't check prereqs of test cases that won't be run anyway","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-11-12T19:28:56Z","receivedAt":"2019-11-12T19:29:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Tue, 12 Nov 2019, SZEDER Gábor wrote:\n\n> With './t1234-foo.sh -r 5,6' we can run only specific test cases in a\n> test script, but our test framwork still evaluates all lazy prereqs\n> that the excluded test cases might depend on.  This is unnecessary and\n> produces verbose and trace output that can be distracting.  This has\n> been an issue ever since the '-r|--run=' options were introduced in\n> 0445e6f0a1 (test-lib: '--run' to run only specific tests, 2014-04-30),\n> because that commit added the check of the list of test cases\n> specified with '-r' after evaluating the prereqs.\n>\n> Avoid this unnecessary prereq evaluation by checking the list of test\n> cases specified with '-r' before looking at the prereqs.\n>\n> Note that GIT_SKIP_TESTS has always been checked before the prereqs,\n> so prereqs necessary for tests skipped that way were not evaluated.\n>\n> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n> ---\n\nThank you for this patch!\nDscho\n\n>\n> Following up on:\n>\n>   https://public-inbox.org/git/20190923165828.GA27068@szeder.dev/\n>\n>  t/test-lib.sh | 12 ++++++------\n>  1 file changed, 6 insertions(+), 6 deletions(-)\n>\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index e06fa02a0e..24b541f494 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1000,6 +1000,12 @@ test_skip () {\n>  \t\tto_skip=t\n>  \t\tskipped_reason=\"GIT_SKIP_TESTS\"\n>  \tfi\n> +\tif test -z \"$to_skip\" && test -n \"$run_list\" &&\n> +\t   ! match_test_selector_list '--run' $test_count \"$run_list\"\n> +\tthen\n> +\t\tto_skip=t\n> +\t\tskipped_reason=\"--run\"\n> +\tfi\n>  \tif test -z \"$to_skip\" && test -n \"$test_prereq\" &&\n>  \t   ! test_have_prereq \"$test_prereq\"\n>  \tthen\n> @@ -1012,12 +1018,6 @@ test_skip () {\n>  \t\tfi\n>  \t\tskipped_reason=\"missing $missing_prereq${of_prereq}\"\n>  \tfi\n> -\tif test -z \"$to_skip\" && test -n \"$run_list\" &&\n> -\t\t! match_test_selector_list '--run' $test_count \"$run_list\"\n> -\tthen\n> -\t\tto_skip=t\n> -\t\tskipped_reason=\"--run\"\n> -\tfi\n>\n>  \tcase \"$to_skip\" in\n>  \tt)\n> --\n> 2.24.0.388.gde53c094ea\n>\n>\n"}]}