Re: [PATCH 2/2] test-lib: GIT_TEST_ONLY to run only specific tests

From: Junio C Hamano <gitster@pobox.com>
Date: Mar 3, 2014, 23:26 UTC
Message-ID: <xmqqob1mn9t6.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CAPig+cT46ekT87TuoTtwvt0G+DraB9cdgW1wd6NsodjJ5FmJrA@mail.gmail.com>

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Mon, Mar 3, 2014 at 5:24 AM, Ilya Bobyr <ilya.bobyr@gmail.com> wrote:
>> This is a counterpart to GIT_SKIP_TESTS.  Mostly useful when debugging.
>
> To be grammatically similar to GIT_SKIP_TESTS, perhaps name it GIT_RUN_TESTS?

I actually do not like the interface to use two variables very much. Can't we just allow negative entries on "to be skipped" list?

That is

	GIT_SKIP_TESTS='t9??? !t91??'

would skip nine-thousand series, but would run 91xx series, and all the others are not excluded.

Simple rules to consider:

 - If the list consists of _only_ negated patterns, pretend that
   there is "unless otherwise specified with negatives, skip all
   tests", i.e. treat GIT_SKIP_TESTS='!t91??' just the same way you
   would treat GIT_SKIP_TESTS='* !t91??'.

 - The orders should not matter for simplicity of the semantics;
   before running each test, check if it matches any negative (and
   run it if it matches, without looking at any positives), and
   otherwise check if it matches any positive (and skip it if it
   does not).

Hmm?

Show 62 quoted lines

>> ---
>>  t/README      |   15 +++++++++++++++
>>  t/test-lib.sh |    8 ++++++++
>>  2 files changed, 23 insertions(+), 0 deletions(-)
>>
>> diff --git a/t/README b/t/README
>> index caeeb9d..f939987 100644
>> --- a/t/README
>> +++ b/t/README
>> @@ -187,6 +187,21 @@ and either can match the "t[0-9]{4}" part to skip the whole
>>  test, or t[0-9]{4} followed by ".$number" to say which
>>  particular test to skip.
>>
>> +Sometimes the opposite is desired - ability to execute only one or
>> +several tests.  Mostly while debugging tests.  For that you can say
>> +
>> +    $ GIT_TEST_ONLY=t9200.8 sh ./t9200-git-cvsexport-commit.sh
>> +
>> +or, similrary to GIT_SKIP_TESTS
>> +
>> +    $ GIT_TEST_ONLY='t[0-4]??? t91?? t9200.8' make
>> +
>> +In additiona to matching against "<test suite number>.<test number>"
>
> s/additiona/addition/
>
> Plus the other typos already mentioned by Philip...
>
>> +GIT_TEST_ONLY is matched against just the test numbes.  This comes
>> +handy when you are running only one test:
>> +
>> +    $ GIT_TEST_ONLY='[0-8]' sh ./t9200-git-cvsexport-commit.sh
>> +
>>  Note that some tests in the existing test suite rely on previous
>>  test item, so you cannot arbitrarily disable one and expect the
>>  remainder of test to check what the test originally was intended
>> diff --git a/t/test-lib.sh b/t/test-lib.sh
>> index 89a405b..12bf436 100644
>> --- a/t/test-lib.sh
>> +++ b/t/test-lib.sh
>> @@ -464,6 +464,14 @@ test_skip () {
>>                 fi
>>                 skipped_reason="missing $missing_prereq${of_prereq}"
>>         fi
>> +       if test -z "$to_skip" && test -n "$GIT_TEST_ONLY" &&
>> +               ! match_pattern_list $this_test.$test_count $GIT_TEST_ONLY &&
>> +               ! match_pattern_list $test_count $GIT_TEST_ONLY
>> +       then
>> +               to_skip=t
>> +               skipped_reason="not in GIT_TEST_ONLY"
>> +       fi
>> +
>>         case "$to_skip" in
>>         t)
>>                 say_color skip >&3 "skipping test: $@"
>> --
>> 1.7.9
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html