Re: [PATCH v2 2/3] t/*: avoid "whitelist"

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jul 19, 2022, 15:26 UTC
Message-ID: <220719.86lespun6l.gmgdl@evledraar.gmail.com>
In-Reply-To: <75cc0f43-d9a5-45d3-5e36-dd20acfebf50@github.com>

On Tue, Jul 19 2022, Derrick Stolee wrote:

Show 25 quoted lines

> On 7/15/2022 7:02 AM, Ævar Arnfjörð Bjarmason wrote:
>>>  GIT_TEST_PASSING_SANITIZE_LEAK=<boolean> when compiled with
>>> -SANITIZE=leak will run only those tests that have whitelisted
>>> -themselves as passing with no memory leaks. Tests can be whitelisted
>>> -by setting "TEST_PASSES_SANITIZE_LEAK=true" before sourcing
>>> -"test-lib.sh" itself at the top of the test script. This test mode is
>>> -used by the "linux-leaks" CI target.
>>> +SANITIZE=leak will run only those tests that have marked themselves as
>>> +passing with no memory leaks by setting "TEST_PASSES_SANITIZE_LEAK=true"
>>> +before sourcing "test-lib.sh" itself at the top of the test script. This
>>> +test mode is used by the "linux-leaks" CI target.
>> 
>> It's hard to improve your own verbage, but I think in this case my
>> original version can be improved still:
>> 
>> 	GIT_TEST_PASSING_SANITIZE_LEAK=<bool> when compiled with
>> 	SANITIZE=leak will, when true, only run those tests that declare
>> 	themselves leak-free by setting "TEST_PASSES_SANITIZE_LEAK=true"
>> 	before sourcing "test-lib.sh". This test mode is used by the
>> 	"linux-leaks" CI target.
>
> Another iteration:
>
>   GIT_TEST_PASSING_SANITIZE_LEAK=<bool> focuses the test suite on finding
>   memory leaks.

In some ways it does the opposite of "focusing on finding memory leaks", we're explicitly finding fewer memory leaks, what we're doing is asserting that the tests we've whitelisted still pass.

>   SANITIZE=leak, only run those tests that declare themselves leak-free by
>   setting "TEST_PASSES_SANITIZE_LEAK=true" before sourcing "test-lib.sh".
>   This test mode is used by the "linux-leaks" CI target.

Perhaps:

	GIT_TEST_PASSING_SANITIZE_LEAK=true skips those tests that
	haven't declared themselves as leak-free by setting
	"TEST_PASSES_SANITIZE_LEAK=true" before sourcing
	"test-lib.sh". This test mode is used by the "linux-leaks" CI
	target.

Anyway, do you mind if this part is dropped from this series?

I have a set of patches I was meaning to submit to add a tri-state [false,true,check] support for this, which this conflicts with.

"check" being a mode where we check that the tests in "false" aren't yet passing. So for that I'll need to re-word this whole thing anyway. I'll rephrase this while I'm at it to something I think you'll be OK with.

Show 12 quoted lines

>>> -test_description='test protocol whitelisting with submodules'
>>> +test_description='test protocol restrictions with submodules'
>> 
>> Minor: I think this shows the awkwardness of using a word derived from
>> "allow". Before we could use "whitelist" and "whitelisting"
>> consistentlry, but now you have "allowed", "allowlist", "restrictions"
>> etc.
>> 
>> I guess you could say "test protocol allowances..." or something? Meh.
>
> Perhaps "filtering" is the best way to describe the higher-level
> feature that these lists help to implement.

*Shrug*. I think "filtering" more naturally refers to the process, i.e. in Makefile syntax:

	$(filter-out $(A),$(LIST))
	$(filter $(B),$(LIST))

I'd call $(B) a whitelist (or whatever), $(A) a blacklist (or whatever), but "filtering" the process of producing them.