Re: [PATCH v3 2/2] tests(mingw): avoid very slow `mingw_test_cmp`

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Nov 19, 2022, 11:54 UTC
Message-ID: <221119.86wn7rdugi.gmgdl@evledraar.gmail.com>
In-Reply-To: <Y3hF1vOQ8waqpvXo@nand.local>

On Fri, Nov 18 2022, Taylor Blau wrote:

Show 22 quoted lines

> On Fri, Nov 18, 2022 at 03:19:55PM -0800, Junio C Hamano wrote:
>> Well "! test_cmp" is wrong anyway, because it _expects_ two files
>> are the same and gives more detailed diagnosis when they differ by
>> giving "diff" output.
>>
>> If you expect them to be different, "! test_cmp" would give
>> "detailed diagnosis" in the wrong case, i.e. the outcome is what we
>> expect.
>
> I agree that "! test_cmp foo bar" will give output about how "foo" is
> different from "bar" (and halt the test only when the two have the same
> contents).
>
>> So the caller must do "test_cmp !" whether the underlying
>> implementation of test_cmp uses "diff -u" or "diff --no-index".
>
> But this confuses me. "git grep 'test_cmp !'" turns up no results, and
> furthermore, test_cmp() itself begins with:
>
>     test "$#" -ne 2 && BUG "2 param"
>
> So I am not sure what you are referring to.

I think we've got got a bit sidetracked here.

Junio's pointing out that a test that would do:

	! diff a b

Is pretty stupid, why would you want to show a diff of differences, if you're expecting it to be different? Just use:

	! cmp a b

Or something. That's correct. So I think probably those "! test_cmp" uses would be good candidates for some clean-up to make the test suite pretty.

But what I'm pointing out is that by having "test_cmp" be something that invokes "git" (or a helper) we *must not* negate it using the shell's negation, we have to use "test_must_fail" for anything we build ourselves.

So, the proposed patch would make the existing cases where we do:

	! test_cmp

Cases where we'd hide a segfault, whereas before it was maybe a bit odd, but harmless, and working as intended in the sense of reliably passing if we had differences.

So no, nothing currently does "test_cmp !", I was suggesting that upthread as the least invasive way of having "test_cmp" safe'd under negation, now that we were having it call out to "git diff" (or a helper...). It's the pattern we use for our own test helpers that invoke our own binaries (and if we don't, that's a bug).