Re: [PATCH v2 2/2] tests: replace mingw_test_cmp with a helper in C

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Sep 7, 2022, 11:57 UTC
Message-ID: <220907.86v8pzl6jz.gmgdl@evledraar.gmail.com>
In-Reply-To: <1f5366f137967cbec30041b40eedd86ce5f6e953.1662469859.git.gitgitgadget@gmail.com>

On Tue, Sep 06 2022, Johannes Schindelin via GitGitGadget wrote:

Show 8 quoted lines

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> [...]
> +++ b/t/helper/test-text-cmp.c
> @@ -0,0 +1,78 @@
> +#include "test-tool.h"
> +#include "git-compat-util.h"
> +#include "strbuf.h"
> +#include "gettext.h"

Superflous header? Compiles without gettext.h for me (and we shouldn't use i18n in test helpers).

Show 8 quoted lines

> [...]
> +int cmd__text_cmp(int argc, const char **argv)
> +{
> +	FILE *f0, *f1;
> +	struct strbuf b0 = STRBUF_INIT, b1 = STRBUF_INIT;
> +
> +	if (argc != 3)
> +		die("Require exactly 2 arguments, got %d", argc);

Here you conflate the argc v.s. arguments minus the "text-cmp", resulting in:

	helper/test-tool text-cmp 2
        fatal: Require exactly 2 arguments, got 2

An argc-- argv++ at the beginning seems like the easiest way out of this. Also s/Require/require/ per CodingGuidelines.

Show 9 quoted lines

> +	if (!strcmp(argv[1], "-") && !strcmp(argv[2], "-"))
> +		die("only one parameter can refer to `stdin` but not both");
> +
> +	if (!(f0 = !strcmp(argv[1], "-") ? stdin : fopen(argv[1], "r")))
> +		return error_errno("could not open '%s'", argv[1]);
> +	if (!(f1 = !strcmp(argv[2], "-") ? stdin : fopen(argv[2], "r"))) {
> +		fclose(f0);
> +		return error_errno("could not open '%s'", argv[2]);
> +	}

Faithfully emulating the old version. I do wonder if we couldn't simply adjust the handful of tests that actually make use of the "-" diff(1) feature. AFAICT there's around 10 of those at most, and they all seem like cases where it would be easy to change:

	(echo foo) | test_cmp - actual

Or whatever, to:

	echo foo >expected &&
	test_cmp expected actual

...

> +			if (!strcmp(argv[1], "-") || !strcmp(argv[2], "-"))
> +				warning("cannot show diff because `stdin` was already consumed");

...

Which means we wouldn't need to punt on this.

> +			else if (!run_diff(argv[1], argv[2]))
> +				die("Huh? 'diff --no-index %s %s' succeeded",
> +				    argv[1], argv[2]);

I tried manually testing this with:

	GIT_TRACE=1 GIT_TEST_CMP="/home/avar/g/git/git diff --no-index --" ./t0021-conversion.sh  -vixd

v.s.:

	GIT_TRACE=1 GIT_TEST_CMP="$PWD/helper/test-tool text-cmp" ./t0021-conversion.sh  -vixd

Your version doesn't get confused by the same, but AFAICT this is by fragile accident.

I.e. you run your own equivalent of "cmp", so because the files are the same in that case we don't run the "diff --no-index".

But the "diff --no-index" in that t0021*.sh case *would* report differences, even though the files are byte-for-byte identical.

So the "cmp"-a-like here isn't just an optimization to avoid forking the "git diff" process, it's an entirely different comparison method in cases where we have a "filter".

It just so happens that our test suite doesn't currently combine them in a way that causes a current failure.

Show 5 quoted lines

>  test_cmp () {
>  	test "$#" -ne 2 && BUG "2 param"
> -	eval "$GIT_TEST_CMP" '"$@"'
> +	GIT_ALLOC_LIMIT=0 eval "$GIT_TEST_CMP" '"$@"'
>  }

Further, we have a clear boundary in the test suite between "git" and "test-tool" things we invoke, and third party tools. The former we put in "test_must_fail_acceptable".

When using this new helper we'd hide potential segfaults and BUGs in any "! test_cmp" invocation..

To avoid the introduction of such a blindspot we'd need to change "test_cmp" to take an optional "!" as the 1st argument, and convert the existing "! test_cmp" to "test_cmp !", then carry some flag to indicate that our "GIT_TEST_CMP" is a git or test-tool invocation, and check it appropriately.

Show 14 quoted lines

> [...]
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 7726d1da88a..0be25ecbd59 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -1546,7 +1546,7 @@ case $uname_s in
>  	test_set_prereq SED_STRIPS_CR
>  	test_set_prereq GREP_STRIPS_CR
>  	test_set_prereq WINDOWS
> -	GIT_TEST_CMP=mingw_test_cmp
> +	GIT_TEST_CMP="test-tool text-cmp"
>  	;;
>  *CYGWIN*)
>  	test_set_prereq POSIXPERM

Not a new problem, but this is incompatible with GIT_TEST_CMP_USE_COPIED_CONTEXT.

What is new though is that with this series there's no longer a good reason AFAICT to carry GIT_TEST_CMP_USE_COPIED_CONTEXT at all. I.e. we have it for a "diff" that doesn't understand "-u".

If (after getting past tho caveats noted above) we could simply invoke our own test-tool we could drop that special-casing & just always invoke our own test_cmp helper.