t7405.17 breakage vanishes with GETTEXT_POISON=1

Something fishy is going on but I don't think I'll spend time hunting it down so I post here in case somebody else is interested. It might also indicate a problem with poison gettext, not the test case too.

On Sun, Oct 28, 2018 at 06:41:06AM +0100, Duy Nguyen wrote:

I haven't actually run the test under GETTEXT_POISON, but I stongly suspect it's the test, or more accurately the helper function 'test_i18ngrep'.

The test in question runs

  test_i18ngrep ! "refusing to lose untracked file at" err

which fails in normal test runs, because 'grep' does find the undesired string; that's the known breakage. Under GETTEXT_POISION, however, 'test_i18ngrep' always succeeds because of this condition:

  if test -n "$GETTEXT_POISON"
  then
          # pretend success
          return 0
  fi

and then in turn the whole test succeeds.

SZEDER Gábor <szeder.dev@gmail.com> writes:

Ah, good spotting.

If a test using "grep" declares that something must not exist in plumbing message, it writes "! grep something", and because "test_i18ngrep something" always pretends that something is found under poisoned build (by the way, would this change when we remove the separate poisoned build?), "test_i18ngrep ! something" must be used for Porcelain messages to say the same thing.

Of course, this has a funny interactions with test_expect_failure. I actually do not think the complexity to work this around is worth it.

Changing behaviour of "test_i18ngrep ! something" to always fail under poisoned build would not work, of course, and changing it to always fail under poisoned build inside test_expect_failure would not be a good idea, either, because the know breakage may be at steps in the same test that is different from the grep, e.g., we may have a "git dothis" command that should keep the HEAD without emitting an error message, and we may test it like so:

	git rev-parse HEAD >old &&
	git dothis >out 2>err &&
	test_i18ngrep ! error: err && # no error should be seen
	git rev-parse HEAD >new &&
	test_cmp old new

but currently the command may have a known bug that it moves HEAD; the command however does not emit an error message.

SO...

On Sun, Oct 28, 2018 at 1:15 PM SZEDER Gábor <szeder.dev@gmail.com> wrote:

And this is something your poison-with-scrambling code does help ;-) ;-)