Re: [Problem] test_must_fail makes possibly questionable assumptions about exit_code.

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Feb 28, 2018, 17:44 UTC
Message-ID: <20180228174402.GC251290@aiede.svl.corp.google.com>
In-Reply-To: <003901d3b0b7$0a144280$1e3cc780$@nexbridge.com>

Randall S. Becker wrote:

Show 6 quoted lines

> The problem is actually in git code in its test suite that uses perl
> inline, not in my test code itself. The difficulty I'm having is
> placing this appropriate so that the signal handler gets used
> throughout the test suite including in the perl -e invocations. This
> is more a lack of my own understanding of plumbing of git test
> framework rather than of using or coding perl.

Can you elaborate with an example? My understanding was that test_must_fail is only for running git. If a test is running perl and wants to check its exit code, the test is supposed to use !, not test_must_fail.

t/README backs me up:

 - use '! git cmd' when you want to make sure the git command exits
   with failure in a controlled way by calling "die()".  Instead,
   use 'test_must_fail git cmd'.  This will signal a failure if git
   dies in an unexpected way (e.g. segfault).

   On the other hand, don't use test_must_fail for running regular
   platform commands; just use '! cmd'.  We are not in the business
   of verifying that the world given to us sanely works.

So I don't consider the initial issue you raised a test issue at all! It's a bug in the git commands, and a fix for it should not be specific to the test suite.

And now it sounds like there is a second issue: the test suite is overusing test_must_fail in some context and that needs to be fixed as well.

Thanks, Jonathan