Re: [PATCH 6/5] t9001: check send-email behavior with implicit sender

From: Junio C Hamano <gitster@pobox.com>
Date: Nov 28, 2012, 20:42 UTC
Message-ID: <7vpq2xtq66.fsf@alter.siamese.dyndns.org>
In-Reply-To: <20121128201713.GA9249@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

Show 13 quoted lines

> On Wed, Nov 28, 2012 at 03:06:26PM -0500, Jeff King wrote:
>
>> Here's a cleaned up version that makes it more obvious the commands are
>> the same (it also fixes a few minor whitespace problems on the
>> indentation, which you can see from the quoting above).
>
> I wondered how painful it would be to actually run the command and then
> conditionally check the results inside the test. I ended up with this:
> ...
> which does work, though it is less nice that the protections and nice
> syntax of "test_must_fail" must be abandoned (unless we want to do
> something horrible like 'test_must_fail sh -c "exit $exit_code"'.
> I could go either way.

A change along that line did cross my mind, and I agree that it does clarify that we are testing the same command produces an expected result, the expectation may be different depending on external conditions, and we can figure out what the expected values are before running the test. I do not think we use such a pattern in very many places, though.

The differences in "expected results" are generally not limited to the final outcome in $? (e.g. your "else" side not just checks $? indicates an error, but makes sure that we got an expected error message), which means that the if statement at the end has to be there in some form anyway, which in turn means that a new test helper function to abstract this pattern out further would not buy us very much (either it would be too complex and bug prone to implement, or it would be too simplistic and limited).

I'd prefer to leave these as two separate tests for now like your previous patch. People interested in consolidating/simplifying can first audit the existing tests to find other instances of this pattern to see if it is worth doing, and then come up with an abstraction to be shared among them.

Thanks.