Re: test &&-chain lint (was: [PATCH 1/5] t5312: test object deletion code paths in a corrupted repository)

> On Thu, Mar 19, 2015 at 10:25:32PM -0400, Jeff King wrote:
>> > diff --git a/t/test-lib.sh b/t/test-lib.sh
>> > index c096778..02a03d5 100644
>> > --- a/t/test-lib.sh
>> > +++ b/t/test-lib.sh
>> > @@ -524,6 +524,21 @@ test_eval_ () {
>> >  test_run_ () {
>> > +   if test -n "$GIT_TEST_CHAIN_LINT"; then
>> > +           # 117 is unlikely to match the exit code of
>> > +           # another part of the chain
>> > +           test_eval_ "(exit 117) && $1"
>> > +           if test "$?" != 117; then
>> > +                   # all bets are off for continuing with other tests;
>> > +                   # we expected none of the rest of the test commands to
>> > +                   # run, but at least some did. Who knows what weird
>> > +                   # state we're in? Just bail, and the user can diagnose
>> > +                   # by running in --verbose mode
>> > +                   error "bug in the test script: broken &&-chain"
>> > +           fi
>> > +   fi

>> > This turns up an appalling number of failures, but AFAICT they are all
>> > "real" in the sense that the &&-chains are broken. In some cases these
>> > are real, but in others the tests are of an older style where they did
>> > not expect some early commands to fail (and we would catch their bogus
>> > output if they did). E.g., in the patch below, I think the first one is
>> > a real potential bug, and the other two are mostly noise. I do not mind
>> > setting a rule and fixing all of them, though.
>
> FWIW, I have spent about a few hours wading through the errors, and am
> about 75% done. There are definitely some broken chains that were
> causing test results to be ignored (as opposed to just minor setup steps
> that we would not expect to fail). In most cases, the tests do passed. I
> have a few that I still need to examine more closely, but there may be
> some where there are actual test failures (but it's possible that I just
> screwed it up while fixing the &&-chaining).
>
> I hope to post something tonight, but I wanted to drop a note on the off
> chance that you were actively looking at it at the same time.