RE: [BUG] test_must_fail: does not correctly detect failures - Was Git 2.16.0-rc2 Test Summary on NonStop

From: Randall S. Becker <rsbecker@nexbridge.com>
Date: Jan 14, 2018, 16:50 UTC
Message-ID: <001a01d38d57$d36c7d10$7a457730$@nexbridge.com>
In-Reply-To: <004a01d38cab$705262a0$50f727e0$@nexbridge.com>

On January 13, 2018 3:17 PM, I wrote:

Show 25 quoted lines

> On January 13, 2018 2:31 PM, I wrote:
> > On January 13, 2018 1:08 PM, I wrote:
> > > Heres where things are. This is probably the best git release so
> > > far
> > (ever).
> > > After applying a4cdf02, I had 6 total breakages. 3 existing, 3 new.
> > > Many reduced. The test took about 24 hours to run on platform, which
> > > is about 2 hours shorter than 2.13.5.
> > >
> > > t1308-config-set.sh (2 already discussed and expecting a fix, both
> > > appear
> > to
> > > be issues in the test script, not code) t1404-update-ref-errors.sh #
> > > 52  reported but not discussed:
> > >    not ok 52 - delete fails cleanly if packed-refs file is locked.
> > >      The lock detection worked, but the test assumed the detection
> > > would occur in a different spot.
> > > t9001-send-email.sh (2 have existed for 2 years. 1 is new. We have
> > > not
> > used
> > > send-email on platform to this point).
> > >    not ok 31 - reject long lines
> > >      This is a new fail since 2.8.5
> > >   not ok 106 - sendemail.transferencoding=7bit fails on 8bit data
> > >      Still to be investigated. This may be a tooling issue on

Platform.

> > >   not ok 107 - --transfer-encoding overrides
> > > sendemail.transferEncoding
> > >      Still to be investigated. This may be a tooling issue on

Platform.

Show 12 quoted lines

> >
> > I missed one:
> > not ok 134 - --dump-aliases must be used alone #
> > #               test_must_fail git send-email --dump-aliases
> > --to=janice@example
> > .com -1 refs/heads/accounting
> 
> Running the tests in debug, I found that they all (1308, 1404, 9001) use
> test_must_fail, and hit similar situations:
> 
> expecting success:
>         test_must_fail git send-email --dump-aliases

--to=janice@example.com

Show 9 quoted lines

> -1 refs/heads/accounting
> --dump-aliases incompatible with other options
> test_must_fail: died by signal 34: git send-email --dump-aliases --
> to=janice@example.com -1 refs/heads/accounting not ok 134 - --dump-
> aliases must be used alone #
> #               test_must_fail git send-email --dump-aliases
> --to=janice@example.com -1 refs/heads/accounting #
> 
> It is looking like git is doing what it is supposed to be doing, but the

test

> scripts are not detecting failures properly. The test_must_fail routine is
> interestingly used in all of the above test cases that are failing. The

actual

> exit_code reported by git was 162, (a.k.a. signal 34 - which is not thrown

on

> the platform. The max signal is 31 (SIGABEND). test_must_fail has a weird
> combination of some errors pass and others don't, but I can't correlate

the

> intent of its use in these tests particularly with no acceptable signals

passed

> in. Adding a return 1 if 162 caused other tests to fail as well, so that's

not the

> fix.

Follow-up: This looks like the completion code from perl on NonStop is not
the same as expected by git in the case of failures. I need to debug this to
get more details to the team. We have had completion issues before relating
to interpretation problems between perl, bash, and git in this platform, so
I'm assuming this to be likely again but need to track down the specifics.
Can anyone point me to where the detection is within git or the execv?

Thanks,
Randall