Re: [PATCH 10/10] t: make 'test_i18ngrep' more informative on failure

From: SZEDER Gábor <szeder.dev@gmail.com>
Date: Jan 26, 2018, 20:26 UTC
Message-ID: <CAM0VKjkXKcwjt1J+KwHYwcaTpb5COXX9ojBWJ4b4b+PRS=AsZQ@mail.gmail.com>
In-Reply-To: <20180126192532.GA29438@sigill.intra.peff.net>

On Fri, Jan 26, 2018 at 8:25 PM, Jeff King <peff@peff.net> wrote:

> On Fri, Jan 26, 2018 at 08:23:24PM +0100, SZEDER Gábor wrote:
>
>> On Fri, Jan 26, 2018 at 7:50 PM, Jeff King <peff@peff.net> wrote:

Show 11 quoted lines

>> > You can also use "-x" to get a better
>> > sense of exactly which command failed,
>>
>> Yeah, I know...  but I have some issues with running tests with '-x'; I
>> suspect PEBKAC, but haven't yet got around to investigate.
>
> Some tests absolutely fail with "-x", due to them caring about the
> stderr output of shell functions. But with the BASH_XTRACEFD stuff, if
> you run suite under bash it should al Just Work (and I recently added
> TEST_SHELL_PATH to use bash just for the test suite without building all
> of the scripts with it).

Yeah, I knew about TEST_SHELL_PATH, but still:

  $ make -j4 TEST_SHELL_PATH=/bin/bash
  <...>
  $ cd t/
  $ for t in t[0-9][0-9][0-9][0-9]-*.sh ; do "./$t" -x ; done >/dev/null 2>&1
  $ grep '^failed [^0]$' test-results/*.counts |wc -l
  44

The worst offender is t0008-ignores with 208 tests failing with '-x'... I suspect a setup test fails for some reason, and (most of) the other failed tests are just fallout; haven't dared to look yet :)