Re: Git in Outreachy December 2019?

On Fri, Sep 27, 2019 at 06:18:58PM -0400, Jeff King wrote:

I try to avoid using TEST_SHELL_PATH at all costs, it is far too keen to not do what I naively expect.

I see it a bit differently. Over a decade we gathered about twenty-something such tests cases: that's about two cases per year. You added three such cases in about a year and a half: that's two cases per year. The numbers add up perfectly, you singlehandedly took care of everything ;)

Anyway, I did some more digging, and, unfortunately, it turned out that Dscho is somewhat right. While the situation is not as bad as he made it look like ("We need Bash!"), it's not as good as I thought it is ("But it Just Works!!") either.

  - Some shells do include file descriptor redirections in the trace
    output, and it varies between implementations to which fd the
    trace of the redirection goes.
    
      - 'ksh/ksh93' and NetBSD's /bin/sh send the trace of
        redirections to the "wrong" fd, in the sense that e.g. the
        trace of commands invoked in 'test_must_fail' goes to the
        function's standard error, and checking its stderr with
        'test_cmp' would then fail.
 
        (But 'ksh/ksh93' doesn't really matter, because they don't
        support the 'local' keyword, so they fail a bunch of tests
        even without '-x' anyway.)

        I don't think we can do anything about these shells.

      - 'mksh/lksh' send the trace of redirections to the "right" fd,
        so they won't pollute the stderr of test helper functions.
        And indeed the test suite passes when run with 'mksh' (well,
        at least the subset of the test suite that I usually run).

  - We do call 'test_have_prereq' from within test cases as well,
    notably from the 'test_i18ngrep', 'test_i18ncmp' and
    'test_ln_s_add' helper functions.  In those cases all trace output
    from 'test_have_prereq' is included in the test case's trace
    output, which means that during the first invocation:

      - there is lots of distracting and confusing trace output, as
        the script evaluating the prereq is passed around to a bunch
        of functions.

      - after running the script evaluating the prereq 'test_eval_'
        does indeed turn off tracing, so there will be no trace from
        the remainder of that test case (except with 'mksh': while it
        does run 'set +x' in 'test_eval_', that somehow doesn't turn
        off tracing...  I have no idea whether that's a bug or a
        feature).

    As far as 'test_i18ngrep' is concerned, which accounts for the
    majority of 'test_have_prereq' invocations within test cases, I
    don't understand why it uses 'test_have_prereq' in the first place
    instead of checking the GIT_TEST_GETTEXT_POISON environment
    variable; and 6cdccfce1e (i18n: make GETTEXT_POISON a runtime
    option, 2018-11-08) doesn't give me any insight.

    I recall that some months ago we discussed the idea of how to
    disable trace output from within test helper functions; that would
    help with this 'test_have_prereq' issue as well, at least in case
    of the more "common" shells.