Re: [RFC PATCH v2 6/6] t/Makefile: run unit tests alongside shell tests

From: Jeff King <peff@peff.net>
Date: Feb 7, 2024, 22:43 UTC
Message-ID: <20240207224350.GA537799@coredump.intra.peff.net>
In-Reply-To: <xmqqzfwc6lle.fsf@gitster.g>

On Wed, Feb 07, 2024 at 12:55:09PM -0800, Junio C Hamano wrote:

Show 14 quoted lines

> > +# A simple wrapper to run shell tests via TEST_SHELL_PATH,
> > +# or exec unit tests directly.
> > +
> > +case "$1" in
> > +*.sh)
> > +	exec ${TEST_SHELL_PATH:-/bin/sh} "$@"
> > +	;;
> > +*)
> > +	exec "$@"
> > +	;;
> > +esac
> 
> Hmph.  This penalizes the non-unit tests by doing an extra "exec",
> once per program?

It does, but IMHO that is not likely to be a problem. It's once per top-level script (so ~1000), and each of those scripts spawns hundreds or thousands of sub-commands. I didn't do any measurements, though.

You can extend "prove" with extra perl modules so that it makes the distinction internally without the extra shell invocation. But when I tried to do it, I found it rather baroque and complicated (I can't remember if I succeeded but found it too gross, or just gave up halfway through trying).

> Of course we cannot run two $(PROVE) invocations serially, one for
> doing $(T) and the other for doing $(UNIT_TESTS)?

Not if they share the same command-line options. If you use something like "--state=slow,save", then the first run will write the list of all tests to ".prove", and then the second will run every test mentioned in .prove (in addition to the unit-tests provided on the command-line).

You should be able to work around it by passing "--statefile". I _think_ it might be OK to just do that unconditionally. Something like:

  prove --exec $(TEST_SHELL_PATH $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)
  prove --statefile=.prove-unit-tests $(GIT_PROVE_OPTS) $(UNIT_TESTS) :: $(GIT_TEST_OPTS)

and then it's just a noop if GIT_PROVE_OPTS doesn't use --state. But I haven't played with it myself.

-Peff