Re: [PATCH 1/1] Makefile: add prove and coverage-prove targets

From: SZEDER Gábor <szeder.dev@gmail.com>
Date: Jan 29, 2019, 15:58 UTC
Message-ID: <20190129155827.GC13764@szeder.dev>
In-Reply-To: <294187c6968eff952e78bcea808c66fbedbf1f90.1548773766.git.gitgitgadget@gmail.com>

On Tue, Jan 29, 2019 at 06:56:08AM -0800, Derrick Stolee via GitGitGadget wrote:

Show 15 quoted lines

> When running the test suite for code coverage using
> 'make coverage-test', a single test failure stops the
> test suite from completing. This leads to significant
> undercounting of covered blocks.
> 
> Add two new targets to the Makefile:
> 
> * 'prove' runs the test suite using 'prove'.
> 
> * 'coverage-prove' compiles the source using the
>   coverage flags, then runs the test suite using
>   'prove'.
> 
> These targets are modeled after the 'test' and
> 'coverage-test' targets.

I think the cover letter would be a better commit message.

Show 16 quoted lines

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  Makefile | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/Makefile b/Makefile
> index 1a44c811aa..ec886635ae 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2665,6 +2665,9 @@ export TEST_NO_MALLOC_CHECK
>  test: all
>  	$(MAKE) -C t/ all
>  
> +prove: all
> +	$(MAKE) -C t/ prove
> +

You don't need this 'prove' target in the "main" Makefile, because 'make test' will run the test suite using DEFAULT_TEST_TARGET anyway.

Show 10 quoted lines

>  perf: all
>  	$(MAKE) -C t/perf/ all
>  
> @@ -3077,6 +3080,10 @@ coverage-test: coverage-clean-results coverage-compile
>  	$(MAKE) CFLAGS="$(COVERAGE_CFLAGS)" LDFLAGS="$(COVERAGE_LDFLAGS)" \
>  		DEFAULT_TEST_TARGET=test -j1 test
>  
> +coverage-prove: coverage-clean-results coverage-compile
> +	$(MAKE) CFLAGS="$(COVERAGE_CFLAGS)" LDFLAGS="$(COVERAGE_LDFLAGS)" \
> +		DEFAULT_TEST_TARGET=prove -j1 prove

First I was wondering why do you need a dedicated 'coverage-prove' target, instead of letting DEFAULT_TEST_TARGET from the environment or from 'config.mak' do its thing. But then I noticed in the hunk context, that, for some reason, the 'coverage-test' target hardcoded 'DEFAULT_TEST_TARGET=test -j1'. Then I was wondering why would it want to do that, and stumbled upon commit c14cc77c11:

    coverage: set DEFAULT_TEST_TARGET to avoid using prove
    
    If the user sets DEFAULT_TEST_TARGET=prove in his config.mak, that
    carries over into the coverage tests.  Which is really bad if he also
    sets GIT_PROVE_OPTS=-j<..> as that completely breaks the coverage
    runs.
    
    Instead of attempting to mess with the GIT_PROVE_OPTS, just force the
    test target to 'test' so that we run under make, like we intended all
    along.

I'm afraid that this issue would badly affect 'coverage-prove' as well (I didn't try). Or if doesn't (anymore?), then that should be mentioned in the commit message, and then perhaps it's time to remove that '-j1' from the 'coverage-test' target as well.