Re: [PATCH] t/Makefile: retain cache t/.prove across prove runs

> On Wed, May 02, 2012 at 05:31:52PM +0200, mhagger@alum.mit.edu wrote:
>> prove(1) can write a summary of its test results and timings into a
>> cache file, t/.prove, then use this information during later runs for
>> various purposes.  But deleting t/.prove after every test run defeats
>> this purpose.  So do not delete t/.prove as part of "make
>> DEFAILT_TEST_TARGET=prove test".  (Continue to delete the file on
>> "make clean".)
>
> Thanks. I had found some odd behaviors with putting "--state" into
> GIT_PROVE_OPTS when I first switched my setup to use prove, but I didn't
> investigate. I think this issue explained some of what I saw.
>
> Do note, though, that we only clean ".prove" on a successful run. So
> something like "--state=hot,save" will still see the cache from the last
> run if there were failures. However, the prove cache is meant to be kept
> across many runs, even successful ones, so I think your patch makes
> sense.
>
>> ---
>> prove(1) can be told to retain information about test runs, and to use
>> it for interesting purposes during future invocations.  For example,
>> it can be told to run tests that failed during the last run:
>>
>>      GIT_PROVE_OPTS = --state=failed,save
>
> I don't think this actually works, because we also feed all of the test
> scripts to prove on the command line. So it will run them all, both
> failed and successful.
>
> I'm not sure if it is worth solving for "make test". I suspect having
> that not test everything would be error prone (i.e., it's too easy to
> accidentally not run the full suite when you meant to, and unexpected
> regressions in other tests will go unnoticed). You can always run
> "prove" yourself from the command line, or we can add a new target to do
> a partial test (with other options set up properly).