Re: [PATCH] test-lib: allow storing counts with test harnesses

>
> Currently, test result files are only stored in test-results/*.counts if
> $HARNESS_ACTIVE is not set.  This dates from 8ef1abe550 (test-lib: Don't
> write test-results when HARNESS_ACTIVE, 2010-08-11), where the
> assumption was that if someone were using a test harness like prove,
> that would track results and the count files wouldn't be required.
> However, as of 49da404070 (test-lib: show missing prereq summary,
> 2021-11-20), those files also store the list of git test prerequisites
> that were missing during the test run, which isn't something that a
> generic test harness like prove can provide.
>
> To allow folk using test harnesses to access the lists of missing
> prerequisites, add a --counts argument to test-lib that will keep these
> counts files even if a test harness is in use.  This means that a
> subsequent call of, say, `make -C t aggregate-results` will report
> useful information.
>
> It might be preferable to do make a wider-ranging change, including

> storing the missing prerequisites separately from the count files, so
> the results can be reported regardless of whether the success/failure
> counts are wanted, but that would be more disruptive and more work for
> relatively little gain.
>
> Signed-off-by: Adam Dinwoodie <adam@dinwoodie.org>
> ---
>
> I submitted this as an RFC back in December, and received no comments,
> so I'm submitting this as an actual patch now.  My key concern was the
> final paragraph above -- embedding using the "count" files for something
> other than counts -- but I've mostly convinced myself that refactoring
> this code to separate that out is unlikely to actually cause significant
> pain.