Re: [PATCH] t/known-leaky: add list of known-leaky test scripts

From: Jeff King <peff@peff.net>
Date: Feb 21, 2018, 18:25 UTC
Message-ID: <20180221182550.GB4333@sigill.intra.peff.net>
In-Reply-To: <xmqqa7w21dz7.fsf@gitster-ct.c.googlers.com>

On Wed, Feb 21, 2018 at 08:53:16AM -0800, Junio C Hamano wrote:

Show 10 quoted lines

> Even though keeping track of list of known-leaky tests may not be so
> interesting, we can still salvage useful pieces from the discussion
> and make them available to developers, e.g.  something like
> 
>     prove --dry --state=failed |
>     perl -lne '/^(t[0-9]{4})-.*\.sh$/ and print $1' | sort >$@+
>     if cmp >/dev/null $@ $@+; then rm $@+; else mv $@+ $@; fi
> 
> could be made into a target to stash away the list of failing tests
> after a test run?

Unfortunately there are some caveats in that snippet:

  1. You are using prove.

  2. You are using --state=save in the initial run.

I think we might be better off having the test scripts write to test-results/*.counts even when run under a TAP harness, and then we can have a consistent way to get the list of failed tests (we already have a "make failed" that works this way).

-Peff