Re: [PATCH] t/known-leaky: add list of known-leaky test scripts

From: Jeff King <peff@peff.net>
Date: Feb 19, 2018, 21:29 UTC
Message-ID: <20180219212937.GB9748@sigill.intra.peff.net>
In-Reply-To: <20180214215637.6462-1-martin.agren@gmail.com>

On Wed, Feb 14, 2018 at 10:56:37PM +0100, Martin Ågren wrote:

Show 6 quoted lines

> Here's what a list of known leaks might look like. It feels a bit
> awkward to post a known-incomplete list (I don't run all tests). Duy
> offered to pick up the ball if I gave up, maybe you could complete and
> post this as your own? :-? Even if I (or others) can't reproduce the
> complete list locally, regressions will be trivial to find, and newly
> leak-free tests fairly easy to notice.

I didn't think about that when I posted my scripts. In general, it's OK to me if you miss a script when you generate the "leaky" list. But if you skip it, you cannot say whether it is leaky or not, and should probably neither add nor remove it from the known-leaky list. So I think the second shell snippet needs to become a little more clever about skipped test scripts.

Even that isn't 100% fool-proof, as some individual tests may be skipped or not skipped on various platforms. But it may be enough in practice (and eventually we'd have no known-leaky tests, of course ;) ).

Or alternatively, we could just not bother with checking this into the repository, and it becomes a local thing for people interested in leak-testing. What's the value in having a shared known-leaky list, especially if we don't expect most people to run it.

I guess it lets us add a Travis job to do the leak-checking, which might get more coverage. So maybe if we do have an in-repo known-leaky, it should match some canonical Travis environment. That won't give us complete coverage, but at this point we're just trying to notice low-hanging fruit.

-Peff