Re: [PATCH v3 3/3] config: add '--show-origin' option to print the origin of a config value

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Feb 15, 2016, 07:53 UTC
Message-ID: <alpine.DEB.2.20.1602150844110.2964@virtualbox>
In-Reply-To: <20160214161816.GC5875@sigill.intra.peff.net>

Hi Peff,

tl;dr let's keep an eye on adding only test cases that do not depend on earlier test cases' output ('setup' excluded, of course).

On Sun, 14 Feb 2016, Jeff King wrote:

> In general, my opinion is that skipping arbitrary leading tests is a
> losing strategy. It's just too easy to introduce hidden dependencies,
> and not worth the programmer time to make sure each test runs in
> isolation. But others on the list may disagree.

Yes, I disagree. And you would, too, if you had to run as many tests as I had to do by way of js/mingw-tests. I did not keep precise track of time, but I am certain that I had to run one of those bloody tests (forgot which one) around 100 times, each taking roughly 3 minutes to complete, and of course, it was the *last* test case failing, and *of course* it depended on earlier tests to run.

It gets even worse when you think about those test cases that depend on some prereq such as SYMLINKS or POSIXPERM. In *most* cases does the developer who adds them not even notice that these prerequisites are required. And subsequent test cases that do *not* share those prerequisites depend (of course!) on the previous test cases' output.

Don't get me wrong, I think it would be too much pain for little gain to clean up our act now. But I think that we have ample evidence that it would be a plenty good idea to try pretty hard to avoid adding to the pile.

Ciao, Dscho