From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Mon, 15 Feb 2016 07:53:17 GMT
Subject: Re: [PATCH v3 3/3] config: add '--show-origin' option to print the origin of a config value
Message-ID: <alpine.DEB.2.20.1602150844110.2964@virtualbox>
In-Reply-To: <20160214161816.GC5875@sigill.intra.peff.net>

Hi Peff,

tl;dr let's keep an eye on adding only test cases that do not depend on
earlier test cases' output ('setup' excluded, of course).

On Sun, 14 Feb 2016, Jeff King wrote:

> In general, my opinion is that skipping arbitrary leading tests is a
> losing strategy. It's just too easy to introduce hidden dependencies,
> and not worth the programmer time to make sure each test runs in
> isolation. But others on the list may disagree.

Yes, I disagree. And you would, too, if you had to run as many tests as I
had to do by way of js/mingw-tests. I did not keep precise track of time,
but I am certain that I had to run one of those bloody tests (forgot which
one) around 100 times, each taking roughly 3 minutes to complete, and of
course, it was the *last* test case failing, and *of course* it depended
on earlier tests to run.

It gets even worse when you think about those test cases that depend on
some prereq such as SYMLINKS or POSIXPERM. In *most* cases does the
developer who adds them not even notice that these prerequisites are
required. And subsequent test cases that do *not* share those
prerequisites depend (of course!) on the previous test cases' output.

Don't get me wrong, I think it would be too much pain for little gain to
clean up our act now. But I think that we have ample evidence that it
would be a plenty good idea to try pretty hard to avoid adding to the
pile.

Ciao,
Dscho