Re: [PATCH v4 8/7] wildmatch test: skip file creation tests on Windows proper

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Jan 6, 2018, 20:46 UTC
Message-ID: <nycvar.QRO.7.76.6.1801062116350.1337@wbunaarf-fpuvaqryva.tvgsbejvaqbjf.bet>
In-Reply-To: <87mv1raz9p.fsf@evledraar.gmail.com>

Hi Ævar,

On Sat, 6 Jan 2018, Ævar Arnfjörð Bjarmason wrote:

> As I explained in 20180105221222.28867-1-avarab@gmail.com the actual
> benefit of this test is that as much as possible is tested
> *somewhere*.

And what I am trying to get across is that your tests are excessive. I do not see the benefit in running such an excessive, overzealous test.

You say that Linux will not overnight stop supporting colons in filenames, which kind of misses the point that the question is more about new platforms needing Git support with a *filesystem* that simply does not meet one of your many tests' expectations.

But what is equally true is that few changes are to be expected in the wildmatch code. And you know as well as I that it is possible to come up with a careful, small set of test cases that will most likely identify breakages.

After all, when there is a bug in some patch to the wildmatch machinery, you do not need 512 test cases to fail. A single failing test case is definitely enough.

I'll test the branch you indicated on Monday, just because you put effort into it.

But please note that it does not change the fact that an effective test suite requires a balance between the need to run fast (if *every* aspect of Git was tested as extensively as your current t3070, even on Linux it would take hours and hours to finish the test suite, making it stupidly expensive for everybody to run) and the need to catch regressions.

It is a well-established rule in effective DevOps that you want lean test suites. Actionable test suites. It is enough for *one* test to fail. When a test fails, it should be as easy to understand as possible what is going wrong. If there are no regressions, the test suite should be passing *fast*.

On Windows, it takes 70-90 minutes *on a fast* machine. That is a huge failure on our part: this is the test suite we designed "to be portable". All I am trying to do here is to prevent even more damage. I will appreciate any assistance to that end. I understand that most developers on this list simply don't care. And that's just too bad, because we could do a lot better. And we should, too.

Ciao, Johannes