Re: [PATCH] ci/install-depends: attempt to fix "brew cask" stuff

From: Jeff King <peff@peff.net>
Date: Jan 27, 2021, 22:21 UTC
Message-ID: <YBHnV80bBQOxZH1O@coredump.intra.peff.net>
In-Reply-To: <830a88ce-1728-a6a5-f60d-59328f85932c@gmail.com>

On Sat, Jan 23, 2021 at 09:41:29PM -0500, Philippe Blain wrote:

Show 14 quoted lines

> > So we could split the macos test out. It would probably involve
> > duplicating a little bit of the content, but we do something similar for
> > the dockerized builds. It might be that there is an option we can set to
> > say "keep building the others even if one of these fails", which would
> > give us the best of both.
> 
> Yes, a quick Google search pointed me to a blog post [1]
> that mentions using 'fail-fast: false' in the test matrix so that
> one failing job does not automatically cancel the rest of the jobs
> in the matrix (the default is 'true') [2].
> 
> If we apply that to all four matrices in the workflow file,
> (windows-test, vs-test, regular and dockerized), it would be
> something like this:

Thanks, that's exactly what I think we'd want.

The downside, of course, is that a failure that will happen on every platform will mean wasted CPU to trigger the same failure over and over. But:

  - I rarely see that myself, because I wouldn't bother pushing up to CI
    until "make test" passed locally. So usually I'm finding portability
    issues via CI. Other people might be different, though.

  - we already have the Windows tests in a separate matrix anyway, so a
    failure on Linux would run the whole Windows suite (which is an
    order of magnitude more expensive)

  - even within the Windows matrix, I think running the rest of the
    tests after a failure is still valuable. If there's a second
    failure, you save a round-trip to CI (so it doesn't reduce CPU, but
    it may help latency to reach a passing state).

-Peff