Re: Travis not looking so good

From: SZEDER Gábor <szeder.dev@gmail.com>
Date: Jun 2, 2019, 11:22 UTC
Message-ID: <20190602112239.GO951@szeder.dev>
In-Reply-To: <20190601004143.GN8616@genre.crustytoothpaste.net>

On Sat, Jun 01, 2019 at 12:41:43AM +0000, brian m. carlson wrote:

Show 20 quoted lines

> On 2019-05-30 at 19:32:41, Johannes Schindelin wrote:
> > Hi Gábor,
> > 
> > do you have any idea why Travis is failing like this in the macOS/gcc
> > job?
> > 
> > > +case "$jobname" in
> > > +brew link gcc@8
> > > Error: No such keg: /usr/local/Cellar/gcc@8
> > > The command "ci/install-dependencies.sh" failed and exited with 1 during .
> > 
> > I usually only look at the Azure Pipelines (which gives me plenty enough
> > to do, what with pu's individual branches being tested individually), but
> > couldn't fail to notice that *all* four branches (maint, master, next and
> > pu) fail in Travis' macOS/gcc job (and only there, the Azure Pipelines are
> > all green):
> > 
> > https://github.com/git/git/branches/all
> > 
> > What's going on?

The usual: Homebrew desperately tries to be overly clever and helpful, but ends up being dumb and annoying. :)

I was hoping that this issue will just solve itself, like several other brew breakages in the past, but apparently it won't...

Show 5 quoted lines

> I'm certainly not an expert on macOS, so I could be getting things wrong
> here, but it looks like we don't ever explicitly install the gcc@8
> package from Homebrew. It may have happened to be installed
> automatically by Travis in the past, but they may have upgraded now that
> GCC 9 has been released.

gcc@8 is still installed in the Travis CI macOS image we use, but it is version 8.2. It's still perfectly usable, but:

When installing dependencies on macOS, we first explicitly run 'brew update', which spends over two minutes to update itself and information about the available packages. This way it learns about GCC 8.3, and after that gcc@8 refers to v8.3, and, unfortunately, 'brew' is just too dumb to be able to do anything with the still installed 8.2 package, and the subsequent 'brew link gcc@8' fails. But hey, even 'brew uninstall gcc@8' fails with the same error after that, which is simply unacceptable from a package manager in my book.

Show 5 quoted lines

> I suspect if we want to use GCC 8, we need to explicitly install it by
> using "brew install gcc@8", or we can just pick the latest released GCC
> by using "brew install gcc" if we like that better. We will still need
> to do "brew link gcc" (or "gcc@8"), since I suspect Homebrew won't
> auto-link it since macOS provides a gcc binary.

Yeah, installing gcc@8 or gcc works. Back in 2c8921db2b (travis-ci: build with the right compiler, 2019-01-17) I opted for simply linking the already installed gcc@8 package, because GCC is big, installing it takes time, and the macOS build jobs have always been prone to exceeding the time limit. (Note that these packages provide 'gcc-8' and 'gcc-9' binaries, not 'gcc', and after 'brew install'-ing them we won't need an additional link step (I'm not sure why linking is necessary with the gcc@8 package already installed in the Travis CI image).)

Another possibilities are:

  - Running 'brew link gcc@8' before 'brew update' works:

      https://travis-ci.org/szeder/git/jobs/540027012#L139

  - Not running 'brew update' at all works as well:
  
      https://travis-ci.org/szeder/git/jobs/514960153#L179
  
    However, it's not as simple as removing the 'brew update' command
    from our ci script, because 'brew' is so very helpful that it
    implicitly runs update for us on the next 'brew install' anyway...
    we have to 'export HOMEBREW_NO_AUTO_UPDATE=1' first to disable
    that.

    As an additional benefit, disabling 'brew update' shaves off over
    2mins from the ~4mins currently spent on installing dependencies
    [1].

    I've been running my somewhat customized builds on Travis CI [2]
    without 'brew update' for about a month now, no ill effects yet.
    Initially I was worried that without 'brew update' a subsequent
    'brew install <pkg>' might fail, if the package repositories
    contain only a newer version of 'pkg'.  I haven't seen any such
    issues yet, and my builds can still 'brew install' Git-LFS 2.5.2
    and Perforce 18.1, although their current versions are 2.7.2 and
    19.1, respectively.

[1] And disabling the recently introduced auto cleanup feature saves
    some more time, and installing all dependencies in the macOS
    builds can be done in ~20-30secs:

      https://travis-ci.org/szeder/git/jobs/514960153#L92

[2] https://travis-ci.org/szeder/git-cooking-topics-for-travis-ci/branches