threads / patch / 50383

patchtravis-ci: make the OSX build jobs' 'brew update' more quiet

Subject: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

## tl;dr

9 messages between Feb 2, 2019 and Feb 6, 2019. Diffs are folded; open one to read it.

replies: 8people: 3as markdown or json

SZEDER Gábor· Feb 2, 2019, 16:34 UTC · lore

Before installing the necessary dependencies, our OSX build jobs run 'brew update --quiet'. This is problematic for two reasons:

  - This '--quiet' flag apparently broke overnight, resulting in
    errored builds:
      +brew update --quiet
      ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
      ######################################################################## 100.0%
      ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
      Usage: brew update_report [--preinstall]
      The Ruby implementation of brew update. Never called manually.
              --preinstall                 Run in 'auto-update' mode (faster, less
                                           output).
          -f, --force                      Override warnings and enable potentially
                                           unsafe operations.
          -d, --debug                      Display any debugging information.
          -v, --verbose                    Make some output more verbose.
          -h, --help                       Show this message.
      Error: invalid option: --quiet
      The command "ci/install-dependencies.sh" failed and exited with 1 during .
    I belive that this breakage will be noticed and fixed soon-ish, so
    we could probably just wait a bit for this issue to solve itself,
    but:
  - 'brew update --quiet' wasn't really quiet in the first place, as
    it listed over about 2000 lines worth of available packages that
    we absolutely don't care about, see e.g. one of the latest
    'master' builds:
      https://travis-ci.org/git/git/jobs/486134962#L113

So drop this '--quiet' option and redirect 'brew update's standard output to /dev/null to make it really quiet, thereby making the OSX builds work again despite the above mentioned breakage.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
Notes:
    There is no conflict with Dscho's Azure Pipelines patch series; the
    patch contexts overlap a bit, but the auto-merging results look good
    to me.
 ci/install-dependencies.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to ci/install-dependencies.sh +1 −1
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index 06c3546e1e..5968efdbbe 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -34,7 +34,7 @@ linux-clang|linux-gcc)
 	popd
 	;;
 osx-clang|osx-gcc)
-	brew update --quiet
+	brew update >/dev/null
 	# Uncomment this if you want to run perf tests:
 	# brew install gnu-time
 	brew install git-lfs gettext

-- 
2.20.1.642.gc55a771460
Johannes Schindelin· Feb 4, 2019, 09:49 UTC · re: SZEDER Gábor · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

Hi Gábor,
On Sat, 2 Feb 2019, SZEDER Gábor wrote:
Show 44 quoted lines
> Before installing the necessary dependencies, our OSX build jobs run
> 'brew update --quiet'.  This is problematic for two reasons:
> 
>   - This '--quiet' flag apparently broke overnight, resulting in
>     errored builds:
> 
>       +brew update --quiet
>       ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
>       ######################################################################## 100.0%
>       ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
>       Usage: brew update_report [--preinstall]
>       The Ruby implementation of brew update. Never called manually.
>               --preinstall                 Run in 'auto-update' mode (faster, less
>                                            output).
>           -f, --force                      Override warnings and enable potentially
>                                            unsafe operations.
>           -d, --debug                      Display any debugging information.
>           -v, --verbose                    Make some output more verbose.
>           -h, --help                       Show this message.
>       Error: invalid option: --quiet
>       The command "ci/install-dependencies.sh" failed and exited with 1 during .
> 
>     I belive that this breakage will be noticed and fixed soon-ish, so
>     we could probably just wait a bit for this issue to solve itself,
>     but:
> 
>   - 'brew update --quiet' wasn't really quiet in the first place, as
>     it listed over about 2000 lines worth of available packages that
>     we absolutely don't care about, see e.g. one of the latest
>     'master' builds:
> 
>       https://travis-ci.org/git/git/jobs/486134962#L113
> 
> So drop this '--quiet' option and redirect 'brew update's standard
> output to /dev/null to make it really quiet, thereby making the OSX
> builds work again despite the above mentioned breakage.
> 
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> ---
> 
> Notes:
>     There is no conflict with Dscho's Azure Pipelines patch series; the
>     patch contexts overlap a bit, but the auto-merging results look good
>     to me.
And this indeed also affects Azure Pipelines.

Since Junio's workflow is very different from GitHub Flow (where this issue would be worked around with a simple, single Pull Request), we have no prayer at a workaround on our side, though: pretty much *all* of the next builds of Junio's branches will be broken, unless he chooses to backport your patch to all of the base commits he chose for those branches.

So our best bet at not getting overwhelmed with failed builds is to help Homebrew get `brew update --quiet` to work again. I just opened a ticket to that end:

	https://github.com/Homebrew/brew/issues/5666

I do like your patch, though, and am very much in favor of fast-tracking it all the way down to `maint`.

Ciao, Dscho

Show 21 quoted lines
>  ci/install-dependencies.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 06c3546e1e..5968efdbbe 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
>  	popd
>  	;;
>  osx-clang|osx-gcc)
> -	brew update --quiet
> +	brew update >/dev/null
>  	# Uncomment this if you want to run perf tests:
>  	# brew install gnu-time
>  	brew install git-lfs gettext
> -- 
> 2.20.1.642.gc55a771460
> 
> 
>
SZEDER Gábor· Feb 4, 2019, 19:38 UTC · re: Johannes Schindelin · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

On Mon, Feb 04, 2019 at 10:49:05AM +0100, Johannes Schindelin wrote:
Show 53 quoted lines
> Hi Gábor,
> 
> On Sat, 2 Feb 2019, SZEDER Gábor wrote:
> 
> > Before installing the necessary dependencies, our OSX build jobs run
> > 'brew update --quiet'.  This is problematic for two reasons:
> > 
> >   - This '--quiet' flag apparently broke overnight, resulting in
> >     errored builds:
> > 
> >       +brew update --quiet
> >       ==> Downloading https://homebrew.bintray.com/bottles-portable-ruby/portable-ruby-2.3.7.mavericks.bottle.tar.gz
> >       ######################################################################## 100.0%
> >       ==> Pouring portable-ruby-2.3.7.mavericks.bottle.tar.gz
> >       Usage: brew update_report [--preinstall]
> >       The Ruby implementation of brew update. Never called manually.
> >               --preinstall                 Run in 'auto-update' mode (faster, less
> >                                            output).
> >           -f, --force                      Override warnings and enable potentially
> >                                            unsafe operations.
> >           -d, --debug                      Display any debugging information.
> >           -v, --verbose                    Make some output more verbose.
> >           -h, --help                       Show this message.
> >       Error: invalid option: --quiet
> >       The command "ci/install-dependencies.sh" failed and exited with 1 during .
> > 
> >     I belive that this breakage will be noticed and fixed soon-ish, so
> >     we could probably just wait a bit for this issue to solve itself,
> >     but:
> > 
> >   - 'brew update --quiet' wasn't really quiet in the first place, as
> >     it listed over about 2000 lines worth of available packages that
> >     we absolutely don't care about, see e.g. one of the latest
> >     'master' builds:
> > 
> >       https://travis-ci.org/git/git/jobs/486134962#L113
> > 
> > So drop this '--quiet' option and redirect 'brew update's standard
> > output to /dev/null to make it really quiet, thereby making the OSX
> > builds work again despite the above mentioned breakage.
> > 
> > Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> > ---
> > 
> > Notes:
> >     There is no conflict with Dscho's Azure Pipelines patch series; the
> >     patch contexts overlap a bit, but the auto-merging results look good
> >     to me.
> 
> And this indeed also affects Azure Pipelines.
> 
> Since Junio's workflow is very different from GitHub Flow (where this
> issue would be worked around with a simple, single Pull Request),

I doubt that a single PR could be sufficient, because it won't help contributors who want to run CI builds in their own forks; they would have to rebase their branches on top of the fix, or cherry-pick it, or whatnot.

Show 7 quoted lines
> we have
> no prayer at a workaround on our side, though: pretty much *all* of the
> next builds of Junio's branches will be broken, unless he chooses to
> backport your patch to all of the base commits he chose for those
> branches.
> 
> So our best bet at not getting overwhelmed with failed builds ...

Oh, in my long-time for-fun semi-automated CI experiment [1] all integration branches and cooking topics are still just fine :) they all have been including this patch (among many others) for quite some time now (since when the list of available Homebrew packages was "only" about 1700 lines... about a year ago).

Unfortunately, I never upstreamed it, because that long list of packages is folded up anyway when viewing the logs, so why bother... In hindsight, I should have. Oh, well.

[1] https://travis-ci.org/szeder/git-cooking-topics-for-travis-ci/branches
Show 30 quoted lines
> is to help
> Homebrew get `brew update --quiet` to work again. I just opened a ticket
> to that end:
> 
> 	https://github.com/Homebrew/brew/issues/5666
> 
> I do like your patch, though, and am very much in favor of fast-tracking
> it all the way down to `maint`.
> 
> Ciao,
> Dscho
> 
> >  ci/install-dependencies.sh | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> > index 06c3546e1e..5968efdbbe 100755
> > --- a/ci/install-dependencies.sh
> > +++ b/ci/install-dependencies.sh
> > @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
> >  	popd
> >  	;;
> >  osx-clang|osx-gcc)
> > -	brew update --quiet
> > +	brew update >/dev/null
> >  	# Uncomment this if you want to run perf tests:
> >  	# brew install gnu-time
> >  	brew install git-lfs gettext
> > -- 
> > 2.20.1.642.gc55a771460
Johannes Schindelin· Feb 5, 2019, 11:31 UTC · re: SZEDER Gábor · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

Hi Gábor,
On Mon, 4 Feb 2019, SZEDER Gábor wrote:
Show 15 quoted lines
> On Mon, Feb 04, 2019 at 10:49:05AM +0100, Johannes Schindelin wrote:
> 
> > So our best bet at not getting overwhelmed with failed builds ...
> 
> Oh, in my long-time for-fun semi-automated CI experiment [1] all
> integration branches and cooking topics are still just fine :) they
> all have been including this patch (among many others) for quite some
> time now (since when the list of available Homebrew packages was
> "only" about 1700 lines... about a year ago).
> 
> Unfortunately, I never upstreamed it, because that long list of
> packages is folded up anyway when viewing the logs, so why bother...
> In hindsight, I should have.  Oh, well.
> 
> [1] https://travis-ci.org/szeder/git-cooking-topics-for-travis-ci/branches

Heh, I feel ya. 70 branches in Git for Windows' patch thicket are waiting (for the most part) to be upstreamed.

> > is to help Homebrew get `brew update --quiet` to work again. I just
> > opened a ticket to that end:
> > 
> > 	https://github.com/Homebrew/brew/issues/5666

And this ticket has been closed with a fix! Yay! I checked, and on Travis, `pu` is now only broken for the FUNNYNAMES issue in Elijah's t4038 that I already reported.

Ciao, Dscho

Junio C Hamano· Feb 4, 2019, 18:26 UTC · re: SZEDER Gábor · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

SZEDER Gábor <szeder.dev@gmail.com> writes:
Show 8 quoted lines
> Before installing the necessary dependencies, our OSX build jobs run
> 'brew update --quiet'.  This is problematic for two reasons:
>
>   - This '--quiet' flag apparently broke overnight, resulting in
>     errored builds:
> ...
>     I belive that this breakage will be noticed and fixed soon-ish, so
>     we could probably just wait a bit for this issue to solve itself,

Yuck. Well, an external influence that can break the automated build job overnight should be capable of fixing it overnight ;-)

Show 12 quoted lines
>     but:
>
>   - 'brew update --quiet' wasn't really quiet in the first place, as
>     it listed over about 2000 lines worth of available packages that
>     we absolutely don't care about, see e.g. one of the latest
>     'master' builds:
>
>       https://travis-ci.org/git/git/jobs/486134962#L113
>
> So drop this '--quiet' option and redirect 'brew update's standard
> output to /dev/null to make it really quiet, thereby making the OSX
> builds work again despite the above mentioned breakage.
Sounds good.
>     There is no conflict with Dscho's Azure Pipelines patch series; the
>     patch contexts overlap a bit, but the auto-merging results look good
>     to me.
Thanks for extra careful.

If this is truly urgent, I could merge this to 'maint' and merge the result upwards to 'pu' and it would hide the issue on my four integration branches. But one thing that makes me wonder is if we can (or want to) do anything to help other people who test build with pull requests. I guess they need to rebase on top of whatever commit that has this fix? That sounds more like a tail wagging a dog, though. I dunno.

Show 16 quoted lines
>  ci/install-dependencies.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 06c3546e1e..5968efdbbe 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -34,7 +34,7 @@ linux-clang|linux-gcc)
>  	popd
>  	;;
>  osx-clang|osx-gcc)
> -	brew update --quiet
> +	brew update >/dev/null
>  	# Uncomment this if you want to run perf tests:
>  	# brew install gnu-time
>  	brew install git-lfs gettext
SZEDER Gábor· Feb 4, 2019, 19:38 UTC · re: Junio C Hamano · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

On Mon, Feb 04, 2019 at 10:26:29AM -0800, Junio C Hamano wrote:
Show 10 quoted lines
> SZEDER Gábor <szeder.dev@gmail.com> writes:
> 
> >   - This '--quiet' flag apparently broke overnight, resulting in
> >     errored builds:
> > ...
> >     I belive that this breakage will be noticed and fixed soon-ish, so
> >     we could probably just wait a bit for this issue to solve itself,
> 
> Yuck.  Well, an external influence that can break the automated
> build job overnight should be capable of fixing it overnight ;-)
Show 7 quoted lines
> If this is truly urgent, I could merge this to 'maint' and merge the
> result upwards to 'pu' and it would hide the issue on my four
> integration branches.  But one thing that makes me wonder is if we
> can (or want to) do anything to help other people who test build
> with pull requests.  I guess they need to rebase on top of whatever
> commit that has this fix?  That sounds more like a tail wagging a
> dog, though.  I dunno.

Contributors don't have to rebase to make their PR builds work. When Travis CI builds a PR, it doesn't build the branch on its own, but the merge of that branch into 'master' (or whatever the default branch is called), assuming can be it merged cleanly. So as soon as this fix lands in master, the PR builds should be fine. I suppose.

However, they will need to rebase their WIP/not PR-ed/not upstreamable branches on top of this fix if they want to run Travis CI builds in their own forks.

Unless, of course, the external influence does manage to fix itself overnight :) Under Dscho's bugreport it looks like they already merged a one-liner fix, but how long will it take to tickle down to Travis CI, I have no idea.

Johannes Schindelin· Feb 5, 2019, 11:41 UTC · re: SZEDER Gábor · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

Hi Gábor,
On Mon, 4 Feb 2019, SZEDER Gábor wrote:
> Under Dscho's bugreport it looks like they already merged a one-liner
> fix, but how long will it take to tickle down to Travis CI, I have no
> idea.

Since the fix affected a Homebrew package, it was not so much about trickling down to Travis, but about trickling down into the public Homebrew repositories. And it seems that that trickling is complete now.

Ciao, Dscho

Junio C Hamano· Feb 5, 2019, 17:53 UTC · re: Johannes Schindelin · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 9 quoted lines
> On Mon, 4 Feb 2019, SZEDER Gábor wrote:
>
>> Under Dscho's bugreport it looks like they already merged a one-liner
>> fix, but how long will it take to tickle down to Travis CI, I have no
>> idea.
>
> Since the fix affected a Homebrew package, it was not so much about
> trickling down to Travis, but about trickling down into the public
> Homebrew repositories. And it seems that that trickling is complete now.
Good news.  Thanks both for reporting and monitoring.

I still think the >/dev/null thing is a good idea, so unless I hear otherwise, let's merge it down to 'next', etc.

Johannes Schindelin· Feb 6, 2019, 11:33 UTC · re: Junio C Hamano · lore

Re: [PATCH] travis-ci: make the OSX build jobs' 'brew update' more quiet

Hi Junio,
On Tue, 5 Feb 2019, Junio C Hamano wrote:
Show 16 quoted lines
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Mon, 4 Feb 2019, SZEDER Gábor wrote:
> >
> >> Under Dscho's bugreport it looks like they already merged a one-liner
> >> fix, but how long will it take to tickle down to Travis CI, I have no
> >> idea.
> >
> > Since the fix affected a Homebrew package, it was not so much about
> > trickling down to Travis, but about trickling down into the public
> > Homebrew repositories. And it seems that that trickling is complete now.
> 
> Good news.  Thanks both for reporting and monitoring.
> 
> I still think the >/dev/null thing is a good idea, so unless I hear
> otherwise, let's merge it down to 'next', etc.
Yes, it is a very good idea.

Ciao, Dscho

← back to recent threads