{"thread":{"id":"56272","subject":"[PATCH] ci: update freebsd 12 cirrus job","startedAt":"2021-08-12T20:23:54Z","lastAt":"2021-08-14T22:29:18Z","messageCount":4,"participants":["Carlo Marcelo Arenas Belón","SZEDER Gábor","Carlo Arenas","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"432601","messageId":"20210812202200.82465-1-carenas@gmail.com","threadId":"56272","inReplyTo":null,"subject":"[PATCH] ci: update freebsd 12 cirrus job","fromName":"Carlo Marcelo Arenas Belón","fromEmail":"carenas@gmail.com","sentAt":"2021-08-12T20:22:00Z","receivedAt":"2021-08-12T20:23:54Z","isPatch":true,"sender":{"key":"carenas@gmail.com","avatar":"https://avatars.githubusercontent.com/u/76036?v=4"},"body":"make sure it uses a supported OS branch and uses all the resources\nthat can be allocated efficiently.\n\nwhile only 1GB of memory is needed, 2GB is the minimum for a 2 CPU\nmachine (the default), but by increasing parallelism wall time has\nbeen reduced by 35%.\n\nSigned-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>\n---\n .cirrus.yml | 9 ++++++++-\n 1 file changed, 8 insertions(+), 1 deletion(-)\n\ndiff --git a/.cirrus.yml b/.cirrus.yml\nindex c2f5fe385a..e114ffee1a 100644\n--- a/.cirrus.yml\n+++ b/.cirrus.yml\n@@ -2,8 +2,15 @@ env:\n   CIRRUS_CLONE_DEPTH: 1\n \n freebsd_12_task:\n+  env:\n+    GIT_PROVE_OPTS: \"--timer --jobs 10\"\n+    GIT_TEST_OPTS: \"--no-chain-lint --no-bin-wrappers\"\n+    MAKEFLAGS: \"-j4\"\n+    DEFAULT_TEST_TARGET: prove\n+    DEVELOPER: 1\n   freebsd_instance:\n-    image: freebsd-12-1-release-amd64\n+    image_family: freebsd-12-2\n+    memory: 2G\n   install_script:\n     pkg install -y gettext gmake perl5\n   create_user_script:\n-- \n2.33.0.rc1.379.g2890ef5eb6\n\n"},{"id":"432667","messageId":"20210813093909.GA3866771@szeder.dev","threadId":"56272","inReplyTo":"20210812202200.82465-1-carenas@gmail.com","subject":"Re: [PATCH] ci: update freebsd 12 cirrus job","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2021-08-13T09:39:09Z","receivedAt":"2021-08-13T09:39:17Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Thu, Aug 12, 2021 at 01:22:00PM -0700, Carlo Marcelo Arenas Belón wrote:\n> make sure it uses a supported OS branch and uses all the resources\n> that can be allocated efficiently.\n> \n> while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU\n> machine (the default), but by increasing parallelism wall time has\n> been reduced by 35%.\n> \n> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>\n> ---\n>  .cirrus.yml | 9 ++++++++-\n>  1 file changed, 8 insertions(+), 1 deletion(-)\n> \n> diff --git a/.cirrus.yml b/.cirrus.yml\n> index c2f5fe385a..e114ffee1a 100644\n> --- a/.cirrus.yml\n> +++ b/.cirrus.yml\n> @@ -2,8 +2,15 @@ env:\n>    CIRRUS_CLONE_DEPTH: 1\n>  \n>  freebsd_12_task:\n> +  env:\n> +    GIT_PROVE_OPTS: \"--timer --jobs 10\"\n\nWhy these prove options?\n\nOn other CI systems we pass 'prove' the option\n'--state=failed,slow,save' as well to reduce runtime.  However, this\nonly works when there is a persistent place for prove's state files,\ne.g. the cache feature of Travis CI.  If Cirrus CI lacks a similar\nfeature, then we can't benefit from this option, but it'd be worth\nmentioning in the commit message.\n\n> +    GIT_TEST_OPTS: \"--no-chain-lint --no-bin-wrappers\"\n\nWhy these test options?\n\nchain-linting is done by a mighty sed script; I think it's worth\nrunning it with FreeBSD's 'sed' as well.\n\nQuoting 't/README', '--no-bin-wrappers' \"can speed up test runs\nespecially on platforms where running shell scripts is expensive\".  Is\nrunning shell scripts really that expensive on FreeBSD?\n\nOTOH, why are there no options that would show us some information\nabout test failures, i.e. why no '--verbose-log -x --immediate' like\non other CI systems?\n\n> +    MAKEFLAGS: \"-j4\"\n> +    DEFAULT_TEST_TARGET: prove\n> +    DEVELOPER: 1\n>    freebsd_instance:\n> -    image: freebsd-12-1-release-amd64\n> +    image_family: freebsd-12-2\n> +    memory: 2G\n>    install_script:\n>      pkg install -y gettext gmake perl5\n>    create_user_script:\n> -- \n> 2.33.0.rc1.379.g2890ef5eb6\n> \n"},{"id":"432723","messageId":"CAPUEspjW3jSUwnz39bWBCmNS8mo_Cfjk7NPA5BXpQvPoypUEMg@mail.gmail.com","threadId":"56272","inReplyTo":"20210813093909.GA3866771@szeder.dev","subject":"Re: [PATCH] ci: update freebsd 12 cirrus job","fromName":"Carlo Arenas","fromEmail":"carenas@gmail.com","sentAt":"2021-08-14T01:42:34Z","receivedAt":"2021-08-14T01:43:24Z","isPatch":true,"sender":{"key":"carenas@gmail.com","avatar":"https://avatars.githubusercontent.com/u/76036?v=4"},"body":"On Fri, Aug 13, 2021 at 2:39 AM SZEDER Gábor <szeder.dev@gmail.com> wrote:\n>\n> On Thu, Aug 12, 2021 at 01:22:00PM -0700, Carlo Marcelo Arenas Belón wrote:\n> > diff --git a/.cirrus.yml b/.cirrus.yml\n> > index c2f5fe385a..e114ffee1a 100644\n> > --- a/.cirrus.yml\n> > +++ b/.cirrus.yml\n> > @@ -2,8 +2,15 @@ env:\n> >    CIRRUS_CLONE_DEPTH: 1\n> >\n> >  freebsd_12_task:\n> > +  env:\n> > +    GIT_PROVE_OPTS: \"--timer --jobs 10\"\n>\n> Why these prove options?\n\nthat line came from the Github Actions part of ci/lib.sh.\n\n--jobs is the one I was going after, but kept --timer for consistency.\n\n> On other CI systems we pass 'prove' the option\n> '--state=failed,slow,save' as well to reduce runtime.  However, this\n> only works when there is a persistent place for prove's state files,\n> e.g. the cache feature of Travis CI.\n\nnote that when GitHub Actions was added[1], it was explicitly excluded\nbecause it wasn't really that effective\n\n> If Cirrus CI lacks a similar\n> feature, then we can't benefit from this option, but it'd be worth\n> mentioning in the commit message.\n\nThey do have the option and will likely be a small change that I could\nsend as a follow up, but wasn't included in this change since it was\n\"controversial\" as pointed above.\n\nnote also that the other uses of it aren't really being exercised,\nsince AFAIK neither azure or travis are running now, and indeed last\ntime I tried to run CI in my own account it didn't end up well, with\nno successful run and running out of credit before I could even\nattempt to fix them all.\n\nindeed, was even going to suggest (after release) that they are\nremoved, which will also make the code cleaner for integrating cirrus\n(which still has a usable opensource plan).\n\n> > +    GIT_TEST_OPTS: \"--no-chain-lint --no-bin-wrappers\"\n> Why these test options?\n\nagain lifted from the \"GitHub Actions\" code just for consistency and\nto avoid breaking if either of those are broken (considering the only\nother running CI does test with those unconditionally), but I realize\nnow that I might have been too conservative since my focus with this\ncommit was \"speed and accuracy\".\n\n> chain-linting is done by a mighty sed script; I think it's worth\n> running it with FreeBSD's 'sed' as well.\n\nindeed, even if I recall correctly it was Solaris sed the one that\nbroke, and I would expect anyone breaking this chainlinting will\nnotice very quickly on their local run.\n\n> Quoting 't/README', '--no-bin-wrappers' \"can speed up test runs\n> especially on platforms where running shell scripts is expensive\".  Is\n> running shell scripts really that expensive on FreeBSD?\n\nno, I am assuming they were added with Windows in mind, but note that\nas configure they also affect all the linux builds through GitHub\nActions.\nBoth these flags might be worth adding back for all non Windows\nsystems, but doing so will be easier when using the \"ci\" scripts for\nall environments and after it has been discussed IMHO.\n\n> OTOH, why are there no options that would show us some information\n> about test failures, i.e. why no '--verbose-log -x --immediate' like\n> on other CI systems?\n\nI wasn't the original author, so can't answer why it wasn't done; but\nI sure think the next step will be to add support for it to the ci/\nscripts and with that make it more consistent, which likely will\ninclude that change.\n\nThis patch was just the minimal change to make sure that it runs\nefficiently and is representative of current users.\n\nCarlo\n\n[1] https://lore.kernel.org/git/40fe4f7e2c34c6997a6824ef0dc6aad7a71a4434.1586538752.git.congdanhqx@gmail.com/\n"},{"id":"432754","messageId":"nycvar.QRO.7.76.6.2108150022160.59@tvgsbejvaqbjf.bet","threadId":"56272","inReplyTo":"20210813093909.GA3866771@szeder.dev","subject":"Re: [PATCH] ci: update freebsd 12 cirrus job","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2021-08-14T22:29:05Z","receivedAt":"2021-08-14T22:29:18Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Fri, 13 Aug 2021, SZEDER Gábor wrote:\n\n> On Thu, Aug 12, 2021 at 01:22:00PM -0700, Carlo Marcelo Arenas Belón wrote:\n> > make sure it uses a supported OS branch and uses all the resources\n> > that can be allocated efficiently.\n> >\n> > while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU\n> > machine (the default), but by increasing parallelism wall time has\n> > been reduced by 35%.\n> >\n> > Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>\n> > ---\n> >  .cirrus.yml | 9 ++++++++-\n> >  1 file changed, 8 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/.cirrus.yml b/.cirrus.yml\n> > index c2f5fe385a..e114ffee1a 100644\n> > --- a/.cirrus.yml\n> > +++ b/.cirrus.yml\n> > @@ -2,8 +2,15 @@ env:\n> >    CIRRUS_CLONE_DEPTH: 1\n> >\n> >  freebsd_12_task:\n> > +  env:\n> > +    GIT_PROVE_OPTS: \"--timer --jobs 10\"\n>\n> Why these prove options?\n\nThe only problem I see with it is that its `--jobs 10` disagrees with the\n`MAKEFLAGS: -j4` below. Either both should say 10, or both should say 4.\n\n> On other CI systems we pass 'prove' the option\n> '--state=failed,slow,save' as well to reduce runtime.  However, this\n> only works when there is a persistent place for prove's state files,\n> e.g. the cache feature of Travis CI.  If Cirrus CI lacks a similar\n> feature, then we can't benefit from this option, but it'd be worth\n> mentioning in the commit message.\n\nWe also don't benefit from this in the GitHub workflow because there is\nlikewise no easy way to persist the state.\n\n> > +    GIT_TEST_OPTS: \"--no-chain-lint --no-bin-wrappers\"\n>\n> Why these test options?\n\nI guess that's for the same reason we use these options in the Windows\ntests: it speeds up things for a historically pretty slow build axis. At\nleast I seem to see FreeBSD runs lagging behind all the time.\n\n> chain-linting is done by a mighty sed script; I think it's worth\n> running it with FreeBSD's 'sed' as well.\n>\n> Quoting 't/README', '--no-bin-wrappers' \"can speed up test runs\n> especially on platforms where running shell scripts is expensive\".  Is\n> running shell scripts really that expensive on FreeBSD?\n\nI don't think it is FreeBSD per se, but the available VMs that make this\nspeed-up worth our while.\n\n> OTOH, why are there no options that would show us some information\n> about test failures, i.e. why no '--verbose-log -x --immediate' like\n> on other CI systems?\n\nThat's because we don't use the `ci/` scripts at all, and the failures are\nnot actually shown.\n\nThis is a mild annoyance to me, I have to admit: there is very little in\nthe way of actionable information whenever the FreeBSD jobs fail.\n\nLucky for me, I've so far only encountered breakages that _also_ affected\nthe macOS build axes, and therefore I could diagnose the issues in the\nGitHub workflow run logs.\n\nBut yes, it would be nice if the FreeBSD CI runs were more helpful in case\nof problems.\n\nAs far as this here patch goes: I am in favor of integrating it as-is.\n\nCiao,\nDscho\n\n>\n> > +    MAKEFLAGS: \"-j4\"\n> > +    DEFAULT_TEST_TARGET: prove\n> > +    DEVELOPER: 1\n> >    freebsd_instance:\n> > -    image: freebsd-12-1-release-amd64\n> > +    image_family: freebsd-12-2\n> > +    memory: 2G\n> >    install_script:\n> >      pkg install -y gettext gmake perl5\n> >    create_user_script:\n> > --\n> > 2.33.0.rc1.379.g2890ef5eb6\n> >\n>\n"}]}