{"thread":{"id":"60745","subject":"[PATCH 0/3] ci: add support for macOS to GitLab CI","startedAt":"2024-01-15T11:44:57Z","lastAt":"2024-01-22T15:44:38Z","messageCount":17,"participants":["Patrick Steinhardt","Phillip Wood","Matthias Aßhauer","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"486807","messageId":"cover.1705318985.git.ps@pks.im","threadId":"60745","inReplyTo":null,"subject":"[PATCH 0/3] ci: add support for macOS to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-15T11:44:52Z","receivedAt":"2024-01-15T11:44:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series extends GitLab CI to also support macOS. Besides\nextending test coverage for GitLab users, this change also has the added\nbenefit that the macOS runners at GitLab are based on Apple silicon,\nwhich to the best of my knowledge is not something we're currently\ntesting on.\n\nThis patch series builds on top of ps/gitlab-ci-static-analysis\n(currently at cd69c635a1 (ci: add job performing static analysis on\nGitLab CI, 2023-12-28)) to avoid a conflict.\n\nPatrick\n\nPatrick Steinhardt (3):\n  ci: make p4 setup on macOS more robust\n  Makefile: detect new Homebrew location for ARM-based Macs\n  ci: add macOS jobs to GitLab CI\n\n .gitlab-ci.yml             | 26 +++++++++++++++++++++++++-\n ci/install-dependencies.sh | 10 ++++------\n ci/lib.sh                  | 12 +++++++++++-\n config.mak.uname           | 13 +++++++++++++\n 4 files changed, 53 insertions(+), 8 deletions(-)\n\n\nbase-commit: cd69c635a1a62b0c8bfdbf221778be8a512ad048\n-- \n2.43.GIT\n\n"},{"id":"486808","messageId":"a5d725bea7b2cc2b8f66682920dbba016e89da9e.1705318985.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705318985.git.ps@pks.im","subject":"[PATCH 1/3] ci: make p4 setup on macOS more robust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-15T11:44:57Z","receivedAt":"2024-01-15T11:45:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When setting up Perforce on macOS we put both `p4` and `p4d` into\n\"$HOME/bin\". On GitHub CI this directory is indeed contained in the PATH\nenvironment variable and thus there is no need for additional setup than\nto put the binaries there. But GitLab CI does not do this, and thus our\nPerforce-based tests would be skipped there even though we download the\nbinaries.\n\nRefactor the setup code to become more robust by downloading binaries\ninto a separate directory which we then manually append to our PATH.\nThis matches what we do on Linux-based jobs.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n ci/install-dependencies.sh | 10 ++++------\n ci/lib.sh                  |  3 +++\n 2 files changed, 7 insertions(+), 6 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 4f407530d3..b4e22de3cb 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -37,15 +37,13 @@ macos-*)\n \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n \tbrew install $BREW_INSTALL_PACKAGES\n \tbrew link --force gettext\n-\tmkdir -p $HOME/bin\n-\t(\n-\t\tcd $HOME/bin\n+\n+\tmkdir -p \"$P4_PATH\"\n+\tpushd \"$P4_PATH\"\n \t\twget -q \"$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz\" &&\n \t\ttar -xf helix-core-server.tgz &&\n \t\tsudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true\n-\t)\n-\tPATH=\"$PATH:${HOME}/bin\"\n-\texport PATH\n+\tpopd\n \n \tif test -n \"$CC_PACKAGE\"\n \tthen\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex c749b21366..f631206a44 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -344,6 +344,9 @@ macos-*)\n \tthen\n \t\tMAKEFLAGS=\"$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes\"\n \tfi\n+\n+\tP4_PATH=\"$HOME/custom/p4\"\n+\texport PATH=\"$P4_PATH:$PATH\"\n \t;;\n esac\n \n-- \n2.43.GIT\n\n"},{"id":"486809","messageId":"3adb0b7ae888db40949f788f9582569617c20c2b.1705318985.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705318985.git.ps@pks.im","subject":"[PATCH 2/3] Makefile: detect new Homebrew location for ARM-based Macs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-15T11:45:01Z","receivedAt":"2024-01-15T11:45:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"With the introduction of the ARM-based Macs the default location for\nHomebrew has changed from \"/usr/local\" to \"/opt/homebrew\". We only\nhandle the former location though, which means that unless the user has\nmanually configured required search paths we won't be able to locate it.\n\nImprove upon this by adding relevant paths to our CFLAGS and LDFLAGS as\nwell as detecting the location of msgfmt(1).\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n config.mak.uname | 13 +++++++++++++\n 1 file changed, 13 insertions(+)\n\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 3bb03f423a..dacc95172d 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -158,6 +158,19 @@ ifeq ($(uname_S),Darwin)\n \t\tifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)\n \t\t\tMSGFMT = /usr/local/opt/gettext/bin/msgfmt\n \t\tendif\n+\t# On newer ARM-based machines the default installation path has changed to\n+\t# /opt/homebrew. Include it in our search paths so that the user does not\n+\t# have to configure this manually.\n+\t#\n+\t# Note that we do not employ the same workaround as above where we manually\n+\t# add gettext. The issue was fixed more than three years ago by now, and at\n+\t# that point there haven't been any ARM-based Macs yet.\n+\telse ifeq ($(shell test -d /opt/homebrew/ && echo y),y)\n+\t\tBASIC_CFLAGS += -I/opt/homebrew/include\n+\t\tBASIC_LDFLAGS += -L/opt/homebrew/lib\n+\t\tifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)\n+\t\t\tMSGFMT = /opt/homebrew/bin/msgfmt\n+\t\tendif\n \tendif\n \n \t# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require\n-- \n2.43.GIT\n\n"},{"id":"486810","messageId":"d196cfd9d01fe3b52c75a1e4e0aca9f67567ab43.1705318985.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705318985.git.ps@pks.im","subject":"[PATCH 3/3] ci: add macOS jobs to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-15T11:45:05Z","receivedAt":"2024-01-15T11:45:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.\nThis matches equivalent jobs we have for GitHub Workflows, except that\nwe use macOS 14 instead of macOS 13.\n\nNote that one test marked as `test_must_fail` is surprisingly passing:\n\n  t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n    TODO passed:   12\n\nThis seems to boil down to an unexpected difference in how regcomp(1)\nworks when matching NUL bytes. Cross-checking with the respective GitHub\njob shows though that this is not an issue unique to the GitLab CI job\nas it passes in the same way there.\n\nFurther note that we do not include the equivalent for the \"osx-gcc\" job\nthat we use with GitHub Workflows. This is because the runner for macOS\non GitLab is running on Apple M1 machines and thus uses the \"arm64\"\narchitecture. GCC does not support this platform yet.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n .gitlab-ci.yml | 26 +++++++++++++++++++++++++-\n ci/lib.sh      |  9 ++++++++-\n 2 files changed, 33 insertions(+), 2 deletions(-)\n\ndiff --git a/.gitlab-ci.yml b/.gitlab-ci.yml\nindex 793243421c..9748970798 100644\n--- a/.gitlab-ci.yml\n+++ b/.gitlab-ci.yml\n@@ -7,7 +7,7 @@ workflow:\n     - if: $CI_COMMIT_TAG\n     - if: $CI_COMMIT_REF_PROTECTED == \"true\"\n \n-test:\n+test:linux:\n   image: $image\n   before_script:\n     - ./ci/install-docker-dependencies.sh\n@@ -52,6 +52,30 @@ test:\n       - t/failed-test-artifacts\n     when: on_failure\n \n+test:osx:\n+  image: $image\n+  tags:\n+    - saas-macos-medium-m1\n+  before_script:\n+    - ./ci/install-dependencies.sh\n+  script:\n+    - ./ci/run-build-and-tests.sh\n+  after_script:\n+    - |\n+      if test \"$CI_JOB_STATUS\" != 'success'\n+      then\n+        ./ci/print-test-failures.sh\n+      fi\n+  parallel:\n+    matrix:\n+      - jobname: osx-clang\n+        image: macos-13-xcode-14\n+        CC: clang\n+  artifacts:\n+    paths:\n+      - t/failed-test-artifacts\n+    when: on_failure\n+\n static-analysis:\n   image: ubuntu:22.04\n   variables:\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex f631206a44..d5dd2f2697 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -252,7 +252,14 @@ then\n \tCI_COMMIT=\"$CI_COMMIT_SHA\"\n \tcase \"$CI_JOB_IMAGE\" in\n \tmacos-*)\n-\t\tCI_OS_NAME=osx;;\n+\t\t# GitLab CI has Python installed via multiple package managers,\n+\t\t# most notably via asdf and Homebrew. Ensure that our builds\n+\t\t# pick up the Homebrew one by prepending it to our PATH as the\n+\t\t# asdf one breaks tests.\n+\t\texport PATH=\"$(brew --prefix)/bin:$PATH\"\n+\n+\t\tCI_OS_NAME=osx\n+\t\t;;\n \talpine:*|fedora:*|ubuntu:*)\n \t\tCI_OS_NAME=linux;;\n \t*)\n-- \n2.43.GIT\n\n"},{"id":"486840","messageId":"61115e68-3d63-4e08-85f7-ae6650f3724e@gmail.com","threadId":"60745","inReplyTo":"d196cfd9d01fe3b52c75a1e4e0aca9f67567ab43.1705318985.git.ps@pks.im","subject":"Re: [PATCH 3/3] ci: add macOS jobs to GitLab CI","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-01-16T14:58:53Z","receivedAt":"2024-01-16T14:58:56Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Patrick\n\nOn 15/01/2024 11:45, Patrick Steinhardt wrote:\n> Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.\n\nThis doesn't match whats in the rest of the commit message where you \nexplain why there is no gcc job. The patch itself looks good to me and \nit is nice that we'll now be testing on arm64 with the GitLab runners.\n\n> This matches equivalent jobs we have for GitHub Workflows, except that\n> we use macOS 14 instead of macOS 13.\n> \n> Note that one test marked as `test_must_fail` is surprisingly passing:\n> \n>    t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n>      TODO passed:   12\n> \n> This seems to boil down to an unexpected difference in how regcomp(1)\n\nnit: regcomp(3)?\n\nBest Wishes\n\nPhillip\n\n> works when matching NUL bytes. Cross-checking with the respective GitHub\n> job shows though that this is not an issue unique to the GitLab CI job\n> as it passes in the same way there.\n> \n> Further note that we do not include the equivalent for the \"osx-gcc\" job\n> that we use with GitHub Workflows. This is because the runner for macOS\n> on GitLab is running on Apple M1 machines and thus uses the \"arm64\"\n> architecture. GCC does not support this platform yet.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>   .gitlab-ci.yml | 26 +++++++++++++++++++++++++-\n>   ci/lib.sh      |  9 ++++++++-\n>   2 files changed, 33 insertions(+), 2 deletions(-)\n> \n> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml\n> index 793243421c..9748970798 100644\n> --- a/.gitlab-ci.yml\n> +++ b/.gitlab-ci.yml\n> @@ -7,7 +7,7 @@ workflow:\n>       - if: $CI_COMMIT_TAG\n>       - if: $CI_COMMIT_REF_PROTECTED == \"true\"\n>   \n> -test:\n> +test:linux:\n>     image: $image\n>     before_script:\n>       - ./ci/install-docker-dependencies.sh\n> @@ -52,6 +52,30 @@ test:\n>         - t/failed-test-artifacts\n>       when: on_failure\n>   \n> +test:osx:\n> +  image: $image\n> +  tags:\n> +    - saas-macos-medium-m1\n> +  before_script:\n> +    - ./ci/install-dependencies.sh\n> +  script:\n> +    - ./ci/run-build-and-tests.sh\n> +  after_script:\n> +    - |\n> +      if test \"$CI_JOB_STATUS\" != 'success'\n> +      then\n> +        ./ci/print-test-failures.sh\n> +      fi\n> +  parallel:\n> +    matrix:\n> +      - jobname: osx-clang\n> +        image: macos-13-xcode-14\n> +        CC: clang\n> +  artifacts:\n> +    paths:\n> +      - t/failed-test-artifacts\n> +    when: on_failure\n> +\n>   static-analysis:\n>     image: ubuntu:22.04\n>     variables:\n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index f631206a44..d5dd2f2697 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -252,7 +252,14 @@ then\n>   \tCI_COMMIT=\"$CI_COMMIT_SHA\"\n>   \tcase \"$CI_JOB_IMAGE\" in\n>   \tmacos-*)\n> -\t\tCI_OS_NAME=osx;;\n> +\t\t# GitLab CI has Python installed via multiple package managers,\n> +\t\t# most notably via asdf and Homebrew. Ensure that our builds\n> +\t\t# pick up the Homebrew one by prepending it to our PATH as the\n> +\t\t# asdf one breaks tests.\n> +\t\texport PATH=\"$(brew --prefix)/bin:$PATH\"\n> +\n> +\t\tCI_OS_NAME=osx\n> +\t\t;;\n>   \talpine:*|fedora:*|ubuntu:*)\n>   \t\tCI_OS_NAME=linux;;\n>   \t*)\n"},{"id":"486910","messageId":"ZaeDBO_8-WONiV0l@tanuki","threadId":"60745","inReplyTo":"61115e68-3d63-4e08-85f7-ae6650f3724e@gmail.com","subject":"Re: [PATCH 3/3] ci: add macOS jobs to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-17T07:34:28Z","receivedAt":"2024-01-17T07:35:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jan 16, 2024 at 02:58:53PM +0000, Phillip Wood wrote:\n> Hi Patrick\n> \n> On 15/01/2024 11:45, Patrick Steinhardt wrote:\n> > Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.\n> \n> This doesn't match whats in the rest of the commit message where you explain\n> why there is no gcc job. The patch itself looks good to me and it is nice\n> that we'll now be testing on arm64 with the GitLab runners.\n\nOops. In my first iterations I still had an osx-gcc job, but I could not\nget it passing due to the architectural difficulties explained in the\nmessage, so I ended up removing it. This sentence is thus a leftover\nfrom previous iterations.\n\n> > This matches equivalent jobs we have for GitHub Workflows, except that\n> > we use macOS 14 instead of macOS 13.\n> > \n> > Note that one test marked as `test_must_fail` is surprisingly passing:\n> > \n> >    t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n> >      TODO passed:   12\n> > \n> > This seems to boil down to an unexpected difference in how regcomp(1)\n> \n> nit: regcomp(3)?\n\nIndeed, thanks!\n\nPatrick\n"},{"id":"486957","messageId":"DB9P250MB0692629640B05593B798E5A0A5712@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"60745","inReplyTo":"a5d725bea7b2cc2b8f66682920dbba016e89da9e.1705318985.git.ps@pks.im","subject":"Re: [PATCH 1/3] ci: make p4 setup on macOS more robust","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2024-01-18T07:19:10Z","receivedAt":"2024-01-18T07:19:14Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Mon, 15 Jan 2024, Patrick Steinhardt wrote:\n\n> When setting up Perforce on macOS we put both `p4` and `p4d` into\n> \"$HOME/bin\". On GitHub CI this directory is indeed contained in the PATH\n> environment variable and thus there is no need for additional setup than\n> to put the binaries there. But GitLab CI does not do this, and thus our\n> Perforce-based tests would be skipped there even though we download the\n> binaries.\n>\n> Refactor the setup code to become more robust by downloading binaries\n> into a separate directory which we then manually append to our PATH.\n> This matches what we do on Linux-based jobs.\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n> ci/install-dependencies.sh | 10 ++++------\n> ci/lib.sh                  |  3 +++\n> 2 files changed, 7 insertions(+), 6 deletions(-)\n>\n> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\n> index 4f407530d3..b4e22de3cb 100755\n> --- a/ci/install-dependencies.sh\n> +++ b/ci/install-dependencies.sh\n> @@ -37,15 +37,13 @@ macos-*)\n> \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n> \tbrew install $BREW_INSTALL_PACKAGES\n> \tbrew link --force gettext\n> -\tmkdir -p $HOME/bin\n> -\t(\n> -\t\tcd $HOME/bin\n> +\n> +\tmkdir -p \"$P4_PATH\"\n> +\tpushd \"$P4_PATH\"\n> \t\twget -q \"$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz\" &&\n> \t\ttar -xf helix-core-server.tgz &&\n> \t\tsudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true\n> -\t)\n> -\tPATH=\"$PATH:${HOME}/bin\"\n> -\texport PATH\n\nDidn't this put \"$HOME/bin\" on the PATH? And isn't the main premise of \nthis patch that \"$HOME/bin\" is not on the PATH?\nor is the issue mainly about where we modify and export PATH and was \nmasked by GitHub Actions already having \"$HOME/bin\" on the PATH?\n\n> +\tpopd\n>\n> \tif test -n \"$CC_PACKAGE\"\n> \tthen\n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index c749b21366..f631206a44 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -344,6 +344,9 @@ macos-*)\n> \tthen\n> \t\tMAKEFLAGS=\"$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes\"\n> \tfi\n> +\n> +\tP4_PATH=\"$HOME/custom/p4\"\n> +\texport PATH=\"$P4_PATH:$PATH\"\n> \t;;\n> esac\n>\n> -- \n> 2.43.GIT\n>\n>\n"},{"id":"486961","messageId":"ZajzDNR1MtbV7zxp@tanuki","threadId":"60745","inReplyTo":"DB9P250MB0692629640B05593B798E5A0A5712@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH 1/3] ci: make p4 setup on macOS more robust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T09:44:44Z","receivedAt":"2024-01-18T09:44:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jan 18, 2024 at 08:19:10AM +0100, Matthias Aßhauer wrote:\n> \n> \n> On Mon, 15 Jan 2024, Patrick Steinhardt wrote:\n> \n> > When setting up Perforce on macOS we put both `p4` and `p4d` into\n> > \"$HOME/bin\". On GitHub CI this directory is indeed contained in the PATH\n> > environment variable and thus there is no need for additional setup than\n> > to put the binaries there. But GitLab CI does not do this, and thus our\n> > Perforce-based tests would be skipped there even though we download the\n> > binaries.\n> > \n> > Refactor the setup code to become more robust by downloading binaries\n> > into a separate directory which we then manually append to our PATH.\n> > This matches what we do on Linux-based jobs.\n> > \n> > Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> > ---\n> > ci/install-dependencies.sh | 10 ++++------\n> > ci/lib.sh                  |  3 +++\n> > 2 files changed, 7 insertions(+), 6 deletions(-)\n> > \n> > diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\n> > index 4f407530d3..b4e22de3cb 100755\n> > --- a/ci/install-dependencies.sh\n> > +++ b/ci/install-dependencies.sh\n> > @@ -37,15 +37,13 @@ macos-*)\n> > \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n> > \tbrew install $BREW_INSTALL_PACKAGES\n> > \tbrew link --force gettext\n> > -\tmkdir -p $HOME/bin\n> > -\t(\n> > -\t\tcd $HOME/bin\n> > +\n> > +\tmkdir -p \"$P4_PATH\"\n> > +\tpushd \"$P4_PATH\"\n> > \t\twget -q \"$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz\" &&\n> > \t\ttar -xf helix-core-server.tgz &&\n> > \t\tsudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true\n> > -\t)\n> > -\tPATH=\"$PATH:${HOME}/bin\"\n> > -\texport PATH\n> \n> Didn't this put \"$HOME/bin\" on the PATH? And isn't the main premise of this\n> patch that \"$HOME/bin\" is not on the PATH?\n> or is the issue mainly about where we modify and export PATH and was masked\n> by GitHub Actions already having \"$HOME/bin\" on the PATH?\n\nYes and no. While these lines put it in PATH, this only works inside of\n\"ci/install-dependencies.sh\". When we call \"ci/run-build-and-test.sh\" we\ndo not source this script though, which means that \"$HOME/bin\" will not\nbe part of PATH during the actual test run unless it was already added\nby the CI.\n\nI'll update the commit message to explain this better.\n\nPatrick\n"},{"id":"486962","messageId":"cover.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705318985.git.ps@pks.im","subject":"[PATCH v2 0/5] ci: add support for macOS to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:22:41Z","receivedAt":"2024-01-18T10:22:47Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis is the second version of my patch series that adds a macOS job to\nGitLab CI. Changes compared to v1:\n\n  - Added a fix for a flaky test in t7527 that caused the pipeline to\n    fail in ~50% of all runs.\n\n  - Improved some commit messages.\n\n  - Tests now write test data into a RAMDisk. This speeds up tests and\n    fixes some hung pipelines I was seeing.\n\nThanks for your reviews so far!\n\nPatrick\n\nPatrick Steinhardt (5):\n  t7527: decrease likelihood of racing with fsmonitor daemon\n  Makefile: detect new Homebrew location for ARM-based Macs\n  ci: handle TEST_OUTPUT_DIRECTORY when printing test failures\n  ci: make p4 setup on macOS more robust\n  ci: add macOS jobs to GitLab CI\n\n .gitlab-ci.yml               | 34 +++++++++++++++++++++++++++++++++-\n ci/install-dependencies.sh   | 10 ++++------\n ci/lib.sh                    | 12 +++++++++++-\n ci/print-test-failures.sh    |  2 +-\n config.mak.uname             | 13 +++++++++++++\n t/t7527-builtin-fsmonitor.sh |  2 +-\n 6 files changed, 63 insertions(+), 10 deletions(-)\n\nRange-diff against v1:\n-:  ---------- > 1:  554b1c8546 t7527: decrease likelihood of racing with fsmonitor daemon\n2:  3adb0b7ae8 = 2:  32d8bd1d78 Makefile: detect new Homebrew location for ARM-based Macs\n-:  ---------- > 3:  d55da77747 ci: handle TEST_OUTPUT_DIRECTORY when printing test failures\n1:  a5d725bea7 ! 4:  1ed6e68650 ci: make p4 setup on macOS more robust\n    @@ Commit message\n         into a separate directory which we then manually append to our PATH.\n         This matches what we do on Linux-based jobs.\n     \n    +    Note that it may seem like we already did append \"$HOME/bin\" to PATH\n    +    because we're actually removing the lines that adapt PATH. But we only\n    +    ever adapted the PATH variable in \"ci/install-dependencies.sh\", and\n    +    didn't adapt it when running \"ci/run-build-and-test.sh\". Consequently,\n    +    the required binaries wouldn't be found during the test run unless the\n    +    CI platform already had the \"$HOME/bin\" in PATH right from the start.\n    +\n         Signed-off-by: Patrick Steinhardt <ps@pks.im>\n     \n      ## ci/install-dependencies.sh ##\n3:  d196cfd9d0 ! 5:  c5ed38f0a6 ci: add macOS jobs to GitLab CI\n    @@ Metadata\n      ## Commit message ##\n         ci: add macOS jobs to GitLab CI\n     \n    -    Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.\n    -    This matches equivalent jobs we have for GitHub Workflows, except that\n    -    we use macOS 14 instead of macOS 13.\n    +    Add a job to GitLab CI which runs tests on macOS, which matches the\n    +    equivalent \"osx-clang\" job that we have for GitHub Workflows. One\n    +    significant difference though is that this new job runs on Apple M1\n    +    machines and thus uses the \"arm64\" architecture. As GCC does not yet\n    +    support this comparatively new architecture we cannot easily include an\n    +    equivalent for the \"osx-gcc\" job that exists in GitHub Workflows.\n     \n         Note that one test marked as `test_must_fail` is surprisingly passing:\n     \n           t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n             TODO passed:   12\n     \n    -    This seems to boil down to an unexpected difference in how regcomp(1)\n    +    This seems to boil down to an unexpected difference in how regcomp(3P)\n         works when matching NUL bytes. Cross-checking with the respective GitHub\n    -    job shows though that this is not an issue unique to the GitLab CI job\n    -    as it passes in the same way there.\n    -\n    -    Further note that we do not include the equivalent for the \"osx-gcc\" job\n    -    that we use with GitHub Workflows. This is because the runner for macOS\n    -    on GitLab is running on Apple M1 machines and thus uses the \"arm64\"\n    -    architecture. GCC does not support this platform yet.\n    +    job shows that this is not an issue unique to the GitLab CI job as it\n    +    passes in the same way there.\n     \n         Signed-off-by: Patrick Steinhardt <ps@pks.im>\n     \n    @@ .gitlab-ci.yml: test:\n     +  image: $image\n     +  tags:\n     +    - saas-macos-medium-m1\n    ++  variables:\n    ++    TEST_OUTPUT_DIRECTORY: \"/Volumes/RAMDisk\"\n     +  before_script:\n    ++    # Create a 4GB RAM disk that we use to store test output on. This small hack\n    ++    # significantly speeds up tests by more than a factor of 2 because the\n    ++    # macOS runners use network-attached storage as disks, which is _really_\n    ++    # slow with the many small writes that our tests do.\n    ++    - sudo diskutil apfs create $(hdiutil attach -nomount ram://8192000) RAMDisk\n     +    - ./ci/install-dependencies.sh\n     +  script:\n     +    - ./ci/run-build-and-tests.sh\n    @@ .gitlab-ci.yml: test:\n     +      if test \"$CI_JOB_STATUS\" != 'success'\n     +      then\n     +        ./ci/print-test-failures.sh\n    ++        mv \"$TEST_OUTPUT_DIRECTORY\"/failed-test-artifacts t/\n     +      fi\n     +  parallel:\n     +    matrix:\n\nbase-commit: cd69c635a1a62b0c8bfdbf221778be8a512ad048\n-- \n2.43.GIT\n\n"},{"id":"486963","messageId":"554b1c85463306809a3503622ec6927165e2f4c2.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"[PATCH v2 1/5] t7527: decrease likelihood of racing with fsmonitor daemon","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:22:45Z","receivedAt":"2024-01-18T10:22:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In t7527, we test that the builtin fsmonitor daemon works well in\nvarious edge cases. One of these tests is frequently failing because\nevents reported by the fsmonitor--daemon are missing an expected event.\nThis failure is essentially a race condition: we do not wait for the\ndaemon to flush out all events before we ask it to quit. Consequently,\nit can happen that we miss some expected events.\n\nIn other testcases we counteract this race by sending a simple query to\nthe daemon. Quoting a comment:\n\n  We run a simple query after modifying the filesystem just to introduce\n  a bit of a delay so that the trace logging from the daemon has time to\n  get flushed to disk.\n\nNow this workaround is not a \"proper\" fix as we do not wait for all\nevents to have been synchronized in a deterministic way. But this fix\nseems to be sufficient for all the other tests to pass, so it must not\nbe all that bad.\n\nConvert the failing test to do the same. While the test was previously\nfailing in about 50% of the test runs, I couldn't reproduce the failure\nafter the change anymore.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n t/t7527-builtin-fsmonitor.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh\nindex 78503158fd..363f9dc0e4 100755\n--- a/t/t7527-builtin-fsmonitor.sh\n+++ b/t/t7527-builtin-fsmonitor.sh\n@@ -978,7 +978,7 @@ test_expect_success !UNICODE_COMPOSITION_SENSITIVE 'Unicode nfc/nfd' '\n \tmkdir test_unicode/nfd &&\n \tmkdir test_unicode/nfd/d_${utf8_nfd} &&\n \n-\tgit -C test_unicode fsmonitor--daemon stop &&\n+\ttest-tool -C test_unicode fsmonitor-client query --token 0 &&\n \n \tif test_have_prereq UNICODE_NFC_PRESERVED\n \tthen\n-- \n2.43.GIT\n\n"},{"id":"486964","messageId":"32d8bd1d783b3f9f2bc31b8a4b3e9de2d13f6b87.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"[PATCH v2 2/5] Makefile: detect new Homebrew location for ARM-based Macs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:22:49Z","receivedAt":"2024-01-18T10:22:53Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"With the introduction of the ARM-based Macs the default location for\nHomebrew has changed from \"/usr/local\" to \"/opt/homebrew\". We only\nhandle the former location though, which means that unless the user has\nmanually configured required search paths we won't be able to locate it.\n\nImprove upon this by adding relevant paths to our CFLAGS and LDFLAGS as\nwell as detecting the location of msgfmt(1).\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n config.mak.uname | 13 +++++++++++++\n 1 file changed, 13 insertions(+)\n\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 3bb03f423a..dacc95172d 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -158,6 +158,19 @@ ifeq ($(uname_S),Darwin)\n \t\tifeq ($(shell test -x /usr/local/opt/gettext/bin/msgfmt && echo y),y)\n \t\t\tMSGFMT = /usr/local/opt/gettext/bin/msgfmt\n \t\tendif\n+\t# On newer ARM-based machines the default installation path has changed to\n+\t# /opt/homebrew. Include it in our search paths so that the user does not\n+\t# have to configure this manually.\n+\t#\n+\t# Note that we do not employ the same workaround as above where we manually\n+\t# add gettext. The issue was fixed more than three years ago by now, and at\n+\t# that point there haven't been any ARM-based Macs yet.\n+\telse ifeq ($(shell test -d /opt/homebrew/ && echo y),y)\n+\t\tBASIC_CFLAGS += -I/opt/homebrew/include\n+\t\tBASIC_LDFLAGS += -L/opt/homebrew/lib\n+\t\tifeq ($(shell test -x /opt/homebrew/bin/msgfmt && echo y),y)\n+\t\t\tMSGFMT = /opt/homebrew/bin/msgfmt\n+\t\tendif\n \tendif\n \n \t# The builtin FSMonitor on MacOS builds upon Simple-IPC.  Both require\n-- \n2.43.GIT\n\n"},{"id":"486965","messageId":"d55da77747d71e6d300ecba5fb3709fb258ef29f.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"[PATCH v2 3/5] ci: handle TEST_OUTPUT_DIRECTORY when printing test failures","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:22:53Z","receivedAt":"2024-01-18T10:22:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The TEST_OUTPUT_DIRECTORY environment variable can be used to instruct\nthe test suite to write test data and test results into a different\nlocation than into \"t/\". The \"ci/print-test-failures.sh\" script does not\nknow to handle this environment variable though, which means that it\nwill search for test results in the wrong location if it was set.\n\nUpdate the script to handle TEST_OUTPUT_DIRECTORY so that we can start\nto set it in our CI.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n ci/print-test-failures.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex c33ad4e3a2..b1f80aeac3 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -8,7 +8,7 @@\n # Tracing executed commands would produce too much noise in the loop below.\n set +x\n \n-cd t/\n+cd \"${TEST_OUTPUT_DIRECTORY:-t/}\"\n \n if ! ls test-results/*.exit >/dev/null 2>/dev/null\n then\n-- \n2.43.GIT\n\n"},{"id":"486966","messageId":"1ed6e6865014b5f24aeadd14505b06a15ed20eb2.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"[PATCH v2 4/5] ci: make p4 setup on macOS more robust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:22:58Z","receivedAt":"2024-01-18T10:23:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When setting up Perforce on macOS we put both `p4` and `p4d` into\n\"$HOME/bin\". On GitHub CI this directory is indeed contained in the PATH\nenvironment variable and thus there is no need for additional setup than\nto put the binaries there. But GitLab CI does not do this, and thus our\nPerforce-based tests would be skipped there even though we download the\nbinaries.\n\nRefactor the setup code to become more robust by downloading binaries\ninto a separate directory which we then manually append to our PATH.\nThis matches what we do on Linux-based jobs.\n\nNote that it may seem like we already did append \"$HOME/bin\" to PATH\nbecause we're actually removing the lines that adapt PATH. But we only\never adapted the PATH variable in \"ci/install-dependencies.sh\", and\ndidn't adapt it when running \"ci/run-build-and-test.sh\". Consequently,\nthe required binaries wouldn't be found during the test run unless the\nCI platform already had the \"$HOME/bin\" in PATH right from the start.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n ci/install-dependencies.sh | 10 ++++------\n ci/lib.sh                  |  3 +++\n 2 files changed, 7 insertions(+), 6 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 4f407530d3..b4e22de3cb 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -37,15 +37,13 @@ macos-*)\n \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n \tbrew install $BREW_INSTALL_PACKAGES\n \tbrew link --force gettext\n-\tmkdir -p $HOME/bin\n-\t(\n-\t\tcd $HOME/bin\n+\n+\tmkdir -p \"$P4_PATH\"\n+\tpushd \"$P4_PATH\"\n \t\twget -q \"$P4WHENCE/bin.macosx1015x86_64/helix-core-server.tgz\" &&\n \t\ttar -xf helix-core-server.tgz &&\n \t\tsudo xattr -d com.apple.quarantine p4 p4d 2>/dev/null || true\n-\t)\n-\tPATH=\"$PATH:${HOME}/bin\"\n-\texport PATH\n+\tpopd\n \n \tif test -n \"$CC_PACKAGE\"\n \tthen\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex c749b21366..f631206a44 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -344,6 +344,9 @@ macos-*)\n \tthen\n \t\tMAKEFLAGS=\"$MAKEFLAGS APPLE_COMMON_CRYPTO_SHA1=Yes\"\n \tfi\n+\n+\tP4_PATH=\"$HOME/custom/p4\"\n+\texport PATH=\"$P4_PATH:$PATH\"\n \t;;\n esac\n \n-- \n2.43.GIT\n\n"},{"id":"486967","messageId":"c5ed38f0a6378297bde8a63bfe8ff8428cc4c966.1705573336.git.ps@pks.im","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"[PATCH v2 5/5] ci: add macOS jobs to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-18T10:23:02Z","receivedAt":"2024-01-18T10:23:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Add a job to GitLab CI which runs tests on macOS, which matches the\nequivalent \"osx-clang\" job that we have for GitHub Workflows. One\nsignificant difference though is that this new job runs on Apple M1\nmachines and thus uses the \"arm64\" architecture. As GCC does not yet\nsupport this comparatively new architecture we cannot easily include an\nequivalent for the \"osx-gcc\" job that exists in GitHub Workflows.\n\nNote that one test marked as `test_must_fail` is surprisingly passing:\n\n  t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n    TODO passed:   12\n\nThis seems to boil down to an unexpected difference in how regcomp(3P)\nworks when matching NUL bytes. Cross-checking with the respective GitHub\njob shows that this is not an issue unique to the GitLab CI job as it\npasses in the same way there.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n .gitlab-ci.yml | 34 +++++++++++++++++++++++++++++++++-\n ci/lib.sh      |  9 ++++++++-\n 2 files changed, 41 insertions(+), 2 deletions(-)\n\ndiff --git a/.gitlab-ci.yml b/.gitlab-ci.yml\nindex 793243421c..43bfbd8834 100644\n--- a/.gitlab-ci.yml\n+++ b/.gitlab-ci.yml\n@@ -7,7 +7,7 @@ workflow:\n     - if: $CI_COMMIT_TAG\n     - if: $CI_COMMIT_REF_PROTECTED == \"true\"\n \n-test:\n+test:linux:\n   image: $image\n   before_script:\n     - ./ci/install-docker-dependencies.sh\n@@ -52,6 +52,38 @@ test:\n       - t/failed-test-artifacts\n     when: on_failure\n \n+test:osx:\n+  image: $image\n+  tags:\n+    - saas-macos-medium-m1\n+  variables:\n+    TEST_OUTPUT_DIRECTORY: \"/Volumes/RAMDisk\"\n+  before_script:\n+    # Create a 4GB RAM disk that we use to store test output on. This small hack\n+    # significantly speeds up tests by more than a factor of 2 because the\n+    # macOS runners use network-attached storage as disks, which is _really_\n+    # slow with the many small writes that our tests do.\n+    - sudo diskutil apfs create $(hdiutil attach -nomount ram://8192000) RAMDisk\n+    - ./ci/install-dependencies.sh\n+  script:\n+    - ./ci/run-build-and-tests.sh\n+  after_script:\n+    - |\n+      if test \"$CI_JOB_STATUS\" != 'success'\n+      then\n+        ./ci/print-test-failures.sh\n+        mv \"$TEST_OUTPUT_DIRECTORY\"/failed-test-artifacts t/\n+      fi\n+  parallel:\n+    matrix:\n+      - jobname: osx-clang\n+        image: macos-13-xcode-14\n+        CC: clang\n+  artifacts:\n+    paths:\n+      - t/failed-test-artifacts\n+    when: on_failure\n+\n static-analysis:\n   image: ubuntu:22.04\n   variables:\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex f631206a44..d5dd2f2697 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -252,7 +252,14 @@ then\n \tCI_COMMIT=\"$CI_COMMIT_SHA\"\n \tcase \"$CI_JOB_IMAGE\" in\n \tmacos-*)\n-\t\tCI_OS_NAME=osx;;\n+\t\t# GitLab CI has Python installed via multiple package managers,\n+\t\t# most notably via asdf and Homebrew. Ensure that our builds\n+\t\t# pick up the Homebrew one by prepending it to our PATH as the\n+\t\t# asdf one breaks tests.\n+\t\texport PATH=\"$(brew --prefix)/bin:$PATH\"\n+\n+\t\tCI_OS_NAME=osx\n+\t\t;;\n \talpine:*|fedora:*|ubuntu:*)\n \t\tCI_OS_NAME=linux;;\n \t*)\n-- \n2.43.GIT\n\n"},{"id":"487178","messageId":"6e190a32-ee45-451b-b841-25cc6eb2c5ab@gmail.com","threadId":"60745","inReplyTo":"cover.1705573336.git.ps@pks.im","subject":"Re: [PATCH v2 0/5] ci: add support for macOS to GitLab CI","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2024-01-21T14:50:05Z","receivedAt":"2024-01-21T14:50:08Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Patrick\n\nOn 18/01/2024 10:22, Patrick Steinhardt wrote:\n> Hi,\n> \n> this is the second version of my patch series that adds a macOS job to\n> GitLab CI. Changes compared to v1:\n> \n>    - Added a fix for a flaky test in t7527 that caused the pipeline to\n>      fail in ~50% of all runs.\n> \n>    - Improved some commit messages.\n> \n>    - Tests now write test data into a RAMDisk. This speeds up tests and\n>      fixes some hung pipelines I was seeing.\n> \n> Thanks for your reviews so far!\n\nI've read though all the patches and they seem sensible to me though I'm \nhardly a macOS expert. I did wonder about the use of pushd/popd in the \nfourth patch as they are bashisms but that matches what we're doing on \nUbuntu already. It's nice to see the GitLab CI running on macOS as well \nas Linux now.\n\nBest Wishes\n\nPhillip\n\n> Patrick\n> \n> Patrick Steinhardt (5):\n>    t7527: decrease likelihood of racing with fsmonitor daemon\n>    Makefile: detect new Homebrew location for ARM-based Macs\n>    ci: handle TEST_OUTPUT_DIRECTORY when printing test failures\n>    ci: make p4 setup on macOS more robust\n>    ci: add macOS jobs to GitLab CI\n> \n>   .gitlab-ci.yml               | 34 +++++++++++++++++++++++++++++++++-\n>   ci/install-dependencies.sh   | 10 ++++------\n>   ci/lib.sh                    | 12 +++++++++++-\n>   ci/print-test-failures.sh    |  2 +-\n>   config.mak.uname             | 13 +++++++++++++\n>   t/t7527-builtin-fsmonitor.sh |  2 +-\n>   6 files changed, 63 insertions(+), 10 deletions(-)\n> \n> Range-diff against v1:\n> -:  ---------- > 1:  554b1c8546 t7527: decrease likelihood of racing with fsmonitor daemon\n> 2:  3adb0b7ae8 = 2:  32d8bd1d78 Makefile: detect new Homebrew location for ARM-based Macs\n> -:  ---------- > 3:  d55da77747 ci: handle TEST_OUTPUT_DIRECTORY when printing test failures\n> 1:  a5d725bea7 ! 4:  1ed6e68650 ci: make p4 setup on macOS more robust\n>      @@ Commit message\n>           into a separate directory which we then manually append to our PATH.\n>           This matches what we do on Linux-based jobs.\n>       \n>      +    Note that it may seem like we already did append \"$HOME/bin\" to PATH\n>      +    because we're actually removing the lines that adapt PATH. But we only\n>      +    ever adapted the PATH variable in \"ci/install-dependencies.sh\", and\n>      +    didn't adapt it when running \"ci/run-build-and-test.sh\". Consequently,\n>      +    the required binaries wouldn't be found during the test run unless the\n>      +    CI platform already had the \"$HOME/bin\" in PATH right from the start.\n>      +\n>           Signed-off-by: Patrick Steinhardt <ps@pks.im>\n>       \n>        ## ci/install-dependencies.sh ##\n> 3:  d196cfd9d0 ! 5:  c5ed38f0a6 ci: add macOS jobs to GitLab CI\n>      @@ Metadata\n>        ## Commit message ##\n>           ci: add macOS jobs to GitLab CI\n>       \n>      -    Add two macOS-based jobs to GitLab CI, one for Clang and one for GCC.\n>      -    This matches equivalent jobs we have for GitHub Workflows, except that\n>      -    we use macOS 14 instead of macOS 13.\n>      +    Add a job to GitLab CI which runs tests on macOS, which matches the\n>      +    equivalent \"osx-clang\" job that we have for GitHub Workflows. One\n>      +    significant difference though is that this new job runs on Apple M1\n>      +    machines and thus uses the \"arm64\" architecture. As GCC does not yet\n>      +    support this comparatively new architecture we cannot easily include an\n>      +    equivalent for the \"osx-gcc\" job that exists in GitHub Workflows.\n>       \n>           Note that one test marked as `test_must_fail` is surprisingly passing:\n>       \n>             t7815-grep-binary.sh                             (Wstat: 0 Tests: 22 Failed: 0)\n>               TODO passed:   12\n>       \n>      -    This seems to boil down to an unexpected difference in how regcomp(1)\n>      +    This seems to boil down to an unexpected difference in how regcomp(3P)\n>           works when matching NUL bytes. Cross-checking with the respective GitHub\n>      -    job shows though that this is not an issue unique to the GitLab CI job\n>      -    as it passes in the same way there.\n>      -\n>      -    Further note that we do not include the equivalent for the \"osx-gcc\" job\n>      -    that we use with GitHub Workflows. This is because the runner for macOS\n>      -    on GitLab is running on Apple M1 machines and thus uses the \"arm64\"\n>      -    architecture. GCC does not support this platform yet.\n>      +    job shows that this is not an issue unique to the GitLab CI job as it\n>      +    passes in the same way there.\n>       \n>           Signed-off-by: Patrick Steinhardt <ps@pks.im>\n>       \n>      @@ .gitlab-ci.yml: test:\n>       +  image: $image\n>       +  tags:\n>       +    - saas-macos-medium-m1\n>      ++  variables:\n>      ++    TEST_OUTPUT_DIRECTORY: \"/Volumes/RAMDisk\"\n>       +  before_script:\n>      ++    # Create a 4GB RAM disk that we use to store test output on. This small hack\n>      ++    # significantly speeds up tests by more than a factor of 2 because the\n>      ++    # macOS runners use network-attached storage as disks, which is _really_\n>      ++    # slow with the many small writes that our tests do.\n>      ++    - sudo diskutil apfs create $(hdiutil attach -nomount ram://8192000) RAMDisk\n>       +    - ./ci/install-dependencies.sh\n>       +  script:\n>       +    - ./ci/run-build-and-tests.sh\n>      @@ .gitlab-ci.yml: test:\n>       +      if test \"$CI_JOB_STATUS\" != 'success'\n>       +      then\n>       +        ./ci/print-test-failures.sh\n>      ++        mv \"$TEST_OUTPUT_DIRECTORY\"/failed-test-artifacts t/\n>       +      fi\n>       +  parallel:\n>       +    matrix:\n> \n> base-commit: cd69c635a1a62b0c8bfdbf221778be8a512ad048\n"},{"id":"487187","messageId":"Za4Hr_IIkgKVh4Go@tanuki","threadId":"60745","inReplyTo":"6e190a32-ee45-451b-b841-25cc6eb2c5ab@gmail.com","subject":"Re: [PATCH v2 0/5] ci: add support for macOS to GitLab CI","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-01-22T06:14:07Z","receivedAt":"2024-01-22T06:14:13Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sun, Jan 21, 2024 at 02:50:05PM +0000, Phillip Wood wrote:\n> Hi Patrick\n> \n> On 18/01/2024 10:22, Patrick Steinhardt wrote:\n> > Hi,\n> > \n> > this is the second version of my patch series that adds a macOS job to\n> > GitLab CI. Changes compared to v1:\n> > \n> >    - Added a fix for a flaky test in t7527 that caused the pipeline to\n> >      fail in ~50% of all runs.\n> > \n> >    - Improved some commit messages.\n> > \n> >    - Tests now write test data into a RAMDisk. This speeds up tests and\n> >      fixes some hung pipelines I was seeing.\n> > \n> > Thanks for your reviews so far!\n> \n> I've read though all the patches and they seem sensible to me though I'm\n> hardly a macOS expert. I did wonder about the use of pushd/popd in the\n> fourth patch as they are bashisms but that matches what we're doing on\n> Ubuntu already. It's nice to see the GitLab CI running on macOS as well as\n> Linux now.\n\nYeah, that part is a bit weird, agreed. As you say, I basically copied\nthe code that we use on Ubuntu, and that is intentional because another\nfollow-up patch series will rip out that part and move the shared code\ninto a common \"install-p4.sh\" script. Like that, we can also easily use\nthis script on the Docker-based Ubuntu jobs.\n\nThanks for your review!\n\nPatrick\n"},{"id":"487206","messageId":"xmqq1qa9e5jz.fsf@gitster.g","threadId":"60745","inReplyTo":"6e190a32-ee45-451b-b841-25cc6eb2c5ab@gmail.com","subject":"Re: [PATCH v2 0/5] ci: add support for macOS to GitLab CI","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-01-22T15:44:32Z","receivedAt":"2024-01-22T15:44:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood <phillip.wood123@gmail.com> writes:\n\n> Hi Patrick\n> ...\n> I've read though all the patches and they seem sensible to me though\n> I'm hardly a macOS expert. I did wonder about the use of pushd/popd in\n> the fourth patch as they are bashisms but that matches what we're\n> doing on Ubuntu already. It's nice to see the GitLab CI running on\n> macOS as well as Linux now.\n\nThanks, both.\n\n"}]}