The Windows builds in GitLab CI use Chocolatey to install dependencies. Unfortunately, Chocolatey seems to be very unreliable, which causes the jobs to fail very regularly. This is a limitation that seems to be somewhat known [1]:
As an organization, you want 100% reliability (or at least that potential), and you may want full trust and control as well. This is something you can get with internally hosted packages, and you are unlikely to achieve from use of the Community Package Repository.
So using the Community Package Repository is kind of discouraged in case one wants reliability. We _do_ want reliability though, and we cannot easily switch to an enterprise license to fix this issue.
Introduce a new script that downloads and installs dependencies directly. This has a couple of benefits:
- We can drop our dependency on Chocolatey completely, thus improving
reliability.- We can easily cache the installers.
- We get direct control over the exact versions we install.
- Installing dependencies is sped up from roundabout 3 minutes to 1
minute.[1]: https://docs.chocolatey.org/en-us/community-repository/community-packages-disclaimer/#summary
Signed-off-by: Patrick Steinhardt <ps@pks.im> --- Hi
I've been quite annoyed recently because our Windows builds in GitLab CI are extremely flakey. All of those flakes come from Chocolatey, which is why this patch moves away from it.
Thanks!
Patrick --- .gitlab-ci.yml | 11 ++++++--- ci/install-dependencies.ps1 | 55 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 3 deletions(-)
Show changes to 2 files +63 −3
.gitlab-ci.yml, ci/install-dependencies.ps1
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e0b9a0d82b..87a5343a94 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -161,11 +161,16 @@ test:mingw64: TEST_OUTPUT_DIRECTORY: "C:/Git-Test" tags: - saas-windows-medium-amd64 + cache: + key: + files: + - ci/install-dependencies.ps1 + paths: + - .dependencies before_script: - *windows_before_script - - choco install -y git meson ninja rust-ms - - Import-Module $env:ChocolateyInstall\helpers\chocolateyProfile.psm1 - - refreshenv + - ./ci/install-dependencies.ps1 + - $env:Path = "C:\Meson;C:\Rust\bin;$env:Path" - New-Item -Path $env:TEST_OUTPUT_DIRECTORY -ItemType Directory build:msvc-meson: diff --git a/ci/install-dependencies.ps1 b/ci/install-dependencies.ps1 new file mode 100755 index 0000000000..e3b367fa54 --- /dev/null +++ b/ci/install-dependencies.ps1 @@ -0,0 +1,55 @@ +param( + [string]$DownloadDirectory = '.dependencies' +) + +$ErrorActionPreference = 'Stop' +$ProgressPreference = 'SilentlyContinue' + +$GitVersion = '2.54.0.windows.1' +$MesonVersion = '1.11.0' +$RustVersion = '1.96.0' + +New-Item -Path $DownloadDirectory -ItemType Directory -Force | Out-Null +New-Item -Path .git/info -ItemType Directory -Force | Out-Null +New-Item -Path .git/info/exclude -ItemType File -Force | Out-Null +Add-Content -Path .git/info/exclude -Value "/$DownloadDirectory" + +function Get-Installer { + param( + [Parameter(Mandatory = $true)][string]$Name, + [Parameter(Mandatory = $true)][string]$Url + ) + + $path = Join-Path $DownloadDirectory $Name + if (-not (Test-Path $path)) { + Write-Host "Downloading $Url" + Invoke-WebRequest $Url -OutFile $path -TimeoutSec 300 + } + return $path +} + +function Invoke-Installer { + param( + [Parameter(Mandatory = $true)][string]$FilePath, + [Parameter(Mandatory = $true)][string[]]$ArgumentList + ) + + Write-Host "Running $FilePath $($ArgumentList -join ' ')" + $process = Start-Process -Wait -PassThru -FilePath $FilePath -ArgumentList $ArgumentList + if ($process.ExitCode -ne 0) { + throw "$FilePath failed with exit code $($process.ExitCode)" + } +} + +$gitAssetVersion = $GitVersion -replace '\.windows\.\d+$', '' +$gitInstaller = Get-Installer "Git-Installer.exe" ` + "https://github.com/git-for-windows/git/releases/download/v$GitVersion/PortableGit-$gitAssetVersion-64-bit.7z.exe" +Invoke-Installer $gitInstaller @('-y', '-o"C:\Program Files\Git"') + +$mesonMsi = Get-Installer "meson.msi" ` + "https://github.com/mesonbuild/meson/releases/download/$MesonVersion/meson-$MesonVersion-64.msi" +Invoke-Installer msiexec.exe @('/i', $mesonMsi, 'INSTALLDIR=C:\Meson', '/quiet', '/norestart') + +$rustMsi = Get-Installer "rust.msi" ` + "https://static.rust-lang.org/dist/rust-$RustVersion-x86_64-pc-windows-msvc.msi" +Invoke-Installer msiexec.exe @('/i', $rustMsi, 'INSTALLDIR=C:\Rust', 'ADDLOCAL=Rustc,Cargo,Std', '/quiet', '/norestart') --- base-commit: ea97ad8d017de0c9037451a78008a0fd60abea0c change-id: 20260615-b4-pks-gitlab-ci-drop-chocolatey-bfe9d4bb1442