From: Patrick Steinhardt Date: Mon, 09 Nov 2026 09:58:21 GMT Subject: [PATCH v2 0/7] ci: some speedups for slow tests Message-ID: <20261109-pks-gitlab-ci-speedups-v2-0-508832602d0b@pks.im> In-Reply-To: <20261106-pks-gitlab-ci-speedups-v1-0-1fa0374c7287@pks.im> Hi, I was staring at CI way more often recently, and that got me annoyed real fast because it's just painstakingly slow. So I decided to have a look and try do something about it, the result of which is this patch series. The series contains a couple of bits that are specific to GitLab, and a couple of fixes that are generic. Some highlights: - Installing the Git for Windows SDK speeds up from 3 minutes to 10 seconds by using tar(1) instead of Extract-Archive. - t0027 speeds up from >10 minutes to 6 minutes on Windows by avoiding the overhead of spawning redundant processes. - t5551 and t5559 both speed up by a factor of 10 on macOS, going from 20 minutes each to 2 minutes each by using reftables in a repo with 100k refs. "osx-meson" and "osx-clang" thus go from 29 minutes to 18 minutes end-to-end. - "fedora-breaking-changes-meson" speeds up by a factor of 2, going from 35 minutes to 16 minutes. Turns out our SHA256 implementation is quite slow with "-O0". Unfortunately, our Windows runners at GitLab are still atrociously slow by themselves. So the critical path only improves by about ~10 minutes, going from 60-70 minutes to 53 minutes end-to-end. But hey, at least the other jobs now give feedback a bit faster. And I guess on GitHub it may be a different story because Windows runners are way better there and are typically not on the critical path. A test run can be found at [1]. Changes in v2: - Drop the cache for the GfW SDK again. - Drop repetitive paths in t0027. - Link to v1: https://patch.msgid.link/20261106-pks-gitlab-ci-speedups-v1-0-1fa0374c7287@pks.im Thanks! Patrick [1]: https://gitlab.com/gitlab-org/git/-/merge_requests/734 --- Patrick Steinhardt (7): gitlab-ci: speed up installation of Git for Windows SDK gitlab-ci: exclude intermediate outputs from Windows artifacts ci: build optimized binaries with Meson t0027: optimize how we reset repository state t0027: cache expected output files t1092: set up repositories from templates t5551: use reftables for repositories with many refs .gitlab-ci.yml | 4 +++- ci/install-sdk.ps1 | 6 +++++- ci/run-build-and-tests.sh | 1 + t/t0027-auto-crlf.sh | 26 +++++++++++++----------- t/t1092-sparse-checkout-compatibility.sh | 34 +++++++++++++++++++------------- t/t5551-http-fetch-smart.sh | 6 ++++-- 6 files changed, 47 insertions(+), 30 deletions(-) Range-diff versus v1: 1: e25adb1b8c8 < -: ----------- gitlab-ci: speed up installation of Git for Windows SDK -: ----------- > 1: 582a9ab6468 gitlab-ci: speed up installation of Git for Windows SDK 2: 116c8b8a716 = 2: bff0a624127 gitlab-ci: exclude intermediate outputs from Windows artifacts 3: a035b8a902c = 3: 713e69bfb59 ci: build optimized binaries with Meson 4: 1a1238f4be7 < -: ----------- t0027: optimize how we reset repository state -: ----------- > 4: e421eb2a275 t0027: optimize how we reset repository state 5: c652f5b380a = 5: 16905b5be62 t0027: cache expected output files 6: 9a90272c620 = 6: 43f355fbb02 t1092: set up repositories from templates 7: 187128e7d5d = 7: 960716da178 t5551: use reftables for repositories with many refs --- base-commit: 0ffa3e35c7c82ab8d0de226149fb03b3f7674d05 change-id: 20261106-pks-gitlab-ci-speedups-09c3841eb178