git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 2/2] ci: use twice the CPU count on both providers

From
Tamir Duberstein <tamird@gmail.com>
Date
Sep 30, 2026, 14:20 UTC
Message-ID
<20260930-ci-large-test-resources-v3-2-d65ac7c21b5f@gmail.com>
In-Reply-To
<20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com>

GitHub Actions sets JOBS to ten regardless of runner size, while GitLab CI uses the detected CPU count. Use twice the CPU count for Make and prove on both providers, doubling GitLab's job count.

Five GitHub Actions attempts per policy, with the long tests enabled, gave these sums of per-job median successful build/test-step times (minutes; four or five samples per job) [1-3]:

                      Fixed 10   CPU count   2x CPU count
  Linux Make             278.9       273.9          259.9
  macOS Make              94.5       119.1           99.8
  Windows Make           102.2       103.8          100.8
Workflow overhead is excluded; Windows runner images varied.

Use twice the CPU count to scale concurrency with runner size while avoiding the larger macOS slowdown observed with one job per CPU. Compared with ten jobs, this trades a lower Linux total for a higher macOS total.

Keep GitLab's Linux and Windows CPU queries. On macOS, use the native sysctl command on both providers so CPU detection does not depend on GNU coreutils.

Link: https://github.com/tamird/git/actions/runs/36070869894/attempts/1 [1]
Link: https://github.com/tamird/git/actions/runs/36070867524/attempts/1 [2]
Link: https://github.com/tamird/git/actions/runs/36070867647/attempts/1 [3]
Assisted-by: LLM
Signed-off-by: Tamir Duberstein <tamird@gmail.com>
---
 ci/lib.sh | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/ci/lib.sh b/ci/lib.sh
index c6ccbf8c17..f0b7f35850 100755
--- a/ci/lib.sh
+++ b/ci/lib.sh
@@ -227,7 +227,6 @@ then
 	cache_dir="$HOME/none"
 
 	GIT_TEST_OPTS="--github-workflow-markup"
-	JOBS=10
 
 	distro=$(echo "$CI_JOB_IMAGE" | tr : -)
 elif test true = "$GITLAB_CI"
@@ -250,7 +249,6 @@ then
 	case "$OS,$CI_JOB_IMAGE" in
 	Windows_NT,*)
 		CI_OS_NAME=windows
-		JOBS=$NUMBER_OF_PROCESSORS
 		;;
 	*,macos-*)
 		# GitLab CI has Python installed via multiple package managers,
@@ -260,11 +258,9 @@ then
 		export PATH="$(brew --prefix)/bin:$PATH"
 
 		CI_OS_NAME=osx
-		JOBS=$(nproc)
 		;;
 	*,almalinux:*|*,alpine:*|*,debian:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*)
 		CI_OS_NAME=linux
-		JOBS=$(nproc)
 		;;
 	*)
 		echo "Could not identify OS image" >&2
@@ -291,6 +287,19 @@ else
 	exit 1
 fi
 
+case "$CI_OS_NAME" in
+windows|windows_nt)
+	JOBS=$NUMBER_OF_PROCESSORS
+	;;
+osx)
+	JOBS=$(sysctl -n hw.logicalcpu)
+	;;
+*)
+	JOBS=$(nproc)
+	;;
+esac
+JOBS=$((2 * JOBS))
+
 MAKEFLAGS="$MAKEFLAGS --jobs=$JOBS"
 GIT_PROVE_OPTS="--timer --jobs $JOBS"
 
-- 
2.56.0.rc2.851.g50a151a6e9.frankengit
Previous: Tamir DubersteinNext: Patrick Steinhardt
Message 21 of 25 in “ci: reduce pressure from large test fixtures”
  1. 0/2 ci: reduce pressure from large test fixturesTamir Duberstein, Sep 23, 2026
  2. 1/2 t4205: compare huge output without diffTamir Duberstein, Sep 23, 2026
  3. Patrick SteinhardtSep 24, 2026
  4. Tamir DubersteinSep 24, 2026
  5. Jeff KingSep 25, 2026
  6. Junio C HamanoSep 25, 2026
  7. Patrick SteinhardtSep 28, 2026
  8. 2/2 ci: match Linux jobs to available CPUsTamir Duberstein, Sep 23, 2026
  9. Patrick SteinhardtSep 24, 2026
  10. Tamir DubersteinSep 25, 2026
  11. Tamir DubersteinSep 23, 2026
  12. 0/2 ci: use cmp and align job-count selectionTamir Duberstein, Sep 25, 2026
  13. 1/2 t4205: compare huge output without diffTamir Duberstein, Sep 25, 2026
  14. Patrick SteinhardtSep 28, 2026
  15. 2/2 ci: align job counts across CI providersTamir Duberstein, Sep 25, 2026
  16. Patrick SteinhardtSep 28, 2026
  17. Tamir DubersteinSep 28, 2026
  18. Patrick SteinhardtSep 28, 2026
  19. 0/2 ci: use cmp and align job-count selectionTamir Duberstein, Sep 30, 2026
  20. 1/2 t4205: compare huge output without diffTamir Duberstein, Sep 30, 2026
  21. 2/2 ci: use twice the CPU count on both providersTamir Duberstein, Sep 30, 2026
  22. Patrick SteinhardtSep 30, 2026
  23. Patrick SteinhardtSep 30, 2026
  24. Tamir DubersteinSep 30, 2026
  25. Junio C HamanoSep 30, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.