From: Tamir Duberstein Date: Fri, 25 Sep 2026 15:03:20 GMT Subject: Re: [PATCH 2/2] ci: match Linux jobs to available CPUs Message-ID: In-Reply-To: On Thu, Sep 24, 2026 at 2:13 AM Patrick Steinhardt wrote: > > On Wed, Sep 23, 2026 at 01:13:29PM -0400, Tamir Duberstein wrote: > > GitHub Actions runs ten Make test suites concurrently even on private > > Linux runners with two CPUs. Pull request runs enable the long tests. > > These runs hit ENOSPC while multiple multi-gigabyte clone and repack > > fixtures were active. > > Again, a link would be appreciated that demonstrates this. Yeah, sorry about that. Again, I suspect GitHub runners on private repos (2 CPUs) make this worse than public repos (4 CPUs). > > Use nproc to choose Make and prove parallelism, as the GitLab CI path > > already does. This reduces overlapping fixtures on small Linux runners > > while keeping the long tests enabled. > > It may avoid overlapping fixtures. But what does CI runtime look like > before and after this change? Does it improve? Does it regress? Would it > maybe make sense to oversubscribe at least a bit? I ran a bunch of tests of this in GitHub and the disappointing answer is that it's not clear what the right choice is: Fixed 10 CPU count 2x CPU count Linux Make 278.9 273.9 259.9 macOS Make 94.5 119.1 99.8 Windows Make 102.2 103.8 100.8 > > > diff --git a/ci/lib.sh b/ci/lib.sh > > index c6ccbf8c17..0855026dad 100755 > > --- a/ci/lib.sh > > +++ b/ci/lib.sh > > @@ -228,6 +228,10 @@ then > > > > GIT_TEST_OPTS="--github-workflow-markup" > > JOBS=10 > > + if test linux = "$CI_OS_NAME" > > + then > > + JOBS=$(nproc) > > + fi > > Makes me wonder whether we should have the same logic on both GitLab and > GitHub going forward. There probably isn't a good reason why these two > should differ from one another. Agreed. I have rewritten this patch to use the same logic across CI providers (1x CPU count) in v2. I'll leave tuning (e.g. moving to 2x) to a future change. > > Patrick