{"thread":{"id":"66378","subject":"[PATCH 0/2] ci: reduce pressure from large test fixtures","startedAt":"2026-09-23T17:13:45Z","lastAt":"2026-09-30T18:04:07Z","messageCount":25,"participants":["Tamir Duberstein","Patrick Steinhardt","Jeff King","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"553078","messageId":"20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com","threadId":"66378","inReplyTo":null,"subject":"[PATCH 0/2] ci: reduce pressure from large test fixtures","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-23T17:13:27Z","receivedAt":"2026-09-23T17:13:45Z","isPatch":true,"body":"Linux pull-request jobs with the long tests enabled hit two resource\nproblems: diff was killed after git log produced its huge output, and\nlarge clone and repack tests hit ENOSPC.\n\nThe first patch compares the huge output with cmp and removes the large\nfiles after success. The second sizes Make and prove parallelism to the\nLinux runner's available CPUs, following the existing GitLab policy.\nBoth changes keep the large test cases enabled.\n\nPrepared with Codex, including review by a separate Codex agent.\n\nSigned-off-by: Tamir Duberstein \n---\nTamir Duberstein (2):\n t4205: compare huge output without diff\n ci: match Linux jobs to available CPUs\n\n ci/lib.sh | 4 ++++\n t/t4205-log-pretty-formats.sh | 3 ++-\n 2 files changed, 6 insertions(+), 1 deletion(-)\n\n\n---\nbase-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7\nchange-id: 20260923-ci-large-test-resources-349cdc95f7f8\n\n"},{"id":"553079","messageId":"20260923-ci-large-test-resources-v1-1-c28416d59475@gmail.com","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com","subject":"[PATCH 1/2] t4205: compare huge output without diff","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-23T17:13:28Z","receivedAt":"2026-09-23T17:13:46Z","isPatch":true,"body":"The huge-commit test compares two files with a line larger than 2 GiB.\nIn Linux GitHub Actions jobs, git log produces its huge output but\nits subsequent diff process is killed with SIGKILL.\n\nUse test_cmp_bin to compare the output byte for byte without constructing\na line-oriented diff. Remove the two large files after a successful\ncomparison, releasing more than 4 GiB before subsequent tests.\n\nSigned-off-by: Tamir Duberstein \n---\n t/t4205-log-pretty-formats.sh | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 4be5c51489..6279a7e9bc 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1189,7 +1189,8 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n \techo 0 >>expect &&\n-\ttest_cmp expect actual\n+\ttest_cmp_bin expect actual &&\n+\trm expect actual\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '\n\n-- \n2.56.0.rc0.807.ga0c0929ce1.frankengit\n\n"},{"id":"553080","messageId":"20260923-ci-large-test-resources-v1-2-c28416d59475@gmail.com","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com","subject":"[PATCH 2/2] ci: match Linux jobs to available CPUs","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-23T17:13:29Z","receivedAt":"2026-09-23T17:13:47Z","isPatch":true,"body":"GitHub Actions runs ten Make test suites concurrently even on private\nLinux runners with two CPUs. Pull request runs enable the long tests.\nThese runs hit ENOSPC while multiple multi-gigabyte clone and repack\nfixtures were active.\n\nUse nproc to choose Make and prove parallelism, as the GitLab CI path\nalready does. This reduces overlapping fixtures on small Linux runners\nwhile keeping the long tests enabled.\n\nSigned-off-by: Tamir Duberstein \n---\n ci/lib.sh | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex c6ccbf8c17..0855026dad 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -228,6 +228,10 @@ then\n \n \tGIT_TEST_OPTS=\"--github-workflow-markup\"\n \tJOBS=10\n+\tif test linux = \"$CI_OS_NAME\"\n+\tthen\n+\t\tJOBS=$(nproc)\n+\tfi\n \n \tdistro=$(echo \"$CI_JOB_IMAGE\" | tr : -)\n elif test true = \"$GITLAB_CI\"\n\n-- \n2.56.0.rc0.807.ga0c0929ce1.frankengit\n\n"},{"id":"553094","messageId":"CAJ-ks9kuQSfxnb8v9pVOkdzkydTjhJchXzxk17SpMF7SJiwcyQ@mail.gmail.com","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com","subject":"Re: [PATCH 0/2] ci: reduce pressure from large test fixtures","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-23T18:36:53Z","receivedAt":"2026-09-23T18:37:33Z","isPatch":true,"body":"On Wed, Sep 23, 2026 at 1:13 PM Tamir Duberstein wrote:\n>\n> Prepared with Codex, including review by a separate Codex agent.\n\nApologies for this inclusion, won't happen again.\n"},{"id":"553143","messageId":"arS_l1hPIr7I2Gn-@pks.im","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-1-c28416d59475@gmail.com","subject":"Re: [PATCH 1/2] t4205: compare huge output without diff","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-24T06:13:43Z","receivedAt":"2026-09-24T06:13:47Z","isPatch":true,"body":"On Wed, Sep 23, 2026 at 01:13:28PM -0400, Tamir Duberstein wrote:\n> The huge-commit test compares two files with a line larger than 2 GiB.\n> In Linux GitHub Actions jobs, git log produces its huge output but\n> its subsequent diff process is killed with SIGKILL.\n\nI've never seen that failure before. Do you maybe have a link to it?\n\n> Use test_cmp_bin to compare the output byte for byte without constructing\n> a line-oriented diff. Remove the two large files after a successful\n> comparison, releasing more than 4 GiB before subsequent tests.\n\nIt would be great to back up the claim that test_cmp_bin is better than\ntest_cmp, e.g. by comparing peak RSS and its runtime.\n\n> Signed-off-by: Tamir Duberstein \n> ---\n> t/t4205-log-pretty-formats.sh | 3 ++-\n> 1 file changed, 2 insertions(+), 1 deletion(-)\n> \n> diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> index 4be5c51489..6279a7e9bc 100755\n> --- a/t/t4205-log-pretty-formats.sh\n> +++ b/t/t4205-log-pretty-formats.sh\n> @@ -1189,7 +1189,8 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n> test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n> \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n> \techo 0 >>expect &&\n> -\ttest_cmp expect actual\n> +\ttest_cmp_bin expect actual &&\n> +\trm expect actual\n> '\n\nHm. Sure, releasing these files isn't a bad idea by itself. But we\nrewrite \"expect\" in the next test anyway, and \"actual\" will be rewritten\ntwo tests further down. So does it really buy us that much...?\n\nPatrick\n"},{"id":"553144","messageId":"arS_nA3g-on2RdIL@pks.im","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-2-c28416d59475@gmail.com","subject":"Re: [PATCH 2/2] ci: match Linux jobs to available CPUs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-24T06:13:48Z","receivedAt":"2026-09-24T06:13:53Z","isPatch":true,"body":"On Wed, Sep 23, 2026 at 01:13:29PM -0400, Tamir Duberstein wrote:\n> GitHub Actions runs ten Make test suites concurrently even on private\n> Linux runners with two CPUs. Pull request runs enable the long tests.\n> These runs hit ENOSPC while multiple multi-gigabyte clone and repack\n> fixtures were active.\n\nAgain, a link would be appreciated that demonstrates this.\n\n> Use nproc to choose Make and prove parallelism, as the GitLab CI path\n> already does. This reduces overlapping fixtures on small Linux runners\n> while keeping the long tests enabled.\n\nIt may avoid overlapping fixtures. But what does CI runtime look like\nbefore and after this change? Does it improve? Does it regress? Would it\nmaybe make sense to oversubscribe at least a bit?\n\n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index c6ccbf8c17..0855026dad 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -228,6 +228,10 @@ then\n> \n> \tGIT_TEST_OPTS=\"--github-workflow-markup\"\n> \tJOBS=10\n> +\tif test linux = \"$CI_OS_NAME\"\n> +\tthen\n> +\t\tJOBS=$(nproc)\n> +\tfi\n\nMakes me wonder whether we should have the same logic on both GitLab and\nGitHub going forward. There probably isn't a good reason why these two\nshould differ from one another.\n\nPatrick\n"},{"id":"553240","messageId":"CAJ-ks9kJWc0e7aEX4vAL-RoJ5kVvfjDABqV86_hZf2Fn-085GA@mail.gmail.com","threadId":"66378","inReplyTo":"arS_l1hPIr7I2Gn-@pks.im","subject":"Re: [PATCH 1/2] t4205: compare huge output without diff","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-24T20:41:05Z","receivedAt":"2026-09-24T20:41:44Z","isPatch":true,"body":"On Thu, Sep 24, 2026 at 2:13 AM Patrick Steinhardt wrote:\n>\n> On Wed, Sep 23, 2026 at 01:13:28PM -0400, Tamir Duberstein wrote:\n> > The huge-commit test compares two files with a line larger than 2 GiB.\n> > In Linux GitHub Actions jobs, git log produces its huge output but\n> > its subsequent diff process is killed with SIGKILL.\n>\n> I've never seen that failure before. Do you maybe have a link to it?\n\nThe failures happened on a private repo that I've since lost access to\n- but I believe it was precipitated by GitHub runners having half the\nmemory in private repos as in public ones [1].\n\n> > Use test_cmp_bin to compare the output byte for byte without constructing\n> > a line-oriented diff. Remove the two large files after a successful\n> > comparison, releasing more than 4 GiB before subsequent tests.\n>\n> It would be great to back up the claim that test_cmp_bin is better than\n> test_cmp, e.g. by comparing peak RSS and its runtime.\n\nAs for the comparison: on Linux arm64 with GNU\ndiffutils 3.8 using two identical files containing 2,147,483,649 \"1\" bytes\nfollowed by \"0\\n\" (matching this test's expected output) gave:\n\nCommand Mean +/- stddev Maximum RSS (KiB)\ndiff -u expect actual 5.276 +/- 0.572 s 4199924\ncmp expect actual 0.506 +/- 0.099 s 1264\n\n>\n> > Signed-off-by: Tamir Duberstein \n> > ---\n> > t/t4205-log-pretty-formats.sh | 3 ++-\n> > 1 file changed, 2 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> > index 4be5c51489..6279a7e9bc 100755\n> > --- a/t/t4205-log-pretty-formats.sh\n> > +++ b/t/t4205-log-pretty-formats.sh\n> > @@ -1189,7 +1189,8 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n> > test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n> > git log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n> > echo 0 >>expect &&\n> > - test_cmp expect actual\n> > + test_cmp_bin expect actual &&\n> > + rm expect actual\n> > '\n>\n> Hm. Sure, releasing these files isn't a bad idea by itself. But we\n> rewrite \"expect\" in the next test anyway, and \"actual\" will be rewritten\n> two tests further down. So does it really buy us that much...?\n\nYou're right, this probably does not buy much.\n\nWould you like me to include the performance comparison in v2? As for\nthe deletion: would you prefer I drop it?\n\nLink: https://docs.github.com/en/actions/reference/runners/github-hosted-runners#standard-github-hosted-runners-for--private-repositories\n[1]\n"},{"id":"553254","messageId":"20260925001332.GA1408107@coredump.intra.peff.net","threadId":"66378","inReplyTo":"CAJ-ks9kJWc0e7aEX4vAL-RoJ5kVvfjDABqV86_hZf2Fn-085GA@mail.gmail.com","subject":"Re: [PATCH 1/2] t4205: compare huge output without diff","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-09-25T00:13:32Z","receivedAt":"2026-09-25T00:13:34Z","isPatch":true,"body":"On Thu, Sep 24, 2026 at 04:41:05PM -0400, Tamir Duberstein wrote:\n\n> > It would be great to back up the claim that test_cmp_bin is better than\n> > test_cmp, e.g. by comparing peak RSS and its runtime.\n> \n> As for the comparison: on Linux arm64 with GNU\n> diffutils 3.8 using two identical files containing 2,147,483,649 \"1\" bytes\n> followed by \"0\\n\" (matching this test's expected output) gave:\n> \n> Command Mean +/- stddev Maximum RSS (KiB)\n> diff -u expect actual 5.276 +/- 0.572 s 4199924\n> cmp expect actual 0.506 +/- 0.099 s 1264\n\nYeah, that's a big difference. This is probably an outlier because of\nthe giant files, but I've wondered if test_cmp() ought to be doing\nsomething like:\n\n # first check quickly if there is any difference\n cmp \"$@\" && return 0\n # if not, then we can spend time to produce a useful output\n diff \"$@\"\n\nBut I never pursued it because I figured most of the things we compare\nare not big enough to matter (and really the comparison itself is\ndwarfed by the process startup time).\n\nSo I am happy just marking this particular case as cmp_bin, too.\n\n-Peff\n"},{"id":"553263","messageId":"xmqq7bk9ygqz.fsf@gitster.g","threadId":"66378","inReplyTo":"20260925001332.GA1408107@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] t4205: compare huge output without diff","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-09-25T06:41:56Z","receivedAt":"2026-09-25T06:41:59Z","isPatch":true,"body":"Jeff King writes:\n\n> # first check quickly if there is any difference\n> cmp \"$@\" && return 0\n> # if not, then we can spend time to produce a useful output\n> diff \"$@\"\n>\n> But I never pursued it because I figured most of the things we compare\n> are not big enough to matter (and really the comparison itself is\n> dwarfed by the process startup time).\n\nYup, that matches my intuition.\n\n> So I am happy just marking this particular case as cmp_bin, too.\n\nMe too.\n\nThanks.\n"},{"id":"553285","messageId":"CAJ-ks9nQF2E9Pp-mXq8VAo=m0ZUi4Am46ysSH9zU8cPK4V-Geg@mail.gmail.com","threadId":"66378","inReplyTo":"arS_nA3g-on2RdIL@pks.im","subject":"Re: [PATCH 2/2] ci: match Linux jobs to available CPUs","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-25T15:03:20Z","receivedAt":"2026-09-25T15:04:02Z","isPatch":true,"body":"On Thu, Sep 24, 2026 at 2:13 AM Patrick Steinhardt wrote:\n>\n> On Wed, Sep 23, 2026 at 01:13:29PM -0400, Tamir Duberstein wrote:\n> > GitHub Actions runs ten Make test suites concurrently even on private\n> > Linux runners with two CPUs. Pull request runs enable the long tests.\n> > These runs hit ENOSPC while multiple multi-gigabyte clone and repack\n> > fixtures were active.\n>\n> Again, a link would be appreciated that demonstrates this.\n\nYeah, sorry about that. Again, I suspect GitHub runners on private\nrepos (2 CPUs) make this worse than public repos (4 CPUs).\n\n> > Use nproc to choose Make and prove parallelism, as the GitLab CI path\n> > already does. This reduces overlapping fixtures on small Linux runners\n> > while keeping the long tests enabled.\n>\n> It may avoid overlapping fixtures. But what does CI runtime look like\n> before and after this change? Does it improve? Does it regress? Would it\n> maybe make sense to oversubscribe at least a bit?\n\nI ran a bunch of tests of this in GitHub and the disappointing answer\nis that it's not clear what the right choice is:\n\n Fixed 10 CPU count 2x CPU count\nLinux Make 278.9 273.9 259.9\nmacOS Make 94.5 119.1 99.8\nWindows Make 102.2 103.8 100.8\n\n>\n> > diff --git a/ci/lib.sh b/ci/lib.sh\n> > index c6ccbf8c17..0855026dad 100755\n> > --- a/ci/lib.sh\n> > +++ b/ci/lib.sh\n> > @@ -228,6 +228,10 @@ then\n> >\n> > GIT_TEST_OPTS=\"--github-workflow-markup\"\n> > JOBS=10\n> > + if test linux = \"$CI_OS_NAME\"\n> > + then\n> > + JOBS=$(nproc)\n> > + fi\n>\n> Makes me wonder whether we should have the same logic on both GitLab and\n> GitHub going forward. There probably isn't a good reason why these two\n> should differ from one another.\n\nAgreed. I have rewritten this patch to use the same logic across CI\nproviders (1x CPU count) in v2. I'll leave tuning (e.g. moving to 2x)\nto a future change.\n\n>\n> Patrick\n"},{"id":"553296","messageId":"20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com","threadId":"66378","inReplyTo":"20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com","subject":"[PATCH v2 0/2] ci: use cmp and align job-count selection","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-25T16:35:37Z","receivedAt":"2026-09-25T16:35:59Z","isPatch":true,"body":"The first patch uses cmp for the huge commit-message output comparison.\nOn this input, GNU diffutils 3.8 on Linux arm64 takes 5.276 seconds with\n4,199,924 KiB peak RSS for diff, versus 0.506 seconds and 1,264 KiB for\ncmp. Patch 1 includes the fixture and measurement details.\n\nThe second patch aligns GitHub Actions' Make and prove job counts with\nGitLab CI's CPU-count policy, replacing GitHub's fixed ten jobs.\n\nSigned-off-by: Tamir Duberstein \n---\nChanges in v2:\n- Replace the unavailable CI failure reference with comparison runtime\n and peak RSS measurements.\n- Drop the file removal; following tests overwrite expect and actual.\n- Share job-count selection between GitHub Actions and GitLab CI.\n- Use native CPU-count queries on macOS and Windows.\n- Link to v1: https://patch.msgid.link/20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com\n\n---\nTamir Duberstein (2):\n t4205: compare huge output without diff\n ci: align job counts across CI providers\n\n ci/lib.sh | 16 ++++++++++++----\n t/t4205-log-pretty-formats.sh | 2 +-\n 2 files changed, 13 insertions(+), 5 deletions(-)\n\nRange-diff versus v1:\n\n1: 5babc36eb5 ! 1: b48c86e164 t4205: compare huge output without diff\n @@ Metadata\n ## Commit message ##\n t4205: compare huge output without diff\n \n - The huge-commit test compares two files with a line larger than 2 GiB.\n - In Linux GitHub Actions jobs, git log produces its huge output but\n - its subsequent diff process is killed with SIGKILL.\n + The huge-commit test compares output containing a line larger than 2 GiB.\n + For two identical files containing 2,147,483,649 \"1\" bytes followed by\n + \"0\\n\", GNU diffutils 3.8 on Linux arm64 gives these measurements:\n \n - Use test_cmp_bin to compare the output byte for byte without constructing\n - a line-oriented diff. Remove the two large files after a successful\n - comparison, releasing more than 4 GiB before subsequent tests.\n + Command Mean +/- stddev Maximum RSS (KiB)\n + diff -u expect actual 5.276 +/- 0.572 s 4199924\n + cmp expect actual 0.506 +/- 0.099 s 1264\n \n + The test needs only an equality check. Use test_cmp_bin, which runs cmp,\n + to compare the output byte for byte with less time and memory.\n +\n + Assisted-by: LLM\n Signed-off-by: Tamir Duberstein \n \n ## t/t4205-log-pretty-formats.sh ##\n @@ t/t4205-log-pretty-formats.sh: test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'se\n \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n \techo 0 >>expect &&\n -\ttest_cmp expect actual\n -+\ttest_cmp_bin expect actual &&\n -+\trm expect actual\n ++\ttest_cmp_bin expect actual\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '\n2: 0836f6b372 < -: ---------- ci: match Linux jobs to available CPUs\n-: ---------- > 2: 8ec0308f6a ci: align job counts across CI providers\n\n---\nbase-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7\nchange-id: 20260923-ci-large-test-resources-349cdc95f7f8\n\n"},{"id":"553297","messageId":"20260925-ci-large-test-resources-v2-1-f632cf319756@gmail.com","threadId":"66378","inReplyTo":"20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com","subject":"[PATCH v2 1/2] t4205: compare huge output without diff","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-25T16:35:38Z","receivedAt":"2026-09-25T16:36:01Z","isPatch":true,"body":"The huge-commit test compares output containing a line larger than 2 GiB.\nFor two identical files containing 2,147,483,649 \"1\" bytes followed by\n\"0\\n\", GNU diffutils 3.8 on Linux arm64 gives these measurements:\n\n Command Mean +/- stddev Maximum RSS (KiB)\n diff -u expect actual 5.276 +/- 0.572 s 4199924\n cmp expect actual 0.506 +/- 0.099 s 1264\n\nThe test needs only an equality check. Use test_cmp_bin, which runs cmp,\nto compare the output byte for byte with less time and memory.\n\nAssisted-by: LLM\nSigned-off-by: Tamir Duberstein \n---\n t/t4205-log-pretty-formats.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 4be5c51489..01b97c8888 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1189,7 +1189,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n \techo 0 >>expect &&\n-\ttest_cmp expect actual\n+\ttest_cmp_bin expect actual\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '\n\n-- \n2.56.0.rc2.815.g5b995412e4.frankengit\n\n"},{"id":"553298","messageId":"20260925-ci-large-test-resources-v2-2-f632cf319756@gmail.com","threadId":"66378","inReplyTo":"20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com","subject":"[PATCH v2 2/2] ci: align job counts across CI providers","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-25T16:35:39Z","receivedAt":"2026-09-25T16:36:03Z","isPatch":true,"body":"GitHub Actions sets JOBS to ten regardless of runner size, while\nGitLab CI uses the detected CPU count. Use the CPU count for Make and\nprove on both providers, selecting JOBS after the operating system\nis identified.\n\nUse nproc on Linux and NUMBER_OF_PROCESSORS on Windows. On macOS, use\nsysctl to avoid requiring nproc before the dependency installer has run;\nGitHub macOS images need not provide GNU coreutils.\n\nAssisted-by: LLM\nSigned-off-by: Tamir Duberstein \n---\n ci/lib.sh | 16 ++++++++++++----\n 1 file changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex c6ccbf8c17..db593cc62c 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -227,7 +227,6 @@ then\n \tcache_dir=\"$HOME/none\"\n \n \tGIT_TEST_OPTS=\"--github-workflow-markup\"\n-\tJOBS=10\n \n \tdistro=$(echo \"$CI_JOB_IMAGE\" | tr : -)\n elif test true = \"$GITLAB_CI\"\n@@ -250,7 +249,6 @@ then\n \tcase \"$OS,$CI_JOB_IMAGE\" in\n \tWindows_NT,*)\n \t\tCI_OS_NAME=windows\n-\t\tJOBS=$NUMBER_OF_PROCESSORS\n \t\t;;\n \t*,macos-*)\n \t\t# GitLab CI has Python installed via multiple package managers,\n@@ -260,11 +258,9 @@ then\n \t\texport PATH=\"$(brew --prefix)/bin:$PATH\"\n \n \t\tCI_OS_NAME=osx\n-\t\tJOBS=$(nproc)\n \t\t;;\n \t*,almalinux:*|*,alpine:*|*,debian:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*)\n \t\tCI_OS_NAME=linux\n-\t\tJOBS=$(nproc)\n \t\t;;\n \t*)\n \t\techo \"Could not identify OS image\" >&2\n@@ -291,6 +287,18 @@ else\n \texit 1\n fi\n \n+case \"$CI_OS_NAME\" in\n+windows|windows_nt)\n+\tJOBS=$NUMBER_OF_PROCESSORS\n+\t;;\n+osx)\n+\tJOBS=$(sysctl -n hw.logicalcpu)\n+\t;;\n+*)\n+\tJOBS=$(nproc)\n+\t;;\n+esac\n+\n MAKEFLAGS=\"$MAKEFLAGS --jobs=$JOBS\"\n GIT_PROVE_OPTS=\"--timer --jobs $JOBS\"\n \n\n-- \n2.56.0.rc2.815.g5b995412e4.frankengit\n\n"},{"id":"553401","messageId":"aroK4d5GZ6VoFUim@pks.im","threadId":"66378","inReplyTo":"CAJ-ks9kJWc0e7aEX4vAL-RoJ5kVvfjDABqV86_hZf2Fn-085GA@mail.gmail.com","subject":"Re: [PATCH 1/2] t4205: compare huge output without diff","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-28T06:36:17Z","receivedAt":"2026-09-28T06:36:23Z","isPatch":true,"body":"On Thu, Sep 24, 2026 at 04:41:05PM -0400, Tamir Duberstein wrote:\n> On Thu, Sep 24, 2026 at 2:13 AM Patrick Steinhardt wrote:\n> >\n> > On Wed, Sep 23, 2026 at 01:13:28PM -0400, Tamir Duberstein wrote:\n> > > The huge-commit test compares two files with a line larger than 2 GiB.\n> > > In Linux GitHub Actions jobs, git log produces its huge output but\n> > > its subsequent diff process is killed with SIGKILL.\n> >\n> > I've never seen that failure before. Do you maybe have a link to it?\n> \n> The failures happened on a private repo that I've since lost access to\n> - but I believe it was precipitated by GitHub runners having half the\n> memory in private repos as in public ones [1].\n\nOkay.\n\n> > > Use test_cmp_bin to compare the output byte for byte without constructing\n> > > a line-oriented diff. Remove the two large files after a successful\n> > > comparison, releasing more than 4 GiB before subsequent tests.\n> >\n> > It would be great to back up the claim that test_cmp_bin is better than\n> > test_cmp, e.g. by comparing peak RSS and its runtime.\n> \n> As for the comparison: on Linux arm64 with GNU\n> diffutils 3.8 using two identical files containing 2,147,483,649 \"1\" bytes\n> followed by \"0\\n\" (matching this test's expected output) gave:\n> \n> Command Mean +/- stddev Maximum RSS (KiB)\n> diff -u expect actual 5.276 +/- 0.572 s 4199924\n> cmp expect actual 0.506 +/- 0.099 s 1264\n\nQuite a significant win indeed.\n\n> > > Signed-off-by: Tamir Duberstein \n> > > ---\n> > > t/t4205-log-pretty-formats.sh | 3 ++-\n> > > 1 file changed, 2 insertions(+), 1 deletion(-)\n> > >\n> > > diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> > > index 4be5c51489..6279a7e9bc 100755\n> > > --- a/t/t4205-log-pretty-formats.sh\n> > > +++ b/t/t4205-log-pretty-formats.sh\n> > > @@ -1189,7 +1189,8 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n> > > test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n> > > git log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n> > > echo 0 >>expect &&\n> > > - test_cmp expect actual\n> > > + test_cmp_bin expect actual &&\n> > > + rm expect actual\n> > > '\n> >\n> > Hm. Sure, releasing these files isn't a bad idea by itself. But we\n> > rewrite \"expect\" in the next test anyway, and \"actual\" will be rewritten\n> > two tests further down. So does it really buy us that much...?\n> \n> You're right, this probably does not buy much.\n> \n> Would you like me to include the performance comparison in v2?\n\nI think that'd be good, yes. Providing context like this to the reviewer\nmakes everyone's life easier :)\n\n> As for the deletion: would you prefer I drop it?\n\nMy personal take is that we can just drop it as it doesn't buy us much.\nIf we want to keep it we should be honest about its effect in the commit\nmessage.\n\nPatrick\n"},{"id":"553402","messageId":"aroK5js5U2t0IUIc@pks.im","threadId":"66378","inReplyTo":"20260925-ci-large-test-resources-v2-1-f632cf319756@gmail.com","subject":"Re: [PATCH v2 1/2] t4205: compare huge output without diff","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-28T06:36:22Z","receivedAt":"2026-09-28T06:36:28Z","isPatch":true,"body":"On Fri, Sep 25, 2026 at 12:35:38PM -0400, Tamir Duberstein wrote:\n> The huge-commit test compares output containing a line larger than 2 GiB.\n> For two identical files containing 2,147,483,649 \"1\" bytes followed by\n> \"0\\n\", GNU diffutils 3.8 on Linux arm64 gives these measurements:\n> \n> Command Mean +/- stddev Maximum RSS (KiB)\n> diff -u expect actual 5.276 +/- 0.572 s 4199924\n> cmp expect actual 0.506 +/- 0.099 s 1264\n> \n> The test needs only an equality check. Use test_cmp_bin, which runs cmp,\n> to compare the output byte for byte with less time and memory.\n\nYup, this is much more compelling as an argument now :)\n\n> diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> index 4be5c51489..01b97c8888 100755\n> --- a/t/t4205-log-pretty-formats.sh\n> +++ b/t/t4205-log-pretty-formats.sh\n> @@ -1189,7 +1189,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n> test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n> \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n> \techo 0 >>expect &&\n> -\ttest_cmp expect actual\n> +\ttest_cmp_bin expect actual\n> '\n\nAnd the patch looks obviously good to me.\n\nPatrick\n"},{"id":"553403","messageId":"aroK7KcRabARSqd8@pks.im","threadId":"66378","inReplyTo":"20260925-ci-large-test-resources-v2-2-f632cf319756@gmail.com","subject":"Re: [PATCH v2 2/2] ci: align job counts across CI providers","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-28T06:36:28Z","receivedAt":"2026-09-28T06:36:33Z","isPatch":true,"body":"On Fri, Sep 25, 2026 at 12:35:39PM -0400, Tamir Duberstein wrote:\n> GitHub Actions sets JOBS to ten regardless of runner size, while\n> GitLab CI uses the detected CPU count. Use the CPU count for Make and\n> prove on both providers, selecting JOBS after the operating system\n> is identified.\n\nAgain, it should be noted here what the effect of this is. In other\nwords, does GitHub slow down as a result? You already showed numbers\nduring the discussion on v1 of this series, and these numbers should\nprobably be included in this message, too.\n\n> Use nproc on Linux and NUMBER_OF_PROCESSORS on Windows. On macOS, use\n> sysctl to avoid requiring nproc before the dependency installer has run;\n> GitHub macOS images need not provide GNU coreutils.\n\nHuh... \"need not\" feels somewhat weird as phrasing. I guess it's rather\n\"does not\", and consequently we have to adapt? I think instead of\ndescribing what you do, I'd directly pinpoint what matters:\n\n Note that we continue to use the same logic to detect the number of\n processors on both Linux and Windows. But on macOS, we cannot continue\n to use nproc(1) because the image used by GitHub does not provide that\n tool. Use sysctl instead, which is available on both GitLab and\n GitHub.\n\nThanks!\n\nPatrick\n"},{"id":"553438","messageId":"CAJ-ks9mWU2cFWLfioSSM_6Ct1ydtxn3i4Dh-L537M5EDiMW3vQ@mail.gmail.com","threadId":"66378","inReplyTo":"aroK7KcRabARSqd8@pks.im","subject":"Re: [PATCH v2 2/2] ci: align job counts across CI providers","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-28T10:16:33Z","receivedAt":"2026-09-28T10:17:11Z","isPatch":true,"body":"On Mon, Sep 28, 2026 at 2:36 AM Patrick Steinhardt wrote:\n>\n> On Fri, Sep 25, 2026 at 12:35:39PM -0400, Tamir Duberstein wrote:\n> > GitHub Actions sets JOBS to ten regardless of runner size, while\n> > GitLab CI uses the detected CPU count. Use the CPU count for Make and\n> > prove on both providers, selecting JOBS after the operating system\n> > is identified.\n>\n> Again, it should be noted here what the effect of this is. In other\n> words, does GitHub slow down as a result? You already showed numbers\n> during the discussion on v1 of this series, and these numbers should\n> probably be included in this message, too.\n\nAgreed, but in this case there was no reliable performance change\nacross 10 runs; I could include that.\n\n>\n> > Use nproc on Linux and NUMBER_OF_PROCESSORS on Windows. On macOS, use\n> > sysctl to avoid requiring nproc before the dependency installer has run;\n> > GitHub macOS images need not provide GNU coreutils.\n>\n> Huh... \"need not\" feels somewhat weird as phrasing. I guess it's rather\n> \"does not\", and consequently we have to adapt? I think instead of\n> describing what you do, I'd directly pinpoint what matters:\n>\n> Note that we continue to use the same logic to detect the number of\n> processors on both Linux and Windows. But on macOS, we cannot continue\n> to use nproc(1) because the image used by GitHub does not provide that\n> tool. Use sysctl instead, which is available on both GitLab and\n> GitHub\n\nAgreed.\n.\n>\n> Thanks!\n>\n> Patrick\n"},{"id":"553444","messageId":"arpMJ4EBomeSF6LS@pks.im","threadId":"66378","inReplyTo":"CAJ-ks9mWU2cFWLfioSSM_6Ct1ydtxn3i4Dh-L537M5EDiMW3vQ@mail.gmail.com","subject":"Re: [PATCH v2 2/2] ci: align job counts across CI providers","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-28T11:14:47Z","receivedAt":"2026-09-28T11:14:54Z","isPatch":true,"body":"On Mon, Sep 28, 2026 at 06:16:33AM -0400, Tamir Duberstein wrote:\n> On Mon, Sep 28, 2026 at 2:36 AM Patrick Steinhardt wrote:\n> >\n> > On Fri, Sep 25, 2026 at 12:35:39PM -0400, Tamir Duberstein wrote:\n> > > GitHub Actions sets JOBS to ten regardless of runner size, while\n> > > GitLab CI uses the detected CPU count. Use the CPU count for Make and\n> > > prove on both providers, selecting JOBS after the operating system\n> > > is identified.\n> >\n> > Again, it should be noted here what the effect of this is. In other\n> > words, does GitHub slow down as a result? You already showed numbers\n> > during the discussion on v1 of this series, and these numbers should\n> > probably be included in this message, too.\n> \n> Agreed, but in this case there was no reliable performance change\n> across 10 runs; I could include that.\n\nI think it should be included, as it's the one thing that people will be\nwondering about when they see this change.\n\nPatrick\n"},{"id":"553703","messageId":"20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com","threadId":"66378","inReplyTo":"20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com","subject":"[PATCH v3 0/2] ci: use cmp and align job-count selection","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-30T14:20:33Z","receivedAt":"2026-09-30T14:20:53Z","isPatch":true,"body":"The first patch uses cmp for the huge commit-message output comparison.\nOn this input, GNU diffutils 3.8 on Linux arm64 takes 5.276 seconds with\n4,199,924 KiB peak RSS for diff, versus 0.506 seconds and 1,264 KiB for\ncmp. Patch 1 includes the fixture and measurement details.\n\nThe second patch uses twice the detected CPU count for Make and prove\non both GitHub Actions and GitLab CI. This replaces GitHub's fixed ten\njobs and doubles GitLab's job count. The GitHub measurements in patch 2\nfavor 2*N over N, with mixed results against ten jobs. GitLab runtime and\nresource use have not been measured.\n\nSigned-off-by: Tamir Duberstein \n---\nThe table in patch 2 uses attempts 1-5 of each linked run. Its rows\naggregate ten Linux jobs, three macOS jobs, and a Windows build plus ten\ntest shards. Failed steps are excluded. Each job has five samples,\nexcept one CPU-count Windows shard and one 2x-CPU Linux job with four.\n\nFive further attempts per policy for osx-clang on three-CPU macOS\nrunners gave these successful build/test-step medians (minutes:seconds):\n\n Jobs 3 6 10\n Median 48:10 36:44 38:05.5\n Passed 5 5 4\n Cancelled 0 0 1\n\nSix jobs had lower times than three in each block. The ten-job\ncancellation followed six hours in the build/test step; its cause is\nunknown. It is excluded from the successful-duration median above.\nThese are attempts 6-10 of the runs linked in patch 2, separate from its\nearlier full CI matrix measurements. Neither experiment measured peak\nmemory or disk use.\n\nChanges in v3:\n- Use twice the CPU count on both providers, instead of adopting\n GitLab's existing one-job-per-CPU policy.\n- Include CI timings and their tradeoffs in patch 2's commit message,\n and explain the use of sysctl directly.\n- Patch 1 is unchanged.\n- Link to v2: https://patch.msgid.link/20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com\n\nChanges in v2:\n- Replace the unavailable CI failure reference with comparison runtime\n and peak RSS measurements.\n- Drop the file removal; following tests overwrite expect and actual.\n- Share job-count selection between GitHub Actions and GitLab CI.\n- Use native CPU-count queries on macOS and Windows.\n- Link to v1: https://patch.msgid.link/20260923-ci-large-test-resources-v1-0-c28416d59475@gmail.com\n\n---\nTamir Duberstein (2):\n t4205: compare huge output without diff\n ci: use twice the CPU count on both providers\n\n ci/lib.sh | 17 +++++++++++++----\n t/t4205-log-pretty-formats.sh | 2 +-\n 2 files changed, 14 insertions(+), 5 deletions(-)\n\n---\nRange-diff versus v2:\n\n1: 5cf348a75c = 1: 4a15b8f17d t4205: compare huge output without diff\n2: 3b9bd2c495 ! 2: d444411070 ci: align job counts across CI providers\n @@ Metadata\n Author: Tamir Duberstein \n \n ## Commit message ##\n - ci: align job counts across CI providers\n + ci: use twice the CPU count on both providers\n \n GitHub Actions sets JOBS to ten regardless of runner size, while\n - GitLab CI uses the detected CPU count. Use the CPU count for Make and\n - prove on both providers, selecting JOBS after the operating system\n - is identified.\n + GitLab CI uses the detected CPU count. Use twice the CPU count for\n + Make and prove on both providers, doubling GitLab's job count.\n \n - Use nproc on Linux and NUMBER_OF_PROCESSORS on Windows. On macOS, use\n - sysctl to avoid requiring nproc before the dependency installer has run;\n - GitHub macOS images need not provide GNU coreutils.\n + Five GitHub Actions attempts per policy, with the long tests enabled,\n + gave these sums of per-job median successful build/test-step times\n + (minutes; four or five samples per job) [1-3]:\n +\n + Fixed 10 CPU count 2x CPU count\n + Linux Make 278.9 273.9 259.9\n + macOS Make 94.5 119.1 99.8\n + Windows Make 102.2 103.8 100.8\n +\n + Workflow overhead is excluded; Windows runner images varied.\n +\n + Use twice the CPU count to scale concurrency with runner size while\n + avoiding the larger macOS slowdown observed with one job per CPU.\n + Compared with ten jobs, this trades a lower Linux total for a higher\n + macOS total.\n +\n + Keep GitLab's Linux and Windows CPU queries. On macOS, use the native\n + sysctl command on both providers so CPU detection does not depend on\n + GNU coreutils.\n +\n + Link: https://github.com/tamird/git/actions/runs/36070869894/attempts/1 [1]\n + Link: https://github.com/tamird/git/actions/runs/36070867524/attempts/1 [2]\n + Link: https://github.com/tamird/git/actions/runs/36070867647/attempts/1 [3]\n \n Assisted-by: LLM\n Signed-off-by: Tamir Duberstein \n @@ ci/lib.sh: else\n +\tJOBS=$(nproc)\n +\t;;\n +esac\n ++JOBS=$((2 * JOBS))\n +\n MAKEFLAGS=\"$MAKEFLAGS --jobs=$JOBS\"\n GIT_PROVE_OPTS=\"--timer --jobs $JOBS\"\n\n---\nbase-commit: 3bc0341126508f78f5869cbfc0005e987efdf0c7\nchange-id: 20260923-ci-large-test-resources-349cdc95f7f8\n\n"},{"id":"553705","messageId":"20260930-ci-large-test-resources-v3-1-d65ac7c21b5f@gmail.com","threadId":"66378","inReplyTo":"20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com","subject":"[PATCH v3 1/2] t4205: compare huge output without diff","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-30T14:20:34Z","receivedAt":"2026-09-30T14:20:55Z","isPatch":true,"body":"The huge-commit test compares output containing a line larger than 2 GiB.\nFor two identical files containing 2,147,483,649 \"1\" bytes followed by\n\"0\\n\", GNU diffutils 3.8 on Linux arm64 gives these measurements:\n\n Command Mean +/- stddev Maximum RSS (KiB)\n diff -u expect actual 5.276 +/- 0.572 s 4199924\n cmp expect actual 0.506 +/- 0.099 s 1264\n\nThe test needs only an equality check. Use test_cmp_bin, which runs cmp,\nto compare the output byte for byte with less time and memory.\n\nAssisted-by: LLM\nSigned-off-by: Tamir Duberstein \n---\n t/t4205-log-pretty-formats.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 4be5c51489..01b97c8888 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -1189,7 +1189,7 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'set up huge commit' '\n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message' '\n \tgit log -1 --format=\"%B%<(1)%x30\" $huge_commit >actual &&\n \techo 0 >>expect &&\n-\ttest_cmp expect actual\n+\ttest_cmp_bin expect actual\n '\n \n test_expect_success EXPENSIVE,SIZE_T_IS_64BIT 'log --pretty with huge commit message does not cause allocation failure' '\n\n-- \n2.56.0.rc2.851.g50a151a6e9.frankengit\n\n"},{"id":"553704","messageId":"20260930-ci-large-test-resources-v3-2-d65ac7c21b5f@gmail.com","threadId":"66378","inReplyTo":"20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com","subject":"[PATCH v3 2/2] ci: use twice the CPU count on both providers","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-30T14:20:35Z","receivedAt":"2026-09-30T14:20:58Z","isPatch":true,"body":"GitHub Actions sets JOBS to ten regardless of runner size, while\nGitLab CI uses the detected CPU count. Use twice the CPU count for\nMake and prove on both providers, doubling GitLab's job count.\n\nFive GitHub Actions attempts per policy, with the long tests enabled,\ngave these sums of per-job median successful build/test-step times\n(minutes; four or five samples per job) [1-3]:\n\n Fixed 10 CPU count 2x CPU count\n Linux Make 278.9 273.9 259.9\n macOS Make 94.5 119.1 99.8\n Windows Make 102.2 103.8 100.8\n\nWorkflow overhead is excluded; Windows runner images varied.\n\nUse twice the CPU count to scale concurrency with runner size while\navoiding the larger macOS slowdown observed with one job per CPU.\nCompared with ten jobs, this trades a lower Linux total for a higher\nmacOS total.\n\nKeep GitLab's Linux and Windows CPU queries. On macOS, use the native\nsysctl command on both providers so CPU detection does not depend on\nGNU coreutils.\n\nLink: https://github.com/tamird/git/actions/runs/36070869894/attempts/1 [1]\nLink: https://github.com/tamird/git/actions/runs/36070867524/attempts/1 [2]\nLink: https://github.com/tamird/git/actions/runs/36070867647/attempts/1 [3]\n\nAssisted-by: LLM\nSigned-off-by: Tamir Duberstein \n---\n ci/lib.sh | 17 +++++++++++++----\n 1 file changed, 13 insertions(+), 4 deletions(-)\n\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex c6ccbf8c17..f0b7f35850 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -227,7 +227,6 @@ then\n \tcache_dir=\"$HOME/none\"\n \n \tGIT_TEST_OPTS=\"--github-workflow-markup\"\n-\tJOBS=10\n \n \tdistro=$(echo \"$CI_JOB_IMAGE\" | tr : -)\n elif test true = \"$GITLAB_CI\"\n@@ -250,7 +249,6 @@ then\n \tcase \"$OS,$CI_JOB_IMAGE\" in\n \tWindows_NT,*)\n \t\tCI_OS_NAME=windows\n-\t\tJOBS=$NUMBER_OF_PROCESSORS\n \t\t;;\n \t*,macos-*)\n \t\t# GitLab CI has Python installed via multiple package managers,\n@@ -260,11 +258,9 @@ then\n \t\texport PATH=\"$(brew --prefix)/bin:$PATH\"\n \n \t\tCI_OS_NAME=osx\n-\t\tJOBS=$(nproc)\n \t\t;;\n \t*,almalinux:*|*,alpine:*|*,debian:*|*,fedora:*|*,ubuntu:*|*,i386/ubuntu:*)\n \t\tCI_OS_NAME=linux\n-\t\tJOBS=$(nproc)\n \t\t;;\n \t*)\n \t\techo \"Could not identify OS image\" >&2\n@@ -291,6 +287,19 @@ else\n \texit 1\n fi\n \n+case \"$CI_OS_NAME\" in\n+windows|windows_nt)\n+\tJOBS=$NUMBER_OF_PROCESSORS\n+\t;;\n+osx)\n+\tJOBS=$(sysctl -n hw.logicalcpu)\n+\t;;\n+*)\n+\tJOBS=$(nproc)\n+\t;;\n+esac\n+JOBS=$((2 * JOBS))\n+\n MAKEFLAGS=\"$MAKEFLAGS --jobs=$JOBS\"\n GIT_PROVE_OPTS=\"--timer --jobs $JOBS\"\n \n\n-- \n2.56.0.rc2.851.g50a151a6e9.frankengit\n\n"},{"id":"553710","messageId":"ar0gnpgH7CJqN_w2@pks.im","threadId":"66378","inReplyTo":"20260930-ci-large-test-resources-v3-2-d65ac7c21b5f@gmail.com","subject":"Re: [PATCH v3 2/2] ci: use twice the CPU count on both providers","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-30T14:45:50Z","receivedAt":"2026-09-30T14:46:02Z","isPatch":true,"body":"On Wed, Sep 30, 2026 at 10:20:35AM -0400, Tamir Duberstein wrote:\n> GitHub Actions sets JOBS to ten regardless of runner size, while\n> GitLab CI uses the detected CPU count. Use twice the CPU count for\n> Make and prove on both providers, doubling GitLab's job count.\n> \n> Five GitHub Actions attempts per policy, with the long tests enabled,\n> gave these sums of per-job median successful build/test-step times\n> (minutes; four or five samples per job) [1-3]:\n> \n> Fixed 10 CPU count 2x CPU count\n> Linux Make 278.9 273.9 259.9\n> macOS Make 94.5 119.1 99.8\n> Windows Make 102.2 103.8 100.8\n> \n> Workflow overhead is excluded; Windows runner images varied.\n> \n> Use twice the CPU count to scale concurrency with runner size while\n> avoiding the larger macOS slowdown observed with one job per CPU.\n> Compared with ten jobs, this trades a lower Linux total for a higher\n> macOS total.\n\nRight. We could of course special-case macOS. But I don't feel like it\nmakes sense to squeeze every single second out of a job that's already\nthe fastest anyway.\n\nPatrick\n"},{"id":"553711","messageId":"ar0gon2VE0RlG_cC@pks.im","threadId":"66378","inReplyTo":"20260930-ci-large-test-resources-v3-0-d65ac7c21b5f@gmail.com","subject":"Re: [PATCH v3 0/2] ci: use cmp and align job-count selection","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-09-30T14:45:54Z","receivedAt":"2026-09-30T14:46:03Z","isPatch":true,"body":"On Wed, Sep 30, 2026 at 10:20:33AM -0400, Tamir Duberstein wrote:\n> Changes in v3:\n> - Use twice the CPU count on both providers, instead of adopting\n> GitLab's existing one-job-per-CPU policy.\n> - Include CI timings and their tradeoffs in patch 2's commit message,\n> and explain the use of sysctl directly.\n> - Patch 1 is unchanged.\n> - Link to v2: https://patch.msgid.link/20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com\n\nThanks, I'm happy with this version.\n\nPatrick\n"},{"id":"553714","messageId":"CAJ-ks9kYC3NM7BY=gYKxVL54nOkH4wp9TdAJye_8ifNPa7oVdg@mail.gmail.com","threadId":"66378","inReplyTo":"ar0gon2VE0RlG_cC@pks.im","subject":"Re: [PATCH v3 0/2] ci: use cmp and align job-count selection","fromName":"Tamir Duberstein","fromEmail":"tamird@gmail.com","sentAt":"2026-09-30T14:58:22Z","receivedAt":"2026-09-30T14:59:02Z","isPatch":true,"body":"On Wed, Sep 30, 2026 at 10:45 AM Patrick Steinhardt wrote:\n>\n> On Wed, Sep 30, 2026 at 10:20:33AM -0400, Tamir Duberstein wrote:\n> > Changes in v3:\n> > - Use twice the CPU count on both providers, instead of adopting\n> > GitLab's existing one-job-per-CPU policy.\n> > - Include CI timings and their tradeoffs in patch 2's commit message,\n> > and explain the use of sysctl directly.\n> > - Patch 1 is unchanged.\n> > - Link to v2: https://patch.msgid.link/20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com\n>\n> Thanks, I'm happy with this version.\n>\n> Patrick\n\nThanks for the reviews!\n"},{"id":"553735","messageId":"xmqq4if6boq3.fsf@gitster.g","threadId":"66378","inReplyTo":"CAJ-ks9kYC3NM7BY=gYKxVL54nOkH4wp9TdAJye_8ifNPa7oVdg@mail.gmail.com","subject":"Re: [PATCH v3 0/2] ci: use cmp and align job-count selection","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-09-30T18:04:04Z","receivedAt":"2026-09-30T18:04:07Z","isPatch":true,"body":"Tamir Duberstein writes:\n\n> On Wed, Sep 30, 2026 at 10:45 AM Patrick Steinhardt wrote:\n>>\n>> On Wed, Sep 30, 2026 at 10:20:33AM -0400, Tamir Duberstein wrote:\n>> > Changes in v3:\n>> > - Use twice the CPU count on both providers, instead of adopting\n>> > GitLab's existing one-job-per-CPU policy.\n>> > - Include CI timings and their tradeoffs in patch 2's commit message,\n>> > and explain the use of sysctl directly.\n>> > - Patch 1 is unchanged.\n>> > - Link to v2: https://patch.msgid.link/20260925-ci-large-test-resources-v2-0-f632cf319756@gmail.com\n>>\n>> Thanks, I'm happy with this version.\n>>\n>> Patrick\n>\n> Thanks for the reviews!\n\nThanks for writing and reviewing these patches, both of you.\n\nLet me mark it for 'next'.\n"}]}