[RFH] Why do osx CI jobs so unreliable?
2 messages between Jun 19, 2026 and Jun 19, 2026, from Junio C Hamano, Patrick Steinhardt.
Plain Markdown or JSON for tools and agents.
I've been observing that in recent push-out to 'master' and 'next', osx-* jobs in GitHub Actions CI keep running for 6 hours and get killed.
What is troubling is that this seems to be very flaky. For example, https://github.com/git/git/actions/runs/27778820659 is testing 95e20213 (Hopefully final batch before -rc2, 2026-06-17) which got killed after wasting 6 hours in osx-clang and osx-gcc jobs.
https://github.com/git/git/actions/runs/27790036076 is testing the same 'master', with a patch to .github/workflows/main.yml to remove everything except for config and osx-* jobs, which succeeded within 30 minutes.
Stumped...
Re: [RFH] Why do osx CI jobs so unreliable?
On Thu, Jun 18, 2026 at 05:35:23PM -0700, Junio C Hamano wrote:
Show 15 quoted lines
> I've been observing that in recent push-out to 'master' and 'next',
> osx-* jobs in GitHub Actions CI keep running for 6 hours and get
> killed.
>
> What is troubling is that this seems to be very flaky. For example,
> https://github.com/git/git/actions/runs/27778820659 is testing
> 95e20213 (Hopefully final batch before -rc2, 2026-06-17) which got
> killed after wasting 6 hours in osx-clang and osx-gcc jobs.
>
> https://github.com/git/git/actions/runs/27790036076 is testing
> the same 'master', with a patch to .github/workflows/main.yml to
> remove everything except for config and osx-* jobs, which succeeded
> within 30 minutes.
>
> Stumped...
So the raw logs have the following trailer:
2026-06-18T23:53:33.2996180Z Cleaning up orphan processes
2026-06-18T23:53:33.7900380Z Terminate orphan process: pid (34022) (git-remote-http)
2026-06-18T23:53:33.9848670Z Terminate orphan process: pid (15488) (httpd)
2026-06-18T23:53:34.0321490Z Terminate orphan process: pid (13146) (httpd)
2026-06-18T23:53:34.0808280Z Terminate orphan process: pid (13145) (httpd)
2026-06-18T23:53:34.1212760Z Terminate orphan process: pid (13144) (httpd)
2026-06-18T23:53:34.1570160Z Terminate orphan process: pid (13141) (httpd)
2026-06-18T23:53:34.1924140Z Terminate orphan process: pid (12553) (bash)
2026-06-18T23:53:34.2472970Z Terminate orphan process: pid (12552) (tee)
2026-06-18T23:53:34.6547890Z Terminate orphan process: pid (21209) (bash)
So I strongly suspect that it most be one of the t555* tests. Furthermore, the t5551 and t5559 (both of which are actually the same test) are the only test suites that use lib-httpd.sh and which are missing in the job logs.
I have not been able to reproduce this hang on my macOS virtual machine though, and on GitLab I didn't notice a similar hang recently. Maybe this is something that's specific to GitHub's environment...? No idea.
Patrick