threads / discuss / 62220

git commit hangs when run concurrently on macos

Subject: git commit hangs when run concurrently on macos

## tl;dr

2 messages between Sep 30, 2024 and Oct 1, 2024.

replies: 1people: 2as markdown or json

Bennet Huber· Sep 30, 2024, 20:58 UTC · lore

Thank you for filling out a Git bug report! Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

Ran several "git commit" commands in parallel on separate repos. Here's a simple test that does that.

```bash
dir-foreach-parallel () {
    subdirs=("${(@f)$(find ./ -maxdepth 1 -mindepth 1 -type d)}")
    for d in $subdirs
    do
        echo "$d"
        pushd "$d" >/dev/null
        $@ &
        popd >/dev/null
    done
}

mkdir -p /tmp/gitTest cd /tmp/gitTest for ((i=0;i<100;i++)) do mkdir git_$i; done

runTest() {
    git init
    USE_FS_MONITOR=true
    echo -e "[user]\n\tname = Gopher\n\temail =
gopher@example.com\n[core]\n\tfsmonitor = $USE_FS_MONITOR\n"
>.git/config
    GIT_TRACE=1 git commit --allow-empty -m `pwd` &>gitci.log
}

dir-foreach-parallel runTest ```

After running this, wait for output to subside, then run `jobs`. On my machine, I usually see 5-10 hung "git commit" jobs. Running `ps -efww | grep "git commit"` will show the repos they belong to, which you can then use to examine the corresponding gitci.log files. The last line of all the files I examined for hung "git commit" commands is:

> 3:39:04.020533 run-command.c:758       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git fsmonitor--daemon run --detach --ipc-threads=8

Setting `USE_FS_MONITOR=false` in the code above consistently results in no hung "git commit" commands, which leads me to suspect the problem has to do with the fsmonitor feature. It's also a decent, though difficult to discover, workaround.

What did you expect to happen? (Expected behavior) Concurrent git commands in separate repos shouldn't interfere with each other.

What happened instead? (Actual behavior) Several "git commit" commands hung and failed to complete due to fsmonitor not starting.

What's different between what you expected and what actually happened? Success

Anything else you want to add:
* I tried to repro this on a linux box and failed. It seems to have
something to do with the macos
fsmonitor implementation.
  * `uname -a` of linux box that succeeds: Linux <hostname>
6.8.0-1016-aws #17-Ubuntu SMP Mon Sep  2 13:48:07 UTC 2024 x86_64
x86_64 x86_64 GNU/Linux
* I'm running an ARM Apple M3 Pro chip, so maybe ARM specific?
* The only mac I have access to is a corporate one, which has a ton of
security monitoring software running on it in the background.  It's
conceivable this could be interfering with fsmonitor somehow.  I
unfortunately can't test this on a clean mac.
* This use case may seem contrived, but I actually lost a lot of time
because of it.  I was trying
  to run some unit tests in parallel which involved running "git
commit" commands.  It took me
a long time to figure out why they were randomly hanging.  In
addition, I've previously used the `dir-foreach-parallel git
commit...` pattern for actual development, when working on software
which requires changes across several dependent git repos.

Please review the rest of the bug report below. You can delete any lines you don't wish to share.

[System Info] git version: git version 2.46.2 cpu: arm64 no commit associated with this build sizeof-long: 8 sizeof-size_t: 8 shell-path: /bin/sh feature: fsmonitor--daemon libcurl: 8.4.0 zlib: 1.2.12 uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6030 arm64 compiler info: clang: 15.0.0 (clang-1500.3.9.4) libc info: no libc information available $SHELL (typically, interactive shell): /bin/zsh

[Enabled Hooks] not run from a git repository - no hooks to show

Eric Sunshine· Oct 1, 2024, 04:29 UTC · re: Bennet Huber · lore

Re: git commit hangs when run concurrently on macos

On Mon, Sep 30, 2024 at 4:58 PM Bennet Huber <bennet.huber@gmail.com> wrote:
Show 9 quoted lines
> After running this, wait for output to subside, then run `jobs`.
> On my machine, I usually see 5-10 hung "git commit" jobs.
>
> > 3:39:04.020533 run-command.c:758       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git fsmonitor--daemon run --detach --ipc-threads=8
>
> Setting `USE_FS_MONITOR=false` in the code above consistently results
> in no hung "git commit" commands, which leads me to suspect the
> problem has to do with the fsmonitor feature.  It's also a decent,
> though difficult to discover, workaround.

I don't use fsmonitor and haven't particularly been following the issue, but there are some active threads on the mailing list regarding fsmonitor hanging on macOS. Perhaps they are relevant to your case.

https://lore.kernel.org/git/20240930194600.GA1805534@coredump.intra.peff.net/T/
https://lore.kernel.org/git/20240509162219.GA1707955@coredump.intra.peff.net/T/
https://lore.kernel.org/git/pull.1802.git.1727577690390.gitgitgadget@gmail.com/T/

← back to recent threads