Re: concurrent access to multiple local git repos is error prone

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Aug 6, 2018, 14:32 UTC
Message-ID: <20180806143202.GA97564@aiede.svl.corp.google.com>
In-Reply-To: <CA+KyZp760a3yTF6ghiNh=c5FoU36MN3cdQU5J8NF2Ss+nwRqyQ@mail.gmail.com>

(administrivia: please don't top-post) Hi Alex,

Alexander Mills wrote:

> Yeah this concurrency problem is real. Not only does it happen with
> `git status` the same thing happens with `git rev-parse
> --show-toplevel`.

Sorry for the confusion --- I didn't mean to claim your experience was not real!

What I wanted to make clear is that

 1. Git is designed to allow concurrent reads of a repository (and
    pushes to a repository).  If it doesn't work, that is a simple bug,
    not a design goal.

 2. Plenty of people rely on concurrently accessing repositories, so
    if it doesn't work, then (i) we definitely want to know and (ii)
    we're going to need a lot of detail to figure out what's happening,
    so we can fix it.

Does that make it clearer?

Show 22 quoted lines

> What happens is that I get no stdout when repos are accessed
> concurrently (and no stderr). If I limit concurrency to 1, the problem
> goes away. When I up the concurrency, the problem is sporadic, which
> is the exact signal for a concurrency/race-condition related issue.
> The signs are damn clear. I have seen this problem on MacOS I think a
> year back on a different project, but I never reported it b/c I hadn't
> really verified it.
>
> Like I said I am on Ubuntu. I have 3 git repos that are incorporated
> into the tool that's generating the problem. For one repo I got this:
>
> $ git fsck
>
> Checking object directories: 100% (256/256), done.
> dangling tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904
>
> For `$ git version --build-options` I have:
>
> git version 2.17.1
> cpu: x86_64
> no commit associated with this build
> sizeof-long: 8

Thanks. My best idea for a next step is that if you can come up with a reproduction recipe, that would be very helpful.

It doesn't have to reproduce 100% of the time, but e.g. if you have a script that reproduces it 50% of the time, I can run that script in a loop.

Thanks, Jonathan