Re: Avoid race condition between fetch and repack/gc?

From: Bryan Turner <bturner@atlassian.com>
Date: Mar 16, 2020, 23:40 UTC
Message-ID: <CAGyf7-HQH7hjuYSmOYeOEvZBWkUzjU4M8Wr50FNPYgG4NZ=5UA@mail.gmail.com>
In-Reply-To: <20200316172729.GA1072073@coredump.intra.peff.net>

On Mon, Mar 16, 2020 at 10:27 AM Jeff King <peff@peff.net> wrote:

Show 7 quoted lines

>
>   - you don't use config like core.packedGitLimit that encourages Git to
>     drop mmap'd windows. On 64-bit systems, this doesn't help at all (we
>     have plenty of address space, and since the maps are read-only, the
>     OS is free to drop the clean pages if there's memory pressure). On
>     32-bit systems, it's a necessity (and I'd expect a 32-bit server to
>     run into this issue a lot for large repositories).

I could be wrong, but I'm going to _guess_ from the :7999 in the example output that the repository is hosted with Bitbucket Server. Assuming my guess is correct, Bitbucket Server _does_ set "core.packedgitlimit=256m" (and "core.packedgitwindowsize=32m", for what it's worth). Those settings are applied specifically because we've found they _do_ impact Git's overall memory usage when serving clones in particular, which is important for cases where the system is serving dozens (or hundreds) of concurrent clones.

Of course, we don't always do a great job of re-testing once-beneficial settings later, which means sometimes they end up being based on outdated observations. Perhaps we should prioritize some additional testing here, especially on 64-bit systems. (We've been setting "core.packedgitlimit" since back when Bitbucket Server was called Stash and supported Git 1.7.6 on the server.)

That said, though, you note "core.packedgitlimit" is necessary on 32-bit servers and, unfortunately, we do still support Bitbucket Server on 32-bit OSes. Maybe we should investigate applying (or not) the flags depending on the platform. Sadly, that's not necessarily simple to do since just because the _OS_ is 64-bit doesn't mean _Git_ is; it's pretty trivial to run 32-bit Git on 64-bit Windows, for example.

Bryan