From: Koji Nakamaru <koji.nakamaru@gree.net>
Date: Wed, 22 Oct 2025 01:21:32 GMT
Subject: Re: [PATCH] macOS: queue for munmap operations
Message-ID: <CAOTNsDxoSFTBwaO0Eqr+r0NQOwdA2Fge3CN7JQtnAWEt2PnDew@mail.gmail.com>
In-Reply-To: <20251021080625.GD259661@coredump.intra.peff.net>

Thank you for pointing out many unusual mmap calls and other details. As
discussed below, the root cause was simply my ~/.gitconfig. This patch
may be useful in some rare/edge cases but a somewhat unusual hack, so
I'm withdrawing it.

On Tue, Oct 21, 2025 at 5:07 PM Jeff King <peff@peff.net> wrote:
>
> On Mon, Oct 20, 2025 at 10:35:02PM +0000, Koji Nakamaru via GitGitGadget wrote:
>
> > From: Koji Nakamaru <koji.nakamaru@gree.net>
> >
> > Executing many mmap/munmap calls alternately can cause a huge load on
> > macOS. In order to reduce it, we should temporarily store munmap
> > operations in a queue and process them all at once when the queue is
> > filled. When the program terminates, we can discard any remaining munmap
> > operations as corresponding mmaped regions are automatically reclaimed.
> >
> > Add a queue for munmap operations to perform them all at once.
> >
> > Here are some example timings. On the Linux kernel repository that
> > requires about 1700 mmap/munmap calls:
> >
> >   time git ls-tree -r -l --full-tree 211ddde > /dev/null
>
> Why is it doing so many mmap calls? Do you have a ton of loose objects?
> We have to mmap loose objects individually (because they're all in
> separate files), but each pack only gets a single map (well, there's a
> window parameter, but it's 1GB on 64-bit systems, so you should get a
> handful of maps at most).
>
> If you run "git gc", how does the resulting ls-tree perform? I have only
> 27 mmap() calls on my system.
>
> I know that running "git gc" is relatively expensive, but it is also
> bringing other optimizations (like the fact that we don't have to open()
> and map each of those files in the first place!).
>
> > On a private repository that requires about 943000 mmap/munmap calls:
> >
> >   time git ls-tree -r -l --full-tree xxxxxxx > /dev/null
>
> Ditto here. I'd be curious how well packed the repo is, and how it does
> after a repack. If it has a very large packfile, you might also try:
>
>   git config core.packedGitWindowSize 4G
>
> or similar (though for just an ls-tree, we should only be looking at
> tree objects, which in general I'd expect to be in a confined area of
> the packfile; so the 1GB window is probably plenty).

Following your suggestion, I investigated the number of mmap calls in
other environments and found much smaller counts. I tracked how
xmmap_gently() was called in packfile.c and found
settings->packed_git_window_size was different between environments. My
~/.gitconfig defined "packedGitLimit = 128m" and this caused many calls.

> > +int git_munmap(void *start, size_t length)
> > +{
> > +     static pthread_mutex_t mutex;
> > +     static struct munmap_queue *queue;
> > +     static int count;
> > +     int i;
> > +
> > +     pthread_mutex_lock(&mutex);
> > +     if (!queue)
> > +             queue = xmalloc(COUNT_MAX * sizeof(struct munmap_queue));
> > +     queue[count].start = start;
> > +     queue[count].length = length;
> > +     if (++count == COUNT_MAX) {
> > +             for (i = 0; i < COUNT_MAX; i++)
> > +                     munmap(queue[i].start, queue[i].length);
> > +             count = 0;
> > +     }
> > +     pthread_mutex_unlock(&mutex);
> > +     return 0;
> > +}
>
> Does batching those unmaps actually make them faster? Or is it just that
> the commands you showed did not fill the queue, so we essentially just
> leaked all of those maps until the program exited?
>
> If the latter, then I'd wonder:
>
>   1. Does this increase memory pressure, since the OS has no idea we're
>      not actually interested in those maps anymore? Some of them can be
>      quite large, if the command is looking at blobs.
>
>   2. How does it perform on a command that actually fills the queue? I
>      guess something like "git log --raw" might do it (though if my
>      guesses above are right, you'd need on the order of 64,000 loose
>      trees).

In my extreme cases, this batching makes them faster. Queue flushing has
occurred several times for the private repository case and not occurred
for the Linux kernel case. Though I haven't investigated in detail,
memory pressure doesn't seem to be critical (and it could also be
possible to adopt smarter thresholds).

I tested git log --raw for the Linux kernel repository. For reference,
the results are shown below:

  without "packedGitLimit = 128m":

        mmap    9

        # without batching
        real    1m3.970s
        user    1m2.232s
        sys     0m1.725s

        # with batching
        real    1m5.991s
        user    0m58.637s
        sys     0m4.315s

  with "packedGitLimit = 128m":

        mmap    3072538

        # without batching
        (It took too long so I stopped the execution)
        real    518m6.928s
        user    0m41.126s
        sys     517m24.072s

        # with batching
        real    2m26.276s
        user    1m8.495s
        sys     1m3.230s