From: Dmitry Potapov <dpotapov@gmail.com>
Date: Mon, 09 Aug 2010 12:35:11 GMT
Subject: Re: Excessive mmap [was Git server eats all memory]
Message-ID: <AANLkTi=6JcwLuyNWq9oYzE_E+7DSn-sEvR-X3AHvXM6U@mail.gmail.com>
In-Reply-To: <wesy6cgm6wd.fsf_-_@kanis.fr>

On Mon, Aug 09, 2010 at 12:12:34PM +0200, Ivan Kanis wrote:
>
> I have clone identical repositories and the system starts to swap. I
> think it shows that cloning two repository doesn't share mmap.

Though Git uses MAP_PRIVATE with mmap, this only marks mapped pages as
copy-on-write. Because cloning does not change the pack file, all those
pages should be shared. So, the only reasons for swapping could be:
- each cloning operation access to different pages at the same time, so
  it means more pages have to be loaded in memory to allow two programs
  to run simultaneously.
- each operation allocates 387Mb (accordingly to your earlier data), so
  it may add more memory pressure.

>
> I saw this constant defined in git-compat-util.h
>
> /* This value must be multiple of (pagesize * 2) */
> #define DEFAULT_PACKED_GIT_WINDOW_SIZE \
>        (sizeof(void*) >= 8 \
>                ?  1 * 1024 * 1024 * 1024 \
>                : 32 * 1024 * 1024)
>
> If I read this correctly git is allocating 1G of mmap on 64 bit
> architecture. Isn't that a bit much?

On 64-bit architecture, you have plenty virtual space, and mapping
a file to memory should not take much physical memory (only space
needed for system tables). You can reduce core.packedGitWindowSize
in the Git configuration to see if it helps, but I doubt that it
will have any noticeable effect.


Dmitry