Re: Excessive mmap [was Git server eats all memory]

From: Dmitry Potapov <dpotapov@gmail.com>
Date: Aug 17, 2010, 17:07 UTC
Message-ID: <AANLkTimDx1Vg6kUaHgap6+khTmJYsxdiSDB6cb88-fnN@mail.gmail.com>
In-Reply-To: <wes4oetv31i.fsf@kanis.fr>

Hi Ivan,

On Tue, Aug 17, 2010 at 02:26:01PM +0200, Ivan Kanis wrote:

Show 6 quoted lines

>
> I have ran the following command for my tests: vmstat -SM -n 60
>
> Here's the stat for one git clone, I see that 4237M of memory is
> consumed. That is roughly the size of the repository. This raises my
> first question: why is the memory not reclaimed, at the end of the run?

It is consumed by system cache to hold the last read data, and it will hold it as long as there is enough free memory. When there is not enough free memory, the system will free some old (long time unused) pages from the system cache. This allows the system to avoid re-reading same files when there is enough memory to keep them.

Show 5 quoted lines

>
> ls -lh objects/pack/*.pack
> 4.2G objects/pack/pack-55ad6d01f37427ca69e6267b0cd4e5257e57272c.pack
>
> Is it a sensible behavior to leave a 4G file lying around?

I am not sure I understand your question. This pack contains all data of your repository. So what did you expect to happen?

> Does it get erased when people are pushing changes in?

No, usually, push adds a new pack. However, when there are too many packs, the garbage collector will try to repack everything in one new pack. You can create an empty file with the same name but with .keep extension to preserve the specified pack from being repacked.

Dmitry