Re: pack operation is thrashing my server

From: Shawn O. Pearce <spearce@spearce.org>
Date: Aug 11, 2008, 03:04 UTC
Message-ID: <20080811030444.GC27195@spearce.org>
In-Reply-To: <a6b6acf60808101247r4fea978ft6d2cdc53e1f99c0e@mail.gmail.com>

Ken Pratt <ken@kenpratt.net> wrote:

Show 8 quoted lines

> I'm having memory issues when trying to clone a remote git repository.
> 
> The remote repository is bare, and is 180MB in size (says du), with
> 1824 objects. The remote (VPS) server is running git version 1.5.6.4
> on Arch Linux on a x86_64 Opteron with 256MB of dedicated RAM.
> 
> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 21782 kenpratt  20   0  444m 212m  272 D    3 83.0   0:04.98 git-pack-object

Well, clearly the server is swapping at this point.  212m resident
for this git-pack-objects process leaves no room available for
anything else.  Git is using too much memory for this system.

Show 7 quoted lines

> I've tried very conservative pack settings:
> 
> [pack]
>         threads = 1
>         windowmemory = 64M
>         deltacachesize = 1M
>         deltacachelimit = 1M

Have you tried something like this?

	[core]
		packedGitWindowSize = 16m
		packedGitLimit = 64m

	[pack]
		threads = 1
		windowMemory = 64m
		deltaCacheSize = 1m

On a 64 bit system packedGitWindowSize and packedGitLimit have very large thresholds which will cause it to mmap in the entire pack file. You may need to try even smaller settings than these; 256m physical memory isn't a lot when dealing with a repository 180m in size. Especially on a 64 bit system.

-- 
Shawn.