Re: Git server eats all memory

From: Avery Pennarun <apenwarr@gmail.com>
Date: Aug 4, 2010, 20:12 UTC
Message-ID: <AANLkTimwy6GumHYSTo2je_hOUO80KEpx4_8z3iOoZyc0@mail.gmail.com>
In-Reply-To: <wesfwyupgrg.fsf@kanis.fr>

On Wed, Aug 4, 2010 at 10:57 AM, Ivan Kanis <expire-by-2010-08-09@kanis.fr> wrote:

Show 8 quoted lines

> I am running git 1.7.0.4 on Debian stable. I have compiled git from
> source. I am cloning using ssh.The client and the server are running
> the same version. The bare repository is 4.5G with various binary files,
> I have 6G of memory on my server.
>
> I am having problem with memory ballooning when receiving object
> from the server. The amount of memory used on the server seems to be same
> size as the object received.

Git works fine with huge repositories; it does not work fine at all with very large individual objects in a repository, and it does what you're experiencing. There are a few minor workarounds (like the repack command someone mentioned) that slightly reduce the symptoms, but the symptoms will crop up again eventually.

There's at least one project intended to solve this that people have linked to: http://caca.zoy.org/wiki/git-bigfiles

...but it's incomplete and it doesn't look like their repo has changed in some time.

I'm not aware of any workaround to this sort of thing other than "don't store large objects in git." You can split the big objects into a bunch of small objects, as bup does, but then you have to reassemble them all, which is inconvenient.

Sorry,

Avery