Re: pack operation is thrashing my server

From: Ken Pratt <ken@kenpratt.net>
Date: Aug 11, 2008, 19:13 UTC
Message-ID: <a6b6acf60808111213j78028c2ercc1199e080eaeccc@mail.gmail.com>
In-Reply-To: <20080811150150.GC26363@spearce.org>

Show 11 quoted lines

> What is the largest object in that repository? Do you have a
> rough guess?  You said earlier:
>
>> The remote repository is bare, and is 180MB in size (says du), with
>> 1824 objects.
>
> That implies there is at least one really large object in that
> repository.  The average of 101KB per object is not going to be
> a correct figure here as most commits and trees are _very_ tiny.
> It must be a large object.  Those big objects are going to consume
> a lot of memory if they get inflated in memory.

Largest object is ~150MB, and there are a couple 5-10MB objects as well.

> You said earlier this was Git 1.5.6.4.  I recently fixed a bug in
> the code that reads data from packs to prevent it from blowing out
> memory usage, but that bug fix was included in 1.5.6.4.

I tried upgrading to 1.5.6.5 as well, but that didn't help.

Show 5 quoted lines

> On the up side, packing should only be consuming huge memory like
> this when it needs to move loose objects into a pack file.  I think
> Martin Langhoff suggested packing this on your laptop then using
> rsync over SSH to copy the pack file and .idx file to the server, so
> the server didn't have to spend time figuring out the deltas itself.

Unfortunately, that will only work as a band-aid solution for my workflow. I think I'll have to limit the file size in the repository to something that the server can handle.