From: Nicolas Pitre <nico@cam.org>
Date: Sun, 07 Sep 2008 17:08:02 GMT
Subject: Re: pack operation is thrashing my server
Message-ID: <alpine.LFD.1.10.0809071304130.23787@xanadu.home>
In-Reply-To: <9e4733910809070658k66e0481fx758e9a365229cf18@mail.gmail.com>

On Sun, 7 Sep 2008, Jon Smirl wrote:

> On 9/7/08, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> >
> >
> >  On Sat, 6 Sep 2008, Jon Smirl wrote:
> >  >
> >
> > > When I was playing with those giant Mozilla packs speed of zlib wasn't
> >  > a big problem. Number one problem was the repack process exceeding 3GB
> >  > which forced me to get 64b hardware and 8GB of memory. If you start
> >  > swapping in a repack, kill it, it will probably take a month to
> >  > finish.
> >
> >
> > .. and you'd make things much much WORSE?
> 
> My observations on the Mozilla packs indicated that the problems were
> elsewhere in git, not in the decompression algorithms. Why does a
> single 2000 delta chain take 15% of the entire pack time? Something
> isn't right when long chains are processed which triggers far more
> decompressions than needed.

Please have a look at commit eac12e2d4d7f.  This fix improved things for 
my gcc repack tests.


Nicolas