Re: Something is broken in repack

From: Nicolas Pitre <nico@cam.org>
Date: Dec 8, 2007, 05:12 UTC
Message-ID: <alpine.LFD.0.99999.0712080004490.555@xanadu.home>
In-Reply-To: <9e4733910712072101k4583c0afsea368253fe1cf706@mail.gmail.com>

On Sat, 8 Dec 2007, Jon Smirl wrote:

Show 16 quoted lines

> On 12/7/07, Nicolas Pitre <nico@cam.org> wrote:
> > On Fri, 7 Dec 2007, Jon Smirl wrote:
> >
> > > Does the gcc repo contain some giant objects? Why wasn't the memory
> > > freed after their chain was processed?
> >
> > It should be.
> >
> > > Most of the last 10% is being done on a single CPU. There must be a
> > > chain of giant objects that is unbalancing everything.
> >
> > I'm about to send a patch to fix the thread balancing for real this
> > time.
> 
> Something is really broken in the last 5% of that repo. I have been
> processing at 97% for 30 minutes without moving to 98%.

This is a clear sign of a problem, indeed.

I'll be away for the weekend, so here's a few things to try out if you feel like it:

1) Make sure the problem occurs with the thread code disabled.  That 
   would eliminate one variable, and will help for #2.

2) Try bissecting the issue.  If you can find an old Git version where 
   the issue doesn't appear then simply run "git bissect" to find the 
   exact commit causing the problem.  Best with a repo that doesn't take
   ages to repack.

3) Compile Git against the dmalloc library in order to identify where
   the huge memory leak is happening.

Nicolas