Re: git-reflog 70 minutes at 100% cpu and counting

From: Eric Paris <eparis@redhat.com>
Date: Dec 15, 2009, 02:11 UTC
Message-ID: <1260843111.9379.86.camel@localhost>
In-Reply-To: <alpine.LFD.2.00.0912141924030.23173@xanadu.home>

On Mon, 2009-12-14 at 19:26 -0500, Nicolas Pitre wrote:

Show 25 quoted lines

> On Mon, 14 Dec 2009, Eric Paris wrote:
> 
> > On Mon, 2009-12-14 at 16:23 -0500, Jeff King wrote:
> > > On Mon, Dec 14, 2009 at 04:20:29PM -0500, Eric Paris wrote:
> > > 
> > > > Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
> > > > for >5 minutes at 100% cpu (I killed it, it didn't finish)
> > > > 
> > > > I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
> > > > to
> > > > 
> > > > http://people.redhat.com/~eparis/git-tar/
> > > 
> > > Wowzers, that's big. Can you send just what's in .git?
> > 
> > So I zipped up just .git   1.2G.  I did a make clean and zipped up the
> > whole repo  1.3G.
> > 
> > Just started pushing the 1.3G file.
> > 
> > Maybe having a .git directory that large is the problem?
> 
> Shouldn't be, unless your repo is really badly packed.
> 
> What's the output of 'git count-objects -v' ?

count: 87065 size: 866744 in-pack: 1203497 packs: 148 size-pack: 976474 prune-packable: 1611 garbage: 0

It's not home movies :) . It's a kernel trees with about 5 'upstream' trees that are remotes, which I update daily. One of the remotes constantly rebases every day starting with Linus' tree and pulling in about 150+ branches of work from others all of which might rebase. I have (needlessly) the tags he keeps of that repo every day. I daily rebase my work on top of that constantly rebasing tree (linux-next) using stgit.

I noticed just blindly poking at sizes in my .git/object/pack that the largest pack is a lot larger than the second and third largest....

-r--r--r-- 1 paris paris 108031039 Feb 12 2009 pack-71a9c0f08c76b8ffd1cf0a14d7cfe991fbc9db80.pack -r--r--r-- 1 paris paris 32670479 Apr 7 2009 pack-5c8333301012d9b70d70648b287cf540afcc63ed.pack -r--r--r-- 1 paris paris 26728958 Dec 30 2008 pack-fb8ceb5a33d9881fe771860c6006f55f73ecdf65.pack

And all total there is almost 1G of data in .git/object/pack

If the answer really is that I just have too much data and it can't be handled, I'm fine exporting my patches getting some clean trees and starting over till I get in this situation again, but if it really is a problem/bug that can be solved, the full tar ball of my repo is at

http://people.redhat.com/~eparis/git-tar/

-Eric