Re: Extremely slow progress during 'git reflog expire --all'

From: Jeff King <peff@peff.net>
Date: Apr 3, 2010, 20:35 UTC
Message-ID: <20100403203507.GA12262@coredump.intra.peff.net>
In-Reply-To: <201004031629.01970.elendil@planet.nl>

[sorry, resend, I managed to totally screw up Shawn's email in the last one. Stupid mutt aliases.]

On Sat, Apr 03, 2010 at 04:29:01PM +0200, Frans Pop wrote:

Show 11 quoted lines

> On Saturday 03 April 2010, Jeff King wrote:
> > > I can make that available, but it's going to take a while to upload
> > > and I don't want to leave it up too long as I'll be abusing a project
> > > service for that. So the people who want to look at it would have grab
> > > it fairly promptly (within 2 days or so).
> 
> The tarball is up at:
> http://alioth.debian.org/~fjp/.tmp/linux-2.6_reflog-issue.tar
> 
> Because of the Easter weekend I'll leave it up a bit longer. I plan to 
> remove it sometime on Thursday.

Thanks, I was able to get it and reproduce your problem. The slowness is in the expire-unreachable code. You can work around it with:

  git config gc.reflogExpireUnreachable never

Obviously that's not really a fix, but it should let your "git gc" work.

It looks like we do two merge-base calculations for each reflog entry, which is what takes so long. Perhaps if we know we are going to do a large number of reachability checks, we can pre-mark all reachable commits, and then each reflog entry would just need to check the commit mark.

I don't have any more time now to look at it, but I am cc'ing Junio (who wrote the original expire-unreachable code) and Shawn (the resident reflog expert), who may have more input.

-Peff