Re: Extremely slow progress during 'git reflog expire --all'

From: Junio C Hamano <gitster@pobox.com>
Date: Apr 4, 2010, 18:22 UTC
Message-ID: <7vy6h36pt1.fsf@alter.siamese.dyndns.org>
In-Reply-To: <20100403203507.GA12262@coredump.intra.peff.net>

Jeff King <peff@peff.net> writes:

Show 12 quoted lines

> Thanks, I was able to get it and reproduce your problem. The slowness is
> in the expire-unreachable code. You can work around it with:
>
>   git config gc.reflogExpireUnreachable never
>
> Obviously that's not really a fix, but it should let your "git gc" work.
>
> It looks like we do two merge-base calculations for each reflog entry,
> which is what takes so long. Perhaps if we know we are going to do a
> large number of reachability checks, we can pre-mark all reachable
> commits, and then each reflog entry would just need to check the commit
> mark.

Thanks for the analysis, but expire_reflog() that is run for each ref already does that, I think. It first runs mark_reachable(), then walks each reflog entry for the ref to call expire_reflog_ent(), which in turn calls unreachable() that first checks if mark_reachable() has marked the commit, and if so we don't run in_merge_bases().

But if the commit in question is not reachable, then we end up running in_merge_bases() to double-check anyway, which is probably the symptom that was observed.

So perhaps this is a workable compromise?

 builtin/reflog.c |    7 +++++++
 1 files changed, 7 insertions(+), 0 deletions(-)

diff --git a/builtin/reflog.c b/builtin/reflog.c
index 64e45bd..7e278b8 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -230,6 +230,13 @@ static int unreachable(struct expire_reflog_cb *cb, struct commit *commit, unsig
 	/* Reachable from the current ref?  Don't prune. */
 	if (commit->object.flags & REACHABLE)
 		return 0;
+	/*
+	 * Unless there was a clock skew, younger ones that are
+	 * reachable should have been marked by mark_reachable().
+	 */
+	if (cb->cmd->expire_total < commit->date)
+		return 1;
+
 	if (in_merge_bases(commit, &cb->ref_commit, 1))
 		return 0;