Re: Git is not scalable with too many refs/*
- From
- Martin Fick <mfick@codeaurora.org>
- Date
- Sep 26, 2011, 15:15 UTC
- Message-ID
- <201109260915.29285.mfick@codeaurora.org>
- In-Reply-To
- <201109251443.28243.mfick@codeaurora.org>
OK, I have found what I believe is another performance regression for large ref counts (~100K).
When I run git br on my repo which only has one branch, but has ~100K refs under ref/changes (a gerrit repo), it takes normally 3-6mins depending on whether my caches are fresh or not. After bisecting some older changes, I noticed that this ref seems to be where things start to get slow: c774aab98ce6c5ef7aaacbef38da0a501eb671d4
commit c774aab98ce6c5ef7aaacbef38da0a501eb671d4 Author: Julian Phillips <julian@quantumfyre.co.uk> Date: Tue Apr 17 02:42:50 2007 +0100
refs.c: add a function to sort a ref list, rather then sorting on add
Rather than sorting the refs list while building it,
sort in one
go after it is built using a merge sort. This has a
large
performance boost with large numbers of refs. It shouldn't happen that we read duplicate entries into
the same
list, but just in case sort_ref_list drops them if the
SHA1s are
the same, or dies, as we have no way of knowing which
one is the
correct one. Signed-off-by: Julian Phillips
<julian@quantumfyre.co.uk>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>which is a bit strange since that commit's purpose was to actually speed things up in the case of many refs. Just to verify, I reverted the commit on 1.7.7.rc0.73 and sure enough, things speed up down to the 14-20s range depending on caching.
If this change does not actually speed things up, should it be reverted? Or was there a bug in the change that makes it not do what it was supposed to do?
Thanks,
-Martin
-- Employee of Qualcomm Innovation Center, Inc. which is a member of Code Aurora Forum