Re: Git is not scalable with too many refs/*
- From
- Martin Fick <mfick@codeaurora.org>
- Date
- Sep 26, 2011, 15:42 UTC
- Message-ID
- <201109260942.08299.mfick@codeaurora.org>
- In-Reply-To
- <4E809AFE.6010901@alum.mit.edu>
On Monday, September 26, 2011 09:32:14 am Michael Haggerty wrote:
Show 26 quoted lines
> On 09/25/2011 10:43 PM, Martin Fick wrote: > > A coworker of mine pointed out to me that a simple > > > > git checkout > > > > can also take rather long periods of time > 3 mins when > > run on a repo with ~100K refs. > > > > While this is not massive like the other problem I > > reported, it still seems like it is more than one > > would expect. So, I tried an older version of git, > > and to my surprise/delight, it was much faster (.2s). > > So, I bisected this issue also, and it seems that the > > "offending" commit is > > > 680955702990c1d4bfb3c6feed6ae9c6cb5c3c07: > I'm still working on changes to store references > hierarchically in the cache and read them lazily. I > hope that it will help some scaling problems with large > number of refs. > > Unfortunately I keep getting tangled up in side issues, > so it is taking a lot longer than expected. But there's > still hope. > > Michael
Thanks Michael, I look forward to those changes. In the meantime however, I will try to take advantage of the current inefficiencies of large ref counts to attempt to find places where there are obvious problems in the code paths. I suspect that there are several commands in git which inadvertently scan all the refs when they probably shouldn't. Since this is likely very slow now, it should be easy to find those, if it were faster, this might get overlooked. I feel like git checkout is one of those cases, it does not seem like git checkout should be affected by the number of refs in a repo?
-Martin
-- Employee of Qualcomm Innovation Center, Inc. which is a member of Code Aurora Forum