Re: Git is not scalable with too many refs/*
- From
- Martin Fick <mfick@codeaurora.org>
- Date
- Sep 26, 2011, 21:52 UTC
- Message-ID
- <201109261552.04946.mfick@codeaurora.org>
- In-Reply-To
- <201109261539.33437.mfick@codeaurora.org>
On Monday, September 26, 2011 03:39:33 pm Martin Fick wrote:
Show 27 quoted lines
> On Monday, September 26, 2011 02:28:53 pm Julian Phillips > wrote: > > >> Random thought. What happens to the with > > >> compression case if you leave the commit in, but > > >> add a sleep(15) to the end of sort_refs_list? > > > > > > Why, what are you thinking? Hmm, I am trying this on > > > the non gced repo and it doesn't seem to be > > > completing (no cpu usage)! It appears that perhaps > > > it is being called many times (the sleeping would > > > explain no cpu usage)?!? This could be a real > > > problem, this should only get called once right? > > > > I was just wondering if the time taken to get the refs > > was changing the interaction with something else. Not > > very likely, but ... > > > > I added a print statement, and it was called four times > > when I had unpacked refs, and once with packed. So, > > maybe you are hitting some nasty case with unpacked > > refs. If you use a print statement instead of a sleep, > > how many times does sort_refs_lists get called in your > > unpacked case? It may well also be worth calculating > > the time taken to do the sort. > > In my case it was called 18785 times! Any other tests I > should run?
Gerrit stores the changes in directories under refs/changes named after the last 2 digits of the change. Then under each change it stores each patchset. So it looks like this: refs/changes/dd/change_num/ps_num
I noticed that:
ls refs/changes/* | wc -l -> 18876
somewhat close, but not super close to 18785, I am not sure if that is a clue. It's almost like each change is causing a re-sort,
-Martin
-- Employee of Qualcomm Innovation Center, Inc. which is a member of Code Aurora Forum