Re: Git is not scalable with too many refs/*

From: Julian Phillips <julian@quantumfyre.co.uk>
Date: Sep 26, 2011, 16:49 UTC
Message-ID: <2b7c307b138bd6061209795056a593ca@quantumfyre.co.uk>
In-Reply-To: <201109261038.34527.mfick@codeaurora.org>

On Mon, 26 Sep 2011 10:38:34 -0600, Martin Fick wrote:

Show 27 quoted lines

> On Monday, September 26, 2011 09:56:50 am Sverre Rabbelier
> wrote:
>> Heya,
>>
>> On Mon, Sep 26, 2011 at 17:48, Martin Fick
> <mfick@codeaurora.org> wrote:
>> > Hmm, I was thinking that too, and I just did a test.
>> >
>> > Instead of storing the changes under refs/changes, I
>> > fetched them under refs/heads/changes and then ran git
>> > 1.7.6 and it took about 3 mins.  Then, I ran the
>> > 1.7.7.rc0.73 with
>> > c774aab98ce6c5ef7aaacbef38da0a501eb671d4 reverted and
>> > it only took 13s!  So, if this indeed tests what you
>> > were suggesting, I think it shows that even in the
>> > intended case this change slowed things down?
>>
>> And if you run 1.7.7 without that commit reverted?
>
> Sorry, I probably confused things by mentioning 1.7.6, the
> bad commit was way before that early 1.5 days...
>
> As for 1.7.7, I don't think that exists yet, so did you mean
> the 1.7.7.rc0.73 version that I mentioned above without the
> revert?  Strangely enough, that ends up being
> 1.7.7.rc0.72.g4b5ea.  That is also slow with
> refs/heads/changes > 3mins.

Hmm ... something interesting is going on.

I created a little test repo with ~100k unpacked refs.

I tried "time git branch" with three versions of git, and I got (hot cache times):

git version 1.7.6.1: ~1.2s git version 1.7.7.rc3: ~1.2s git version 1.7.7.rc3.1.gbc93f: ~40s

Where the third was with the commit reverted. That was almost 40s of 100% CPU - my poor laptop had to turn the fans up to noisy ...

> -Martin

-- 
Julian