Re: git status takes 30 seconds on Windows 7. Why?

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Mar 27, 2013, 20:12 UTC
Message-ID: <CA+55aFzu72fxhms_azREiLz+hLDzT8XKJfJuYHZc=AQcZxmknw@mail.gmail.com>
In-Reply-To: <7vy5d8lijp.fsf@alter.siamese.dyndns.org>

On Wed, Mar 27, 2013 at 1:00 PM, Junio C Hamano <gitster@pobox.com> wrote:

Show 6 quoted lines

>
> Given that we haven't tweaked the parallelism or thread-cost
> parameters since the inception of the mechanism in Nov 2008, I
> suspect that we would see praises from some and grievances from
> other corners of the user base for a while until we find acceptable
> values for them

Looking at the parameters again, I really think they are pretty sane,
and I don't think the numbers are all that likely to have shifted from
2008. The maximum thread value is quite reasonable: twenty threads is
sufficient to cover quite a bit of latency, and brings "several
seconds" down to "under half a second" for any truly IO-limited load,
while not being disastrous for the case where everything is in cache
and we only have a limited number of CPU cores.

And the "at least 500 files per thread" limit is eminently reasonable too - smaller projects like git won't have more than five or so threads.

So I'd be very surprised if the values need much tweaking. Sure, there might be some extreme cases that might tune for some particular patterns, and maybe we should make the values be tunable rather than totally hardcoded, but I suspect there's limited up-side.

It might be interesting for the people who really like tuning, though. So in addition to "index.preload=true", maybe an extended config format like "index_preload=50,200" to say "maximum of fifty threads, for every 200 files" could be done just so people could play around with the numbers and see how much (if at all) they actually matter.

But I really don't think the original 20/500 rule is likely to be all that bad for anybody. Unless there is some *really* sucky thread library out there (ie fully user-space threads, so filename lookup isn't actually parallelised at all), but at least for that case the fix is to just say "ok, your threads aren't real threads, so just disable index preloading entirely).

                 Linus