Re: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 18, 2017, 05:56 UTC
Message-ID: <xmqqvas8m499.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <20170215164416.tekykkzhm6qlj2h2@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

Show 12 quoted lines

> On Wed, Feb 15, 2017 at 09:27:53AM -0500, Jeff Hostetler wrote:
>
>> I have some informal numbers in a spreadsheet.  I was seeing
>> a 8-9% speed up on a status on my gigantic repo.
>> 
>> I'll try to put together a before/after perf-test to better
>> demonstrate this.
>
> Thanks. What I'm mostly curious about is how much each individual step
> buys. Sometimes when doing a long optimization series, I find that some
> of the optimizations make other ones somewhat redundant (e.g., if patch
> 2 causes us to call the optimized code from patch 3 less often).

I am curious too.

To me 1/5 (reduction of redundant calls), 4/5 (correctly size the hash that would grow to a known size anyway) and 5/5 (take advantage of the fact that adjacent cache entries are often in the same directory) look like no brainers to take, regardless of the others (including themselves).

It is not clear to me if 3/5 (preload-index uses available cores to compute hashes) is an unconditional win (an operation that is pathspec limited may need hashes for only a small fraction of the index---would it still be a win to compute the hash for all entries upon loading the index, even if we are using otherwise-idel cores?).

Of course 2/5 is a prerequisite step for 3/5 and 5/5, so if we want either of the latter two, we cannot avoid it.