Re: [PATCH v1] read-cache: speed up index load through parallelization
- From
Ben Peart <peartben@gmail.com>
- Date
- Aug 24, 2018, 17:28 UTC
- Message-ID
- <3e82ee23-6a04-6714-4d51-68bb2ef4a123@gmail.com>
- In-Reply-To
- <20180824155734.GA6170@duynguyen.home>
On 8/24/2018 11:57 AM, Duy Nguyen wrote:
Show 28 quoted lines
> On Fri, Aug 24, 2018 at 05:37:20PM +0200, Duy Nguyen wrote:
>> Since we're cutting corners to speed things up, could you try
>> something like this?
>>
>> I notice that reading v4 is significantly slower than v2 and
>> apparently strlen() (at least from glibc) is much cleverer and at
>> least gives me a few percentage time saving.
>>
>> diff --git a/read-cache.c b/read-cache.c
>> index 7b1354d759..d10cccaed0 100644
>> --- a/read-cache.c
>> +++ b/read-cache.c
>> @@ -1755,8 +1755,7 @@ static unsigned long expand_name_field(struct
>> strbuf *name, const char *cp_)
>> if (name->len < len)
>> die("malformed name field in the index");
>> strbuf_remove(name, name->len - len, len);
>> - for (ep = cp; *ep; ep++)
>> - ; /* find the end */
>> + ep = cp + strlen(cp);
>> strbuf_add(name, cp, ep - cp);
>> return (const char *)ep + 1 - cp_;
>> }
>
> No try this instead. It's half way back to v2 numbers for me (tested
> with "test-tool read-cache 100" on webkit.git). For the record, v4 is
> about 30% slower than v2 in my tests.
> Thanks Duy, this helped on my system as well.
Interestingly, simply reading the cache tree extension in read_one() now takes about double the CPU on the primary thread as does load_cache_entries().
Hmm, that gives me an idea. I could kick off another thread to load that extension in parallel and cut off another ~160 ms. I'll add that to my list of future patches to investigate...
Show 23 quoted lines
> We could probably do better too. Instead of preparing the string in a
> separate buffer (previous_name_buf), we could just assemble it directly
> to the newly allocated "ce".
>
> -- 8< --
> diff --git a/read-cache.c b/read-cache.c
> index 7b1354d759..237f60a76c 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -1754,9 +1754,8 @@ static unsigned long expand_name_field(struct strbuf *name, const char *cp_)
>
> if (name->len < len)
> die("malformed name field in the index");
> - strbuf_remove(name, name->len - len, len);
> - for (ep = cp; *ep; ep++)
> - ; /* find the end */
> + strbuf_setlen(name, name->len - len);
> + ep = cp + strlen(cp);
> strbuf_add(name, cp, ep - cp);
> return (const char *)ep + 1 - cp_;
> }
> -- 8< --
>