From: Ben Peart Date: Mon, 17 Sep 2018 17:25:42 GMT Subject: Re: [PATCH v5 3/5] read-cache: load cache entries on worker threads Message-ID: In-Reply-To: On 9/15/2018 6:31 AM, Duy Nguyen wrote: > On Wed, Sep 12, 2018 at 6:18 PM Ben Peart wrote: >> >> This patch helps address the CPU cost of loading the index by creating >> multiple threads to divide the work of loading and converting the cache >> entries across all available CPU cores. >> >> It accomplishes this by having the primary thread loop across the index file >> tracking the offset and (for V4 indexes) expanding the name. It creates a >> thread to process each block of entries as it comes to them. >> >> I used p0002-read-cache.sh to generate some performance data: >> >> Test w/100,000 files Baseline Parallel entries >> --------------------------------------------------------------------------- >> read_cache/discard_cache 1000 times 14.08(0.01+0.10) 9.72(0.03+0.06) -31.0% >> >> Test w/1,000,000 files Baseline Parallel entries >> ------------------------------------------------------------------------------ >> read_cache/discard_cache 1000 times 202.95(0.01+0.07) 154.14(0.03+0.06) -24.1% > > The numbers here and the previous patch to load extensions in parallel > are exactly the same. What do these numbers mean? With both changes? > It means I messed up when creating my commit message for the extension patch and copy/pasted the wrong numbers. Yes, these numbers are with both changes (the correct numbers for the extension only are not as good).