Re: [PATCH v5 3/5] read-cache: load cache entries on worker threads
- From
Duy Nguyen <pclouds@gmail.com>
- Date
- Sep 15, 2018, 10:31 UTC
- Message-ID
- <CACsJy8CjWD_CHwv5NoURLt9is7-UUDpKDo-3EcM28imznZAOpA@mail.gmail.com>
- In-Reply-To
- <20180912161832.55324-4-benpeart@microsoft.com>
On Wed, Sep 12, 2018 at 6:18 PM Ben Peart <benpeart@microsoft.com> wrote:
Show 18 quoted lines
> > This patch helps address the CPU cost of loading the index by creating > multiple threads to divide the work of loading and converting the cache > entries across all available CPU cores. > > It accomplishes this by having the primary thread loop across the index file > tracking the offset and (for V4 indexes) expanding the name. It creates a > thread to process each block of entries as it comes to them. > > I used p0002-read-cache.sh to generate some performance data: > > Test w/100,000 files Baseline Parallel entries > --------------------------------------------------------------------------- > read_cache/discard_cache 1000 times 14.08(0.01+0.10) 9.72(0.03+0.06) -31.0% > > Test w/1,000,000 files Baseline Parallel entries > ------------------------------------------------------------------------------ > read_cache/discard_cache 1000 times 202.95(0.01+0.07) 154.14(0.03+0.06) -24.1%
The numbers here and the previous patch to load extensions in parallel are exactly the same. What do these numbers mean? With both changes?
-- Duy