Re: [PATCH v5 3/5] read-cache: load cache entries on worker threads

From: Duy Nguyen <pclouds@gmail.com>
Date: Sep 15, 2018, 11:29 UTC
Message-ID: <CACsJy8AhNQhFa1ONsmnLOjznbZss2=L=xCPD5fa2vdHw79x0ag@mail.gmail.com>
In-Reply-To: <20180912161832.55324-4-benpeart@microsoft.com>

On Wed, Sep 12, 2018 at 6:18 PM Ben Peart <benpeart@microsoft.com> wrote:

Show 9 quoted lines

>  #ifndef NO_PTHREADS
>         nr_threads = git_config_get_index_threads();
> -       if (!nr_threads)
> -               nr_threads = online_cpus();
> +       if (!nr_threads) {
> +               cpus = online_cpus();
> +               nr_threads = istate->cache_nr / THREAD_COST;
> +               if (nr_threads > cpus)
> +                       nr_threads = cpus;

It seems like overcommitting cpu does reduce time. With this patch (and a 4 core system), I got

$ test-tool read-cache 100 real 0m36.270s user 0m54.193s sys 0m17.346s

if I force nr_threads to 9 (even though cpus is 4)

$ test-tool read-cache 100 real 0m33.592s user 1m4.230s sys 0m18.380s

Even though we use more cpus, real time is shorter. I guess these threads still sleep a bit due to I/O and having more threads than cores will utilize those idle cycles. -- Duy