Re: [PATCH v2 2/2] index-pack: support multithreaded delta resolving

From: Thomas Rast <trast@inf.ethz.ch>
Date: Mar 12, 2012, 10:57 UTC
Message-ID: <87y5r684t6.fsf@thomas.inf.ethz.ch>
In-Reply-To: <1331519549-28090-3-git-send-email-pclouds@gmail.com>

Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:

> This puts delta resolving on each base on a separate thread, one base
> cache per thread. Per-thread data is grouped in struct thread_local.
> When running with nr_threads == 1, no pthreads calls are made. The
> system essentially runs in non-thread mode.

As discussed when we took the git-grep measurements, it may be interesting to have a way to run 1 thread. Can you put in such an option?

Show 14 quoted lines

> An experiment on a Xeon 24 core machine with linux-2.6.git shows that
> performance does not increase proportional to the number of cores. So
> by default, we use maximum 3 cores. Some numbers with --threads from 1
> to 16:
>
> 1..4
> real    1m16.310s  0m48.183s  0m37.866s  0m32.834s
> user    1m13.773s  1m15.537s  1m15.781s  1m16.233s
> sys     0m2.480s   0m3.936s   0m4.448s   0m4.852s
>
> 5..8
> real    0m33.170s  0m30.369s  0m28.406s  0m26.968s
> user    1m31.474s  1m30.322s  1m29.562s  1m28.694s
> sys     0m6.096s   0m6.268s   0m6.684s   0m7.172s

Interesting. Is this a real 24-core machine or 12*2 hyperthreaded? Does it use Turbo Boost and how far (how fast and on how many cores simultaneously) does that go?

I'm asking because if Turbo Boost starts to wear off around 4 cores, like these measurements suggest, then it may not be beneficial to spawn threads on 2*2HT CPUs (found in many laptops) where Turbo Boost only really works if you only use a single core.

Oh, and could you write a perf test for this? :-)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch