Re: [PATCH v2 2/2] index-pack: support multithreaded delta resolving

From: Thomas Rast <trast@inf.ethz.ch>
Date: Mar 12, 2012, 11:47 UTC
Message-ID: <87boo282hg.fsf@thomas.inf.ethz.ch>
In-Reply-To: <CACsJy8CyLgKrEAriS-uLU9qG6ahBeQ6qLbvHiLvsbTjMZsMW+g@mail.gmail.com>

Nguyen Thai Ngoc Duy <pclouds@gmail.com> writes:

Show 14 quoted lines

> 2012/3/12 Thomas Rast <trast@inf.ethz.ch>:
>> Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
>>
>>> This puts delta resolving on each base on a separate thread, one base
>>> cache per thread. Per-thread data is grouped in struct thread_local.
>>> When running with nr_threads == 1, no pthreads calls are made. The
>>> system essentially runs in non-thread mode.
>>
>> As discussed when we took the git-grep measurements, it may be
>> interesting to have a way to run 1 thread.  Can you put in such an
>> option?
>
> Sorry I wasn't clear, nr_threads == 1 is equivalent to --threads=1. So
> yes it supports running in non-thread mode.

Well, in that case I wasn't clear: I meant that there should be a way to run with the whole threading machinery enabled, but still only have one thread (doing the work, possibly having another that fills the queue).

That allows us to see how big the overhead is.

Show 13 quoted lines

>> Oh, and could you write a perf test for this? :-)
>
> Yeah, about that, index-pack is mostly used as part of git-fetch or
> git-clone. Maybe we need to add --threads to those commands too, then
> we can see how clone/fetch performs. I'll need such tests anyway if
> I'm going to push for cheaper connectivity check in git-fetch in
> another thread.
>
> I guess one test with --threads=1, one with threads=2 and one without
> --threads. Any ideas? We can try testing it on half available cores,
> all cores, double available cores, but that would require exporting
> online_cpus(), perhaps via test command. I didn't see grep --threads
> perf test either (wanted to use it as template..)

A simple one is in t/perf/p7810-grep.sh (in master already). It doesn't test threads though. For index-pack you'll also have to find a good way to choose a pack, perhaps simply the biggest one in the repo.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch