Re: [PATCH v3] Threaded grep

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Jan 19, 2010, 15:41 UTC
Message-ID: <alpine.LFD.2.00.1001190728300.13231@localhost.localdomain>
In-Reply-To: <20100119073454.GA15575@fredrik-laptop>

On Tue, 19 Jan 2010, Fredrik Kuivinen wrote:

Show 12 quoted lines

> > 
> >  - git grep --no-ext-grep function
> > 
> > 	real	0m0.241s
> > 	user	0m1.436s
> > 	sys	0m0.216s
> 
> I haven't seen this overhead during my tests. But I'm _guessing_ that
> the problem is that the mutex grep_lock is held while the result is
> written to stdout. So when we are writing, no significant amount of
> work can be done by any thread. Here is a patch to fix this (applies
> on to of v3).

No, I get basically the same timings with this patch:

	real	0m0.239s
	user	0m1.372s
	sys	0m0.280s

(that _may_ be a slight real improvement, but it's more likely that the changing in user/sys time is just noise).

But I do think that you're right that there's a difference in my timings, and I am comparing to the one that uses strstr(). Your patches didn't have a "disable threads" option that I could see, so I just compared against my old numbers.

So I guess a better example is to use a "complex" grep pattern like 'function.*a' which also gets a lot of hits, and the threaded case does look much better in comparison. Pre-threaded:

	real	0m0.921s
	user	0m0.728s
	sys	0m0.184s

so it's less than 2x the CPU time, and a 3.85x real-time improvement. And that "less than 2x the CPU time" is the factor I'd expect from HT.

It's also worth noting that at least _some_ versions of glibc would actually use different versions of subroutines for the threaded vs non-threaded case, ie they'd avoid locking overhead when they know they aren't running with threads. So things like stdio actually got slower when you did any pthreads things, because suddenly glibc knew that it needed to do locking around the functions.

			Linus