Re: [PATCH v3] Threaded grep

On Mon, 18 Jan 2010, Fredrik Kuivinen wrote:

Ok, this is much better.

On my machine (4 cores with HT, so 8 threads total), I get the following:

 [ Note: the --no-ext-grep is because I'm comparing with a git that has 
  the original grep optimization, but hasn't removed the external grep 
  functionality yet. I just used the same command line when then comparing 
  to next+your patch, so don't mind it.

  Also, the three examples are chosen to be: "trivial single match", 
  "regex single match", and "trivial lots of matches" ]

Before (all best-of-five), with the non-threaded internal grep:

 - git grep --no-ext-grep qwerty

	real	0m0.311s
	user	0m0.164s
	sys	0m0.144s

 - git grep --no-ext-grep qwerty.*as

	real	0m0.555s
	user	0m0.416s
	sys	0m0.132s

 - git grep --no-ext-grep function

	real	0m0.461s
	user	0m0.276s
	sys	0m0.180s

After, with threading:

 - git grep --no-ext-grep qwerty

	real	0m0.152s
	user	0m0.788s
	sys	0m0.212s

 - git grep --no-ext-grep qwerty.*as

	real	0m0.148s
	user	0m0.724s
	sys	0m0.284s

 - git grep --no-ext-grep function

	real	0m0.241s
	user	0m1.436s
	sys	0m0.216s

so now it's a clear win in all cases. And as you'd expect, the "complex case with single output" is the one that wins most, since it's the one that should have the most CPU usage, with the least synchronization.

That said, it still does waste quite a bit of time doing this, and while it almost doubles performance in the last case too, it does so at the cost of quite a bit of overhead.

(Some overhead is natural, especially since I have HT. Running two threads on the same core does _not_ give double the performance, so the CPU time almost doubling - because the threads themselves run slower - is not unexpected. It's when the CPU time more than quadruples that I suspect that it could be improved a bit).

		Linus