[PATCH v3 0/3] grep attributes and multithreading

From: Thomas Rast <trast@student.ethz.ch>
Date: Dec 12, 2011, 21:16 UTC
Message-ID: <cover.1323723759.git.trast@student.ethz.ch>
In-Reply-To: <4ED8F9AE.8030605@lsrfire.ath.cx>

I think we should finish up these three patches for the next release. The first two are unchanged from last time; nobody seemed to have any objections.

On the third one I'm following René's argument:

Show 10 quoted lines

> Am 02.12.2011 14:07, schrieb Thomas Rast:
> > So disable threading entirely when not scanning the worktree, to get
> > the NO_PTHREADS performance in that case.  This obsoletes all code
> > related to grep_sha1_async.  The thread startup must be delayed until
> > after all arguments have been parsed, but this does not have a
> > measurable effect.
> 
> This is a bit radical.  I think the underlying issue that 
> read_sha1_file() is not thread-safe can be solved eventually and then 
> we'd need to readd that code.

I already posted a bunch of POC patches, but doing the timing manually has been getting on my nerves lately. So I would first like to formalize some of the performance testing, perhaps along lines already drawn up by Michael Hagger, perhaps not. Then we can revisit the issue of grep performance. But I would prefer not to block the -W fix and two easy and confirmed speedups on that.

I dropped this part entirely:

> How about adding a parameter to control the number of threads 
> (--threads?) instead that defaults to eight (or five) for the worktree 
> and one for the rest?  That would also make benchmarking easier.

It does make testing a lot easier, but the interface is IMHO not fit for users and I have a feeling that the "right" for-debugging interface will end up falling out of the performance testing work (probably an environment variable). The end-user option should be a config setting, if any.

Thomas Rast (3):
  grep: load funcname patterns for -W
  grep: enable threading with -p and -W using lazy attribute lookup
  grep: disable threading in non-worktree case

 builtin/grep.c  |   34 +++++++++++++++----------
 grep.c          |   73 ++++++++++++++++++++++++++++++++++---------------------
 grep.h          |    7 +++++
 t/t7810-grep.sh |   14 ++++++++++
 4 files changed, 86 insertions(+), 42 deletions(-)

-- 
1.7.8.431.g2abf2