From: Nguyen Thai Ngoc Duy Date: Sat, 24 Dec 2011 10:49:42 GMT Subject: Re: [PATCH v2 3/3] grep: disable threading in all but worktree case Message-ID: In-Reply-To: <20111224070715.GA32267@sigill.intra.peff.net> On Sat, Dec 24, 2011 at 2:07 PM, Jeff King wrote: > I tried to get some timings for this, but ran across some quite > surprising results. Here's a simple grep of the linux-2.6 working tree, > using a single-threaded grep: > >  $ time git grep SIMPLE >/dev/null >  real    0m0.439s >  user    0m0.272s >  sys     0m0.160s > > and then the same thing, via xargs, without even turning on > parallelization. This should give us a measurement of the overhead for > going through xargs at all. We'd expect it to be slower, but not too > much so: > >  $ time git ls-files | xargs git grep SIMPLE -- >/dev/null >  real    0m11.989s >  user    0m11.769s >  sys     0m0.268s > > Twenty-five times slower! Running 'perf' reports the culprit as pathspec > matching: > >  +  63.23%    git  git                 [.] match_pathspec_depth >  +  28.60%    git  libc-2.13.so        [.] __strncmp_sse42 >  +   2.22%    git  git                 [.] strncmp@plt >  +   1.67%    git  git                 [.] kwsexec > > where the strncmps are called as part of match_pathspec_depth. So over > 90% of the CPU time is spent on matching the pathspecs, compared to less > than 2% actually grepping. > > Which really makes me wonder if our pathspec matching could stand to be > faster. True, giving a bunch of single files is the least efficient way > to use pathspecs, but that's pretty amazingly slow. We could eliminate get_pathspec_depth() in grep_directory() when read_directory() learns to filter path properly using (and at the cost of) tree_entry_interesting(). The latter function has more optimizaions built in and should be faster than the former. This is a good test case for my read_directory() rewrite. Thanks. get_pathspec_depth() can still use some optimizations though for grep_cache() case. -- Duy