From: Jeff King <peff@peff.net>
Date: Mon, 04 Jan 2010 08:09:40 GMT
Subject: Re: [PATCH] grep: do not do external grep on skip-worktree entries
Message-ID: <20100104080940.GA4815@coredump.intra.peff.net>
In-Reply-To: <20100104072659.GA10611@coredump.intra.peff.net>

On Mon, Jan 04, 2010 at 02:26:59AM -0500, Jeff King wrote:

> The pcre analysis there came from just using the "pcreposix" header, I
> think. From my limited research, modern pcre may have some tuning
> options (including a DFA engine!) that could do a lot better.

Hmm. I was able to get some improvements by using pcre_dfa_exec, but
still not as good as external grep. For "git grep 'foo.*bar'" in the
linux-2.6 repo, I got:

  external grep: 0.76s
  pcre_dfa_exec: 1.85s
      pcre_exec: 3.21s
          glibc: 4.00s

However, gprof reports that for the pcre dfa case, we spend more time in
grep.c:end_of_line than we do actually running the regex. So clearly
there are some other micro-optimizations in GNU grep that are making a
difference, too.

By the way, you can see the abysmal performance of our internal code by
doing a "git grep foo". It uses the "fixed" internal engine and weighs
in at 3.24s on the same machine, _slower_ than pcre doing an actual
regex.

-Peff