git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 3/8] grep: stop using a custom JIT stack with PCRE v1

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Jul 26, 2019, 15:08 UTC
Message-ID
<20190726150818.6373-4-avarab@gmail.com>
In-Reply-To
<20190724151415.3698-1-avarab@gmail.com>

Simplify the PCRE v1 code for the same reasons as for the PCRE v2 code in the last commit. Unlike with v2 we actually used the custom stack in v1, but let's use PCRE's built-in 32 KB one instead, since experience with v2 shows that's enough. Most distros are already using v2 as a default, and the underlying sljit code is the same.

Unfortunately we can't just pass a NULL to pcre_jit_exec() as with pcre2_jit_match(). Unlike the v2 function it doesn't support that. Instead we need to use the fatter pcre_exec() if we'd like the same behavior.

This will make things slightly slower than on the fast-path function, but it's OK since we care less about v1 performance these days since we have and recommend v2. Running a similar performance test as what I ran in fbaceaac47 ("grep: add support for the PCRE v1 JIT API", 2017-05-25) via:

    GIT_PERF_REPEAT_COUNT=30 GIT_PERF_LARGE_REPO=~/g/linux GIT_PERF_MAKE_OPTS='-j8 USE_LIBPCRE1=Y CFLAGS=-O3 LIBPCREDIR=/home/avar/g/pcre/inst' ./run HEAD~ HEAD p7820-grep-engines.sh
Gives us this, just the /perl/ results:
    Test                                            HEAD~             HEAD
    ---------------------------------------------------------------------------------------
    7820.3: perl grep 'how.to'                      0.19(0.67+0.52)   0.19(0.65+0.52) +0.0%
    7820.7: perl grep '^how to'                     0.19(0.78+0.44)   0.19(0.72+0.49) +0.0%
    7820.11: perl grep '[how] to'                   0.39(2.13+0.43)   0.40(2.10+0.46) +2.6%
    7820.15: perl grep '(e.t[^ ]*|v.ry) rare'       0.44(2.55+0.37)   0.45(2.47+0.41) +2.3%
    7820.19: perl grep 'm(ú|u)lt.b(æ|y)te'          0.23(1.06+0.42)   0.22(1.03+0.43) -4.3%

It will also implicitly re-enable UTF-8 validation for PCRE v1. As noted in [1] we now have cases as a result where PCRE v1 is more eager to error out. Subsequent patches will fix that for v2, and I think it's fair to tell v1 users "just upgrade" and not worry about that edge case for v1.

1.  https://public-inbox.org/git/CAPUEsphZJ_Uv9o1-yDpjNLA_q-f7gWXz9g1gCY2pYAYN8ri40g@mail.gmail.com/
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 grep.c | 28 +++++-----------------------
 grep.h |  5 -----
 2 files changed, 5 insertions(+), 28 deletions(-)
diff --git a/grep.c b/grep.c
index 4b1e917ac5..9c2b259771 100644
--- a/grep.c
+++ b/grep.c
@@ -394,12 +394,6 @@ static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)
 
 #ifdef GIT_PCRE1_USE_JIT
 	pcre_config(PCRE_CONFIG_JIT, &p->pcre1_jit_on);
-	if (p->pcre1_jit_on) {
-		p->pcre1_jit_stack = pcre_jit_stack_alloc(1, 1024 * 1024);
-		if (!p->pcre1_jit_stack)
-			die("Couldn't allocate PCRE JIT stack");
-		pcre_assign_jit_stack(p->pcre1_extra_info, NULL, p->pcre1_jit_stack);
-	}
 #endif
 }
 
@@ -411,18 +405,9 @@ static int pcre1match(struct grep_pat *p, const char *line, const char *eol,
 	if (eflags & REG_NOTBOL)
 		flags |= PCRE_NOTBOL;
 
-#ifdef GIT_PCRE1_USE_JIT
-	if (p->pcre1_jit_on) {
-		ret = pcre_jit_exec(p->pcre1_regexp, p->pcre1_extra_info, line,
-				    eol - line, 0, flags, ovector,
-				    ARRAY_SIZE(ovector), p->pcre1_jit_stack);
-	} else
-#endif
-	{
-		ret = pcre_exec(p->pcre1_regexp, p->pcre1_extra_info, line,
-				eol - line, 0, flags, ovector,
-				ARRAY_SIZE(ovector));
-	}
+	ret = pcre_exec(p->pcre1_regexp, p->pcre1_extra_info, line,
+			eol - line, 0, flags, ovector,
+			ARRAY_SIZE(ovector));
 
 	if (ret < 0 && ret != PCRE_ERROR_NOMATCH)
 		die("pcre_exec failed with error code %d", ret);
@@ -439,14 +424,11 @@ static void free_pcre1_regexp(struct grep_pat *p)
 {
 	pcre_free(p->pcre1_regexp);
 #ifdef GIT_PCRE1_USE_JIT
-	if (p->pcre1_jit_on) {
+	if (p->pcre1_jit_on)
 		pcre_free_study(p->pcre1_extra_info);
-		pcre_jit_stack_free(p->pcre1_jit_stack);
-	} else
+	else
 #endif
-	{
 		pcre_free(p->pcre1_extra_info);
-	}
 	pcre_free((void *)p->pcre1_tables);
 }
 #else /* !USE_LIBPCRE1 */
diff --git a/grep.h b/grep.h
index 4d8e300175..ce2d72571f 100644
--- a/grep.h
+++ b/grep.h
@@ -14,13 +14,9 @@
 #ifndef GIT_PCRE_STUDY_JIT_COMPILE
 #define GIT_PCRE_STUDY_JIT_COMPILE 0
 #endif
-#if PCRE_MAJOR <= 8 && PCRE_MINOR < 20
-typedef int pcre_jit_stack;
-#endif
 #else
 typedef int pcre;
 typedef int pcre_extra;
-typedef int pcre_jit_stack;
 #endif
 #ifdef USE_LIBPCRE2
 #define PCRE2_CODE_UNIT_WIDTH 8
@@ -85,7 +81,6 @@ struct grep_pat {
 	regex_t regexp;
 	pcre *pcre1_regexp;
 	pcre_extra *pcre1_extra_info;
-	pcre_jit_stack *pcre1_jit_stack;
 	const unsigned char *pcre1_tables;
 	int pcre1_jit_on;
 	pcre2_code *pcre2_pattern;
-- 
2.22.0.455.g172b71a6c5
Previous: Carlo ArenasNext: Carlo Arenas
Message 8 of 70 in “grep: use custom JIT stack with pcre2”
  1. grep: use custom JIT stack with pcre2Carlo Marcelo Arenas Belón, Jul 21, 2019
  2. 0/3 grep: PCRE JIT fixesÆvar Arnfjörð Bjarmason, Jul 24, 2019
  3. Junio C HamanoJul 24, 2019
  4. Ævar Arnfjörð BjarmasonJul 24, 2019
  5. 1/8 grep: remove overly paranoid BUG(...) codeÆvar Arnfjörð Bjarmason, Jul 26, 2019
  6. 2/8 grep: stop "using" a custom JIT stack with PCRE v2Ævar Arnfjörð Bjarmason, Jul 26, 2019
  7. Carlo ArenasJul 29, 2019
  8. 3/8 grep: stop using a custom JIT stack with PCRE v1Ævar Arnfjörð Bjarmason, Jul 26, 2019
  9. Carlo ArenasJul 29, 2019
  10. 4/8 grep: consistently use "p->fixed" in compile_regexp()Ævar Arnfjörð Bjarmason, Jul 26, 2019
  11. Carlo ArenasJul 29, 2019
  12. Ævar Arnfjörð BjarmasonJul 29, 2019
  13. Ævar Arnfjörð BjarmasonJul 29, 2019
  14. Junio C HamanoJul 29, 2019
  15. 5/8 grep: create a "is_fixed" member in "grep_pat"Ævar Arnfjörð Bjarmason, Jul 26, 2019
  16. 7/8 grep: do not enter PCRE2_UTF mode on fixed matchingÆvar Arnfjörð Bjarmason, Jul 26, 2019
  17. Junio C HamanoJul 26, 2019
  18. 6/8 grep: stess test PCRE v2 on invalid UTF-8 dataÆvar Arnfjörð Bjarmason, Jul 26, 2019
  19. Junio C HamanoJul 26, 2019
  20. Ævar Arnfjörð BjarmasonJul 26, 2019
  21. Carlo ArenasJul 29, 2019
  22. t7812: add missing redirectsAndreas Schwab, Nov 26, 2019
  23. Johannes SchindelinNov 26, 2019
  24. Andreas SchwabNov 26, 2019
  25. Jeff KingNov 27, 2019
  26. t7812: expect failure for grep -i with invalid UTF-8 dataTodd Zullinger, Nov 30, 2019
  27. Andreas SchwabNov 30, 2019
  28. Junio C HamanoDec 1, 2019
  29. Andreas SchwabDec 1, 2019
  30. Todd ZullingerDec 1, 2019
  31. Junio C HamanoDec 2, 2019
  32. 0/8 grep: PCRE JIT fixes + ab/no-kwset fixÆvar Arnfjörð Bjarmason, Jul 26, 2019
  33. Junio C HamanoJul 26, 2019
  34. Ævar Arnfjörð BjarmasonJul 29, 2019
  35. Junio C HamanoJul 29, 2019
  36. 8/8 grep: optimistically use PCRE2_MATCH_INVALID_UTFÆvar Arnfjörð Bjarmason, Jul 26, 2019
  37. Junio C HamanoJul 26, 2019
  38. Ævar Arnfjörð BjarmasonJul 26, 2019
  39. Ævar Arnfjörð BjarmasonJul 26, 2019
  40. 0/4 grep: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  41. 1/2 grep/pcre2 tests: don't rely on invalid UTF-8 data testÆvar Arnfjörð Bjarmason, Jan 24, 2021
  42. 2/2 grep/pcre2: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  43. Ramsay JonesJan 24, 2021
  44. Ramsay JonesJan 24, 2021
  45. Ævar Arnfjörð BjarmasonJan 24, 2021
  46. Ramsay JonesJan 24, 2021
  47. Ævar Arnfjörð BjarmasonJan 24, 2021
  48. 0/2 grep: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  49. 0/2 grep: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  50. 1/2 grep/pcre2 tests: don't rely on invalid UTF-8 data testÆvar Arnfjörð Bjarmason, Jan 24, 2021
  51. 2/2 grep/pcre2: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  52. 1/4 grep/pcre2 tests: don't rely on invalid UTF-8 data testÆvar Arnfjörð Bjarmason, Jan 24, 2021
  53. 4/4 grep/pcre2: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason, Jan 24, 2021
  54. 3/4 grep/pcre2: further simplify boolean spaghettiÆvar Arnfjörð Bjarmason, Jan 24, 2021
  55. 2/4 grep/pcre2: simplify boolean spaghettiÆvar Arnfjörð Bjarmason, Jan 24, 2021
  56. Junio C HamanoJan 24, 2021
  57. Johannes SixtJan 24, 2021
  58. 2/3 grep: stop "using" a custom JIT stack with PCRE v2Ævar Arnfjörð Bjarmason, Jul 24, 2019
  59. Junio C HamanoJul 24, 2019
  60. Ævar Arnfjörð BjarmasonJul 24, 2019
  61. Carlo ArenasJul 25, 2019
  62. 1/3 grep: remove overly paranoid BUG(...) codeÆvar Arnfjörð Bjarmason, Jul 24, 2019
  63. 3/3 grep: stop using a custom JIT stack with PCRE v1Ævar Arnfjörð Bjarmason, Jul 24, 2019
  64. Carlo ArenasJul 26, 2019
  65. Ævar Arnfjörð BjarmasonJul 26, 2019
  66. Carlo ArenasJul 26, 2019
  67. Ævar Arnfjörð BjarmasonJul 26, 2019
  68. 0/2 PCRE1 cleanupCarlo Marcelo Arenas Belón, Jul 26, 2019
  69. 1/2 grep: make sure NO_LIBPCRE1_JIT disable JIT in PCRE1Carlo Marcelo Arenas Belón, Jul 26, 2019
  70. 2/2 grep: refactor and simplify PCRE1 supportCarlo Marcelo Arenas Belón, Jul 26, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.