Re: [v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)

From: René Scharfe <l.s.r@web.de>
Date: Jan 30, 2022, 13:32 UTC
Message-ID: <b74f781c-548b-5254-d3d1-fc1873c70e10@web.de>
In-Reply-To: <20220130090422.GA4769@szeder.dev>

Am 30.01.22 um 10:04 schrieb SZEDER Gábor:

Show 17 quoted lines

> On Sun, Jan 30, 2022 at 08:55:02AM +0100, René Scharfe wrote:
>> e0c6029 (Fix inifinite loop when a single byte newline is searched in
>> JIT., 2020-05-29) [1] sounds like it might have fixed it.  It's part of
>> version 10.36.
>
> I saw this hang on two Ubuntu 20.04 based boxes, which predate that
> fix you mention only by a month or two, and apparently the almost two
> years since then was not enough for this fix to trickle down into
> updated 20.04 pcre packages, because:
>
>> Do you still get the error when you disable JIT, i.e. when you use the
>> pattern "(*NO_JIT)^\s" instead?
>
> No, with this pattern it works as expected.
>
> So is there a more convenient way to disable PCRE JIT in Git?  FWIW,
> (non-git) 'grep -P' works with the same patterns.

I don't know a better way.  We could do it automatically, though:

--- >8 ---
Subject: [PATCH] grep: disable JIT on PCRE2 before 10.36 to avoid endless loop

Commit e0c6029 (Fix inifinite loop when a single byte newline is searched in JIT., 2020-05-29) of PCRE2 adds the following point to its ChangeLog for version 10.36:

  2. Fix inifinite loop when a single byte newline is searched in JIT when
  invalid utf8 mode is enabled.

Avoid that bug on older versions (which are still reportedly found in the wild) by disabling the JIT when handling UTF-8.

Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
Not sure how to test it.  Killing git grep after a second or so seems a
bit clumsy.  timeout(1) from GNU coreutils at least allows doing that
from the shell, but it's not a standard tool.  Perhaps we need a new
test helper for that purpose?

 grep.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/grep.c b/grep.c
index 7bb0360869..16629a2301 100644
--- a/grep.c
+++ b/grep.c
@@ -406,6 +406,14 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
 	}

 	pcre2_config(PCRE2_CONFIG_JIT, &p->pcre2_jit_on);
+#ifndef GIT_PCRE2_VERSION_10_36_OR_HIGHER
+	/*
+	 * Work around the bug fixed by e0c6029 (Fix inifinite loop when a
+	 * single byte newline is searched in JIT., 2020-05-29).
+	 */
+	if (options & PCRE2_MATCH_INVALID_UTF)
+		p->pcre2_jit_on = 0;
+#endif
 	if (p->pcre2_jit_on) {
 		jitret = pcre2_jit_compile(p->pcre2_pattern, PCRE2_JIT_COMPLETE);
 		if (jitret)
--
2.35.0