git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Jan 31, 2022, 21:01 UTC
Message-ID
<220131.868ruvbovo.gmgdl@evledraar.gmail.com>
In-Reply-To
<b74f781c-548b-5254-d3d1-fc1873c70e10@web.de>
On Sun, Jan 30 2022, René Scharfe wrote:
Show 56 quoted lines
> Am 30.01.22 um 10:04 schrieb SZEDER Gábor:
>> On Sun, Jan 30, 2022 at 08:55:02AM +0100, René Scharfe wrote:
>>> e0c6029 (Fix inifinite loop when a single byte newline is searched in
>>> JIT., 2020-05-29) [1] sounds like it might have fixed it.  It's part of
>>> version 10.36.
>>
>> I saw this hang on two Ubuntu 20.04 based boxes, which predate that
>> fix you mention only by a month or two, and apparently the almost two
>> years since then was not enough for this fix to trickle down into
>> updated 20.04 pcre packages, because:
>>
>>> Do you still get the error when you disable JIT, i.e. when you use the
>>> pattern "(*NO_JIT)^\s" instead?
>>
>> No, with this pattern it works as expected.
>>
>> So is there a more convenient way to disable PCRE JIT in Git?  FWIW,
>> (non-git) 'grep -P' works with the same patterns.
>
> I don't know a better way.  We could do it automatically, though:
>
> --- >8 ---
> Subject: [PATCH] grep: disable JIT on PCRE2 before 10.36 to avoid endless loop
>
> Commit e0c6029 (Fix inifinite loop when a single byte newline is
> searched in JIT., 2020-05-29) of PCRE2 adds the following point to its
> ChangeLog for version 10.36:
>
>   2. Fix inifinite loop when a single byte newline is searched in JIT when
>   invalid utf8 mode is enabled.
>
> Avoid that bug on older versions (which are still reportedly found in
> the wild) by disabling the JIT when handling UTF-8.
>
> Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
> Not sure how to test it.  Killing git grep after a second or so seems a
> bit clumsy.  timeout(1) from GNU coreutils at least allows doing that
> from the shell, but it's not a standard tool.  Perhaps we need a new
> test helper for that purpose?
>
>  grep.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/grep.c b/grep.c
> index 7bb0360869..16629a2301 100644
> --- a/grep.c
> +++ b/grep.c
> @@ -406,6 +406,14 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
>  	}
>
>  	pcre2_config(PCRE2_CONFIG_JIT, &p->pcre2_jit_on);
> +#ifndef GIT_PCRE2_VERSION_10_36_OR_HIGHER
> +	/*
> +	 * Work around the bug fixed by e0c6029 (Fix inifinite loop when a

Better to quote this as PhilipHazel/pcre2@e0c6029 or something, i.e. to indicate that it's not git.git's commit.

> +	 * single byte newline is searched in JIT., 2020-05-29).
> +	 */
> +	if (options & PCRE2_MATCH_INVALID_UTF)
> +		p->pcre2_jit_on = 0;

It seems rather heavy-hande, but I can't think of a better way to deal with this, i.e. if we selectively use JIT on older versions, surely we run into the match-bytes-but-want-chars bug you were fixing.

> +#endif
>  	if (p->pcre2_jit_on) {
>  		jitret = pcre2_jit_compile(p->pcre2_pattern, PCRE2_JIT_COMPLETE);
>  		if (jitret)
Previous: René ScharfeNext: René Scharfe
Message 12 of 17 in “grep/pcre2: use PCRE2_UTF even with ASCII patterns”
  1. 1/2 grep/pcre2: use PCRE2_UTF even with ASCII patternsRené Scharfe, Dec 18, 2021
  2. 2/2 grep/pcre2: factor out literal variableRené Scharfe, Dec 18, 2021
  3. Ævar Arnfjörð BjarmasonDec 19, 2021
  4. Junio C HamanoDec 20, 2021
  5. Ævar Arnfjörð BjarmasonDec 20, 2021
  6. Junio C HamanoDec 20, 2021
  7. Junio C HamanoDec 20, 2021
  8. [v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)SZEDER Gábor, Jan 29, 2022
  9. René ScharfeJan 30, 2022
  10. SZEDER GáborJan 30, 2022
  11. René ScharfeJan 30, 2022
  12. Ævar Arnfjörð BjarmasonJan 31, 2022
  13. René ScharfeFeb 5, 2022
  14. SZEDER GáborFeb 6, 2022
  15. Ævar Arnfjörð BjarmasonFeb 12, 2022
  16. René ScharfeFeb 17, 2022
  17. Junio C HamanoFeb 17, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.