Re: [v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)

>
> On Sun, Jan 30 2022, René Scharfe wrote:
>
>> Am 30.01.22 um 10:04 schrieb SZEDER Gábor:
>>> On Sun, Jan 30, 2022 at 08:55:02AM +0100, René Scharfe wrote:
>>>> e0c6029 (Fix inifinite loop when a single byte newline is searched in
>>>> JIT., 2020-05-29) [1] sounds like it might have fixed it.  It's part of
>>>> version 10.36.
>>>
>>> I saw this hang on two Ubuntu 20.04 based boxes, which predate that
>>> fix you mention only by a month or two, and apparently the almost two
>>> years since then was not enough for this fix to trickle down into
>>> updated 20.04 pcre packages, because:
>>>
>>>> Do you still get the error when you disable JIT, i.e. when you use the
>>>> pattern "(*NO_JIT)^\s" instead?
>>>
>>> No, with this pattern it works as expected.
>>>
>>> So is there a more convenient way to disable PCRE JIT in Git?  FWIW,
>>> (non-git) 'grep -P' works with the same patterns.
>>
>> I don't know a better way.  We could do it automatically, though:
>>
>> --- >8 ---
>> Subject: [PATCH] grep: disable JIT on PCRE2 before 10.36 to avoid endless loop
>>
>> Commit e0c6029 (Fix inifinite loop when a single byte newline is
>> searched in JIT., 2020-05-29) of PCRE2 adds the following point to its
>> ChangeLog for version 10.36:
>>
>>   2. Fix inifinite loop when a single byte newline is searched in JIT when
>>   invalid utf8 mode is enabled.
>>
>> Avoid that bug on older versions (which are still reportedly found in
>> the wild) by disabling the JIT when handling UTF-8.
>>
>> Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
>> Signed-off-by: René Scharfe <l.s.r@web.de>
>> ---
>> Not sure how to test it.  Killing git grep after a second or so seems a
>> bit clumsy.  timeout(1) from GNU coreutils at least allows doing that
>> from the shell, but it's not a standard tool.  Perhaps we need a new
>> test helper for that purpose?

>>
>>  grep.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/grep.c b/grep.c
>> index 7bb0360869..16629a2301 100644
>> --- a/grep.c
>> +++ b/grep.c
>> @@ -406,6 +406,14 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
>>  	}
>>
>>  	pcre2_config(PCRE2_CONFIG_JIT, &p->pcre2_jit_on);
>> +#ifndef GIT_PCRE2_VERSION_10_36_OR_HIGHER
>> +	/*
>> +	 * Work around the bug fixed by e0c6029 (Fix inifinite loop when a
>
> Better to quote this as PhilipHazel/pcre2@e0c6029 or something, i.e. to
> indicate that it's not git.git's commit.

>
>> +	 * single byte newline is searched in JIT., 2020-05-29).
>> +	 */
>> +	if (options & PCRE2_MATCH_INVALID_UTF)
>> +		p->pcre2_jit_on = 0;
>
> It seems rather heavy-hande, but I can't think of a better way to deal
> with this, i.e. if we selectively use JIT on older versions, surely we
> run into the match-bytes-but-want-chars bug you were fixing.
>
>> +#endif
>>  	if (p->pcre2_jit_on) {
>>  		jitret = pcre2_jit_compile(p->pcre2_pattern, PCRE2_JIT_COMPLETE);
>>  		if (jitret)
>