[v2.35.0 regression] some PCRE hangs under UTF-8 locale (was: [PATCH 1/2] grep/pcre2: use PCRE2_UTF even with ASCII patterns)

From: SZEDER Gábor <szeder.dev@gmail.com>
Date: Jan 29, 2022, 17:25 UTC
Message-ID: <20220129172542.GB2581@szeder.dev>
In-Reply-To: <5fa6962e-3c1c-6dbc-f6d7-589151a9baec@web.de>

On Sat, Dec 18, 2021 at 08:50:02PM +0100, René Scharfe wrote:

Show 30 quoted lines

> compile_pcre2_pattern() currently uses the option PCRE2_UTF only for
> patterns with non-ASCII characters.  Patterns with ASCII wildcards can
> match non-ASCII strings, though.  Without that option PCRE2 mishandles
> UTF-8 input, though -- it matches parts of multi-byte characters.  Fix
> that by using PCRE2_UTF even for ASCII-only patterns.
> 
> This is a remake of the reverted ae39ba431a (grep/pcre2: fix an edge
> case concerning ascii patterns and UTF-8 data, 2021-10-15).  The change
> to the condition and the test are simplified and more targeted.
> 
> Original-patch-by: Hamza Mahfooz <someguy@effective-light.com>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
>  grep.c                          | 2 +-
>  t/t7812-grep-icase-non-ascii.sh | 6 ++++++
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/grep.c b/grep.c
> index fe847a0111..5badb6d851 100644
> --- a/grep.c
> +++ b/grep.c
> @@ -382,7 +382,7 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
>  		}
>  		options |= PCRE2_CASELESS;
>  	}
> -	if (!opt->ignore_locale && is_utf8_locale() && has_non_ascii(p->pattern) &&
> +	if (!opt->ignore_locale && is_utf8_locale() &&
>  	    !(!opt->ignore_case && (p->fixed || p->is_fixed)))
>  		options |= (PCRE2_UTF | PCRE2_MATCH_INVALID_UTF);
>

I tried to use 'git grep -P' for the first time ever, and it hung right away, spinning all CPUs at 100%. I could narrow it down, both the complexity of the pattern and the size of the input, see the test below, and it bisects to this patch.

  ---   >8   ---

#!/bin/sh

test_description='test'

. ./test-lib.sh

test_expect_success PCRE 'test' '
	# LC_ALL=C works
	LC_ALL=en_US.UTF-8 &&
	cat >ascii <<-\EOF &&
	foo
	 bar
	 baz
	EOF
	cat >utf8 <<-\EOF &&
	foo
	 bar
	 báz
	EOF
	git add ascii utf8 &&

	# These all work as expected:
	git grep --threads=1 -P " " ascii &&
	git grep --threads=1 -P "^ " ascii &&
	git grep --threads=1 -P "\s" ascii &&
	git grep --threads=1 -P "^\s" ascii &&
	git grep --threads=1 -P " " utf8 &&
	git grep --threads=1 -P "^ " utf8 &&
	git grep --threads=1 -P "\s" utf8 &&

	# This hangs (but it does work with basic and extended regexp):
	git grep --threads=1 -P "^\s" utf8
'

test_done