Re: bug#60690: [PATCH v2] grep: correctly identify utf-8 characters with \{b, w} in -P

From: Paul Eggert <eggert@cs.ucla.edu>
Date: Jan 9, 2023, 23:12 UTC
Message-ID: <80b42740-c85b-cef2-622c-c5b2450e264c@cs.ucla.edu>
In-Reply-To: <230109.865ydf1mdu.gmgdl@evledraar.gmail.com>

On 1/9/23 11:51, Ævar Arnfjörð Bjarmason wrote:

Show 15 quoted lines

> 	/b:
> 	155781
> 	(*UCP)/b:
> 	46035
> 	/s:
> 	0
> 	(*UCP)/s:
> 	0
> 	/w:
> 	142468
> 	(*UCP)/w:
> 	9706
> 
> So the output still differs, and some of those differences may or may
> not be wanted.

I took a look at the output, and by and large I'd want the differences; that is, I'd want the UCP version, which generates less output. This is because several Emacs source files are not UTF-8, and \b has nonsense matches when searching text files encoded via Shift-JIS or Big 5 or whatever. For this sort of thing, the fewer matches the better.

> If all you're doing is matching either ASCII or Japanese text and you
> want "locale-aware numbers" it might do the wrong thing.

I'm not seeing much of a problem here. When searching Japanese text, I would expect \d and [0-9０-９] (using both ASCII and full-width digits) to be equivalent so (assuming UCP) it's not a big deal as to which regex you use, since Japanese text won't contain Bengali (or whatever) digits. And when searching binary data, I'd expect a bunch of garbage no matter how \d is interpreted.

Here I'm assuming [０-９] (using full-width digits) has the expected meaning in PCRE2, i.e., that PCRE2 didn't make the same mistake that POSIX made.