Re: bug#60690: -P '\d' in GNU and git grep

From: Paul Eggert <eggert@cs.ucla.edu>
Date: Apr 4, 2023, 06:46 UTC
Message-ID: <920dcc8d-9e45-a03e-af06-6b420c6e0f81@cs.ucla.edu>
In-Reply-To: <CA+8g5KHuE-kQqmi9cVjeJbpyt54v9m9omh9A9we1zmR0+aTDHg@mail.gmail.com>

On 2023-04-03 20:30, Jim Meyering wrote:

> have you seen justification
> (other than for compatibility with some other tool or language) for
> allowing \d to match non-ASCII by default, in spite of the risks?

In the example Ævar supplied in <https://bugs.gnu.org/60690>, my impression was that it was better when \d matched non-ASCII digits. That is, in a UTF-8 locale it's better when \d finds matches in these lines:

>> 	> git-gui/po/ja.po:"- 第１行: 何をしたか、を１行で要約。\n"
>> 	> git-gui/po/ja.po:"- 第２行: 空白\n"

because they contain the Japanese digits "１" and "２". This was the only example I recall being given.

Also, I find it odd that grep -P '^[\w\d]*$' matches lines containing any sort of Arabic word characters, but it rejects lines containing Arabic digits like "٣" that are perfectly reasonable in Arabic-language text. I also find it odd that [\d] and [[:digit:]] mean different things.

There are arguments on the other side, otherwise we wouldn't be having this discussion. And it's true that grep -P '\d' formerly rejected Arabic digits (though it's also true that grep -P '\w' formerly rejected Arabic letters...). Still, the cure's oddness and incompatibility with Git, Perl, etc. appears to me to be worse than the disease of dealing with grep -P invocations that need to use [0-9] or LC_ALL="C" anyway if they want to be portable to any program other than GNU grep.