RE: grep: fix multibyte regex handling under macOS (1819ad327b7a1f19540a819813b70a0e8a7f798f)
- From
D. Ben Knoble <ben.knoble@gmail.com>
- Date
- Feb 1, 2023, 15:18 UTC
- Message-ID
- <CALnO6CAZtwfGY4SYeOuKqdP9+e_0EYNf4F703DRQB7UUfd_bUg@mail.gmail.com>
I recently updated to git 2.39.1 and noticed today that `git diff --word-diff` fails for files with `diff=scheme`. I was able to narrow the failure down to the inclusion of control characters \xc0, \xff, \x80, \xbf by https://github.com/git/git/blob/2fc9e9ca3c7505bc60069f11e7ef09b1aeeee473/userdiff.c#L17 in the definition of the scheme diff pattern (really, all patterns).
I suspect the commit referenced in the subject, given that it messes with regex handling on macOS.
Relevant environment that I can think of: ``` # locale LANG="fr_FR.UTF-8" LC_COLLATE="fr_FR.UTF-8" LC_CTYPE="fr_FR.UTF-8" LC_MESSAGES="fr_FR.UTF-8" LC_MONETARY="fr_FR.UTF-8" LC_NUMERIC="fr_FR.UTF-8" LC_TIME="fr_FR.UTF-8" LC_ALL="fr_FR.UTF-8" ```
I'm on macOS 11.7.
Failure (using Zsh to produce the characters; I think there's a Bash equivalent): ``` # git diff --word-diff --word-diff-regex=$'[\xc0-\xff][\x80-\xbf]+' fatal¬†: invalid regular expression: [¿-ˇ][Ä-ø]+ ``` (Looks like the output is a bit scrambled; here's the hexdump) ``` # !! |& xxd 00000000: 6661 7461 6cc2 a03a 2069 6e76 616c 6964 fatal..: invalid 00000010: 2072 6567 756c 6172 2065 7870 7265 7373 regular express 00000020: 696f 6e3a 205b c02d ff5d 5b80 2dbf 5d2b ion: [.-.][.-.]+ 00000030: 0a . ```
-- D. Ben Knoble