git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: grep: fix multibyte regex handling under macOS (1819ad327b7a1f19540a819813b70a0e8a7f798f)

From
Ddemerphq <demerphq@gmail.com>
Date
Feb 1, 2023, 16:09 UTC
Message-ID
<CANgJU+X_e0owKC3uWPaA_gVP54syF1+MJ-cTn+fjPrNS5LDsMA@mail.gmail.com>
In-Reply-To
<CALnO6CAZtwfGY4SYeOuKqdP9+e_0EYNf4F703DRQB7UUfd_bUg@mail.gmail.com>
On Wed, 1 Feb 2023 at 16:25, D. Ben Knoble <ben.knoble@gmail.com> wrote:
Show 31 quoted lines
>
> I recently updated to git 2.39.1 and noticed today that `git diff
> --word-diff` fails for files with `diff=scheme`. I was able to narrow
> the failure down to the inclusion of control characters \xc0, \xff,
> \x80, \xbf by https://github.com/git/git/blob/2fc9e9ca3c7505bc60069f11e7ef09b1aeeee473/userdiff.c#L17
> in the definition of the scheme diff pattern (really, all patterns).
>
> I suspect the commit referenced in the subject, given that it messes
> with regex handling on macOS.
>
> Relevant environment that I can think of:
> ```
> # locale
> LANG="fr_FR.UTF-8"
> LC_COLLATE="fr_FR.UTF-8"
> LC_CTYPE="fr_FR.UTF-8"
> LC_MESSAGES="fr_FR.UTF-8"
> LC_MONETARY="fr_FR.UTF-8"
> LC_NUMERIC="fr_FR.UTF-8"
> LC_TIME="fr_FR.UTF-8"
> LC_ALL="fr_FR.UTF-8"
> ```
>
> I'm on macOS 11.7.
>
> Failure (using Zsh to produce the characters; I think there's a Bash
> equivalent):
> ```
> # git diff --word-diff --word-diff-regex=$'[\xc0-\xff][\x80-\xbf]+'
> fatal¬†: invalid regular expression: [¿-ˇ][Ä-ø]+
> ```

FWIW that looks pretty weird to me, like the escapes in the charclass were interpolated before being fed to the regex engine. Are you sure you tested the right thing?

Yves

-- perl -Mre=debug -e "/just|another|perl|hacker/"

Previous: D. Ben KnobleNext: D. Ben Knoble
Message 2 of 22 in “RE: grep: fix multibyte regex handling under macOS (1819ad327b7a1f19540a819813b70a0e8a7f798f)”
  1. D. Ben KnobleFeb 1, 2023
  2. demerphqFeb 1, 2023
  3. D. Ben KnobleFeb 1, 2023
  4. demerphqFeb 1, 2023
  5. Junio C HamanoFeb 1, 2023
  6. D. Ben KnobleFeb 1, 2023
  7. D. Ben KnobleFeb 1, 2023
  8. Junio C HamanoFeb 1, 2023
  9. Jeff KingFeb 1, 2023
  10. demerphqFeb 2, 2023
  11. D. Ben KnobleFeb 2, 2023
  12. Jeff KingFeb 3, 2023
  13. Ævar Arnfjörð BjarmasonFeb 3, 2023
  14. Jeff KingFeb 4, 2023
  15. demerphqFeb 4, 2023
  16. D. Ben KnobleFeb 5, 2023
  17. Jeff KingFeb 7, 2023
  18. D. Ben KnobleFeb 7, 2023
  19. Jeff KingFeb 7, 2023
  20. D. Ben KnobleFeb 2, 2023
  21. Jeff KingFeb 3, 2023
  22. D. Ben KnobleFeb 3, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.