git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: grep: fix multibyte regex handling under macOS (1819ad327b7a1f19540a819813b70a0e8a7f798f)

From
D. Ben Knoble <ben.knoble@gmail.com>
Date
Feb 1, 2023, 15:18 UTC
Message-ID
<CALnO6CAZtwfGY4SYeOuKqdP9+e_0EYNf4F703DRQB7UUfd_bUg@mail.gmail.com>

I recently updated to git 2.39.1 and noticed today that `git diff --word-diff` fails for files with `diff=scheme`. I was able to narrow the failure down to the inclusion of control characters \xc0, \xff, \x80, \xbf by https://github.com/git/git/blob/2fc9e9ca3c7505bc60069f11e7ef09b1aeeee473/userdiff.c#L17 in the definition of the scheme diff pattern (really, all patterns).

I suspect the commit referenced in the subject, given that it messes with regex handling on macOS.

Relevant environment that I can think of: ``` # locale LANG="fr_FR.UTF-8" LC_COLLATE="fr_FR.UTF-8" LC_CTYPE="fr_FR.UTF-8" LC_MESSAGES="fr_FR.UTF-8" LC_MONETARY="fr_FR.UTF-8" LC_NUMERIC="fr_FR.UTF-8" LC_TIME="fr_FR.UTF-8" LC_ALL="fr_FR.UTF-8" ```

I'm on macOS 11.7.

Failure (using Zsh to produce the characters; I think there's a Bash equivalent): ``` # git diff --word-diff --word-diff-regex=$'[\xc0-\xff][\x80-\xbf]+' fatal¬†: invalid regular expression: [¿-ˇ][Ä-ø]+ ``` (Looks like the output is a bit scrambled; here's the hexdump) ``` # !! |& xxd 00000000: 6661 7461 6cc2 a03a 2069 6e76 616c 6964 fatal..: invalid 00000010: 2072 6567 756c 6172 2065 7870 7265 7373 regular express 00000020: 696f 6e3a 205b c02d ff5d 5b80 2dbf 5d2b ion: [.-.][.-.]+ 00000030: 0a . ```

-- 
D. Ben Knoble
Next: demerphq
Message 1 of 22 in “RE: grep: fix multibyte regex handling under macOS (1819ad327b7a1f19540a819813b70a0e8a7f798f)”
  1. D. Ben KnobleFeb 1, 2023
  2. demerphqFeb 1, 2023
  3. D. Ben KnobleFeb 1, 2023
  4. demerphqFeb 1, 2023
  5. Junio C HamanoFeb 1, 2023
  6. D. Ben KnobleFeb 1, 2023
  7. D. Ben KnobleFeb 1, 2023
  8. Junio C HamanoFeb 1, 2023
  9. Jeff KingFeb 1, 2023
  10. demerphqFeb 2, 2023
  11. D. Ben KnobleFeb 2, 2023
  12. Jeff KingFeb 3, 2023
  13. Ævar Arnfjörð BjarmasonFeb 3, 2023
  14. Jeff KingFeb 4, 2023
  15. demerphqFeb 4, 2023
  16. D. Ben KnobleFeb 5, 2023
  17. Jeff KingFeb 7, 2023
  18. D. Ben KnobleFeb 7, 2023
  19. Jeff KingFeb 7, 2023
  20. D. Ben KnobleFeb 2, 2023
  21. Jeff KingFeb 3, 2023
  22. D. Ben KnobleFeb 3, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.