git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] grep: skip UTF8 checks explicitly

From
Carlo Marcelo Arenas Belón <carenas@gmail.com>
Date
Jul 22, 2019, 14:43 UTC
Message-ID
<20190722144350.46458-1-carenas@gmail.com>
In-Reply-To
<nycvar.QRO.7.76.6.1907221340320.47@tvgsbejvaqbjf.bet>

Usually PCRE is compiled with JIT support, and therefore the code path used includes calling pcre2_jit_match (for PCRE2), that ignores invalid UTF-8 in the corpus.

Make that option explicit so it can be also used when JIT is not enabled and pcre2_match is called instead, preventing `git grep` to abort when hitting the first binary blob in a fixed match after ed0479ce3d ("Merge branch 'ab/no-kwset' into next", 2019-07-15)

Reviewed-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
V2: spelling fixes from Eric Sunshine
 grep.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/grep.c b/grep.c
index fc0ed73ef3..146093f590 100644
--- a/grep.c
+++ b/grep.c
@@ -409,7 +409,7 @@ static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)
 static int pcre1match(struct grep_pat *p, const char *line, const char *eol,
 		regmatch_t *match, int eflags)
 {
-	int ovector[30], ret, flags = 0;
+	int ovector[30], ret, flags = PCRE_NO_UTF8_CHECK;
 
 	if (eflags & REG_NOTBOL)
 		flags |= PCRE_NOTBOL;
@@ -554,7 +554,7 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
 static int pcre2match(struct grep_pat *p, const char *line, const char *eol,
 		regmatch_t *match, int eflags)
 {
-	int ret, flags = 0;
+	int ret, flags = PCRE2_NO_UTF_CHECK;
 	PCRE2_SIZE *ovector;
 	PCRE2_UCHAR errbuf[256];
 
-- 
2.22.0
Previous: Johannes SchindelinNext: Junio C Hamano
Message 4 of 26 in “grep: skip UTF8 checks explicitally”
  1. grep: skip UTF8 checks explicitallyCarlo Marcelo Arenas Belón, Jul 21, 2019
  2. Eric SunshineJul 21, 2019
  3. Johannes SchindelinJul 22, 2019
  4. grep: skip UTF8 checks explicitlyCarlo Marcelo Arenas Belón, Jul 22, 2019
  5. Junio C HamanoJul 24, 2019
  6. Johannes SchindelinJul 24, 2019
  7. Junio C HamanoJul 24, 2019
  8. grep: skip UTF8 checks explicitlyCarlo Marcelo Arenas Belón, Aug 28, 2019
  9. Carlo ArenasAug 28, 2019
  10. Carlo ArenasSep 9, 2019
  11. Junio C HamanoSep 9, 2019
  12. Ævar Arnfjörð BjarmasonJul 22, 2019
  13. Carlo ArenasJul 23, 2019
  14. Johannes SchindelinJul 23, 2019
  15. Carlo ArenasJul 24, 2019
  16. Johannes SchindelinJul 24, 2019
  17. Ævar Arnfjörð BjarmasonJul 24, 2019
  18. Junio C HamanoJul 24, 2019
  19. Johannes SchindelinJul 25, 2019
  20. Junio C HamanoJul 25, 2019
  21. Johannes SchindelinJul 25, 2019
  22. Ævar Arnfjörð BjarmasonJul 26, 2019
  23. Carlo ArenasJul 26, 2019
  24. Ævar Arnfjörð BjarmasonJul 26, 2019
  25. Junio C HamanoJul 26, 2019
  26. Ævar Arnfjörð BjarmasonJul 26, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.