git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH v2] grep: allow for run time disabling of JIT in PCRE

From
Carlo Marcelo Arenas Belón <carenas@gmail.com>
Date
Jul 29, 2019, 10:59 UTC
Message-ID
<20190729105955.44390-1-carenas@gmail.com>
In-Reply-To
<20190728235427.41425-1-carenas@gmail.com>

PCRE1 allowed for a compile time flag to disable JIT, but PCRE2 never had one, forcing the use of JIT if -P was requested.

After ed0479ce3d (Merge branch 'ab/no-kwset' into next, 2019-07-15) the PCRE2 engine will be used more broadly and therefore adding this knob will allow users a escape from situations where JIT might be problematic.

JIT will be used by default but it can be disabled with the --no-pcre-jit option in `git grep` or by setting 0/false into the pcre.jit config.

If a value of -1 is used instead then the following error is prevented by using the interpreter when a JIT failure consistent with known security restrictions is found at regex compilation time.

  $ git grep 'foo bar'
  fatal: Couldn't JIT the PCRE2 pattern 'foo bar', got '-48'
Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
V2: add command line to grep as suggested by Junio
 Documentation/git-grep.txt | 11 +++++++++++
 builtin/grep.c             |  4 ++++
 grep.c                     | 30 ++++++++++++++++++++++++++----
 grep.h                     |  1 +
 4 files changed, 42 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index c89fb569e3..895c6b34ec 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -13,6 +13,7 @@ SYNOPSIS
 	   [-v | --invert-match] [-h|-H] [--full-name]
 	   [-E | --extended-regexp] [-G | --basic-regexp]
 	   [-P | --perl-regexp]
+	   [-j | --[no]-pcre-jit]
 	   [-F | --fixed-strings] [-n | --line-number] [--column]
 	   [-l | --files-with-matches] [-L | --files-without-match]
 	   [(-O | --open-files-in-pager) [<pager>]]
@@ -69,6 +70,12 @@ grep.fallbackToNoIndex::
 	If set to true, fall back to git grep --no-index if git grep
 	is executed outside of a git repository.  Defaults to false.
 
+pcre.jit::
+	If set to false, disable JIT when using PCRE.  Defaults to
+	true.
+	if set to -1 will try first to use JIT and fallback to the
+	interpreter instead of returning an error.
+
 
 OPTIONS
 -------
@@ -175,6 +182,10 @@ providing this option will cause it to die.
 	Use fixed strings for patterns (don't interpret pattern
 	as a regex).
 
+-j::
+--[no-]pcre-jit::
+	Diable JIT in PCRE with --no-pcre-jit.
+
 -n::
 --line-number::
 	Prefix the line number to matching lines.
diff --git a/builtin/grep.c b/builtin/grep.c
index 580fd38f41..b0e94875b2 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -923,6 +923,10 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		OPT_BOOL_F(0, "ext-grep", &external_grep_allowed__ignored,
 			   N_("allow calling of grep(1) (ignored by this build)"),
 			   PARSE_OPT_NOCOMPLETE),
+		OPT_GROUP("PCRE"),
+		OPT_SET_INT('j', "pcre-jit", &opt.pcre_jit,
+			N_("when to use JIT with PCRE"),
+			1),
 		OPT_END()
 	};
 
diff --git a/grep.c b/grep.c
index c7c06ae08d..d58cad0257 100644
--- a/grep.c
+++ b/grep.c
@@ -56,6 +56,7 @@ void init_grep_defaults(struct repository *repo)
 	opt->repo = repo;
 	opt->relative = 1;
 	opt->pathname = 1;
+	opt->pcre_jit = 1;
 	opt->max_depth = -1;
 	opt->pattern_type_option = GREP_PATTERN_TYPE_UNSPECIFIED;
 	color_set(opt->colors[GREP_COLOR_CONTEXT], "");
@@ -125,6 +126,12 @@ int grep_config(const char *var, const char *value, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(var, "pcre.jit")) {
+		int is_bool;
+		opt->pcre_jit = git_config_bool_or_int(var, value, &is_bool);
+		return 0;
+	}
+
 	if (!strcmp(var, "color.grep"))
 		opt->color = git_config_colorbool(var, value);
 	if (!strcmp(var, "color.grep.match")) {
@@ -163,6 +170,7 @@ void grep_init(struct grep_opt *opt, struct repository *repo, const char *prefix
 	opt->pattern_tail = &opt->pattern_list;
 	opt->header_tail = &opt->header_list;
 
+	opt->pcre_jit = def->pcre_jit;
 	opt->only_matching = def->only_matching;
 	opt->color = def->color;
 	opt->extended_regexp_option = def->extended_regexp_option;
@@ -393,7 +401,8 @@ static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)
 		die("%s", error);
 
 #ifdef GIT_PCRE1_USE_JIT
-	pcre_config(PCRE_CONFIG_JIT, &p->pcre1_jit_on);
+	if (opt->pcre_jit)
+		pcre_config(PCRE_CONFIG_JIT, &p->pcre1_jit_on);
 #endif
 }
 
@@ -489,11 +498,24 @@ static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt
 		compile_regexp_failed(p, (const char *)&errbuf);
 	}
 
-	pcre2_config(PCRE2_CONFIG_JIT, &p->pcre2_jit_on);
+	if (opt->pcre_jit)
+		pcre2_config(PCRE2_CONFIG_JIT, &p->pcre2_jit_on);
+
 	if (p->pcre2_jit_on) {
 		jitret = pcre2_jit_compile(p->pcre2_pattern, PCRE2_JIT_COMPLETE);
-		if (jitret)
-			die("Couldn't JIT the PCRE2 pattern '%s', got '%d'\n", p->pattern, jitret);
+		if (jitret) {
+			if ((opt->pcre_jit < 0) &&
+				jitret == PCRE2_ERROR_NOMEMORY) {
+				/*
+				 * JIT compiler isn't available but we can
+				 * still fallback to the interpreter
+				 */
+				p->pcre2_jit_on = 0;
+				return;
+			}
+			else
+				die("Couldn't JIT the PCRE2 pattern '%s', got '%d'\n", p->pattern, jitret);
+		}
 
 		/*
 		 * The pcre2_config(PCRE2_CONFIG_JIT, ...) call just
diff --git a/grep.h b/grep.h
index c0c71eb4a9..fff152e606 100644
--- a/grep.h
+++ b/grep.h
@@ -151,6 +151,7 @@ struct grep_opt {
 	int allow_textconv;
 	int extended;
 	int use_reflog_filter;
+	int pcre_jit;
 	int pcre1;
 	int pcre2;
 	int relative;

base-commit: 870eea81669bfff4333b37b11fedd870cd05fd90

-- 
2.22.0
Previous: Carlo ArenasNext: Carlo Arenas
Message 9 of 23 in “grep: allow for run time disabling of JIT in PCRE”
  1. grep: allow for run time disabling of JIT in PCRECarlo Marcelo Arenas Belón, Jul 28, 2019
  2. Carlo ArenasJul 29, 2019
  3. Junio C HamanoJul 29, 2019
  4. Carlo ArenasJul 29, 2019
  5. Ævar Arnfjörð BjarmasonJul 29, 2019
  6. Carlo ArenasJul 29, 2019
  7. Ævar Arnfjörð BjarmasonJul 29, 2019
  8. Carlo ArenasJul 30, 2019
  9. grep: allow for run time disabling of JIT in PCRECarlo Marcelo Arenas Belón, Jul 29, 2019
  10. Carlo ArenasJul 29, 2019
  11. René ScharfeJul 29, 2019
  12. Junio C HamanoJul 29, 2019
  13. Carlo ArenasJul 30, 2019
  14. René ScharfeJul 30, 2019
  15. Johannes SchindelinJul 31, 2019
  16. Junio C HamanoJul 31, 2019
  17. Johannes SchindelinJul 31, 2019
  18. Ævar Arnfjörð BjarmasonJul 31, 2019
  19. Carlo ArenasAug 4, 2019
  20. grep: treat PCRE2 jit compilation memory error as non fatalCarlo Marcelo Arenas Belón, Aug 4, 2019
  21. Carlo ArenasAug 4, 2019
  22. Junio C HamanoAug 5, 2019
  23. Johannes SchindelinJul 31, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.