git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 25/25] pickaxe -G: terminate early on matching lines

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 3, 2021, 03:28 UTC
Message-ID
<20210203032811.14979-26-avarab@gmail.com>
In-Reply-To
<20210203032811.14979-1-avarab@gmail.com>

Solve a long-standing item for "git log -Grx" of us e.g. finding "+ str" in the diff context and noting that we had a "hit", but xdiff diligently continuing to generate and spew the rest of the diff at us.

The TODO item has been there since "git log -G" was implemented. See f506b8e8b5f (git log/diff: add -G<regexp> that greps in the patch text, 2010-08-23).

Our xdiff interface also had the limitation of not being able to abort early since the beginning, see d9ea73e0564 (combine-diff: refactor built-in xdiff interface., 2006-04-05). Although at that time "xdiff_emit_line_fn" was called "xdiff_emit_consume_fn", and "xdiff_emit_hunk_fn" didn't exist yet.

But now with the support added in the preceding ""xdiff-interface: allow early return from xdiff_emit_{line,hunk}_fn" commit we can return early, and furthermore test the functionality of the new early-exit xdiff-interface by having a BUG() call here to die if it ever starts handing us needless work again.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 diffcore-pickaxe.c | 29 +++++++++++++++++++----------
 xdiff-interface.h  |  5 +++++
 2 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c
index 21f9d66b6a..e773fa69a2 100644
--- a/diffcore-pickaxe.c
+++ b/diffcore-pickaxe.c
@@ -29,12 +29,11 @@ static int diffgrep_consume(void *priv, char *line, unsigned long len)
 	if (line[0] != '+' && line[0] != '-')
 		return 0;
 	if (data->hit)
-		/*
-		 * NEEDSWORK: we should have a way to terminate the
-		 * caller early.
-		 */
-		return 0;
-	data->hit = patmatch(grep_pat, line + 1, line + len + 1, &regmatch, 0);
+		BUG("Already matched in diffgrep_consume! Broken xdiff_emit_line_fn?");
+	if (patmatch(grep_pat, line + 1, line + len + 1, &regmatch, 0)) {
+		data->hit = 1;
+		return -1;
+	}
 	return 0;
 }
 
@@ -47,6 +46,7 @@ static int diff_grep(mmfile_t *one, mmfile_t *two,
 	xdemitconf_t xecfg;
 	regmatch_t regmatch;
 	struct grep_pat *grep_pat = grep_filter->pattern_list;
+	int ret;
 
 	if (!one)
 		return patmatch(grep_pat, two->ptr, two->ptr + two->size,
@@ -65,10 +65,19 @@ static int diff_grep(mmfile_t *one, mmfile_t *two,
 	ecbdata.hit = 0;
 	xecfg.ctxlen = 0;
 	xecfg.interhunkctxlen = o->interhunkcontext;
-	if (xdi_diff_outf(one, two, discard_hunk_line, diffgrep_consume,
-			  &ecbdata, &xpp, &xecfg))
-		return 0;
-	return ecbdata.hit;
+
+	/*
+	 * An xdiff error might be our "data->hit" from above. See the
+	 * comment for xdiff_emit_{line,hunk}_fn in xdiff-interface.h
+	 * for why.
+	 */
+	ret = xdi_diff_outf(one, two, discard_hunk_line, diffgrep_consume,
+			    &ecbdata, &xpp, &xecfg);
+	if (ecbdata.hit)
+		return 1;
+	if (ret)
+		return ret;
+	return 0;
 }
 
 static unsigned int contains(mmfile_t *mf, struct grep_opt *grep_filter)
diff --git a/xdiff-interface.h b/xdiff-interface.h
index 1b27d6104c..347d8a4425 100644
--- a/xdiff-interface.h
+++ b/xdiff-interface.h
@@ -25,6 +25,11 @@
  * granular return values, but for now use it carefully, or consider
  * e.g. using discard_hunk_line() if you say just don't care about
  * hunk headers.
+ *
+ * Note that just returning -1 will make your early return
+ * indistinguishable from an error internal to xdiff. See "diff_grep"
+ * in diffcore-pickaxe.c for a trick to work around this, i.e. using
+ * the "consume_callback_data" to note the desired early return.
  */
 typedef int (*xdiff_emit_line_fn)(void *, char *, unsigned long);
 typedef int (*xdiff_emit_hunk_fn)(void *data,
-- 
2.30.0.284.gd98b1dd5eaa7
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 35 of 99 in “grep: PCREv2 fixes, remove kwset.[ch]”
  1. 00/25 grep: PCREv2 fixes, remove kwset.[ch]Ævar Arnfjörð Bjarmason, Feb 3, 2021
  2. 01/25 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Feb 3, 2021
  3. 03/25 grep/pcre2: drop needless assignment to NULLÆvar Arnfjörð Bjarmason, Feb 3, 2021
  4. 02/25 grep/pcre2: drop needless assignment + assert() on opt->pcre2Ævar Arnfjörð Bjarmason, Feb 3, 2021
  5. 04/25 grep/pcre2: correct reference to grep_init() in commentÆvar Arnfjörð Bjarmason, Feb 3, 2021
  6. 05/25 grep/pcre2: prepare to add debugging to pcre2_malloc()Ævar Arnfjörð Bjarmason, Feb 3, 2021
  7. 06/25 grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug modeÆvar Arnfjörð Bjarmason, Feb 3, 2021
  8. 07/25 grep/pcre2: use compile-time PCREv2 version testÆvar Arnfjörð Bjarmason, Feb 3, 2021
  9. 08/25 grep/pcre2: use pcre2_maketables_free() functionÆvar Arnfjörð Bjarmason, Feb 3, 2021
  10. 09/25 grep/pcre2: actually make pcre2 use custom allocatorÆvar Arnfjörð Bjarmason, Feb 3, 2021
  11. 10/25 grep/pcre2: move back to thread-only PCREv2 structuresÆvar Arnfjörð Bjarmason, Feb 3, 2021
  12. 12/25 pickaxe tests: refactor to use test_commit --appendÆvar Arnfjörð Bjarmason, Feb 3, 2021
  13. 11/25 grep/pcre2: move definitions of pcre2_{malloc,free}Ævar Arnfjörð Bjarmason, Feb 3, 2021
  14. 13/25 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Feb 3, 2021
  15. 14/25 pickaxe -S: remove redundant "sz" check in while-loopÆvar Arnfjörð Bjarmason, Feb 3, 2021
  16. René ScharfeFeb 4, 2021
  17. Junio C HamanoFeb 4, 2021
  18. Ævar Arnfjörð BjarmasonFeb 4, 2021
  19. 16/25 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Feb 3, 2021
  20. 18/25 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Feb 3, 2021
  21. 20/25 grep.h: make patmatch() a public functionÆvar Arnfjörð Bjarmason, Feb 3, 2021
  22. 23/25 xdiff-interface: allow early return from xdiff_emit_{line,hunk}_fnÆvar Arnfjörð Bjarmason, Feb 3, 2021
  23. 24/25 xdiff-interface: support early exit in xdiff_outf()Ævar Arnfjörð Bjarmason, Feb 3, 2021
  24. Junio C HamanoFeb 4, 2021
  25. 17/25 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Feb 3, 2021
  26. 19/25 pickaxe -G: set -U0 for diff generationÆvar Arnfjörð Bjarmason, Feb 3, 2021
  27. Ævar Arnfjörð BjarmasonFeb 3, 2021
  28. Junio C HamanoFeb 3, 2021
  29. 15/25 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Feb 3, 2021
  30. 21/25 pickaxe: use PCREv2 for -G and -SÆvar Arnfjörð Bjarmason, Feb 3, 2021
  31. Ævar Arnfjörð BjarmasonFeb 3, 2021
  32. Junio C HamanoFeb 4, 2021
  33. Junio C HamanoFeb 4, 2021
  34. 22/25 Remove unused kwset.[ch]Ævar Arnfjörð Bjarmason, Feb 3, 2021
  35. 25/25 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Feb 3, 2021
  36. Ævar Arnfjörð BjarmasonFeb 3, 2021
  37. 00/22 pickaxe: test and refactoring for follow-up changesÆvar Arnfjörð Bjarmason, Feb 16, 2021
  38. Junio C HamanoFeb 16, 2021
  39. Junio C HamanoFeb 17, 2021
  40. 00/22 pickaxe: test and refactoring for future PCRE backendÆvar Arnfjörð Bjarmason, Apr 12, 2021
  41. 01/22 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Apr 12, 2021
  42. 02/22 pickaxe tests: refactor to use test_commit --append --printfÆvar Arnfjörð Bjarmason, Apr 12, 2021
  43. 03/22 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Apr 12, 2021
  44. 04/22 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Apr 12, 2021
  45. 05/22 pickaxe tests: test for -G, -S and --find-object incompatibilityÆvar Arnfjörð Bjarmason, Apr 12, 2021
  46. 06/22 pickaxe tests: add missing test for --no-pickaxe-regex being an errorÆvar Arnfjörð Bjarmason, Apr 12, 2021
  47. 07/22 pickaxe: die when -G and --pickaxe-regex are combinedÆvar Arnfjörð Bjarmason, Apr 12, 2021
  48. 08/22 pickaxe: die when --find-object and --pickaxe-all are combinedÆvar Arnfjörð Bjarmason, Apr 12, 2021
  49. 09/22 diff.h: move pickaxe fields together againÆvar Arnfjörð Bjarmason, Apr 12, 2021
  50. 10/22 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Apr 12, 2021
  51. 11/22 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Apr 12, 2021
  52. 12/22 pickaxe: refactor function selection in diffcore-pickaxe()Ævar Arnfjörð Bjarmason, Apr 12, 2021
  53. 13/22 pickaxe: assert that we must have a needle under -G or -SÆvar Arnfjörð Bjarmason, Apr 12, 2021
  54. 14/22 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Apr 12, 2021
  55. 15/22 pickaxe: rename variables in has_changes() for brevityÆvar Arnfjörð Bjarmason, Apr 12, 2021
  56. 16/22 pickaxe -S: slightly optimize contains()Ævar Arnfjörð Bjarmason, Apr 12, 2021
  57. 17/22 xdiff-interface: prepare for allowing early returnÆvar Arnfjörð Bjarmason, Apr 12, 2021
  58. 18/22 xdiff-interface: allow early return from xdiff_emit_line_fnÆvar Arnfjörð Bjarmason, Apr 12, 2021
  59. 19/22 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Apr 12, 2021
  60. 20/22 pickaxe -G: don't special-case create/deleteÆvar Arnfjörð Bjarmason, Apr 12, 2021
  61. 21/22 xdiff users: use designated initializers for out_lineÆvar Arnfjörð Bjarmason, Apr 12, 2021
  62. 22/22 xdiff-interface: replace discard_hunk_line() with a flagÆvar Arnfjörð Bjarmason, Apr 12, 2021
  63. 01/22 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Feb 16, 2021
  64. 02/22 test-lib-functions: document and test test_commit --no-tagÆvar Arnfjörð Bjarmason, Feb 16, 2021
  65. Junio C HamanoMar 30, 2021
  66. 03/22 test-lib-functions: reword "test_commit --append" docsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  67. 04/22 test-lib functions: add --printf option to test_commitÆvar Arnfjörð Bjarmason, Feb 16, 2021
  68. Junio C HamanoMar 30, 2021
  69. Ævar Arnfjörð BjarmasonApr 12, 2021
  70. 05/22 pickaxe tests: refactor to use test_commit --append --printfÆvar Arnfjörð Bjarmason, Feb 16, 2021
  71. Junio C HamanoMar 30, 2021
  72. 06/22 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  73. 08/22 pickaxe tests: test for -G, -S and --find-object incompatibilityÆvar Arnfjörð Bjarmason, Feb 16, 2021
  74. Junio C HamanoMar 30, 2021
  75. 07/22 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Feb 16, 2021
  76. 09/22 pickaxe: die when -G and --pickaxe-regex are combinedÆvar Arnfjörð Bjarmason, Feb 16, 2021
  77. Junio C HamanoMar 30, 2021
  78. 11/22 diff.h: move pickaxe fields together againÆvar Arnfjörð Bjarmason, Feb 16, 2021
  79. 10/22 pickaxe: die when --find-object and --pickaxe-all are combinedÆvar Arnfjörð Bjarmason, Feb 16, 2021
  80. 14/22 pickaxe: refactor function selection in diffcore-pickaxe()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  81. Junio C HamanoMar 30, 2021
  82. 12/22 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  83. 15/22 pickaxe: assert that we must have a needle under -G or -SÆvar Arnfjörð Bjarmason, Feb 16, 2021
  84. Junio C HamanoMar 30, 2021
  85. 17/22 pickaxe: rename variables in has_changes() for brevityÆvar Arnfjörð Bjarmason, Feb 16, 2021
  86. 13/22 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Feb 16, 2021
  87. 21/22 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Feb 16, 2021
  88. Junio C HamanoMar 31, 2021
  89. 16/22 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Feb 16, 2021
  90. Junio C HamanoMar 30, 2021
  91. 22/22 pickaxe -G: don't special-case create/deleteÆvar Arnfjörð Bjarmason, Feb 16, 2021
  92. Junio C HamanoMar 31, 2021
  93. 18/22 pickaxe -S: slightly optimize contains()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  94. Junio C HamanoMar 30, 2021
  95. 20/22 xdiff-interface: support early exit in xdiff_outf()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  96. 19/22 xdiff-interface: allow early return from xdiff_emit_{line,hunk}_fnÆvar Arnfjörð Bjarmason, Feb 16, 2021
  97. Junio C HamanoMar 31, 2021
  98. Ævar Arnfjörð BjarmasonFeb 3, 2021
  99. Ævar Arnfjörð BjarmasonFeb 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.