git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 18/22] pickaxe -S: slightly optimize contains()

From
Junio C Hamano <gitster@pobox.com>
Date
Mar 30, 2021, 23:58 UTC
Message-ID
<xmqqeefw9ost.fsf@gitster.g>
In-Reply-To
<20210216115801.4773-19-avarab@gmail.com>
Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
> When the "log -S<pat>" switch counts occurrences of <pat> on the
> pre-image and post-image of a change. As soon as we know we had e.g. 1
> before and 2 now we can stop, we don't need to keep counting past 2.
Logical.

I do not know how much difference this would make in practice, though. The performance characteristics between "diff A B" and "diff B A" with the same pickaxe -Sneedle would be asymmetric with this optimization, which is a bit curious (but not incorrect).

I wonder if there is a good heuristics to decide which, between one and two, blob to start counting. Obviously, scanning the one that is likely to contain fewer occurrence of the needle, before we can employ this optimization to the other side, is what we want.

Show 50 quoted lines
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  diffcore-pickaxe.c | 13 ++++++++++---
>  1 file changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/diffcore-pickaxe.c b/diffcore-pickaxe.c
> index 66e34d254f1..76c178bae2b 100644
> --- a/diffcore-pickaxe.c
> +++ b/diffcore-pickaxe.c
> @@ -68,7 +68,8 @@ static int diff_grep(mmfile_t *one, mmfile_t *two,
>  	return ecbdata.hit;
>  }
>  
> -static unsigned int contains(mmfile_t *mf, regex_t *regexp, kwset_t kws)
> +static unsigned int contains(mmfile_t *mf, regex_t *regexp, kwset_t kws,
> +			     unsigned int limit)
>  {
>  	unsigned int cnt = 0;
>  	unsigned long sz = mf->size;
> @@ -88,6 +89,9 @@ static unsigned int contains(mmfile_t *mf, regex_t *regexp, kwset_t kws)
>  				sz--;
>  			}
>  			cnt++;
> +
> +			if (limit && cnt == limit)
> +				return cnt;
>  		}
>  
>  	} else { /* Classic exact string match */
> @@ -99,6 +103,9 @@ static unsigned int contains(mmfile_t *mf, regex_t *regexp, kwset_t kws)
>  			sz -= offset + kwsm.size[0];
>  			data += offset + kwsm.size[0];
>  			cnt++;
> +
> +			if (limit && cnt == limit)
> +				return cnt;
>  		}
>  	}
>  	return cnt;
> @@ -108,8 +115,8 @@ static int has_changes(mmfile_t *one, mmfile_t *two,
>  		       struct diff_options *o,
>  		       regex_t *regexp, kwset_t kws)
>  {
> -	unsigned int c1 = one ? contains(one, regexp, kws) : 0;
> -	unsigned int c2 = two ? contains(two, regexp, kws) : 0;
> +	unsigned int c1 = one ? contains(one, regexp, kws, 0) : 0;
> +	unsigned int c2 = two ? contains(two, regexp, kws, c1 + 1) : 0;
>  	return c1 != c2;
>  }
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 94 of 99 in “grep: PCREv2 fixes, remove kwset.[ch]”
  1. 00/25 grep: PCREv2 fixes, remove kwset.[ch]Ævar Arnfjörð Bjarmason, Feb 3, 2021
  2. 01/25 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Feb 3, 2021
  3. 03/25 grep/pcre2: drop needless assignment to NULLÆvar Arnfjörð Bjarmason, Feb 3, 2021
  4. 02/25 grep/pcre2: drop needless assignment + assert() on opt->pcre2Ævar Arnfjörð Bjarmason, Feb 3, 2021
  5. 04/25 grep/pcre2: correct reference to grep_init() in commentÆvar Arnfjörð Bjarmason, Feb 3, 2021
  6. 05/25 grep/pcre2: prepare to add debugging to pcre2_malloc()Ævar Arnfjörð Bjarmason, Feb 3, 2021
  7. 06/25 grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug modeÆvar Arnfjörð Bjarmason, Feb 3, 2021
  8. 07/25 grep/pcre2: use compile-time PCREv2 version testÆvar Arnfjörð Bjarmason, Feb 3, 2021
  9. 08/25 grep/pcre2: use pcre2_maketables_free() functionÆvar Arnfjörð Bjarmason, Feb 3, 2021
  10. 09/25 grep/pcre2: actually make pcre2 use custom allocatorÆvar Arnfjörð Bjarmason, Feb 3, 2021
  11. 10/25 grep/pcre2: move back to thread-only PCREv2 structuresÆvar Arnfjörð Bjarmason, Feb 3, 2021
  12. 12/25 pickaxe tests: refactor to use test_commit --appendÆvar Arnfjörð Bjarmason, Feb 3, 2021
  13. 11/25 grep/pcre2: move definitions of pcre2_{malloc,free}Ævar Arnfjörð Bjarmason, Feb 3, 2021
  14. 13/25 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Feb 3, 2021
  15. 14/25 pickaxe -S: remove redundant "sz" check in while-loopÆvar Arnfjörð Bjarmason, Feb 3, 2021
  16. René ScharfeFeb 4, 2021
  17. Junio C HamanoFeb 4, 2021
  18. Ævar Arnfjörð BjarmasonFeb 4, 2021
  19. 16/25 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Feb 3, 2021
  20. 18/25 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Feb 3, 2021
  21. 20/25 grep.h: make patmatch() a public functionÆvar Arnfjörð Bjarmason, Feb 3, 2021
  22. 23/25 xdiff-interface: allow early return from xdiff_emit_{line,hunk}_fnÆvar Arnfjörð Bjarmason, Feb 3, 2021
  23. 24/25 xdiff-interface: support early exit in xdiff_outf()Ævar Arnfjörð Bjarmason, Feb 3, 2021
  24. Junio C HamanoFeb 4, 2021
  25. 17/25 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Feb 3, 2021
  26. 19/25 pickaxe -G: set -U0 for diff generationÆvar Arnfjörð Bjarmason, Feb 3, 2021
  27. Ævar Arnfjörð BjarmasonFeb 3, 2021
  28. Junio C HamanoFeb 3, 2021
  29. 15/25 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Feb 3, 2021
  30. 21/25 pickaxe: use PCREv2 for -G and -SÆvar Arnfjörð Bjarmason, Feb 3, 2021
  31. Ævar Arnfjörð BjarmasonFeb 3, 2021
  32. Junio C HamanoFeb 4, 2021
  33. Junio C HamanoFeb 4, 2021
  34. 22/25 Remove unused kwset.[ch]Ævar Arnfjörð Bjarmason, Feb 3, 2021
  35. 25/25 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Feb 3, 2021
  36. Ævar Arnfjörð BjarmasonFeb 3, 2021
  37. 00/22 pickaxe: test and refactoring for follow-up changesÆvar Arnfjörð Bjarmason, Feb 16, 2021
  38. Junio C HamanoFeb 16, 2021
  39. Junio C HamanoFeb 17, 2021
  40. 00/22 pickaxe: test and refactoring for future PCRE backendÆvar Arnfjörð Bjarmason, Apr 12, 2021
  41. 01/22 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Apr 12, 2021
  42. 02/22 pickaxe tests: refactor to use test_commit --append --printfÆvar Arnfjörð Bjarmason, Apr 12, 2021
  43. 03/22 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Apr 12, 2021
  44. 04/22 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Apr 12, 2021
  45. 05/22 pickaxe tests: test for -G, -S and --find-object incompatibilityÆvar Arnfjörð Bjarmason, Apr 12, 2021
  46. 06/22 pickaxe tests: add missing test for --no-pickaxe-regex being an errorÆvar Arnfjörð Bjarmason, Apr 12, 2021
  47. 07/22 pickaxe: die when -G and --pickaxe-regex are combinedÆvar Arnfjörð Bjarmason, Apr 12, 2021
  48. 08/22 pickaxe: die when --find-object and --pickaxe-all are combinedÆvar Arnfjörð Bjarmason, Apr 12, 2021
  49. 09/22 diff.h: move pickaxe fields together againÆvar Arnfjörð Bjarmason, Apr 12, 2021
  50. 10/22 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Apr 12, 2021
  51. 11/22 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Apr 12, 2021
  52. 12/22 pickaxe: refactor function selection in diffcore-pickaxe()Ævar Arnfjörð Bjarmason, Apr 12, 2021
  53. 13/22 pickaxe: assert that we must have a needle under -G or -SÆvar Arnfjörð Bjarmason, Apr 12, 2021
  54. 14/22 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Apr 12, 2021
  55. 15/22 pickaxe: rename variables in has_changes() for brevityÆvar Arnfjörð Bjarmason, Apr 12, 2021
  56. 16/22 pickaxe -S: slightly optimize contains()Ævar Arnfjörð Bjarmason, Apr 12, 2021
  57. 17/22 xdiff-interface: prepare for allowing early returnÆvar Arnfjörð Bjarmason, Apr 12, 2021
  58. 18/22 xdiff-interface: allow early return from xdiff_emit_line_fnÆvar Arnfjörð Bjarmason, Apr 12, 2021
  59. 19/22 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Apr 12, 2021
  60. 20/22 pickaxe -G: don't special-case create/deleteÆvar Arnfjörð Bjarmason, Apr 12, 2021
  61. 21/22 xdiff users: use designated initializers for out_lineÆvar Arnfjörð Bjarmason, Apr 12, 2021
  62. 22/22 xdiff-interface: replace discard_hunk_line() with a flagÆvar Arnfjörð Bjarmason, Apr 12, 2021
  63. 01/22 grep/pcre2 tests: reword comments referring to kwsetÆvar Arnfjörð Bjarmason, Feb 16, 2021
  64. 02/22 test-lib-functions: document and test test_commit --no-tagÆvar Arnfjörð Bjarmason, Feb 16, 2021
  65. Junio C HamanoMar 30, 2021
  66. 03/22 test-lib-functions: reword "test_commit --append" docsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  67. 04/22 test-lib functions: add --printf option to test_commitÆvar Arnfjörð Bjarmason, Feb 16, 2021
  68. Junio C HamanoMar 30, 2021
  69. Ævar Arnfjörð BjarmasonApr 12, 2021
  70. 05/22 pickaxe tests: refactor to use test_commit --append --printfÆvar Arnfjörð Bjarmason, Feb 16, 2021
  71. Junio C HamanoMar 30, 2021
  72. 06/22 pickaxe tests: add test for diffgrep_consume() internalsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  73. 08/22 pickaxe tests: test for -G, -S and --find-object incompatibilityÆvar Arnfjörð Bjarmason, Feb 16, 2021
  74. Junio C HamanoMar 30, 2021
  75. 07/22 pickaxe tests: add test for "log -S" not being a regexÆvar Arnfjörð Bjarmason, Feb 16, 2021
  76. 09/22 pickaxe: die when -G and --pickaxe-regex are combinedÆvar Arnfjörð Bjarmason, Feb 16, 2021
  77. Junio C HamanoMar 30, 2021
  78. 11/22 diff.h: move pickaxe fields together againÆvar Arnfjörð Bjarmason, Feb 16, 2021
  79. 10/22 pickaxe: die when --find-object and --pickaxe-all are combinedÆvar Arnfjörð Bjarmason, Feb 16, 2021
  80. 14/22 pickaxe: refactor function selection in diffcore-pickaxe()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  81. Junio C HamanoMar 30, 2021
  82. 12/22 pickaxe/style: consolidate declarations and assignmentsÆvar Arnfjörð Bjarmason, Feb 16, 2021
  83. 15/22 pickaxe: assert that we must have a needle under -G or -SÆvar Arnfjörð Bjarmason, Feb 16, 2021
  84. Junio C HamanoMar 30, 2021
  85. 17/22 pickaxe: rename variables in has_changes() for brevityÆvar Arnfjörð Bjarmason, Feb 16, 2021
  86. 13/22 perf: add performance test for pickaxeÆvar Arnfjörð Bjarmason, Feb 16, 2021
  87. 21/22 pickaxe -G: terminate early on matching linesÆvar Arnfjörð Bjarmason, Feb 16, 2021
  88. Junio C HamanoMar 31, 2021
  89. 16/22 pickaxe -S: support content with NULs under --pickaxe-regexÆvar Arnfjörð Bjarmason, Feb 16, 2021
  90. Junio C HamanoMar 30, 2021
  91. 22/22 pickaxe -G: don't special-case create/deleteÆvar Arnfjörð Bjarmason, Feb 16, 2021
  92. Junio C HamanoMar 31, 2021
  93. 18/22 pickaxe -S: slightly optimize contains()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  94. Junio C HamanoMar 30, 2021
  95. 20/22 xdiff-interface: support early exit in xdiff_outf()Ævar Arnfjörð Bjarmason, Feb 16, 2021
  96. 19/22 xdiff-interface: allow early return from xdiff_emit_{line,hunk}_fnÆvar Arnfjörð Bjarmason, Feb 16, 2021
  97. Junio C HamanoMar 31, 2021
  98. Ævar Arnfjörð BjarmasonFeb 3, 2021
  99. Ævar Arnfjörð BjarmasonFeb 3, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.