git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 7/7] grep: simplify config parsing and option parsing

From
Junio C Hamano <gitster@pobox.com>
Date
Dec 27, 2021, 18:51 UTC
Message-ID
<xmqq4k6tyj8r.fsf@gitster.g>
In-Reply-To
<xmqqpmpid1jc.fsf@gitster.g>
Junio C Hamano <gitster@pobox.com> writes:
Show 14 quoted lines
> The above does not look correct at all.
>
> What happens when the configuration parser sees these configuration
> variables in this sequence:
>
>  - grep.patternType set to say "pcre" (or anything not "default").
>  - grep.extendedRegexp set to "true".
>  - grep.patternType set to "default".
>
> After these three variable definitions with the usual "last one
> wins" (for each variable independently), the last value for the
> grep.patternType variable is "default", and the last value for
> the grep.extendedRegexp variable is "true".  The user wants to use
> the ERE patterns.

By the way, the example I gave you for the previous round, and similarly the one in the message I am responding to were all written to help you realize that it is simply a broken approach if we do not keep "default" as default and instead resolve it to either "basic" or "extended" too early. The goal of these examples was *NOT* to tell you "this single thing is broken with the code in this round so let's fix it".

It seems I am not succeeding in conveying that point, and specially I smell that in the change between v5 and v6.

So let me try to be a bit more explicit. Let's not do another round of "I think this is a moral equivalent of what you want, even though it is not done the way you suggested." I think we wasted a reroll or three with that attitude in changes leading to v6 already, after I gave my review to v5, and I think the v5 review essentially was a repeat of my review for v3's 3/7, so if I conveyed the point clearly enough back then, perhaps we didn't have to waste your time on v4 and v5, either. Sorry about that.

So, here is what this step of the series SHOULD do:
 * Use two members to keep track of the final configuration value we
   saw for grep.patternTYpe and grep.extendedRegexp independently.
   The existing .fixed and .pcre2 fields are superfluous.  But no
   more "ah, we see patternType so let's ignore extendedRegexp"
   games.
 * When parsing the command line options -G, -E, etc., update the
   .patternType member with the value found.  We do not want to and
   need to touch .extendedRegexp member, whose SOLE purpose should
   be to keep track of "what the last value we saw for
   grep.extendedRegexp configuration variable".
 * Do ALL THE ABOVE while keeping "default" in the .patternType
   member as "default" as-is given by the user; do not turn it into
   "basic" or "extended" in config callback at all.
 * At some point of your choice between the time we finished parsing
   both configuration variables and command line options and the
   time we compile the pattern string to regexp objects of various
   types, look at the .patternType member and resolve it into
   basic/extended IFF it is set to default, using .extendedRegexp
   member (for this to work correctly, it is important not to let
   -E/-G command like options to touch .extendedRegexp member---it
   should be used ONLY to keep track of "what the last value we saw
   for grep.extendedRegexp configuration variable").
 * After the above step is done, .extendedRegexp member is no longer
   needed and we can compile the pattern using only the value in
   .patternType member.

The penultimate bullet point gives us a wiggle room to lose the "commit" thing and delay it until the very last moment, the function that decides to call which regexp engine's regcomp. The important thing is that we cannot lose the value "default" from .patternType field or lose the last value given to .extendedRegexp field too early, namely, before we have read all the configurtion streams and know the last value for these two variables.

Thanks.  Hopefully I was clear enough this time.
----- >8 ---- ----- >8 ---- ----- >8 ---- ----- >8 ---- ----- >8 -----
Subject: [PATCH] fixup! grep tests: add missing "grep.patternType" config tests
---
 t/t7810-grep.sh | 10 ++++++++++
 1 file changed, 10 insertions(+)
diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh
index 664f884e12..2e2829ee55 100755
--- a/t/t7810-grep.sh
+++ b/t/t7810-grep.sh
@@ -471,6 +471,16 @@ do
 		test_cmp expected actual
 	'
 
+	test_expect_success "grep $L with grep.extendedRegexp and grep.patternType are both last-one-wins independently" '
+		echo "${HC}ab:abc" >expected &&
+		git \
+			-c grep.patternType=fixed \
+			-c grep.extendedRegexp=true \
+			-c grep.patternType=default \
+			grep "a+b*c" $H ab >actual &&
+		test_cmp expected actual
+	'
+
 	test_expect_success "grep $L with grep.patternType=extended and grep.patternType=default" '
 		echo "${HC}ab:a+bc" >expected &&
 		git \
-- 
2.34.1-568-g69e9fd72b5
Previous: Junio C HamanoNext: Ævar Arnfjörð Bjarmason
Message 86 of 151 in “grep: simplify & delete code by changing obscure cfg variable behavior”
  1. 0/8 grep: simplify & delete code by changing obscure cfg variable behaviorÆvar Arnfjörð Bjarmason, Nov 6, 2021
  2. 1/8 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Nov 6, 2021
  3. 2/8 git.c & grep.c: assert that "prefix" is NULL or non-zero stringÆvar Arnfjörð Bjarmason, Nov 6, 2021
  4. Taylor BlauNov 8, 2021
  5. 3/8 grep: remove unused "prefix_length" memberÆvar Arnfjörð Bjarmason, Nov 6, 2021
  6. Taylor BlauNov 8, 2021
  7. 4/8 grep.c: move "prefix" out of "struct grep_opt"Ævar Arnfjörð Bjarmason, Nov 6, 2021
  8. Taylor BlauNov 8, 2021
  9. Ævar Arnfjörð BjarmasonNov 9, 2021
  10. Taylor BlauNov 10, 2021
  11. 5/8 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Nov 6, 2021
  12. 6/8 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Nov 6, 2021
  13. Taylor BlauNov 8, 2021
  14. Ævar Arnfjörð BjarmasonNov 9, 2021
  15. Taylor BlauNov 10, 2021
  16. 7/8 grep: simplify config parsing, change grep.<rx config> interactionÆvar Arnfjörð Bjarmason, Nov 6, 2021
  17. Taylor BlauNov 8, 2021
  18. Ævar Arnfjörð BjarmasonNov 9, 2021
  19. Taylor BlauNov 10, 2021
  20. 8/8 grep: make "extendedRegexp=true" the same as "patternType=extended"Ævar Arnfjörð Bjarmason, Nov 6, 2021
  21. 0/8 grep: simplify & delete code by changing obscure cfg variable behaviorÆvar Arnfjörð Bjarmason, Nov 10, 2021
  22. 1/8 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Nov 10, 2021
  23. Junio C HamanoNov 12, 2021
  24. 2/8 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Nov 10, 2021
  25. Junio C HamanoNov 12, 2021
  26. 3/8 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Nov 10, 2021
  27. Junio C HamanoNov 12, 2021
  28. 4/8 grep docs: de-duplicate configuration sectionsÆvar Arnfjörð Bjarmason, Nov 10, 2021
  29. Junio C HamanoNov 12, 2021
  30. 5/8 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Nov 10, 2021
  31. Junio C HamanoNov 12, 2021
  32. 6/8 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Nov 10, 2021
  33. Junio C HamanoNov 12, 2021
  34. 7/8 grep: simplify config parsing, change grep.<rx config> interactionÆvar Arnfjörð Bjarmason, Nov 10, 2021
  35. Junio C HamanoNov 12, 2021
  36. Ævar Arnfjörð BjarmasonNov 13, 2021
  37. 8/8 grep: make "extendedRegexp=true" the same as "patternType=extended"Ævar Arnfjörð Bjarmason, Nov 10, 2021
  38. Junio C HamanoNov 12, 2021
  39. Taylor BlauNov 10, 2021
  40. 0/7 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Nov 29, 2021
  41. 1/7 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Nov 29, 2021
  42. 2/7 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Nov 29, 2021
  43. Tests in t4202 are aborted early, was: Re: [PATCH v3 2/7] logFabian Stelzer, Mar 4, 2022
  44. log tests: fix "abort tests early" regression in ff37a60c369Ævar Arnfjörð Bjarmason, Mar 4, 2022
  45. 4/7 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Nov 29, 2021
  46. 3/7 grep tests: add missing "grep.patternType" config testÆvar Arnfjörð Bjarmason, Nov 29, 2021
  47. Junio C HamanoNov 29, 2021
  48. Junio C HamanoDec 3, 2021
  49. 6/7 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Nov 29, 2021
  50. 5/7 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Nov 29, 2021
  51. 7/7 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Nov 29, 2021
  52. Junio C HamanoNov 29, 2021
  53. Junio C HamanoNov 29, 2021
  54. 0/7 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Dec 3, 2021
  55. 1/7 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Dec 3, 2021
  56. 2/7 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Dec 3, 2021
  57. 3/7 grep tests: add missing "grep.patternType" config testÆvar Arnfjörð Bjarmason, Dec 3, 2021
  58. 4/7 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Dec 3, 2021
  59. 5/7 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Dec 3, 2021
  60. 6/7 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Dec 3, 2021
  61. 7/7 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Dec 3, 2021
  62. 0/7 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Dec 22, 2021
  63. 1/7 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Dec 22, 2021
  64. 2/7 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Dec 22, 2021
  65. 3/7 grep tests: add missing "grep.patternType" config testÆvar Arnfjörð Bjarmason, Dec 22, 2021
  66. Junio C HamanoDec 23, 2021
  67. Re* [PATCH v5 3/7] grep tests: add missing "grep.patternType" config testJunio C Hamano, Dec 25, 2021
  68. grep: allow scripts to ignore configured pattern typeJunio C Hamano, Dec 25, 2021
  69. Ævar Arnfjörð BjarmasonDec 26, 2021
  70. Junio C HamanoDec 25, 2021
  71. 4/7 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Dec 22, 2021
  72. 5/7 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Dec 22, 2021
  73. 6/7 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Dec 22, 2021
  74. 7/7 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Dec 22, 2021
  75. Junio C HamanoDec 23, 2021
  76. Junio C HamanoDec 23, 2021
  77. 0/7 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Dec 26, 2021
  78. 1/7 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Dec 26, 2021
  79. 2/7 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Dec 26, 2021
  80. 3/7 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Dec 26, 2021
  81. 4/7 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Dec 26, 2021
  82. 5/7 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Dec 26, 2021
  83. 6/7 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Dec 26, 2021
  84. 7/7 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Dec 26, 2021
  85. Junio C HamanoDec 27, 2021
  86. Junio C HamanoDec 27, 2021
  87. 00/10 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Dec 28, 2021
  88. 01/10 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Dec 28, 2021
  89. 02/10 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Dec 28, 2021
  90. 03/10 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Dec 28, 2021
  91. 04/10 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Dec 28, 2021
  92. 05/10 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Dec 28, 2021
  93. 06/10 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Dec 28, 2021
  94. 10/10 grep.[ch]: remove GREP_PATTERN_TYPE_UNSPECIFIEDÆvar Arnfjörð Bjarmason, Dec 28, 2021
  95. 09/10 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Dec 28, 2021
  96. 07/10 grep.h: make "grep_opt.pattern_type_option" use its enumÆvar Arnfjörð Bjarmason, Dec 28, 2021
  97. 08/10 grep.c: do "if (bool && memchr())" not "if (memchr() && bool)"Ævar Arnfjörð Bjarmason, Dec 28, 2021
  98. 00/10 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Jan 18, 2022
  99. 01/10 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Jan 18, 2022
  100. 02/10 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Jan 18, 2022
  101. 03/10 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Jan 18, 2022
  102. 04/10 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Jan 18, 2022
  103. 05/10 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Jan 18, 2022
  104. 06/10 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Jan 18, 2022
  105. 07/10 grep.h: make "grep_opt.pattern_type_option" use its enumÆvar Arnfjörð Bjarmason, Jan 18, 2022
  106. 08/10 grep.c: do "if (bool && memchr())" not "if (memchr() && bool)"Ævar Arnfjörð Bjarmason, Jan 18, 2022
  107. 10/10 grep.[ch]: remove GREP_PATTERN_TYPE_UNSPECIFIEDÆvar Arnfjörð Bjarmason, Jan 18, 2022
  108. 09/10 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Jan 18, 2022
  109. Junio C HamanoJan 18, 2022
  110. Junio C HamanoJan 18, 2022
  111. Ævar Arnfjörð BjarmasonJan 19, 2022
  112. Junio C HamanoJan 19, 2022
  113. Ævar Arnfjörð BjarmasonJan 19, 2022
  114. 0/9 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Jan 27, 2022
  115. 1/9 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Jan 27, 2022
  116. 2/9 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Jan 27, 2022
  117. 3/9 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Jan 27, 2022
  118. 4/9 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Jan 27, 2022
  119. 5/9 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Jan 27, 2022
  120. 6/9 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Jan 27, 2022
  121. 7/9 grep.h: make "grep_opt.pattern_type_option" use its enumÆvar Arnfjörð Bjarmason, Jan 27, 2022
  122. 9/9 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Jan 27, 2022
  123. Junio C HamanoJan 27, 2022
  124. Junio C HamanoJan 27, 2022
  125. Junio C HamanoJan 27, 2022
  126. 8/9 grep.c: do "if (bool && memchr())" not "if (memchr() && bool)"Ævar Arnfjörð Bjarmason, Jan 27, 2022
  127. 0/9 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Feb 4, 2022
  128. 1/9 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Feb 4, 2022
  129. 3/9 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Feb 4, 2022
  130. Junio C HamanoFeb 4, 2022
  131. Junio C HamanoFeb 4, 2022
  132. 2/9 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Feb 4, 2022
  133. 5/9 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Feb 4, 2022
  134. 4/9 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Feb 4, 2022
  135. 7/9 grep.h: make "grep_opt.pattern_type_option" use its enumÆvar Arnfjörð Bjarmason, Feb 4, 2022
  136. 8/9 grep.c: do "if (bool && memchr())" not "if (memchr() && bool)"Ævar Arnfjörð Bjarmason, Feb 4, 2022
  137. 6/9 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Feb 4, 2022
  138. 9/9 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Feb 4, 2022
  139. Junio C HamanoFeb 4, 2022
  140. 00/10 grep: simplify & delete "init" & "config" codeÆvar Arnfjörð Bjarmason, Feb 16, 2022
  141. 01/10 grep.h: remove unused "regex_t regexp" from grep_optÆvar Arnfjörð Bjarmason, Feb 16, 2022
  142. 02/10 log tests: check if grep_config() is called by "log"-like cmdsÆvar Arnfjörð Bjarmason, Feb 16, 2022
  143. 05/10 built-ins: trust the "prefix" from run_builtin()Ævar Arnfjörð Bjarmason, Feb 16, 2022
  144. 03/10 grep tests: create a helper function for "BRE" or "ERE"Ævar Arnfjörð Bjarmason, Feb 16, 2022
  145. 04/10 grep tests: add missing "grep.patternType" config testsÆvar Arnfjörð Bjarmason, Feb 16, 2022
  146. 06/10 grep.c: don't pass along NULL callback valueÆvar Arnfjörð Bjarmason, Feb 16, 2022
  147. 08/10 grep.h: make "grep_opt.pattern_type_option" use its enumÆvar Arnfjörð Bjarmason, Feb 16, 2022
  148. 07/10 grep API: call grep_config() after grep_init()Ævar Arnfjörð Bjarmason, Feb 16, 2022
  149. 09/10 grep.c: do "if (bool && memchr())" not "if (memchr() && bool)"Ævar Arnfjörð Bjarmason, Feb 16, 2022
  150. 10/10 grep: simplify config parsing and option parsingÆvar Arnfjörð Bjarmason, Feb 16, 2022
  151. Junio C HamanoFeb 16, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.