{"thread":{"id":"45765","subject":"[PATCH v3 00/18] PCRE v1 improvements & PCRE v2 support","startedAt":"2017-04-20T21:24:15Z","lastAt":"2017-04-20T22:27:57Z","messageCount":22,"participants":["Ævar Arnfjörð Bjarmason","Brandon Williams","Jeff King"],"isPatch":true,"patchVersion":3,"patchTotal":18},"messages":[{"id":"317403","messageId":"20170420212345.7408-1-avarab@gmail.com","threadId":"45765","inReplyTo":null,"subject":"[PATCH v3 00/18] PCRE v1 improvements & PCRE v2 support","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:27Z","receivedAt":"2017-04-20T21:24:15Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Sorry about the high volume sending. I thought I wouldn't have time to\nwork on v3 for a while, but here it is a day later. I promise to hold\noff on further sending for a bit.\n\nThis, unlike v2, addresses all the outstanding comments the series\nhad. Most importantly I added a patch at the end to remove the\nconcurrent support for v1 & v2 of the library as Jeff & Junio\nsuggested.\n\nSee the v2 coverletter in <20170419224053.8920-1-avarab@gmail.com> for\nchanges since v1, changes since v2 noted below:\n\nÆvar Arnfjörð Bjarmason (18):\n grep: amend submodule recursion test in preparation for rx engine\n testing\n grep: add tests for grep pattern types being passed to submodules\n grep: submodule-related case statements should die if new fields are\n added\n\nI thought there was a bug in v2 still where submodule grepping\nwouldn't properly pass along e.g. `-c grep.patternType=pcre2`, this\nturned out to not be the case since we pass -c options to subprocesses\nvia the GIT_CONFIG_PARAMETERS env var, which I didn't know about.\n\nBut in preparing to fix that I wrote these tests which improve on some\nblind spots in the tests Brandon initially added for the submodule\ngrep. It's part of this series because it would conflict with\nsubsequent patches if sent stand-alone, but is otherwise unrelated.\n\n grep: remove redundant regflags assignment under PCRE\n Makefile & configure: reword outdated comment about PCRE\n\nNo changes.\n\n grep: add a test for backreferences in PCRE patterns\n\ns/PCRE/LIBPCRE/ for the prereq check. That's changed later, but this\npatch didn't work properly stand-alone, error introduced in earlier\nv1-era rebasing.\n\n log: add exhaustive tests for pattern style options & config\n log: add -P as a synonym for --perl-regexp\n\nNo changes.\n\n grep & rev-list doc: stop promising libpcre for --perl-regexp\n\nI've reworded these docs to make even weaker promises about what's a\n\"Perl-compatible regular expression\". As noted in the commit message\nthe motivation is that maybe in the future we'd like to provide -P via\nsome entirely different library, e.g. Intel's hyperscan. This leaves\nthat door open without breaking existing promises in the\ndocumentation.\n\n grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"\n\nNow with tests for recursive submodule grepping.\n\n test-lib: rename the LIBPCRE prerequisite to PCRE\n\nNow with s/LIBPCRE/PCRE/ back for the s/PCRE/LIBPCRE/ change noted earlier.\n\n grep: change the internal PCRE macro names to be PCRE1\n grep: change the internal PCRE code & header names to be PCRE1\n perf: add a performance comparison test of grep -E and -P\n grep: add support for the PCRE v1 JIT API\n\nNo changes.\n\n grep: add support for PCRE v2\n\nFix one s/LIBPCREDIR/LIBPCRE2DIR/ in a ./configure notice which I\nmissed & tests & minor changes for recursive submodule grepping.\n\n grep: remove support concurrent use of both PCRE v1 & v2\n\nNEW: Removes grep.patternType=[pcre|pcre1|pcre2] & makes trying to\ncompile git with both v1 & v2 an error, but either one will work, with\nv1 being the default still.\n\nAfter writing this up & seeing what the change is I also agree that\nit's a good idea to apply this. It makes the user-facing docs simpler,\nbut also the various tests which previously had to worry about\n[pcre|pcre1|pcre2] which now just test \"perl\".\n\n Makefile & configure: make PCRE v2 the default PCRE implementation\n\nNEW: This changes the default PCRE implementation to v2, and makes\nUSE_LIBPCRE mean USE_LIBPCRE2, but USE_LIBPCRE1 is still available.\n\nI originally wrote this as something to keep for some future\nsubmission, but come to think of it I can't see why it shouldn't be\napplied.\n\nThe v2 PCRE is stable & end-user compatible, all this change does is\nchange the default, someone building a new git is likely to also have\npackaged PCRE v2 sometime in the last 2 years since it was released,\nand if not they can choose to use the legacy v2 library by making the\ntrivial s/USE_LIBPCRE/USE_LIBPCRE1/ change, or package up PCRE v2.\n\n Documentation/git-grep.txt | 7 +-\n Documentation/rev-list-options.txt | 9 +-\n Makefile | 39 +++++--\n builtin/grep.c | 4 +\n configure.ac | 81 +++++++++++---\n grep.c | 214 ++++++++++++++++++++++++++++++------\n grep.h | 32 +++++-\n revision.c | 2 +-\n t/README | 4 +-\n t/perf/p7820-grep-engines.sh | 25 +++++\n t/t4202-log.sh | 86 ++++++++++++++-\n t/t7810-grep.sh | 41 ++++---\n t/t7812-grep-icase-non-ascii.sh | 4 +-\n t/t7813-grep-icase-iso.sh | 11 +-\n t/t7814-grep-recurse-submodules.sh | 215 +++++++++++++++++++++++--------------\n t/test-lib.sh | 3 +-\n 16 files changed, 608 insertions(+), 169 deletions(-)\n create mode 100755 t/perf/p7820-grep-engines.sh\n\n-- \n2.11.0\n\n"},{"id":"317404","messageId":"20170420212345.7408-2-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 01/18] grep: amend submodule recursion test in preparation for rx engine testing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:28Z","receivedAt":"2017-04-20T21:24:17Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Amend the submodule recursion test added in commit 0281e487fd (\"grep:\noptionally recurse into submodules\", 2016-12-16) to prepare it for\nsubsequent tests of whether it passes along the grep.patternType to\nthe submodule greps.\n\nThis is just the result of searching & replacing:\n\n foobar -> (1|2)d(3|4)\n foo -> (1|2)\n bar -> (3|4)\n\nCurrently there's no tests for whether e.g. -P or -E is correctly\npassed along, tests for that will be added in a follow-up change, but\nfirst add content to the tests which will match differently under\ndifferent regex engines.\n\nReuse the pattern established in an earlier commit of mine in this\nseries (\"log: add exhaustive tests for pattern style options &\nconfig\", 2017-04-07). The pattern \"(.|.)[\\d]\" will match this content\ndifferently under fixed/basic/extended & perl.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t7814-grep-recurse-submodules.sh | 166 ++++++++++++++++++-------------------\n 1 file changed, 83 insertions(+), 83 deletions(-)\n\ndiff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh\nindex 5b6eb3a65e..3c580b38ba 100755\n--- a/t/t7814-grep-recurse-submodules.sh\n+++ b/t/t7814-grep-recurse-submodules.sh\n@@ -9,13 +9,13 @@ submodules.\n . ./test-lib.sh\n \n test_expect_success 'setup directory structure and submodule' '\n-\techo \"foobar\" >a &&\n+\techo \"(1|2)d(3|4)\" >a &&\n \tmkdir b &&\n-\techo \"bar\" >b/b &&\n+\techo \"(3|4)\" >b/b &&\n \tgit add a b &&\n \tgit commit -m \"add a and b\" &&\n \tgit init submodule &&\n-\techo \"foobar\" >submodule/a &&\n+\techo \"(1|2)d(3|4)\" >submodule/a &&\n \tgit -C submodule add a &&\n \tgit -C submodule commit -m \"add a\" &&\n \tgit submodule add ./submodule &&\n@@ -24,18 +24,18 @@ test_expect_success 'setup directory structure and submodule' '\n \n test_expect_success 'grep correctly finds patterns in a submodule' '\n \tcat >expect <<-\\EOF &&\n-\ta:foobar\n-\tb/b:bar\n-\tsubmodule/a:foobar\n+\ta:(1|2)d(3|4)\n+\tb/b:(3|4)\n+\tsubmodule/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep and basic pathspecs' '\n \tcat >expect <<-\\EOF &&\n-\tsubmodule/a:foobar\n+\tsubmodule/a:(1|2)d(3|4)\n \tEOF\n \n \tgit grep -e. --recurse-submodules -- submodule >actual &&\n@@ -44,7 +44,7 @@ test_expect_success 'grep and basic pathspecs' '\n \n test_expect_success 'grep and nested submodules' '\n \tgit init submodule/sub &&\n-\techo \"foobar\" >submodule/sub/a &&\n+\techo \"(1|2)d(3|4)\" >submodule/sub/a &&\n \tgit -C submodule/sub add a &&\n \tgit -C submodule/sub commit -m \"add a\" &&\n \tgit -C submodule submodule add ./sub &&\n@@ -54,117 +54,117 @@ test_expect_success 'grep and nested submodules' '\n \tgit commit -m \"updated submodule\" &&\n \n \tcat >expect <<-\\EOF &&\n-\ta:foobar\n-\tb/b:bar\n-\tsubmodule/a:foobar\n-\tsubmodule/sub/a:foobar\n+\ta:(1|2)d(3|4)\n+\tb/b:(3|4)\n+\tsubmodule/a:(1|2)d(3|4)\n+\tsubmodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep and multiple patterns' '\n \tcat >expect <<-\\EOF &&\n-\ta:foobar\n-\tsubmodule/a:foobar\n-\tsubmodule/sub/a:foobar\n+\ta:(1|2)d(3|4)\n+\tsubmodule/a:(1|2)d(3|4)\n+\tsubmodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --and -e \"foo\" --recurse-submodules >actual &&\n+\tgit grep -e \"(3|4)\" --and -e \"(1|2)d\" --recurse-submodules >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep and multiple patterns' '\n \tcat >expect <<-\\EOF &&\n-\tb/b:bar\n+\tb/b:(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --and --not -e \"foo\" --recurse-submodules >actual &&\n+\tgit grep -e \"(3|4)\" --and --not -e \"(1|2)d\" --recurse-submodules >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'basic grep tree' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD:a:foobar\n-\tHEAD:b/b:bar\n-\tHEAD:submodule/a:foobar\n-\tHEAD:submodule/sub/a:foobar\n+\tHEAD:a:(1|2)d(3|4)\n+\tHEAD:b/b:(3|4)\n+\tHEAD:submodule/a:(1|2)d(3|4)\n+\tHEAD:submodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree HEAD^' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD^:a:foobar\n-\tHEAD^:b/b:bar\n-\tHEAD^:submodule/a:foobar\n+\tHEAD^:a:(1|2)d(3|4)\n+\tHEAD^:b/b:(3|4)\n+\tHEAD^:submodule/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD^ >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD^ >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree HEAD^^' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD^^:a:foobar\n-\tHEAD^^:b/b:bar\n+\tHEAD^^:a:(1|2)d(3|4)\n+\tHEAD^^:b/b:(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD^^ >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD^^ >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree and pathspecs' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD:submodule/a:foobar\n-\tHEAD:submodule/sub/a:foobar\n+\tHEAD:submodule/a:(1|2)d(3|4)\n+\tHEAD:submodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD -- submodule >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD -- submodule >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree and pathspecs' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD:submodule/a:foobar\n-\tHEAD:submodule/sub/a:foobar\n+\tHEAD:submodule/a:(1|2)d(3|4)\n+\tHEAD:submodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD -- \"submodule*a\" >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD -- \"submodule*a\" >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree and more pathspecs' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD:submodule/a:foobar\n+\tHEAD:submodule/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD -- \"submodul?/a\" >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD -- \"submodul?/a\" >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep tree and more pathspecs' '\n \tcat >expect <<-\\EOF &&\n-\tHEAD:submodule/sub/a:foobar\n+\tHEAD:submodule/sub/a:(1|2)d(3|4)\n \tEOF\n \n-\tgit grep -e \"bar\" --recurse-submodules HEAD -- \"submodul*/sub/a\" >actual &&\n+\tgit grep -e \"(3|4)\" --recurse-submodules HEAD -- \"submodul*/sub/a\" >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success !MINGW 'grep recurse submodule colon in name' '\n \tgit init parent &&\n \ttest_when_finished \"rm -rf parent\" &&\n-\techo \"foobar\" >\"parent/fi:le\" &&\n+\techo \"(1|2)d(3|4)\" >\"parent/fi:le\" &&\n \tgit -C parent add \"fi:le\" &&\n \tgit -C parent commit -m \"add fi:le\" &&\n \n \tgit init \"su:b\" &&\n \ttest_when_finished \"rm -rf su:b\" &&\n-\techo \"foobar\" >\"su:b/fi:le\" &&\n+\techo \"(1|2)d(3|4)\" >\"su:b/fi:le\" &&\n \tgit -C \"su:b\" add \"fi:le\" &&\n \tgit -C \"su:b\" commit -m \"add fi:le\" &&\n \n@@ -172,30 +172,30 @@ test_expect_success !MINGW 'grep recurse submodule colon in name' '\n \tgit -C parent commit -m \"add submodule\" &&\n \n \tcat >expect <<-\\EOF &&\n-\tfi:le:foobar\n-\tsu:b/fi:le:foobar\n+\tfi:le:(1|2)d(3|4)\n+\tsu:b/fi:le:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep -e \"foobar\" --recurse-submodules >actual &&\n+\tgit -C parent grep -e \"(1|2)d(3|4)\" --recurse-submodules >actual &&\n \ttest_cmp expect actual &&\n \n \tcat >expect <<-\\EOF &&\n-\tHEAD:fi:le:foobar\n-\tHEAD:su:b/fi:le:foobar\n+\tHEAD:fi:le:(1|2)d(3|4)\n+\tHEAD:su:b/fi:le:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep -e \"foobar\" --recurse-submodules HEAD >actual &&\n+\tgit -C parent grep -e \"(1|2)d(3|4)\" --recurse-submodules HEAD >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep history with moved submoules' '\n \tgit init parent &&\n \ttest_when_finished \"rm -rf parent\" &&\n-\techo \"foobar\" >parent/file &&\n+\techo \"(1|2)d(3|4)\" >parent/file &&\n \tgit -C parent add file &&\n \tgit -C parent commit -m \"add file\" &&\n \n \tgit init sub &&\n \ttest_when_finished \"rm -rf sub\" &&\n-\techo \"foobar\" >sub/file &&\n+\techo \"(1|2)d(3|4)\" >sub/file &&\n \tgit -C sub add file &&\n \tgit -C sub commit -m \"add file\" &&\n \n@@ -203,82 +203,82 @@ test_expect_success 'grep history with moved submoules' '\n \tgit -C parent commit -m \"add submodule\" &&\n \n \tcat >expect <<-\\EOF &&\n-\tdir/sub/file:foobar\n-\tfile:foobar\n+\tdir/sub/file:(1|2)d(3|4)\n+\tfile:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep -e \"foobar\" --recurse-submodules >actual &&\n+\tgit -C parent grep -e \"(1|2)d(3|4)\" --recurse-submodules >actual &&\n \ttest_cmp expect actual &&\n \n \tgit -C parent mv dir/sub sub-moved &&\n \tgit -C parent commit -m \"moved submodule\" &&\n \n \tcat >expect <<-\\EOF &&\n-\tfile:foobar\n-\tsub-moved/file:foobar\n+\tfile:(1|2)d(3|4)\n+\tsub-moved/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep -e \"foobar\" --recurse-submodules >actual &&\n+\tgit -C parent grep -e \"(1|2)d(3|4)\" --recurse-submodules >actual &&\n \ttest_cmp expect actual &&\n \n \tcat >expect <<-\\EOF &&\n-\tHEAD^:dir/sub/file:foobar\n-\tHEAD^:file:foobar\n+\tHEAD^:dir/sub/file:(1|2)d(3|4)\n+\tHEAD^:file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep -e \"foobar\" --recurse-submodules HEAD^ >actual &&\n+\tgit -C parent grep -e \"(1|2)d(3|4)\" --recurse-submodules HEAD^ >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep using relative path' '\n \ttest_when_finished \"rm -rf parent sub\" &&\n \tgit init sub &&\n-\techo \"foobar\" >sub/file &&\n+\techo \"(1|2)d(3|4)\" >sub/file &&\n \tgit -C sub add file &&\n \tgit -C sub commit -m \"add file\" &&\n \n \tgit init parent &&\n-\techo \"foobar\" >parent/file &&\n+\techo \"(1|2)d(3|4)\" >parent/file &&\n \tgit -C parent add file &&\n \tmkdir parent/src &&\n-\techo \"foobar\" >parent/src/file2 &&\n+\techo \"(1|2)d(3|4)\" >parent/src/file2 &&\n \tgit -C parent add src/file2 &&\n \tgit -C parent submodule add ../sub &&\n \tgit -C parent commit -m \"add files and submodule\" &&\n \n \t# From top works\n \tcat >expect <<-\\EOF &&\n-\tfile:foobar\n-\tsrc/file2:foobar\n-\tsub/file:foobar\n+\tfile:(1|2)d(3|4)\n+\tsrc/file2:(1|2)d(3|4)\n+\tsub/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep --recurse-submodules -e \"foobar\" >actual &&\n+\tgit -C parent grep --recurse-submodules -e \"(1|2)d(3|4)\" >actual &&\n \ttest_cmp expect actual &&\n \n \t# Relative path to top\n \tcat >expect <<-\\EOF &&\n-\t../file:foobar\n-\tfile2:foobar\n-\t../sub/file:foobar\n+\t../file:(1|2)d(3|4)\n+\tfile2:(1|2)d(3|4)\n+\t../sub/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent/src grep --recurse-submodules -e \"foobar\" -- .. >actual &&\n+\tgit -C parent/src grep --recurse-submodules -e \"(1|2)d(3|4)\" -- .. >actual &&\n \ttest_cmp expect actual &&\n \n \t# Relative path to submodule\n \tcat >expect <<-\\EOF &&\n-\t../sub/file:foobar\n+\t../sub/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent/src grep --recurse-submodules -e \"foobar\" -- ../sub >actual &&\n+\tgit -C parent/src grep --recurse-submodules -e \"(1|2)d(3|4)\" -- ../sub >actual &&\n \ttest_cmp expect actual\n '\n \n test_expect_success 'grep from a subdir' '\n \ttest_when_finished \"rm -rf parent sub\" &&\n \tgit init sub &&\n-\techo \"foobar\" >sub/file &&\n+\techo \"(1|2)d(3|4)\" >sub/file &&\n \tgit -C sub add file &&\n \tgit -C sub commit -m \"add file\" &&\n \n \tgit init parent &&\n \tmkdir parent/src &&\n-\techo \"foobar\" >parent/src/file &&\n+\techo \"(1|2)d(3|4)\" >parent/src/file &&\n \tgit -C parent add src/file &&\n \tgit -C parent submodule add ../sub src/sub &&\n \tgit -C parent submodule add ../sub sub &&\n@@ -286,19 +286,19 @@ test_expect_success 'grep from a subdir' '\n \n \t# Verify grep from root works\n \tcat >expect <<-\\EOF &&\n-\tsrc/file:foobar\n-\tsrc/sub/file:foobar\n-\tsub/file:foobar\n+\tsrc/file:(1|2)d(3|4)\n+\tsrc/sub/file:(1|2)d(3|4)\n+\tsub/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent grep --recurse-submodules -e \"foobar\" >actual &&\n+\tgit -C parent grep --recurse-submodules -e \"(1|2)d(3|4)\" >actual &&\n \ttest_cmp expect actual &&\n \n \t# Verify grep from a subdir works\n \tcat >expect <<-\\EOF &&\n-\tfile:foobar\n-\tsub/file:foobar\n+\tfile:(1|2)d(3|4)\n+\tsub/file:(1|2)d(3|4)\n \tEOF\n-\tgit -C parent/src grep --recurse-submodules -e \"foobar\" >actual &&\n+\tgit -C parent/src grep --recurse-submodules -e \"(1|2)d(3|4)\" >actual &&\n \ttest_cmp expect actual\n '\n \n-- \n2.11.0\n\n"},{"id":"317405","messageId":"20170420212345.7408-3-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 02/18] grep: add tests for grep pattern types being passed to submodules","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:29Z","receivedAt":"2017-04-20T21:24:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add testing for grep pattern types being correctly passed to\nsubmodules. The pattern \"(.|.)[\\d]\" matches differently under\nfixed (not at all), and then matches different lines under\nbasic/extended & perl regular expressions, so this change asserts that\nthe pattern type is passed along correctly.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t7814-grep-recurse-submodules.sh | 49 ++++++++++++++++++++++++++++++++++++++\n 1 file changed, 49 insertions(+)\n\ndiff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh\nindex 3c580b38ba..ef658b7899 100755\n--- a/t/t7814-grep-recurse-submodules.sh\n+++ b/t/t7814-grep-recurse-submodules.sh\n@@ -313,4 +313,53 @@ test_incompatible_with_recurse_submodules ()\n test_incompatible_with_recurse_submodules --untracked\n test_incompatible_with_recurse_submodules --no-index\n \n+test_expect_success 'grep --recurse-submodules should pass the pattern type along' '\n+\t# Fixed\n+\ttest_must_fail git grep -F --recurse-submodules -e \"(.|.)[\\d]\" &&\n+\ttest_must_fail git -c grep.patternType=fixed grep --recurse-submodules -e \"(.|.)[\\d]\" &&\n+\n+\t# Basic\n+\tgit grep -G --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\ta:(1|2)d(3|4)\n+\tsubmodule/a:(1|2)d(3|4)\n+\tsubmodule/sub/a:(1|2)d(3|4)\n+\tEOF\n+\ttest_cmp expect actual &&\n+\tgit -c grep.patternType=basic grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\t# Extended\n+\tgit grep -E --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules:[submodule \"submodule\"]\n+\t.gitmodules:\tpath = submodule\n+\t.gitmodules:\turl = ./submodule\n+\ta:(1|2)d(3|4)\n+\tsubmodule/.gitmodules:[submodule \"sub\"]\n+\tsubmodule/a:(1|2)d(3|4)\n+\tsubmodule/sub/a:(1|2)d(3|4)\n+\tEOF\n+\ttest_cmp expect actual &&\n+\tgit -c grep.patternType=extended grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit -c grep.extendedRegexp=true grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\ttest_cmp expect actual &&\n+\n+\t# Perl\n+\tif test_have_prereq PCRE\n+\tthen\n+\t\tgit grep -P --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\tcat >expect <<-\\EOF &&\n+\t\ta:(1|2)d(3|4)\n+\t\tb/b:(3|4)\n+\t\tsubmodule/a:(1|2)d(3|4)\n+\t\tsubmodule/sub/a:(1|2)d(3|4)\n+\t\tEOF\n+\t\ttest_cmp expect actual &&\n+\t\tgit -c grep.patternType=perl grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\ttest_cmp expect actual\n+\tfi\n+'\n+\n test_done\n-- \n2.11.0\n\n"},{"id":"317406","messageId":"20170420212345.7408-4-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 03/18] grep: submodule-related case statements should die if new fields are added","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:30Z","receivedAt":"2017-04-20T21:24:22Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change two case statements added in commit 0281e487fd (\"grep:\noptionally recurse into submodules\", 2016-12-16) so that they die if\nnew GREP_PATTERN_* enum fields are added without updating them.\n\nThese case statements currently check for an exhaustive list of\nfields, but if a new field is added it's easy to introduce a bug here\nwhere the code will start subtly doing the wrong thing, e.g. if a new\npattern type is added we'll fall through to\nGREP_PATTERN_TYPE_UNSPECIFIED, i.e. the \"basic\" POSIX regular\nexpressions.\n\nThis should arguably be done for the switch(opt->binary)\ncase-statement as well, but isn't trivial to add since that code isn't\ncurrently working with an exhaustive list.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/grep.c | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 3ffb5b4e81..be3dbd6957 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -495,6 +495,8 @@ static void compile_submodule_options(const struct grep_opt *opt,\n \t\tbreak;\n \tcase GREP_PATTERN_TYPE_UNSPECIFIED:\n \t\tbreak;\n+\tdefault:\n+\t\tdie(\"BUG: Added a new grep pattern type without updating switch statement\");\n \t}\n \n \tfor (pattern = opt->pattern_list; pattern != NULL;\n@@ -515,6 +517,8 @@ static void compile_submodule_options(const struct grep_opt *opt,\n \t\tcase GREP_PATTERN_BODY:\n \t\tcase GREP_PATTERN_HEAD:\n \t\t\tbreak;\n+\t\tdefault:\n+\t\t\tdie(\"BUG: Added a new grep token type without updating case statement\");\n \t\t}\n \t}\n \n-- \n2.11.0\n\n"},{"id":"317407","messageId":"20170420212345.7408-5-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 04/18] grep: remove redundant regflags assignment under PCRE","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:31Z","receivedAt":"2017-04-20T21:24:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove a redundant assignment to the \"regflags\" variable. This\nvariable is only used for POSIX regular expression matching, not when\nthe PCRE library is used.\n\nThis redundant assignment was added as a result of copy/paste\nprogramming in commit 84befcd0a4 (\"grep: add a grep.patternType\nconfiguration setting\", 2012-08-03). That commit modified already\nworking code in commit cca2c172e0 (\"git-grep: do not die upon -F/-P\nwhen grep.extendedRegexp is set.\", 2011-05-09) which didn't assign to\nregflags when under PCRE.\n\nRevert back to that behavior, more to reduce \"wait this is used under\nPCRE how?\" confusion when reading the code, than to to save ourselves\ntrivial CPU cycles by removing one assignment.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n grep.c | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/grep.c b/grep.c\nindex 47cee45067..59ae7809f2 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -197,7 +197,6 @@ static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, st\n \tcase GREP_PATTERN_TYPE_PCRE:\n \t\topt->fixed = 0;\n \t\topt->pcre = 1;\n-\t\topt->regflags &= ~REG_EXTENDED;\n \t\tbreak;\n \t}\n }\n-- \n2.11.0\n\n"},{"id":"317409","messageId":"20170420212345.7408-6-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 05/18] Makefile & configure: reword outdated comment about PCRE","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:32Z","receivedAt":"2017-04-20T21:24:40Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Reword an outdated comment which suggests that only git-grep can use\nPCRE.\n\nThis comment was added back when PCRE support was initially added in\ncommit 63e7e9d8b6 (\"git-grep: Learn PCRE\", 2011-05-09), and was true\nat the time.\n\nIt hasn't been telling the full truth since git-log learned to use\nPCRE with --grep in commit 727b6fc3ed (\"log --grep: accept\n--basic-regexp and --perl-regexp\", 2012-10-03), and more importantly\nis likely to get more inaccurate over time as more use is made of PCRE\nin other areas.\n\nReword it to be more future-proof, and to more clearly explain that\nthis enables user-initiated runtime behavior.\n\nCopy/pasting this so much in configure.ac is lame, these Makefile-like\nflags aren't even used by autoconf, just the corresponding\n--with[out]-* options. But copy/pasting the comments that make sense\nfor the Makefile to configure.ac where they make less sence is the\npattern everything else follows in that file. I'm not going to war\nagainst that as part of this change, just following the existing\npattern.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 6 ++++--\n configure.ac | 12 ++++++++----\n 2 files changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex eb1a1a7cff..2e63b1cfcc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -24,8 +24,10 @@ all::\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n # This also implies BLK_SHA1.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. git-grep will be\n-# able to use Perl-compatible regular expressions.\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n #\n # Define LIBPCREDIR=/foo/bar if your libpcre header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\ndiff --git a/configure.ac b/configure.ac\nindex 128165529f..deeb968daa 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -250,8 +250,10 @@ AS_HELP_STRING([--with-openssl],[use OpenSSL library (default is YES)])\n AS_HELP_STRING([], [ARG can be prefix for openssl library and headers]),\n GIT_PARSE_WITH([openssl]))\n \n-# Define USE_LIBPCRE if you have and want to use libpcre. git-grep will be\n-# able to use Perl-compatible regular expressions.\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n #\n # Define LIBPCREDIR=/foo/bar if your libpcre header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\n@@ -499,8 +501,10 @@ GIT_CONF_SUBST([NEEDS_SSL_WITH_CRYPTO])\n GIT_CONF_SUBST([NO_OPENSSL])\n \n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. git-grep will be\n-# able to use Perl-compatible regular expressions.\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n #\n \n if test -n \"$USE_LIBPCRE\"; then\n-- \n2.11.0\n\n"},{"id":"317410","messageId":"20170420212345.7408-7-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 06/18] grep: add a test for backreferences in PCRE patterns","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:33Z","receivedAt":"2017-04-20T21:24:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a test for backreferences such as (.)\\1 in PCRE patterns. This\ntest ensures that the PCRE_NO_AUTO_CAPTURE option isn't turned\non. Before this change turning it on would break these sort of\npatterns, but wouldn't break any tests.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t7810-grep.sh | 7 +++++++\n 1 file changed, 7 insertions(+)\n\ndiff --git a/t/t7810-grep.sh b/t/t7810-grep.sh\nindex cee42097b0..ec8fe585a7 100755\n--- a/t/t7810-grep.sh\n+++ b/t/t7810-grep.sh\n@@ -1102,6 +1102,13 @@ test_expect_success LIBPCRE 'grep -P -w pattern' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success LIBPCRE 'grep -P backreferences work (the PCRE NO_AUTO_CAPTURE flag is not set)' '\n+\tgit grep -P -h \"(?P.)(?P=one)\" hello_world >actual &&\n+\ttest_cmp hello_world actual &&\n+\tgit grep -P -h \"(.)\\1\" hello_world >actual &&\n+\ttest_cmp hello_world actual\n+'\n+\n test_expect_success 'grep -G invalidpattern properly dies ' '\n \ttest_must_fail git grep -G \"a[\"\n '\n-- \n2.11.0\n\n"},{"id":"317411","messageId":"20170420212345.7408-9-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 08/18] log: add -P as a synonym for --perl-regexp","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:35Z","receivedAt":"2017-04-20T21:24:51Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add a short -P option as a synonym for the longer --perl-regexp, for\nconsistency with the options the corresponding grep invocations\naccept.\n\nThis was intentionally omitted in commit 727b6fc3ed (\"log --grep:\naccept --basic-regexp and --perl-regexp\", 2012-10-03) for unspecified\nfuture use.\n\nSince nothing has come along in over 4 1/2 years that's wanted to use\nit, it's more valuable to make it consistent with \"grep\" than to keep\nit open for future use, and to avoid the confusion of -P meaning\ndifferent things for grep & log, as is the case with the -G option.\n\nAs noted in the aforementioned commit the --basic-regexp option can't\nhave a corresponding -G argument, as the log command already uses that\nfor -G.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Documentation/rev-list-options.txt | 1 +\n revision.c | 2 +-\n t/t4202-log.sh | 9 +++++++++\n 3 files changed, 11 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex a02f7324c0..5b7fa49a7f 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -91,6 +91,7 @@ endif::git-rev-list[]\n \tConsider the limiting patterns to be fixed strings (don't interpret\n \tpattern as a regular expression).\n \n+-P::\n --perl-regexp::\n \tConsider the limiting patterns to be Perl-compatible regular expressions.\n \tRequires libpcre to be compiled in.\ndiff --git a/revision.c b/revision.c\nindex 7ff61ff5f7..03a3a012de 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1995,7 +1995,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t\tDIFF_OPT_SET(&revs->diffopt, PICKAXE_IGNORE_CASE);\n \t} else if (!strcmp(arg, \"--fixed-strings\") || !strcmp(arg, \"-F\")) {\n \t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_FIXED;\n-\t} else if (!strcmp(arg, \"--perl-regexp\")) {\n+\t} else if (!strcmp(arg, \"--perl-regexp\") || !strcmp(arg, \"-P\")) {\n \t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_PCRE;\n \t} else if (!strcmp(arg, \"--all-match\")) {\n \t\trevs->grep_filter.all_match = 1;\ndiff --git a/t/t4202-log.sh b/t/t4202-log.sh\nindex fc186f10ea..06acc848b8 100755\n--- a/t/t4202-log.sh\n+++ b/t/t4202-log.sh\n@@ -331,8 +331,17 @@ test_expect_success 'log with various grep.patternType configurations & command-\n \t\t\t--grep=\"(1|2)\" >actual.fixed.short-arg &&\n \t\tgit log --pretty=tformat:%s -E \\\n \t\t\t--grep=\"\\|2\" >actual.extended.short-arg &&\n+\t\tif test_have_prereq PCRE\n+\t\tthen\n+\t\t\tgit log --pretty=tformat:%s -P \\\n+\t\t\t\t--grep=\"[\\d]\\|\" >actual.perl.short-arg\n+\t\tfi &&\n \t\ttest_cmp expect.fixed actual.fixed.short-arg &&\n \t\ttest_cmp expect.extended actual.extended.short-arg &&\n+\t\tif test_have_prereq PCRE\n+\t\tthen\n+\t\t\ttest_cmp expect.perl actual.perl.short-arg\n+\t\tfi &&\n \n \t\tgit log --pretty=tformat:%s --fixed-strings \\\n \t\t\t--grep=\"(1|2)\" >actual.fixed.long-arg &&\n-- \n2.11.0\n\n"},{"id":"317412","messageId":"20170420212345.7408-10-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 09/18] grep & rev-list doc: stop promising libpcre for --perl-regexp","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:36Z","receivedAt":"2017-04-20T21:25:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Stop promising in our grep & rev-list options documentation that we're\nalways going to be using libpcre when given the --perl-regexp option.\n\nInstead talk about using \"Perl-compatible regular expressions\" and\nusing these types of patterns using \"a compile-time dependency\".\n\nSaying \"libpcre\" strongly suggests that we might be talking about\nlibpcre.so, which is always going to be v1. This change is part of a\nseries to add support for libpcre2 which comes with v2 of PCRE.\n\nIn the future we might use some completely unrelated library to\nprovide perl-compatible regular expression support. By wording the\ndocumentation differently and not promising any specific version of\nPCRE or ever PCRE at all we have more wiggle room to change the\nimplementation.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Documentation/git-grep.txt | 7 +++++--\n Documentation/rev-list-options.txt | 8 ++++++--\n 2 files changed, 11 insertions(+), 4 deletions(-)\n\ndiff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt\nindex 71f32f3508..5033483db4 100644\n--- a/Documentation/git-grep.txt\n+++ b/Documentation/git-grep.txt\n@@ -161,8 +161,11 @@ OPTIONS\n \n -P::\n --perl-regexp::\n-\tUse Perl-compatible regexp for patterns. Requires libpcre to be\n-\tcompiled in.\n+\tUse Perl-compatible regular expressions for patterns.\n++\n+Support for these types of regular expressions is an optional\n+compile-time dependency. If Git wasn't compiled with support for them\n+providing this option will cause it to die.\n \n -F::\n --fixed-strings::\ndiff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt\nindex 5b7fa49a7f..9c44eae55d 100644\n--- a/Documentation/rev-list-options.txt\n+++ b/Documentation/rev-list-options.txt\n@@ -93,8 +93,12 @@ endif::git-rev-list[]\n \n -P::\n --perl-regexp::\n-\tConsider the limiting patterns to be Perl-compatible regular expressions.\n-\tRequires libpcre to be compiled in.\n+\tConsider the limiting patterns to be Perl-compatible regular\n+\texpressions.\n++\n+Support for these types of regular expressions is an optional\n+compile-time dependency. If Git wasn't compiled with support for them\n+providing this option will cause it to die.\n \n --remove-empty::\n \tStop when a given path disappears from the tree.\n-- \n2.11.0\n\n"},{"id":"317413","messageId":"20170420212345.7408-11-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 10/18] grep: make grep.patternType=[pcre|pcre1] a synonym for \"perl\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:37Z","receivedAt":"2017-04-20T21:25:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Make the pattern types \"pcre\" & \"pcre1\" synonyms for the long-standing\n\"perl\" grep.patternType.\n\nThis change is part of a longer patch series to add pcre2 support to\nGit. It's nice to be able to performance test PCRE v1 v.s. v2 without\nhaving to recompile git, and doing that via grep.patternType makes\nsense.\n\nHowever, just adding \"pcre2\" when we only have \"perl\" would be\nconfusing, so start by adding a \"pcre\" & \"pcre1\" synonym.\n\nIn the future \"perl\" and \"pcre\" might be changed to default to \"pcre2\"\ninstead of \"pcre1\", and depending on how Git is compiled the more\nspecific \"pcre1\" or \"pcre2\" pattern types might produce an error.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Documentation/config.txt | 9 +++++++++\n grep.c | 4 +++-\n t/t7810-grep.sh | 10 ++++++++++\n t/t7814-grep-recurse-submodules.sh | 4 ++++\n 4 files changed, 26 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 475e874d51..5ef12d0694 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1624,6 +1624,15 @@ grep.patternType::\n \t'fixed', or 'perl' will enable the `--basic-regexp`, `--extended-regexp`,\n \t`--fixed-strings`, or `--perl-regexp` option accordingly, while the\n \tvalue 'default' will return to the default matching behavior.\n++\n+The 'pcre' and 'pcre1' values are synonyms for 'perl'. The other\n+values starting with 'pcre' are reserved for future use, e.g. if we'd\n+like to use 'pcre2' for the PCRE v2 library.\n++\n+In the future 'perl' and 'pcre' might become synonyms for some other\n+implementation or PCRE version, such as 'pcre2', while the more\n+specific 'pcre1' & 'pcre2' might throw errors depending on whether git\n+is compiled to include those libraries.\n \n grep.extendedRegexp::\n \tIf set to true, enable `--extended-regexp` option by default. This\ndiff --git a/grep.c b/grep.c\nindex 59ae7809f2..506545c0ee 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -60,7 +60,9 @@ static int parse_pattern_type_arg(const char *opt, const char *arg)\n \t\treturn GREP_PATTERN_TYPE_ERE;\n \telse if (!strcmp(arg, \"fixed\"))\n \t\treturn GREP_PATTERN_TYPE_FIXED;\n-\telse if (!strcmp(arg, \"perl\"))\n+\telse if (!strcmp(arg, \"perl\") ||\n+\t\t !strcmp(arg, \"pcre\") ||\n+\t\t !strcmp(arg, \"pcre1\"))\n \t\treturn GREP_PATTERN_TYPE_PCRE;\n \tdie(\"bad %s argument: %s\", opt, arg);\n }\ndiff --git a/t/t7810-grep.sh b/t/t7810-grep.sh\nindex ec8fe585a7..7199356d35 100755\n--- a/t/t7810-grep.sh\n+++ b/t/t7810-grep.sh\n@@ -1512,4 +1512,14 @@ test_expect_success 'grep does not report i-t-a and assume unchanged with -L' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success LIBPCRE \"grep with grep.patternType synonyms perl/pcre/pcre1\" '\n+\techo \"#include \" >expected &&\n+\tgit -c grep.patternType=perl grep -h --no-line-number \"st(?=dio)\" >actual &&\n+\ttest_cmp expected actual &&\n+\tgit -c grep.patternType=pcre grep -h --no-line-number \"st(?=dio)\" >actual &&\n+\ttest_cmp expected actual &&\n+\tgit -c grep.patternType=pcre1 grep -h --no-line-number \"st(?=dio)\" >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_done\ndiff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh\nindex ef658b7899..dc6cf771ec 100755\n--- a/t/t7814-grep-recurse-submodules.sh\n+++ b/t/t7814-grep-recurse-submodules.sh\n@@ -358,6 +358,10 @@ test_expect_success 'grep --recurse-submodules should pass the pattern type alon\n \t\tEOF\n \t\ttest_cmp expect actual &&\n \t\tgit -c grep.patternType=perl grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit -c grep.patternType=pcre grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\ttest_cmp expect actual &&\n+\t\tgit -c grep.patternType=pcre1 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n \t\ttest_cmp expect actual\n \tfi\n '\n-- \n2.11.0\n\n"},{"id":"317414","messageId":"20170420212345.7408-12-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 11/18] test-lib: rename the LIBPCRE prerequisite to PCRE","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:38Z","receivedAt":"2017-04-20T21:25:06Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Rename the LIBPCRE prerequisite to PCRE. This is for preparation for\nlibpcre2 support, where having just \"LIBPCRE\" would be confusing as it\nimplies v1 of the library.\n\nNone of these tests are incompatible between versions 1 & 2 of\nlibpcre, it's less confusing to give them a more general name to make\nit clear that they work on both library versions.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/README | 4 ++--\n t/t4202-log.sh | 10 +++++-----\n t/t7810-grep.sh | 32 ++++++++++++++++----------------\n t/t7812-grep-icase-non-ascii.sh | 4 ++--\n t/t7813-grep-icase-iso.sh | 2 +-\n t/test-lib.sh | 2 +-\n 6 files changed, 27 insertions(+), 27 deletions(-)\n\ndiff --git a/t/README b/t/README\nindex ab386c3681..a90cb62583 100644\n--- a/t/README\n+++ b/t/README\n@@ -803,9 +803,9 @@ use these, and \"test_set_prereq\" for how to define your own.\n Test is not run by root user, and an attempt to write to an\n unwritable file is expected to fail correctly.\n \n- - LIBPCRE\n+ - PCRE\n \n- Git was compiled with USE_LIBPCRE=YesPlease. Wrap any tests\n+ Git was compiled with support for PCRE. Wrap any tests\n that use git-grep --perl-regexp or git-grep -P in these.\n \n - CASE_INSENSITIVE_FS\ndiff --git a/t/t4202-log.sh b/t/t4202-log.sh\nindex 06acc848b8..0b5f808ca3 100755\n--- a/t/t4202-log.sh\n+++ b/t/t4202-log.sh\n@@ -266,7 +266,7 @@ test_expect_success 'log -F -E --grep= uses ere' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success LIBPCRE 'log -F -E --perl-regexp --grep= uses PCRE' '\n+test_expect_success PCRE 'log -F -E --perl-regexp --grep= uses PCRE' '\n \ttest_when_finished \"rm -rf num_commits\" &&\n \tgit init num_commits &&\n \t(\n@@ -314,7 +314,7 @@ test_expect_success 'log with various grep.patternType configurations & command-\n \t\t\t--grep=\"(.|.)\" >actual.basic &&\n \t\tgit -c grep.patternType=extended log --pretty=tformat:%s \\\n \t\t\t--grep=\"\\|2\" >actual.extended &&\n-\t\tif test_have_prereq LIBPCRE\n+\t\tif test_have_prereq PCRE\n \t\tthen\n \t\t\tgit -c grep.patternType=perl log --pretty=tformat:%s \\\n \t\t\t\t--grep=\"[\\d]\\|\" >actual.perl\n@@ -322,7 +322,7 @@ test_expect_success 'log with various grep.patternType configurations & command-\n \t\ttest_cmp expect.fixed actual.fixed &&\n \t\ttest_cmp expect.basic actual.basic &&\n \t\ttest_cmp expect.extended actual.extended &&\n-\t\tif test_have_prereq LIBPCRE\n+\t\tif test_have_prereq PCRE\n \t\tthen\n \t\t\ttest_cmp expect.perl actual.perl\n \t\tfi &&\n@@ -349,7 +349,7 @@ test_expect_success 'log with various grep.patternType configurations & command-\n \t\t\t--grep=\"(.|.)\" >actual.basic.long-arg &&\n \t\tgit log --pretty=tformat:%s --extended-regexp \\\n \t\t\t--grep=\"\\|2\" >actual.extended.long-arg &&\n-\t\tif test_have_prereq LIBPCRE\n+\t\tif test_have_prereq PCRE\n \t\tthen\n \t\t\tgit log --pretty=tformat:%s --perl-regexp \\\n \t\t\t\t--grep=\"[\\d]\\|\" >actual.perl.long-arg\n@@ -357,7 +357,7 @@ test_expect_success 'log with various grep.patternType configurations & command-\n \t\ttest_cmp expect.fixed actual.fixed.long-arg &&\n \t\ttest_cmp expect.basic actual.basic.long-arg &&\n \t\ttest_cmp expect.extended actual.extended.long-arg &&\n-\t\tif test_have_prereq LIBPCRE\n+\t\tif test_have_prereq PCRE\n \t\tthen\n \t\t\ttest_cmp expect.perl actual.perl.long-arg\n \t\tfi\ndiff --git a/t/t7810-grep.sh b/t/t7810-grep.sh\nindex 7199356d35..f929b447e9 100755\n--- a/t/t7810-grep.sh\n+++ b/t/t7810-grep.sh\n@@ -275,7 +275,7 @@ do\n \t\ttest_cmp expected actual\n \t'\n \n-\ttest_expect_success LIBPCRE \"grep $L with grep.patterntype=perl\" '\n+\ttest_expect_success PCRE \"grep $L with grep.patterntype=perl\" '\n \t\techo \"${HC}ab:a+b*c\" >expected &&\n \t\tgit -c grep.patterntype=perl grep \"a\\x{2b}b\\x{2a}c\" $H ab >actual &&\n \t\ttest_cmp expected actual\n@@ -1053,12 +1053,12 @@ hello.c:int main(int argc, const char **argv)\n hello.c:\tprintf(\"Hello world.\\n\");\n EOF\n \n-test_expect_success LIBPCRE 'grep --perl-regexp pattern' '\n+test_expect_success PCRE 'grep --perl-regexp pattern' '\n \tgit grep --perl-regexp \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P pattern' '\n+test_expect_success PCRE 'grep -P pattern' '\n \tgit grep -P \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n \ttest_cmp expected actual\n '\n@@ -1070,13 +1070,13 @@ test_expect_success 'grep pattern with grep.extendedRegexp=true' '\n \ttest_cmp empty actual\n '\n \n-test_expect_success LIBPCRE 'grep -P pattern with grep.extendedRegexp=true' '\n+test_expect_success PCRE 'grep -P pattern with grep.extendedRegexp=true' '\n \tgit -c grep.extendedregexp=true \\\n \t\tgrep -P \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P -v pattern' '\n+test_expect_success PCRE 'grep -P -v pattern' '\n \t{\n \t\techo \"ab:a+b*c\"\n \t\techo \"ab:a+bc\"\n@@ -1085,7 +1085,7 @@ test_expect_success LIBPCRE 'grep -P -v pattern' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P -i pattern' '\n+test_expect_success PCRE 'grep -P -i pattern' '\n \tcat >expected <<-EOF &&\n \thello.c:\tprintf(\"Hello world.\\n\");\n \tEOF\n@@ -1093,7 +1093,7 @@ test_expect_success LIBPCRE 'grep -P -i pattern' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P -w pattern' '\n+test_expect_success PCRE 'grep -P -w pattern' '\n \t{\n \t\techo \"hello_world:Hello world\"\n \t\techo \"hello_world:HeLLo world\"\n@@ -1102,7 +1102,7 @@ test_expect_success LIBPCRE 'grep -P -w pattern' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P backreferences work (the PCRE NO_AUTO_CAPTURE flag is not set)' '\n+test_expect_success PCRE 'grep -P backreferences work (the PCRE NO_AUTO_CAPTURE flag is not set)' '\n \tgit grep -P -h \"(?P.)(?P=one)\" hello_world >actual &&\n \ttest_cmp hello_world actual &&\n \tgit grep -P -h \"(.)\\1\" hello_world >actual &&\n@@ -1125,11 +1125,11 @@ test_expect_success 'grep invalidpattern properly dies with grep.patternType=ext\n \ttest_must_fail git -c grep.patterntype=extended grep \"a[\"\n '\n \n-test_expect_success LIBPCRE 'grep -P invalidpattern properly dies ' '\n+test_expect_success PCRE 'grep -P invalidpattern properly dies ' '\n \ttest_must_fail git grep -P \"a[\"\n '\n \n-test_expect_success LIBPCRE 'grep invalidpattern properly dies with grep.patternType=perl' '\n+test_expect_success PCRE 'grep invalidpattern properly dies with grep.patternType=perl' '\n \ttest_must_fail git -c grep.patterntype=perl grep \"a[\"\n '\n \n@@ -1198,13 +1198,13 @@ test_expect_success 'grep pattern with grep.patternType=fixed, =basic, =perl, =e\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -G -F -E -P pattern' '\n+test_expect_success PCRE 'grep -G -F -E -P pattern' '\n \techo \"d0:0\" >expected &&\n \tgit grep -G -F -E -P \"[\\d]\" d0 >actual &&\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep pattern with grep.patternType=fixed, =basic, =extended, =perl' '\n+test_expect_success PCRE 'grep pattern with grep.patternType=fixed, =basic, =extended, =perl' '\n \techo \"d0:0\" >expected &&\n \tgit \\\n \t\t-c grep.patterntype=fixed \\\n@@ -1215,7 +1215,7 @@ test_expect_success LIBPCRE 'grep pattern with grep.patternType=fixed, =basic, =\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P pattern with grep.patternType=fixed' '\n+test_expect_success PCRE 'grep -P pattern with grep.patternType=fixed' '\n \techo \"ab:a+b*c\" >expected &&\n \tgit \\\n \t\t-c grep.patterntype=fixed \\\n@@ -1350,12 +1350,12 @@ space: line with leading space2\n space: line with leading space3\n EOF\n \n-test_expect_success LIBPCRE 'grep -E \"^ \"' '\n+test_expect_success PCRE 'grep -E \"^ \"' '\n \tgit grep -E \"^ \" space >actual &&\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE 'grep -P \"^ \"' '\n+test_expect_success PCRE 'grep -P \"^ \"' '\n \tgit grep -P \"^ \" space >actual &&\n \ttest_cmp expected actual\n '\n@@ -1512,7 +1512,7 @@ test_expect_success 'grep does not report i-t-a and assume unchanged with -L' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE \"grep with grep.patternType synonyms perl/pcre/pcre1\" '\n+test_expect_success PCRE \"grep with grep.patternType synonyms perl/pcre/pcre1\" '\n \techo \"#include \" >expected &&\n \tgit -c grep.patternType=perl grep -h --no-line-number \"st(?=dio)\" >actual &&\n \ttest_cmp expected actual &&\ndiff --git a/t/t7812-grep-icase-non-ascii.sh b/t/t7812-grep-icase-non-ascii.sh\nindex 169fd8d706..04a61cb8e0 100755\n--- a/t/t7812-grep-icase-non-ascii.sh\n+++ b/t/t7812-grep-icase-non-ascii.sh\n@@ -20,13 +20,13 @@ test_expect_success REGEX_LOCALE 'grep literal string, no -F' '\n \tgit grep -i \"TILRAUN: HALLÓ HEIMUR!\"\n '\n \n-test_expect_success GETTEXT_LOCALE,LIBPCRE 'grep pcre utf-8 icase' '\n+test_expect_success GETTEXT_LOCALE,PCRE 'grep pcre utf-8 icase' '\n \tgit grep --perl-regexp \"TILRAUN: H.lló Heimur!\" &&\n \tgit grep --perl-regexp -i \"TILRAUN: H.lló Heimur!\" &&\n \tgit grep --perl-regexp -i \"TILRAUN: H.LLÓ HEIMUR!\"\n '\n \n-test_expect_success GETTEXT_LOCALE,LIBPCRE 'grep pcre utf-8 string with \"+\"' '\n+test_expect_success GETTEXT_LOCALE,PCRE 'grep pcre utf-8 string with \"+\"' '\n \ttest_write_lines \"TILRAUN: Hallóó Heimur!\" >file2 &&\n \tgit add file2 &&\n \tgit grep -l --perl-regexp \"TILRAUN: H.lló+ Heimur!\" >actual &&\ndiff --git a/t/t7813-grep-icase-iso.sh b/t/t7813-grep-icase-iso.sh\nindex efef7fb81f..701e08a8e5 100755\n--- a/t/t7813-grep-icase-iso.sh\n+++ b/t/t7813-grep-icase-iso.sh\n@@ -11,7 +11,7 @@ test_expect_success GETTEXT_ISO_LOCALE 'setup' '\n \texport LC_ALL\n '\n \n-test_expect_success GETTEXT_ISO_LOCALE,LIBPCRE 'grep pcre string' '\n+test_expect_success GETTEXT_ISO_LOCALE,PCRE 'grep pcre string' '\n \tgit grep --perl-regexp -i \"TILRAUN: H.ll� Heimur!\" &&\n \tgit grep --perl-regexp -i \"TILRAUN: H.LL� HEIMUR!\"\n '\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 13b5696822..6abf1d1918 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1010,7 +1010,7 @@ esac\n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_PERL\" && test_set_prereq PERL\n test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n-test -n \"$USE_LIBPCRE\" && test_set_prereq LIBPCRE\n+test -n \"$USE_LIBPCRE\" && test_set_prereq PCRE\n test -z \"$NO_GETTEXT\" && test_set_prereq GETTEXT\n \n # Can we rely on git's output in the C locale?\n-- \n2.11.0\n\n"},{"id":"317415","messageId":"20170420212345.7408-13-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 12/18] grep: change the internal PCRE macro names to be PCRE1","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:39Z","receivedAt":"2017-04-20T21:25:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the internal USE_LIBPCRE define, & build options flag to use a\nnaming convention ending in PCRE1, without changing the long-standing\nUSE_LIBPCRE Makefile flag which enables this code.\n\nThis is for preparation for libpcre2 support where having things like\nUSE_LIBPCRE and USE_LIBPCRE2 in any more places than we absolutely\nneed to for backwards compatibility with old Makefile arguments would\nbe confusing.\n\nIn some ways it would be better to change everything that now uses\nUSE_LIBPCRE to use USE_LIBPCRE1, and to make specifying\nUSE_LIBPCRE (or --with-pcre) an error. This would impose a one-time\nburden on packagers of git to s/USE_LIBPCRE/USE_LIBPCRE1/ in their\nbuild scripts.\n\nHowever I'd like to leave the door open to making\nUSE_LIBPCRE=YesPlease eventually mean USE_LIBPCRE2=YesPlease,\ni.e. once PCRE v2 is ubiquitous enough that it makes sense to make it\nthe default.\n\nThis code and the USE_LIBPCRE Makefile argument was added in commit\n63e7e9d8b6 (\"git-grep: Learn PCRE\", 2011-05-09). At the time there was\nno indication that the PCRE project would release an entirely new &\nincompatible API around 3 years later.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 4 ++--\n grep.c | 6 +++---\n grep.h | 2 +-\n t/test-lib.sh | 2 +-\n 4 files changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 2e63b1cfcc..aecdfdcfe6 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1086,7 +1086,7 @@ ifdef NO_LIBGEN_H\n endif\n \n ifdef USE_LIBPCRE\n-\tBASIC_CFLAGS += -DUSE_LIBPCRE\n+\tBASIC_CFLAGS += -DUSE_LIBPCRE1\n \tifdef LIBPCREDIR\n \t\tBASIC_CFLAGS += -I$(LIBPCREDIR)/include\n \t\tEXTLIBS += -L$(LIBPCREDIR)/$(lib) $(CC_LD_DYNPATH)$(LIBPCREDIR)/$(lib)\n@@ -2238,7 +2238,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo TAR=\\''$(subst ','\\'',$(subst ','\\'',$(TAR)))'\\' >>$@+\n \t@echo NO_CURL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_CURL)))'\\' >>$@+\n \t@echo NO_EXPAT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_EXPAT)))'\\' >>$@+\n-\t@echo USE_LIBPCRE=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBPCRE)))'\\' >>$@+\n+\t@echo USE_LIBPCRE1=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBPCRE)))'\\' >>$@+\n \t@echo NO_PERL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PERL)))'\\' >>$@+\n \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\ndiff --git a/grep.c b/grep.c\nindex 506545c0ee..2535abd214 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -325,7 +325,7 @@ static NORETURN void compile_regexp_failed(const struct grep_pat *p,\n \tdie(\"%s'%s': %s\", where, p->pattern, error);\n }\n \n-#ifdef USE_LIBPCRE\n+#ifdef USE_LIBPCRE1\n static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)\n {\n \tconst char *error;\n@@ -377,7 +377,7 @@ static void free_pcre_regexp(struct grep_pat *p)\n \tpcre_free(p->pcre_extra_info);\n \tpcre_free((void *)p->pcre_tables);\n }\n-#else /* !USE_LIBPCRE */\n+#else /* !USE_LIBPCRE1 */\n static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)\n {\n \tdie(\"cannot use Perl-compatible regexes when not compiled with USE_LIBPCRE\");\n@@ -392,7 +392,7 @@ static int pcrematch(struct grep_pat *p, const char *line, const char *eol,\n static void free_pcre_regexp(struct grep_pat *p)\n {\n }\n-#endif /* !USE_LIBPCRE */\n+#endif /* !USE_LIBPCRE1 */\n \n static int is_fixed(const char *s, size_t len)\n {\ndiff --git a/grep.h b/grep.h\nindex 267534ca24..073b0e4c92 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -1,7 +1,7 @@\n #ifndef GREP_H\n #define GREP_H\n #include \"color.h\"\n-#ifdef USE_LIBPCRE\n+#ifdef USE_LIBPCRE1\n #include \n #else\n typedef int pcre;\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 6abf1d1918..e5cfbcc36b 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1010,7 +1010,7 @@ esac\n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_PERL\" && test_set_prereq PERL\n test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n-test -n \"$USE_LIBPCRE\" && test_set_prereq PCRE\n+test -n \"$USE_LIBPCRE1\" && test_set_prereq PCRE\n test -z \"$NO_GETTEXT\" && test_set_prereq GETTEXT\n \n # Can we rely on git's output in the C locale?\n-- \n2.11.0\n\n"},{"id":"317416","messageId":"20170420212345.7408-14-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 13/18] grep: change the internal PCRE code & header names to be PCRE1","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:40Z","receivedAt":"2017-04-20T21:25:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the internal PCRE variable & function names to have a \"1\"\nsuffix. This is for preparation for libpcre2 support, where having\nnon-versioned names would be confusing.\n\nThe earlier \"grep: change the internal PCRE macro names to be PCRE1\"\nchange elaborates on the motivations behind this commit.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n builtin/grep.c | 4 ++--\n grep.c | 56 ++++++++++++++++++++++++++++----------------------------\n grep.h | 10 +++++-----\n revision.c | 2 +-\n 4 files changed, 36 insertions(+), 36 deletions(-)\n\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex be3dbd6957..28e0dd3236 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -490,7 +490,7 @@ static void compile_submodule_options(const struct grep_opt *opt,\n \tcase GREP_PATTERN_TYPE_FIXED:\n \t\targv_array_push(&submodule_options, \"-F\");\n \t\tbreak;\n-\tcase GREP_PATTERN_TYPE_PCRE:\n+\tcase GREP_PATTERN_TYPE_PCRE1:\n \t\targv_array_push(&submodule_options, \"-P\");\n \t\tbreak;\n \tcase GREP_PATTERN_TYPE_UNSPECIFIED:\n@@ -1022,7 +1022,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)\n \t\t\t GREP_PATTERN_TYPE_FIXED),\n \t\tOPT_SET_INT('P', \"perl-regexp\", &pattern_type_arg,\n \t\t\t N_(\"use Perl-compatible regular expressions\"),\n-\t\t\t GREP_PATTERN_TYPE_PCRE),\n+\t\t\t GREP_PATTERN_TYPE_PCRE1),\n \t\tOPT_GROUP(\"\"),\n \t\tOPT_BOOL('n', \"line-number\", &opt.linenum, N_(\"show line numbers\")),\n \t\tOPT_NEGBIT('h', NULL, &opt.pathname, N_(\"don't show filenames\"), 1),\ndiff --git a/grep.c b/grep.c\nindex 2535abd214..d2c87ee2c3 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -63,7 +63,7 @@ static int parse_pattern_type_arg(const char *opt, const char *arg)\n \telse if (!strcmp(arg, \"perl\") ||\n \t\t !strcmp(arg, \"pcre\") ||\n \t\t !strcmp(arg, \"pcre1\"))\n-\t\treturn GREP_PATTERN_TYPE_PCRE;\n+\t\treturn GREP_PATTERN_TYPE_PCRE1;\n \tdie(\"bad %s argument: %s\", opt, arg);\n }\n \n@@ -180,25 +180,25 @@ static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, st\n \n \tcase GREP_PATTERN_TYPE_BRE:\n \t\topt->fixed = 0;\n-\t\topt->pcre = 0;\n+\t\topt->pcre1 = 0;\n \t\topt->regflags &= ~REG_EXTENDED;\n \t\tbreak;\n \n \tcase GREP_PATTERN_TYPE_ERE:\n \t\topt->fixed = 0;\n-\t\topt->pcre = 0;\n+\t\topt->pcre1 = 0;\n \t\topt->regflags |= REG_EXTENDED;\n \t\tbreak;\n \n \tcase GREP_PATTERN_TYPE_FIXED:\n \t\topt->fixed = 1;\n-\t\topt->pcre = 0;\n+\t\topt->pcre1 = 0;\n \t\topt->regflags &= ~REG_EXTENDED;\n \t\tbreak;\n \n-\tcase GREP_PATTERN_TYPE_PCRE:\n+\tcase GREP_PATTERN_TYPE_PCRE1:\n \t\topt->fixed = 0;\n-\t\topt->pcre = 1;\n+\t\topt->pcre1 = 1;\n \t\tbreak;\n \t}\n }\n@@ -326,7 +326,7 @@ static NORETURN void compile_regexp_failed(const struct grep_pat *p,\n }\n \n #ifdef USE_LIBPCRE1\n-static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)\n+static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)\n {\n \tconst char *error;\n \tint erroffset;\n@@ -334,23 +334,23 @@ static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)\n \n \tif (opt->ignore_case) {\n \t\tif (has_non_ascii(p->pattern))\n-\t\t\tp->pcre_tables = pcre_maketables();\n+\t\t\tp->pcre1_tables = pcre_maketables();\n \t\toptions |= PCRE_CASELESS;\n \t}\n \tif (is_utf8_locale() && has_non_ascii(p->pattern))\n \t\toptions |= PCRE_UTF8;\n \n-\tp->pcre_regexp = pcre_compile(p->pattern, options, &error, &erroffset,\n-\t\t\t\t p->pcre_tables);\n-\tif (!p->pcre_regexp)\n+\tp->pcre1_regexp = pcre_compile(p->pattern, options, &error, &erroffset,\n+\t\t\t\t p->pcre1_tables);\n+\tif (!p->pcre1_regexp)\n \t\tcompile_regexp_failed(p, error);\n \n-\tp->pcre_extra_info = pcre_study(p->pcre_regexp, 0, &error);\n-\tif (!p->pcre_extra_info && error)\n+\tp->pcre1_extra_info = pcre_study(p->pcre1_regexp, 0, &error);\n+\tif (!p->pcre1_extra_info && error)\n \t\tdie(\"%s\", error);\n }\n \n-static int pcrematch(struct grep_pat *p, const char *line, const char *eol,\n+static int pcre1match(struct grep_pat *p, const char *line, const char *eol,\n \t\tregmatch_t *match, int eflags)\n {\n \tint ovector[30], ret, flags = 0;\n@@ -358,7 +358,7 @@ static int pcrematch(struct grep_pat *p, const char *line, const char *eol,\n \tif (eflags & REG_NOTBOL)\n \t\tflags |= PCRE_NOTBOL;\n \n-\tret = pcre_exec(p->pcre_regexp, p->pcre_extra_info, line, eol - line,\n+\tret = pcre_exec(p->pcre1_regexp, p->pcre1_extra_info, line, eol - line,\n \t\t\t0, flags, ovector, ARRAY_SIZE(ovector));\n \tif (ret < 0 && ret != PCRE_ERROR_NOMATCH)\n \t\tdie(\"pcre_exec failed with error code %d\", ret);\n@@ -371,25 +371,25 @@ static int pcrematch(struct grep_pat *p, const char *line, const char *eol,\n \treturn ret;\n }\n \n-static void free_pcre_regexp(struct grep_pat *p)\n+static void free_pcre1_regexp(struct grep_pat *p)\n {\n-\tpcre_free(p->pcre_regexp);\n-\tpcre_free(p->pcre_extra_info);\n-\tpcre_free((void *)p->pcre_tables);\n+\tpcre_free(p->pcre1_regexp);\n+\tpcre_free(p->pcre1_extra_info);\n+\tpcre_free((void *)p->pcre1_tables);\n }\n #else /* !USE_LIBPCRE1 */\n-static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)\n+static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)\n {\n \tdie(\"cannot use Perl-compatible regexes when not compiled with USE_LIBPCRE\");\n }\n \n-static int pcrematch(struct grep_pat *p, const char *line, const char *eol,\n+static int pcre1match(struct grep_pat *p, const char *line, const char *eol,\n \t\tregmatch_t *match, int eflags)\n {\n \treturn 1;\n }\n \n-static void free_pcre_regexp(struct grep_pat *p)\n+static void free_pcre1_regexp(struct grep_pat *p)\n {\n }\n #endif /* !USE_LIBPCRE1 */\n@@ -476,8 +476,8 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt)\n \t\treturn;\n \t}\n \n-\tif (opt->pcre) {\n-\t\tcompile_pcre_regexp(p, opt);\n+\tif (opt->pcre1) {\n+\t\tcompile_pcre1_regexp(p, opt);\n \t\treturn;\n \t}\n \n@@ -833,8 +833,8 @@ void free_grep_patterns(struct grep_opt *opt)\n \t\tcase GREP_PATTERN_BODY:\n \t\t\tif (p->kws)\n \t\t\t\tkwsfree(p->kws);\n-\t\t\telse if (p->pcre_regexp)\n-\t\t\t\tfree_pcre_regexp(p);\n+\t\t\telse if (p->pcre1_regexp)\n+\t\t\t\tfree_pcre1_regexp(p);\n \t\t\telse\n \t\t\t\tregfree(&p->regexp);\n \t\t\tfree(p->pattern);\n@@ -913,8 +913,8 @@ static int patmatch(struct grep_pat *p, char *line, char *eol,\n \n \tif (p->fixed)\n \t\thit = !fixmatch(p, line, eol, match);\n-\telse if (p->pcre_regexp)\n-\t\thit = !pcrematch(p, line, eol, match, eflags);\n+\telse if (p->pcre1_regexp)\n+\t\thit = !pcre1match(p, line, eol, match, eflags);\n \telse\n \t\thit = !regexec_buf(&p->regexp, line, eol - line, 1, match,\n \t\t\t\t eflags);\ndiff --git a/grep.h b/grep.h\nindex 073b0e4c92..fa2ab9485f 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -46,9 +46,9 @@ struct grep_pat {\n \tsize_t patternlen;\n \tenum grep_header_field field;\n \tregex_t regexp;\n-\tpcre *pcre_regexp;\n-\tpcre_extra *pcre_extra_info;\n-\tconst unsigned char *pcre_tables;\n+\tpcre *pcre1_regexp;\n+\tpcre_extra *pcre1_extra_info;\n+\tconst unsigned char *pcre1_tables;\n \tkwset_t kws;\n \tunsigned fixed:1;\n \tunsigned ignore_case:1;\n@@ -68,7 +68,7 @@ enum grep_pattern_type {\n \tGREP_PATTERN_TYPE_BRE,\n \tGREP_PATTERN_TYPE_ERE,\n \tGREP_PATTERN_TYPE_FIXED,\n-\tGREP_PATTERN_TYPE_PCRE\n+\tGREP_PATTERN_TYPE_PCRE1\n };\n \n struct grep_expr {\n@@ -111,7 +111,7 @@ struct grep_opt {\n \tint allow_textconv;\n \tint extended;\n \tint use_reflog_filter;\n-\tint pcre;\n+\tint pcre1;\n \tint relative;\n \tint pathname;\n \tint null_following_name;\ndiff --git a/revision.c b/revision.c\nindex 03a3a012de..7a10a8570a 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1996,7 +1996,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t} else if (!strcmp(arg, \"--fixed-strings\") || !strcmp(arg, \"-F\")) {\n \t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_FIXED;\n \t} else if (!strcmp(arg, \"--perl-regexp\") || !strcmp(arg, \"-P\")) {\n-\t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_PCRE;\n+\t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_PCRE1;\n \t} else if (!strcmp(arg, \"--all-match\")) {\n \t\trevs->grep_filter.all_match = 1;\n \t} else if (!strcmp(arg, \"--invert-grep\")) {\n-- \n2.11.0\n\n"},{"id":"317417","messageId":"20170420212345.7408-8-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 07/18] log: add exhaustive tests for pattern style options & config","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:34Z","receivedAt":"2017-04-20T21:25:23Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add exhaustive tests for how the different grep.patternType options &\nthe corresponding command-line options affect git-log.\n\nBefore this change it was possible to patch revision.c so that the\n--basic-regexp option was synonymous with --extended-regexp, and\n--perl-regexp wasn't recognized at all, and still have 100% of the\ntest suite pass.\n\nThis was because the first test being modified here, added in commit\n34a4ae55b2 (\"log --grep: use the same helper to set -E/-F options as\n\"git grep\"\", 2012-10-03), didn't actually check whether we'd enabled\nextended regular expressions as distinct from re-toggling non-fixed\nstring support.\n\nFix that by changing the pattern to a pattern that'll only match if\n--extended-regexp option is provided, but won't match under the\ndefault --basic-regexp option.\n\nOther potential regressions were possible since there were no tests\nfor the rest of the combinations of grep.patternType configuration\ntoggles & corresponding git-log command-line options. Add exhaustive\ntests for those.\n\nThe patterns being passed to fixed/basic/extended/PCRE are carefully\ncrafted to return the wrong thing if the grep engine were to pick any\nother matching method than the one it's told to use.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n t/t4202-log.sh | 77 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 1 file changed, 76 insertions(+), 1 deletion(-)\n\ndiff --git a/t/t4202-log.sh b/t/t4202-log.sh\nindex f577990716..fc186f10ea 100755\n--- a/t/t4202-log.sh\n+++ b/t/t4202-log.sh\n@@ -262,7 +262,23 @@ test_expect_success 'log --grep -i' '\n \n test_expect_success 'log -F -E --grep= uses ere' '\n \techo second >expect &&\n-\tgit log -1 --pretty=\"tformat:%s\" -F -E --grep=s.c.nd >actual &&\n+\tgit log -1 --pretty=\"tformat:%s\" -F -E --grep=\"(s).c.nd\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success LIBPCRE 'log -F -E --perl-regexp --grep= uses PCRE' '\n+\ttest_when_finished \"rm -rf num_commits\" &&\n+\tgit init num_commits &&\n+\t(\n+\t\tcd num_commits &&\n+\t\ttest_commit 1d &&\n+\t\ttest_commit 2e\n+\t) &&\n+\techo 2e >expect &&\n+\tgit -C num_commits log -1 --pretty=\"tformat:%s\" -F -E --perl-regexp --grep=\"[\\d]\" >actual &&\n+\ttest_cmp expect actual &&\n+\techo 1d >expect &&\n+\tgit -C num_commits log -1 --pretty=\"tformat:%s\" -F -E --grep=\"[\\d]\" >actual &&\n \ttest_cmp expect actual\n '\n \n@@ -280,6 +296,65 @@ test_expect_success 'log with grep.patternType configuration and command line' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'log with various grep.patternType configurations & command-lines' '\n+\tgit init pattern-type &&\n+\t(\n+\t\tcd pattern-type &&\n+\t\ttest_commit 1 file A &&\n+\t\ttest_commit \"(1|2)\" file B &&\n+\n+\t\techo \"(1|2)\" >expect.fixed &&\n+\t\tcp expect.fixed expect.basic &&\n+\t\tcp expect.fixed expect.extended &&\n+\t\tcp expect.fixed expect.perl &&\n+\n+\t\tgit -c grep.patternType=fixed log --pretty=tformat:%s \\\n+\t\t\t--grep=\"(1|2)\" >actual.fixed &&\n+\t\tgit -c grep.patternType=basic log --pretty=tformat:%s \\\n+\t\t\t--grep=\"(.|.)\" >actual.basic &&\n+\t\tgit -c grep.patternType=extended log --pretty=tformat:%s \\\n+\t\t\t--grep=\"\\|2\" >actual.extended &&\n+\t\tif test_have_prereq LIBPCRE\n+\t\tthen\n+\t\t\tgit -c grep.patternType=perl log --pretty=tformat:%s \\\n+\t\t\t\t--grep=\"[\\d]\\|\" >actual.perl\n+\t\tfi &&\n+\t\ttest_cmp expect.fixed actual.fixed &&\n+\t\ttest_cmp expect.basic actual.basic &&\n+\t\ttest_cmp expect.extended actual.extended &&\n+\t\tif test_have_prereq LIBPCRE\n+\t\tthen\n+\t\t\ttest_cmp expect.perl actual.perl\n+\t\tfi &&\n+\n+\t\tgit log --pretty=tformat:%s -F \\\n+\t\t\t--grep=\"(1|2)\" >actual.fixed.short-arg &&\n+\t\tgit log --pretty=tformat:%s -E \\\n+\t\t\t--grep=\"\\|2\" >actual.extended.short-arg &&\n+\t\ttest_cmp expect.fixed actual.fixed.short-arg &&\n+\t\ttest_cmp expect.extended actual.extended.short-arg &&\n+\n+\t\tgit log --pretty=tformat:%s --fixed-strings \\\n+\t\t\t--grep=\"(1|2)\" >actual.fixed.long-arg &&\n+\t\tgit log --pretty=tformat:%s --basic-regexp \\\n+\t\t\t--grep=\"(.|.)\" >actual.basic.long-arg &&\n+\t\tgit log --pretty=tformat:%s --extended-regexp \\\n+\t\t\t--grep=\"\\|2\" >actual.extended.long-arg &&\n+\t\tif test_have_prereq LIBPCRE\n+\t\tthen\n+\t\t\tgit log --pretty=tformat:%s --perl-regexp \\\n+\t\t\t\t--grep=\"[\\d]\\|\" >actual.perl.long-arg\n+\t\tfi &&\n+\t\ttest_cmp expect.fixed actual.fixed.long-arg &&\n+\t\ttest_cmp expect.basic actual.basic.long-arg &&\n+\t\ttest_cmp expect.extended actual.extended.long-arg &&\n+\t\tif test_have_prereq LIBPCRE\n+\t\tthen\n+\t\t\ttest_cmp expect.perl actual.perl.long-arg\n+\t\tfi\n+\t)\n+'\n+\n cat > expect <\n---\n t/perf/p7820-grep-engines.sh | 25 +++++++++++++++++++++++++\n 1 file changed, 25 insertions(+)\n create mode 100755 t/perf/p7820-grep-engines.sh\n\ndiff --git a/t/perf/p7820-grep-engines.sh b/t/perf/p7820-grep-engines.sh\nnew file mode 100755\nindex 0000000000..5ae42ceccc\n--- /dev/null\n+++ b/t/perf/p7820-grep-engines.sh\n@@ -0,0 +1,25 @@\n+#!/bin/sh\n+\n+test_description=\"Comparison of git-grep's regex engines\"\n+\n+. ./perf-lib.sh\n+\n+test_perf_large_repo\n+test_checkout_worktree\n+\n+for engine in extended pcre1\n+do\n+\t# Patterns stolen from http://sljit.sourceforge.net/pcre.html\n+\tfor pattern in \\\n+\t\t'how.to' \\\n+\t\t'^how to' \\\n+\t\t'\\w+our\\w*' \\\n+\t\t'-?-?-?-?-?-?-?-?-?-?-?-----------$'\n+\tdo\n+\t\ttest_perf \"$engine with $pattern\" \"\n+\t\t\tgit -c grep.patternType=$engine grep -- '$pattern' || :\n+\t\t\"\n+\tdone\n+done\n+\n+test_done\n-- \n2.11.0\n\n"},{"id":"317419","messageId":"20170420212345.7408-19-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 18/18] Makefile & configure: make PCRE v2 the default PCRE implementation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:45Z","receivedAt":"2017-04-20T21:25:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the USE_LIBPCRE=YesPlease & --with-libpcre flags to the\nMakefile & configure script, respectively, to mean use PCRE v2, not\nPCRE v1. The legacy library is still available on request via\nUSE_LIBPCRE1=YesPlease or --with-libpcre1.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 21 ++++++++++-----------\n configure.ac | 18 +++++++++---------\n 2 files changed, 19 insertions(+), 20 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex a792f206b9..42b09f9632 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -29,10 +29,10 @@ all::\n # Perl-compatible regular expressions instead of standard or extended\n # POSIX regular expressions.\n #\n-# Currently USE_LIBPCRE is a synonym for USE_LIBPCRE1, define\n-# USE_LIBPCRE2 instead if you'd like to use version 2 of the PCRE\n-# library. The USE_LIBPCRE flag will likely be changed to mean v2 by\n-# default in future releases.\n+# The USE_LIBPCRE flag is a synonym for USE_LIBPCRE2, in previous\n+# versions it meant the same thing USE_LIBPCRE1 does now. Define\n+# USE_LIBPCRE1 instead if you'd like to use the legacy version 1 of\n+# the PCRE library.\n #\n # Define LIBPCREDIR=/foo/bar if your PCRE header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\n@@ -1090,18 +1090,17 @@ ifdef NO_LIBGEN_H\n \tCOMPAT_OBJS += compat/basename.o\n endif\n \n-USE_LIBPCRE1 ?= $(USE_LIBPCRE)\n-\n-ifneq (,$(USE_LIBPCRE1))\n-\tifdef USE_LIBPCRE2\n-$(error Only set USE_LIBPCRE1 (or its alias USE_LIBPCRE) or USE_LIBPCRE2, not both!)\n-\tendif\n+USE_LIBPCRE2 ?= $(USE_LIBPCRE)\n \n+ifdef USE_LIBPCRE1\n \tBASIC_CFLAGS += -DUSE_LIBPCRE1\n \tEXTLIBS += -lpcre\n endif\n \n-ifdef USE_LIBPCRE2\n+ifneq (,$(USE_LIBPCRE2))\n+\tifdef USE_LIBPCRE1\n+$(error Only set USE_LIBPCRE2 (or its alias USE_LIBPCRE) or USE_LIBPCRE1, not both!)\n+\tendif\n \tBASIC_CFLAGS += -DUSE_LIBPCRE2\n \tEXTLIBS += -lpcre2-8\n endif\ndiff --git a/configure.ac b/configure.ac\nindex 11d083fbe0..f9659daeb7 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -255,25 +255,25 @@ GIT_PARSE_WITH([openssl]))\n # Perl-compatible regular expressions instead of standard or extended\n # POSIX regular expressions.\n #\n-# Currently USE_LIBPCRE is a synonym for USE_LIBPCRE1, define\n-# USE_LIBPCRE2 instead if you'd like to use version 2 of the PCRE\n-# library. The USE_LIBPCRE flag will likely be changed to mean v2 by\n-# default in future releases.\n+# The USE_LIBPCRE flag is a synonym for USE_LIBPCRE2, in previous\n+# versions it meant the same thing USE_LIBPCRE1 does now. Define\n+# USE_LIBPCRE1 instead if you'd like to use the legacy version 1 of\n+# the PCRE library.\n #\n # Define LIBPCREDIR=/foo/bar if your PCRE header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\n #\n AC_ARG_WITH(libpcre,\n-AS_HELP_STRING([--with-libpcre],[synonym for --with-libpcre1]),\n+AS_HELP_STRING([--with-libpcre],[synonym for --with-libpcre2]),\n if test \"$withval\" = \"no\"; then\n-\tUSE_LIBPCRE1=\n+\tUSE_LIBPCRE2=\n elif test \"$withval\" = \"yes\"; then\n-\tUSE_LIBPCRE1=YesPlease\n+\tUSE_LIBPCRE2=YesPlease\n else\n-\tUSE_LIBPCRE1=YesPlease\n+\tUSE_LIBPCRE2=YesPlease\n \tLIBPCREDIR=$withval\n \tAC_MSG_NOTICE([Setting LIBPCREDIR to $LIBPCREDIR])\n- dnl USE_LIBPCRE1 can still be modified below, so don't substitute\n+ dnl USE_LIBPCRE2 can still be modified below, so don't substitute\n dnl it yet.\n \tGIT_CONF_SUBST([LIBPCREDIR])\n fi)\n-- \n2.11.0\n\n"},{"id":"317420","messageId":"20170420212345.7408-17-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 16/18] grep: add support for PCRE v2","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:43Z","receivedAt":"2017-04-20T21:25:32Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Add support for v2 of the PCRE API. This is a new major version of\nPCRE that came out in early 2015[1].\n\nThe regular expression syntax is the same, but while the API is\nsimilar-ish, pretty much every function is either renamed or takes\ndifferent arguments. Thus using it via entirely new functions makes\nsense, as opposed to trying to e.g. have one compile_pcre_pattern()\nthat would call either PCRE v1 or v2 functions.\n\nGit can now be compiled with any combination of\nUSE_LIBPCRE=[YesPlease|] & USE_LIBPCRE2=[YesPlease|]. If both are\nprovided the version of the PCRE library can be selected at runtime\nwith grep.PatternType, but the default (for now) is v1.\n\nThis table shows what the various combinations do, depending on what\nlibraries Git is compiled against:\n\n |------------------+-----+-----+----------|\n | grep.PatternType | v1 | v2 | v1 && v2 |\n |------------------+-----+-----+----------|\n | perl | v1 | v2 | v1 |\n | pcre | v1 | v2 | v1 |\n | pcre1 | v1 | ERR | v1 |\n | pcre2 | ERR | v2 | v2 |\n |------------------+-----+-----+----------|\n\nWhen Git is only compiled with v2 grep.PatternType=perl, --perl-regexp\n& -P will use v2. All tests pass with this new PCRE version. When Git\nis compiled with both v1 & v2 most of the tests will only test v1, but\nthere are some v2-specific tests that will be run.\n\nWith earlier patches to enable JIT for PCRE v1 the performance of the\nrelease versions of both libraries have almost exactly the same\nperformance, with PCRE v2 being around 1% slower.\n\nHowever after I reported this to the pcre-dev mailing list[2] I got a\nlot of help with the API use from Zoltán Herczeg, he\nsubsequently optimized some of the JIT functionality in v2 of the\nlibrary.\n\nRunning the p7820-grep-engines.sh performance test against the latest\nSubversion trunk of both, with both them and git compiled as -O3, and\nthe test run against linux.git, gives the following results:\n\n 7820.1: extended with how.to 0.27(1.22+0.34)\n 7820.2: extended with ^how to 0.27(1.18+0.36)\n 7820.3: extended with \\w+our\\w* 6.09(38.64+0.32)\n 7820.4: extended with -?-?-?-?-?-?-?-?-?-?-?-----------$ 0.38(1.69+0.28)\n 7820.5: pcre1 with how.to 0.19(0.42+0.53)\n 7820.6: pcre1 with ^how to 0.23(0.58+0.50)\n 7820.7: pcre1 with \\w+our\\w* 0.50(2.93+0.34)\n 7820.8: pcre1 with -?-?-?-?-?-?-?-?-?-?-?-----------$ 5.12(19.32+0.38)\n 7820.9: pcre2 with how.to 0.19(0.34+0.57)\n 7820.10: pcre2 with ^how to 0.19(0.29+0.60)\n 7820.11: pcre2 with \\w+our\\w* 0.51(2.85+0.41)\n 7820.12: pcre2 with -?-?-?-?-?-?-?-?-?-?-?-----------$ 5.04(19.27+0.31)\n\nI.e. the two are neck-to-neck, but PCRE v2 usually pulls ahead, when\nit does it's up to 20% faster.\n\nA brief note on thread safety: As noted in pcre2api(3) & pcre2jit(3)\nthe compiled pattern can be shared between threads, but not some of\nthe JIT context, however the grep threading support does all pattern &\nJIT compilation in separate threads, so this code doesn't need to\nconcern itself with thread safety.\n\nSee commit 63e7e9d8b6 (\"git-grep: Learn PCRE\", 2011-05-09) for the\ninitial addition of PCRE v1. This change follows some of the same\npatterns it did (and which were discussed on list at the time),\ne.g. mocking up types with typedef instead of ifdef-ing them out when\nUSE_LIBPCRE2 isn't defined. This adds some trivial memory use to the\nprogram, but makes the code look nicer.\n\n1. https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html\n2. https://lists.exim.org/lurker/thread/20170419.172322.833ee099.en.html\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Documentation/config.txt | 14 ++--\n Makefile | 18 +++++\n builtin/grep.c | 7 +-\n configure.ac | 49 +++++++++++-\n grep.c | 149 ++++++++++++++++++++++++++++++++++++-\n grep.h | 21 +++++-\n revision.c | 2 +-\n t/README | 12 +++\n t/perf/p7820-grep-engines.sh | 2 +-\n t/t7810-grep.sh | 30 +++++++-\n t/t7813-grep-icase-iso.sh | 11 ++-\n t/t7814-grep-recurse-submodules.sh | 12 ++-\n t/test-lib.sh | 4 +-\n 13 files changed, 304 insertions(+), 27 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 5ef12d0694..a5fc482495 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1625,14 +1625,12 @@ grep.patternType::\n \t`--fixed-strings`, or `--perl-regexp` option accordingly, while the\n \tvalue 'default' will return to the default matching behavior.\n +\n-The 'pcre' and 'pcre1' values are synonyms for 'perl'. The other\n-values starting with 'pcre' are reserved for future use, e.g. if we'd\n-like to use 'pcre2' for the PCRE v2 library.\n-+\n-In the future 'perl' and 'pcre' might become synonyms for some other\n-implementation or PCRE version, such as 'pcre2', while the more\n-specific 'pcre1' & 'pcre2' might throw errors depending on whether git\n-is compiled to include those libraries.\n+The 'perl' and 'pcre' values are synonyms. Depending on which PCRE\n+library Git was compiled with either or both of 'pcre1' and 'pcre2'\n+might also be available.\n++\n+If both are available Git currently defaults to 'pcre1', but this\n+might change in future versions.\n \n grep.extendedRegexp::\n \tIf set to true, enable `--extended-regexp` option by default. This\ndiff --git a/Makefile b/Makefile\nindex aecdfdcfe6..afdde49cda 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -32,6 +32,14 @@ all::\n # Define LIBPCREDIR=/foo/bar if your libpcre header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\n #\n+# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Define LIBPCRE2DIR=/foo/bar if your libpcre2 header and library\n+# files are in /foo/bar/include and /foo/bar/lib directories.\n+#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n # Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n@@ -1094,6 +1102,15 @@ ifdef USE_LIBPCRE\n \tEXTLIBS += -lpcre\n endif\n \n+ifdef USE_LIBPCRE2\n+\tBASIC_CFLAGS += -DUSE_LIBPCRE2\n+\tifdef LIBPCRE2DIR\n+\t\tBASIC_CFLAGS += -I$(LIBPCRE2DIR)/include\n+\t\tEXTLIBS += -L$(LIBPCRE2DIR)/$(lib) $(CC_LD_DYNPATH)$(LIBPCR2EDIR)/$(lib)\n+\tendif\n+\tEXTLIBS += -lpcre2-8\n+endif\n+\n ifdef HAVE_ALLOCA_H\n \tBASIC_CFLAGS += -DHAVE_ALLOCA_H\n endif\n@@ -2239,6 +2256,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_CURL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_CURL)))'\\' >>$@+\n \t@echo NO_EXPAT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_EXPAT)))'\\' >>$@+\n \t@echo USE_LIBPCRE1=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBPCRE)))'\\' >>$@+\n+\t@echo USE_LIBPCRE2=\\''$(subst ','\\'',$(subst ','\\'',$(USE_LIBPCRE2)))'\\' >>$@+\n \t@echo NO_PERL=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PERL)))'\\' >>$@+\n \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 28e0dd3236..178b10aa6f 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -490,11 +490,14 @@ static void compile_submodule_options(const struct grep_opt *opt,\n \tcase GREP_PATTERN_TYPE_FIXED:\n \t\targv_array_push(&submodule_options, \"-F\");\n \t\tbreak;\n-\tcase GREP_PATTERN_TYPE_PCRE1:\n+\tcase GREP_PATTERN_TYPE_PCRE:\n \t\targv_array_push(&submodule_options, \"-P\");\n \t\tbreak;\n \tcase GREP_PATTERN_TYPE_UNSPECIFIED:\n \t\tbreak;\n+\tcase GREP_PATTERN_TYPE_PCRE1:\n+\tcase GREP_PATTERN_TYPE_PCRE2:\n+\t\tdie(\"BUG: Command-line option for pcre1 or pcre2 added without updating switch statement\");\n \tdefault:\n \t\tdie(\"BUG: Added a new grep pattern type without updating switch statement\");\n \t}\n@@ -1022,7 +1025,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)\n \t\t\t GREP_PATTERN_TYPE_FIXED),\n \t\tOPT_SET_INT('P', \"perl-regexp\", &pattern_type_arg,\n \t\t\t N_(\"use Perl-compatible regular expressions\"),\n-\t\t\t GREP_PATTERN_TYPE_PCRE1),\n+\t\t\t GREP_PATTERN_TYPE_PCRE),\n \t\tOPT_GROUP(\"\"),\n \t\tOPT_BOOL('n', \"line-number\", &opt.linenum, N_(\"show line numbers\")),\n \t\tOPT_NEGBIT('h', NULL, &opt.pathname, N_(\"don't show filenames\"), 1),\ndiff --git a/configure.ac b/configure.ac\nindex deeb968daa..7ceb22ed03 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -259,8 +259,8 @@ GIT_PARSE_WITH([openssl]))\n # /foo/bar/include and /foo/bar/lib directories.\n #\n AC_ARG_WITH(libpcre,\n-AS_HELP_STRING([--with-libpcre],[support Perl-compatible regexes (default is NO)])\n-AS_HELP_STRING([], [ARG can be also prefix for libpcre library and headers]),\n+AS_HELP_STRING([--with-libpcre],[support Perl-compatible regexes via libpcre1 (default is NO)])\n+AS_HELP_STRING([], [ARG can be also prefix for libpcre1 library and headers]),\n if test \"$withval\" = \"no\"; then\n \tUSE_LIBPCRE=\n elif test \"$withval\" = \"yes\"; then\n@@ -273,6 +273,30 @@ AS_HELP_STRING([], [ARG can be also prefix for libpcre library and hea\n dnl it yet.\n \tGIT_CONF_SUBST([LIBPCREDIR])\n fi)\n+\n+# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Define LIBPCR2EDIR=/foo/bar if your libpcre2 header and library\n+# files are in /foo/bar/include and /foo/bar/lib directories.\n+#\n+AC_ARG_WITH(libpcre2,\n+AS_HELP_STRING([--with-libpcre2],[support Perl-compatible regexes via libpcre2 (default is NO)])\n+AS_HELP_STRING([], [ARG can be also prefix for libpcre library and headers]),\n+ if test \"$withval\" = \"no\"; then\n+\tUSE_LIBPCRE2=\n+ elif test \"$withval\" = \"yes\"; then\n+\tUSE_LIBPCRE2=YesPlease\n+ else\n+\tUSE_LIBPCRE2=YesPlease\n+\tLIBPCRE2DIR=$withval\n+\tAC_MSG_NOTICE([Setting LIBPCRE2DIR to $LIBPCRE2DIR])\n+ dnl USE_LIBPCRE2 can still be modified below, so don't substitute\n+ dnl it yet.\n+\tGIT_CONF_SUBST([LIBPCRE2DIR])\n+ fi)\n #\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n AC_FUNC_ALLOCA\n@@ -522,6 +546,27 @@ GIT_CONF_SUBST([USE_LIBPCRE])\n fi\n \n #\n+# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+\n+if test -n \"$USE_LIBPCRE2\"; then\n+\n+GIT_STASH_FLAGS($LIBPCRE2DIR)\n+\n+AC_CHECK_LIB([pcre2-8], [pcre2_config_8],\n+[USE_LIBPCRE2=YesPlease],\n+[USE_LIBPCRE2=])\n+\n+GIT_UNSTASH_FLAGS($LIBPCRE2DIR)\n+\n+GIT_CONF_SUBST([USE_LIBPCRE2])\n+\n+fi\n+\n+#\n # Define NO_CURL if you do not have libcurl installed. git-http-pull and\n # git-http-push are not built, and you cannot use http:// and https://\n # transports.\ndiff --git a/grep.c b/grep.c\nindex eb68bdaa2a..39955f5745 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -61,9 +61,12 @@ static int parse_pattern_type_arg(const char *opt, const char *arg)\n \telse if (!strcmp(arg, \"fixed\"))\n \t\treturn GREP_PATTERN_TYPE_FIXED;\n \telse if (!strcmp(arg, \"perl\") ||\n-\t\t !strcmp(arg, \"pcre\") ||\n-\t\t !strcmp(arg, \"pcre1\"))\n+\t\t !strcmp(arg, \"pcre\"))\n+\t\treturn GREP_PATTERN_TYPE_PCRE;\n+\telse if (!strcmp(arg, \"pcre1\"))\n \t\treturn GREP_PATTERN_TYPE_PCRE1;\n+\telse if (!strcmp(arg, \"pcre2\"))\n+\t\treturn GREP_PATTERN_TYPE_PCRE2;\n \tdie(\"bad %s argument: %s\", opt, arg);\n }\n \n@@ -181,24 +184,48 @@ static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, st\n \tcase GREP_PATTERN_TYPE_BRE:\n \t\topt->fixed = 0;\n \t\topt->pcre1 = 0;\n+\t\topt->pcre2 = 0;\n \t\topt->regflags &= ~REG_EXTENDED;\n \t\tbreak;\n \n \tcase GREP_PATTERN_TYPE_ERE:\n \t\topt->fixed = 0;\n \t\topt->pcre1 = 0;\n+\t\topt->pcre2 = 0;\n \t\topt->regflags |= REG_EXTENDED;\n \t\tbreak;\n \n \tcase GREP_PATTERN_TYPE_FIXED:\n \t\topt->fixed = 1;\n \t\topt->pcre1 = 0;\n+\t\topt->pcre2 = 0;\n \t\topt->regflags &= ~REG_EXTENDED;\n \t\tbreak;\n \n+\tcase GREP_PATTERN_TYPE_PCRE:\n+\t\topt->fixed = 0;\n+\t\t/* We default to pcre1 if we have both versions. This\n+\t\t * may change in future versions.\n+\t\t */\n+#ifdef USE_LIBPCRE1\n+\t\topt->pcre1 = 1;\n+\t\topt->pcre2 = 0;\n+#elif USE_LIBPCRE2\n+\t\topt->pcre1 = 0;\n+\t\topt->pcre2 = 1;\n+#endif\n+\t\tbreak;\n+\n \tcase GREP_PATTERN_TYPE_PCRE1:\n \t\topt->fixed = 0;\n \t\topt->pcre1 = 1;\n+\t\topt->pcre2 = 0;\n+\t\tbreak;\n+\n+\tcase GREP_PATTERN_TYPE_PCRE2:\n+\t\topt->fixed = 0;\n+\t\topt->pcre1 = 0;\n+\t\topt->pcre2 = 1;\n \t\tbreak;\n \t}\n }\n@@ -419,6 +446,115 @@ static void free_pcre1_regexp(struct grep_pat *p)\n }\n #endif /* !USE_LIBPCRE1 */\n \n+#ifdef USE_LIBPCRE2\n+static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt)\n+{\n+\tint error;\n+\tPCRE2_UCHAR errbuf[256];\n+\tPCRE2_SIZE erroffset;\n+\tint options = PCRE2_MULTILINE;\n+\tconst uint8_t *character_tables = NULL;\n+\tuint32_t canjit;\n+\tint jitret;\n+\n+\tp->pcre2_ccontext = NULL;\n+\n+\tif (opt->ignore_case) {\n+\t\tif (has_non_ascii(p->pattern)) {\n+\t\t\tcharacter_tables = pcre2_maketables(NULL);\n+\t\t\tp->pcre2_ccontext = pcre2_compile_context_create(NULL);\n+\t\t\tpcre2_set_character_tables(p->pcre2_ccontext, character_tables);\n+\t\t}\n+\t\toptions |= PCRE2_CASELESS;\n+\t}\n+\tif (is_utf8_locale() && has_non_ascii(p->pattern))\n+\t\toptions |= PCRE2_UTF;\n+\n+\tp->pcre2_pattern = pcre2_compile((PCRE2_SPTR)p->pattern,\n+\t\t\t\t\t p->patternlen, options, &error, &erroffset,\n+\t\t\t\t\t p->pcre2_ccontext);\n+\n+\tif (p->pcre2_pattern) {\n+\t\tp->pcre2_match_data = pcre2_match_data_create_from_pattern(p->pcre2_pattern, NULL);\n+\t\tif (!p->pcre2_match_data)\n+\t\t\tdie(\"BUG: Couldn't allocate PCRE2 match data\");\n+\t} else {\n+\t\tpcre2_get_error_message(error, errbuf, sizeof(errbuf));\n+\t\tcompile_regexp_failed(p, (const char *)&errbuf);\n+\t}\n+\n+\tpcre2_config(PCRE2_CONFIG_JIT, &canjit);\n+\tif (canjit == 1) {\n+\t\tjitret = pcre2_jit_compile(p->pcre2_pattern, PCRE2_JIT_COMPLETE);\n+\t\tif (!jitret)\n+\t\t\tp->pcre2_jit_on = 1;\n+\t\telse\n+\t\t\tdie(\"BUG: Couldn't JIT the PCRE2 pattern '%s', got '%d'\\n\", p->pattern, jitret);\n+\t\tp->pcre2_jit_stack = pcre2_jit_stack_create(1, 1024 * 1024, NULL);\n+\t\tif (!p->pcre2_jit_stack)\n+\t\t\tdie(\"BUG: Couldn't allocate PCRE2 JIT stack\");\n+\t\tp->pcre2_match_context = pcre2_match_context_create(NULL);\n+\t\tif (!p->pcre2_jit_stack)\n+\t\t\tdie(\"BUG: Couldn't allocate PCRE2 match context\");\n+\t\tpcre2_jit_stack_assign(p->pcre2_match_context, NULL, p->pcre2_jit_stack);\n+\t}\n+}\n+\n+static int pcre2match(struct grep_pat *p, const char *line, const char *eol,\n+\t\tregmatch_t *match, int eflags)\n+{\n+\tint ret, flags = 0;\n+\tPCRE2_SIZE *ovector;\n+\n+\tif (eflags & REG_NOTBOL)\n+\t\tflags |= PCRE2_NOTBOL;\n+\n+\tif (p->pcre2_jit_on)\n+\t\tret = pcre2_jit_match(p->pcre2_pattern, (unsigned char *)line,\n+\t\t\t\t eol - line, 0, flags, p->pcre2_match_data,\n+\t\t\t\t NULL);\n+\telse\n+\t\tret = pcre2_match(p->pcre2_pattern, (unsigned char *)line,\n+\t\t\t\t eol - line, 0, flags, p->pcre2_match_data,\n+\t\t\t\t NULL);\n+\n+\tif (ret < 0 && ret != PCRE2_ERROR_NOMATCH)\n+\t\tdie(\"pcre_exec failed with error code %d\", ret);\n+\tif (ret > 0) {\n+\t\tovector = pcre2_get_ovector_pointer(p->pcre2_match_data);\n+\t\tret = 0;\n+\t\tmatch->rm_so = (int)ovector[0];\n+\t\tmatch->rm_eo = (int)ovector[1];\n+\t}\n+\n+\treturn ret;\n+}\n+\n+static void free_pcre2_pattern(struct grep_pat *p)\n+{\n+\tpcre2_compile_context_free(p->pcre2_ccontext);\n+\tpcre2_code_free(p->pcre2_pattern);\n+\tpcre2_match_data_free(p->pcre2_match_data);\n+\tpcre2_jit_stack_free(p->pcre2_jit_stack);\n+\tpcre2_match_context_free(p->pcre2_match_context);\n+}\n+#else /* !USE_LIBPCRE2 */\n+static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_opt *opt)\n+{\n+\tdie(\"cannot use Perl-compatible regexes when not compiled with USE_LIBPCRE2\");\n+}\n+\n+static int pcre2match(struct grep_pat *p, const char *line, const char *eol,\n+\t\tregmatch_t *match, int eflags)\n+{\n+\treturn 1;\n+}\n+\n+static void free_pcre2_pattern(struct grep_pat *p)\n+{\n+}\n+#endif /* !USE_LIBPCRE2 */\n+\n static int is_fixed(const char *s, size_t len)\n {\n \tsize_t i;\n@@ -506,6 +642,11 @@ static void compile_regexp(struct grep_pat *p, struct grep_opt *opt)\n \t\treturn;\n \t}\n \n+\tif (opt->pcre2) {\n+\t\tcompile_pcre2_pattern(p, opt);\n+\t\treturn;\n+\t}\n+\n \terr = regcomp(&p->regexp, p->pattern, opt->regflags);\n \tif (err) {\n \t\tchar errbuf[1024];\n@@ -860,6 +1001,8 @@ void free_grep_patterns(struct grep_opt *opt)\n \t\t\t\tkwsfree(p->kws);\n \t\t\telse if (p->pcre1_regexp)\n \t\t\t\tfree_pcre1_regexp(p);\n+\t\t\telse if (p->pcre2_pattern)\n+\t\t\t\tfree_pcre2_pattern(p);\n \t\t\telse\n \t\t\t\tregfree(&p->regexp);\n \t\t\tfree(p->pattern);\n@@ -940,6 +1083,8 @@ static int patmatch(struct grep_pat *p, char *line, char *eol,\n \t\thit = !fixmatch(p, line, eol, match);\n \telse if (p->pcre1_regexp)\n \t\thit = !pcre1match(p, line, eol, match, eflags);\n+\telse if (p->pcre2_pattern)\n+\t\thit = !pcre2match(p, line, eol, match, eflags);\n \telse\n \t\thit = !regexec_buf(&p->regexp, line, eol - line, 1, match,\n \t\t\t\t eflags);\ndiff --git a/grep.h b/grep.h\nindex 29e20bf837..dd16b5fd53 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -11,6 +11,16 @@ typedef int pcre;\n typedef int pcre_extra;\n typedef int pcre_jit_stack;\n #endif\n+#ifdef USE_LIBPCRE2\n+#define PCRE2_CODE_UNIT_WIDTH 8\n+#include \n+#else\n+typedef int pcre2_code;\n+typedef int pcre2_match_data;\n+typedef int pcre2_compile_context;\n+typedef int pcre2_match_context;\n+typedef int pcre2_jit_stack;\n+#endif\n #include \"kwset.h\"\n #include \"thread-utils.h\"\n #include \"userdiff.h\"\n@@ -54,6 +64,12 @@ struct grep_pat {\n \tpcre_extra *pcre1_extra_info;\n \tpcre_jit_stack *pcre1_jit_stack;\n \tconst unsigned char *pcre1_tables;\n+\tpcre2_code *pcre2_pattern;\n+\tpcre2_match_data *pcre2_match_data;\n+\tpcre2_compile_context *pcre2_ccontext;\n+\tpcre2_match_context *pcre2_match_context;\n+\tpcre2_jit_stack *pcre2_jit_stack;\n+\tint pcre2_jit_on;\n \tkwset_t kws;\n \tunsigned fixed:1;\n \tunsigned ignore_case:1;\n@@ -73,7 +89,9 @@ enum grep_pattern_type {\n \tGREP_PATTERN_TYPE_BRE,\n \tGREP_PATTERN_TYPE_ERE,\n \tGREP_PATTERN_TYPE_FIXED,\n-\tGREP_PATTERN_TYPE_PCRE1\n+\tGREP_PATTERN_TYPE_PCRE,\n+\tGREP_PATTERN_TYPE_PCRE1,\n+\tGREP_PATTERN_TYPE_PCRE2\n };\n \n struct grep_expr {\n@@ -117,6 +135,7 @@ struct grep_opt {\n \tint extended;\n \tint use_reflog_filter;\n \tint pcre1;\n+\tint pcre2;\n \tint relative;\n \tint pathname;\n \tint null_following_name;\ndiff --git a/revision.c b/revision.c\nindex 7a10a8570a..03a3a012de 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1996,7 +1996,7 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg\n \t} else if (!strcmp(arg, \"--fixed-strings\") || !strcmp(arg, \"-F\")) {\n \t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_FIXED;\n \t} else if (!strcmp(arg, \"--perl-regexp\") || !strcmp(arg, \"-P\")) {\n-\t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_PCRE1;\n+\t\trevs->grep_filter.pattern_type_option = GREP_PATTERN_TYPE_PCRE;\n \t} else if (!strcmp(arg, \"--all-match\")) {\n \t\trevs->grep_filter.all_match = 1;\n \t} else if (!strcmp(arg, \"--invert-grep\")) {\ndiff --git a/t/README b/t/README\nindex a90cb62583..547b06e700 100644\n--- a/t/README\n+++ b/t/README\n@@ -808,6 +808,18 @@ use these, and \"test_set_prereq\" for how to define your own.\n Git was compiled with support for PCRE. Wrap any tests\n that use git-grep --perl-regexp or git-grep -P in these.\n \n+ - LIBPCRE1\n+\n+ Git was compiled with PCRE v1 support via\n+ USE_LIBPCRE=YesPlease. Wrap any PCRE using tests that for some\n+ reason need v1 of the PCRE library instead of v2 in these.\n+\n+ - LIBPCRE2\n+\n+ Git was compiled with PCRE v2 support via\n+ USE_LIBPCRE2=YesPlease. Wrap any PCRE using tests that for some\n+ reason need v2 of the PCRE library instead of v1 in these.\n+\n - CASE_INSENSITIVE_FS\n \n Test is run on a case insensitive file system.\ndiff --git a/t/perf/p7820-grep-engines.sh b/t/perf/p7820-grep-engines.sh\nindex 5ae42ceccc..70c5a5ca32 100755\n--- a/t/perf/p7820-grep-engines.sh\n+++ b/t/perf/p7820-grep-engines.sh\n@@ -7,7 +7,7 @@ test_description=\"Comparison of git-grep's regex engines\"\n test_perf_large_repo\n test_checkout_worktree\n \n-for engine in extended pcre1\n+for engine in extended pcre1 pcre2\n do\n \t# Patterns stolen from http://sljit.sourceforge.net/pcre.html\n \tfor pattern in \\\ndiff --git a/t/t7810-grep.sh b/t/t7810-grep.sh\nindex f929b447e9..bae7e524b9 100755\n--- a/t/t7810-grep.sh\n+++ b/t/t7810-grep.sh\n@@ -1063,6 +1063,16 @@ test_expect_success PCRE 'grep -P pattern' '\n \ttest_cmp expected actual\n '\n \n+test_expect_success LIBPCRE1 'grep libpcre1 pattern' '\n+\tgit -c grep.patternType=pcre1 grep \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success LIBPCRE2 'grep libpcre2 pattern' '\n+\tgit -c grep.patternType=pcre2 grep \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n+\ttest_cmp expected actual\n+'\n+\n test_expect_success 'grep pattern with grep.extendedRegexp=true' '\n \t>empty &&\n \ttest_must_fail git -c grep.extendedregexp=true \\\n@@ -1512,14 +1522,28 @@ test_expect_success 'grep does not report i-t-a and assume unchanged with -L' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success PCRE \"grep with grep.patternType synonyms perl/pcre/pcre1\" '\n+test_expect_success PCRE \"grep with grep.patternType synonyms perl/pcre\" '\n \techo \"#include \" >expected &&\n \tgit -c grep.patternType=perl grep -h --no-line-number \"st(?=dio)\" >actual &&\n \ttest_cmp expected actual &&\n \tgit -c grep.patternType=pcre grep -h --no-line-number \"st(?=dio)\" >actual &&\n-\ttest_cmp expected actual &&\n-\tgit -c grep.patternType=pcre1 grep -h --no-line-number \"st(?=dio)\" >actual &&\n \ttest_cmp expected actual\n '\n \n+test_expect_success LIBPCRE1 \"grep with grep.patternType=pcre1\" '\n+\techo \"#include \" >expected &&\n+\tgit -c grep.patternType=pcre1 grep -h --no-line-number \"st(?=dio)\" >actual &&\n+\ttest_cmp expected actual &&\n+\ttest_must_fail git -c grep.patternType=pcre1 grep \"foo(?+bar)\" 2>error &&\n+\ttest_i18ngrep -q \"digit expected after\" error\n+'\n+\n+test_expect_success LIBPCRE2 \"grep with grep.patternType=pcre2\" '\n+\techo \"#include \" >expected &&\n+\tgit -c grep.patternType=pcre2 grep -h --no-line-number \"st(?=dio)\" >actual &&\n+\ttest_cmp expected actual &&\n+\ttest_must_fail git -c grep.patternType=pcre2 grep \"foo(?+bar)\" 2>error &&\n+\ttest_i18ngrep -q \"digit expected after\" error\n+'\n+\n test_done\ndiff --git a/t/t7813-grep-icase-iso.sh b/t/t7813-grep-icase-iso.sh\nindex 701e08a8e5..e16570690c 100755\n--- a/t/t7813-grep-icase-iso.sh\n+++ b/t/t7813-grep-icase-iso.sh\n@@ -11,9 +11,12 @@ test_expect_success GETTEXT_ISO_LOCALE 'setup' '\n \texport LC_ALL\n '\n \n-test_expect_success GETTEXT_ISO_LOCALE,PCRE 'grep pcre string' '\n-\tgit grep --perl-regexp -i \"TILRAUN: H.ll� Heimur!\" &&\n-\tgit grep --perl-regexp -i \"TILRAUN: H.LL� HEIMUR!\"\n-'\n+for pcrev in 1 2\n+do\n+\ttest_expect_success GETTEXT_ISO_LOCALE,LIBPCRE$pcrev \"grep -i with i18n string using libpcre$pcrev\" \"\n+\t\tgit -c grep.patternType=pcre$pcrev grep -i \\\"TILRAUN: H.ll� Heimur!\\\" &&\n+\t\tgit -c grep.patternType=pcre$pcrev grep -i \\\"TILRAUN: H.LL� HEIMUR!\\\"\n+\t\"\n+done\n \n test_done\ndiff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh\nindex dc6cf771ec..7a8a48493b 100755\n--- a/t/t7814-grep-recurse-submodules.sh\n+++ b/t/t7814-grep-recurse-submodules.sh\n@@ -361,8 +361,16 @@ test_expect_success 'grep --recurse-submodules should pass the pattern type alon\n \t\ttest_cmp expect actual &&\n \t\tgit -c grep.patternType=pcre grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n \t\ttest_cmp expect actual &&\n-\t\tgit -c grep.patternType=pcre1 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n-\t\ttest_cmp expect actual\n+\t\tif test_have_prereq LIBPCRE1\n+\t\tthen\n+\t\t\tgit -c grep.patternType=pcre1 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\t\ttest_cmp expect actual\n+\t\tfi &&\n+\t\tif test_have_prereq LIBPCRE2\n+\t\tthen\n+\t\t\tgit -c grep.patternType=pcre2 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n+\t\t\ttest_cmp expect actual\n+\t\tfi\n \tfi\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex e5cfbcc36b..6f873de3e7 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1010,7 +1010,9 @@ esac\n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_PERL\" && test_set_prereq PERL\n test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n-test -n \"$USE_LIBPCRE1\" && test_set_prereq PCRE\n+test -n \"$USE_LIBPCRE1$USE_LIBPCRE2\" && test_set_prereq PCRE\n+test -n \"$USE_LIBPCRE1\" && test_set_prereq LIBPCRE1\n+test -n \"$USE_LIBPCRE2\" && test_set_prereq LIBPCRE2\n test -z \"$NO_GETTEXT\" && test_set_prereq GETTEXT\n \n # Can we rely on git's output in the C locale?\n-- \n2.11.0\n\n"},{"id":"317421","messageId":"20170420212345.7408-18-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 17/18] grep: remove support concurrent use of both PCRE v1 & v2","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:44Z","receivedAt":"2017-04-20T21:25:34Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the support for concurrently using PCRE v1 & v2 by compiling\nGit with support for both.\n\nHaving access to both at runtime via grep.patternType=[pcre1|pcre2]\nmakes it easier for the developer hacking on the PCRE implementations\nto test them concurrently, but adds confusion for everyone else,\nparticularly Git users who have no reason to concurrently use both\nlibraries.\n\nNow either USE_LIBPCRE1=YesPlease (or its alias USE_LIBPCRE) or\nUSE_LIBPCRE2=YesPlease can be supplied when building git, but\nproviding both will yield an error, similarly providing both\n--with-libpcre1 & --with-libpcre2 to the configure script will produce\nan error.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Documentation/config.txt | 7 ----\n Makefile | 36 +++++++++----------\n builtin/grep.c | 3 --\n configure.ac | 72 +++++++++++++++++++++-----------------\n grep.c | 19 +---------\n grep.h | 4 +--\n t/README | 12 -------\n t/perf/p7820-grep-engines.sh | 2 +-\n t/t7810-grep.sh | 30 +---------------\n t/t7814-grep-recurse-submodules.sh | 14 +-------\n t/test-lib.sh | 5 ++-\n 11 files changed, 65 insertions(+), 139 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex a5fc482495..475e874d51 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -1624,13 +1624,6 @@ grep.patternType::\n \t'fixed', or 'perl' will enable the `--basic-regexp`, `--extended-regexp`,\n \t`--fixed-strings`, or `--perl-regexp` option accordingly, while the\n \tvalue 'default' will return to the default matching behavior.\n-+\n-The 'perl' and 'pcre' values are synonyms. Depending on which PCRE\n-library Git was compiled with either or both of 'pcre1' and 'pcre2'\n-might also be available.\n-+\n-If both are available Git currently defaults to 'pcre1', but this\n-might change in future versions.\n \n grep.extendedRegexp::\n \tIf set to true, enable `--extended-regexp` option by default. This\ndiff --git a/Makefile b/Makefile\nindex afdde49cda..a792f206b9 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -29,16 +29,13 @@ all::\n # Perl-compatible regular expressions instead of standard or extended\n # POSIX regular expressions.\n #\n-# Define LIBPCREDIR=/foo/bar if your libpcre header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n+# Currently USE_LIBPCRE is a synonym for USE_LIBPCRE1, define\n+# USE_LIBPCRE2 instead if you'd like to use version 2 of the PCRE\n+# library. The USE_LIBPCRE flag will likely be changed to mean v2 by\n+# default in future releases.\n #\n-# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Define LIBPCRE2DIR=/foo/bar if your libpcre2 header and library\n-# files are in /foo/bar/include and /foo/bar/lib directories.\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n #\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n@@ -1093,24 +1090,27 @@ ifdef NO_LIBGEN_H\n \tCOMPAT_OBJS += compat/basename.o\n endif\n \n-ifdef USE_LIBPCRE\n-\tBASIC_CFLAGS += -DUSE_LIBPCRE1\n-\tifdef LIBPCREDIR\n-\t\tBASIC_CFLAGS += -I$(LIBPCREDIR)/include\n-\t\tEXTLIBS += -L$(LIBPCREDIR)/$(lib) $(CC_LD_DYNPATH)$(LIBPCREDIR)/$(lib)\n+USE_LIBPCRE1 ?= $(USE_LIBPCRE)\n+\n+ifneq (,$(USE_LIBPCRE1))\n+\tifdef USE_LIBPCRE2\n+$(error Only set USE_LIBPCRE1 (or its alias USE_LIBPCRE) or USE_LIBPCRE2, not both!)\n \tendif\n+\n+\tBASIC_CFLAGS += -DUSE_LIBPCRE1\n \tEXTLIBS += -lpcre\n endif\n \n ifdef USE_LIBPCRE2\n \tBASIC_CFLAGS += -DUSE_LIBPCRE2\n-\tifdef LIBPCRE2DIR\n-\t\tBASIC_CFLAGS += -I$(LIBPCRE2DIR)/include\n-\t\tEXTLIBS += -L$(LIBPCRE2DIR)/$(lib) $(CC_LD_DYNPATH)$(LIBPCR2EDIR)/$(lib)\n-\tendif\n \tEXTLIBS += -lpcre2-8\n endif\n \n+ifdef LIBPCREDIR\n+\tBASIC_CFLAGS += -I$(LIBPCREDIR)/include\n+\tEXTLIBS += -L$(LIBPCREDIR)/$(lib) $(CC_LD_DYNPATH)$(LIBPCREDIR)/$(lib)\n+endif\n+\n ifdef HAVE_ALLOCA_H\n \tBASIC_CFLAGS += -DHAVE_ALLOCA_H\n endif\ndiff --git a/builtin/grep.c b/builtin/grep.c\nindex 178b10aa6f..be3dbd6957 100644\n--- a/builtin/grep.c\n+++ b/builtin/grep.c\n@@ -495,9 +495,6 @@ static void compile_submodule_options(const struct grep_opt *opt,\n \t\tbreak;\n \tcase GREP_PATTERN_TYPE_UNSPECIFIED:\n \t\tbreak;\n-\tcase GREP_PATTERN_TYPE_PCRE1:\n-\tcase GREP_PATTERN_TYPE_PCRE2:\n-\t\tdie(\"BUG: Command-line option for pcre1 or pcre2 added without updating switch statement\");\n \tdefault:\n \t\tdie(\"BUG: Added a new grep pattern type without updating switch statement\");\n \t}\ndiff --git a/configure.ac b/configure.ac\nindex 7ceb22ed03..11d083fbe0 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -255,47 +255,63 @@ GIT_PARSE_WITH([openssl]))\n # Perl-compatible regular expressions instead of standard or extended\n # POSIX regular expressions.\n #\n-# Define LIBPCREDIR=/foo/bar if your libpcre header and library files are in\n+# Currently USE_LIBPCRE is a synonym for USE_LIBPCRE1, define\n+# USE_LIBPCRE2 instead if you'd like to use version 2 of the PCRE\n+# library. The USE_LIBPCRE flag will likely be changed to mean v2 by\n+# default in future releases.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are in\n # /foo/bar/include and /foo/bar/lib directories.\n #\n AC_ARG_WITH(libpcre,\n-AS_HELP_STRING([--with-libpcre],[support Perl-compatible regexes via libpcre1 (default is NO)])\n-AS_HELP_STRING([], [ARG can be also prefix for libpcre1 library and headers]),\n+AS_HELP_STRING([--with-libpcre],[synonym for --with-libpcre1]),\n if test \"$withval\" = \"no\"; then\n-\tUSE_LIBPCRE=\n+\tUSE_LIBPCRE1=\n elif test \"$withval\" = \"yes\"; then\n-\tUSE_LIBPCRE=YesPlease\n+\tUSE_LIBPCRE1=YesPlease\n else\n-\tUSE_LIBPCRE=YesPlease\n+\tUSE_LIBPCRE1=YesPlease\n \tLIBPCREDIR=$withval\n \tAC_MSG_NOTICE([Setting LIBPCREDIR to $LIBPCREDIR])\n- dnl USE_LIBPCRE can still be modified below, so don't substitute\n+ dnl USE_LIBPCRE1 can still be modified below, so don't substitute\n+ dnl it yet.\n+\tGIT_CONF_SUBST([LIBPCREDIR])\n+ fi)\n+\n+AC_ARG_WITH(libpcre1,\n+AS_HELP_STRING([--with-libpcre1],[support Perl-compatible regexes via libpcre1 (default is NO)])\n+AS_HELP_STRING([], [ARG can be also prefix for libpcre library and headers]),\n+ if test \"$withval\" = \"no\"; then\n+\tUSE_LIBPCRE1=\n+ elif test \"$withval\" = \"yes\"; then\n+\tUSE_LIBPCRE1=YesPlease\n+ else\n+\tUSE_LIBPCRE1=YesPlease\n+\tLIBPCREDIR=$withval\n+\tAC_MSG_NOTICE([Setting LIBPCREDIR to $LIBPCREDIR])\n+ dnl USE_LIBPCRE1 can still be modified below, so don't substitute\n dnl it yet.\n \tGIT_CONF_SUBST([LIBPCREDIR])\n fi)\n \n-# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Define LIBPCR2EDIR=/foo/bar if your libpcre2 header and library\n-# files are in /foo/bar/include and /foo/bar/lib directories.\n-#\n AC_ARG_WITH(libpcre2,\n AS_HELP_STRING([--with-libpcre2],[support Perl-compatible regexes via libpcre2 (default is NO)])\n AS_HELP_STRING([], [ARG can be also prefix for libpcre library and headers]),\n+ if test -n \"$USE_LIBPCRE1\"; then\n+ AC_MSG_ERROR([Only supply one of --with-libpcre1 or --with-libpcre2!])\n+ fi\n+\n if test \"$withval\" = \"no\"; then\n \tUSE_LIBPCRE2=\n elif test \"$withval\" = \"yes\"; then\n \tUSE_LIBPCRE2=YesPlease\n else\n \tUSE_LIBPCRE2=YesPlease\n-\tLIBPCRE2DIR=$withval\n-\tAC_MSG_NOTICE([Setting LIBPCRE2DIR to $LIBPCRE2DIR])\n+\tLIBPCREDIR=$withval\n+\tAC_MSG_NOTICE([Setting LIBPCREDIR to $LIBPCREDIR])\n dnl USE_LIBPCRE2 can still be modified below, so don't substitute\n dnl it yet.\n-\tGIT_CONF_SUBST([LIBPCRE2DIR])\n+\tGIT_CONF_SUBST([LIBPCREDIR])\n fi)\n #\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n@@ -525,13 +541,11 @@ GIT_CONF_SUBST([NEEDS_SSL_WITH_CRYPTO])\n GIT_CONF_SUBST([NO_OPENSSL])\n \n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n+# Handle the USE_LIBPCRE1 and USE_LIBPCRE2 options potentially set\n+# above.\n #\n \n-if test -n \"$USE_LIBPCRE\"; then\n+if test -n \"$USE_LIBPCRE1\"; then\n \n GIT_STASH_FLAGS($LIBPCREDIR)\n \n@@ -541,26 +555,20 @@ AC_CHECK_LIB([pcre], [pcre_version],\n \n GIT_UNSTASH_FLAGS($LIBPCREDIR)\n \n-GIT_CONF_SUBST([USE_LIBPCRE])\n+GIT_CONF_SUBST([USE_LIBPCRE1])\n \n fi\n \n-#\n-# Define USE_LIBPCRE2 if you have and want to use libpcre2. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n \n if test -n \"$USE_LIBPCRE2\"; then\n \n-GIT_STASH_FLAGS($LIBPCRE2DIR)\n+GIT_STASH_FLAGS($LIBPCREDIR)\n \n AC_CHECK_LIB([pcre2-8], [pcre2_config_8],\n [USE_LIBPCRE2=YesPlease],\n [USE_LIBPCRE2=])\n \n-GIT_UNSTASH_FLAGS($LIBPCRE2DIR)\n+GIT_UNSTASH_FLAGS($LIBPCREDIR)\n \n GIT_CONF_SUBST([USE_LIBPCRE2])\n \ndiff --git a/grep.c b/grep.c\nindex 39955f5745..1c8d47b615 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -60,13 +60,8 @@ static int parse_pattern_type_arg(const char *opt, const char *arg)\n \t\treturn GREP_PATTERN_TYPE_ERE;\n \telse if (!strcmp(arg, \"fixed\"))\n \t\treturn GREP_PATTERN_TYPE_FIXED;\n-\telse if (!strcmp(arg, \"perl\") ||\n-\t\t !strcmp(arg, \"pcre\"))\n+\telse if (!strcmp(arg, \"perl\"))\n \t\treturn GREP_PATTERN_TYPE_PCRE;\n-\telse if (!strcmp(arg, \"pcre1\"))\n-\t\treturn GREP_PATTERN_TYPE_PCRE1;\n-\telse if (!strcmp(arg, \"pcre2\"))\n-\t\treturn GREP_PATTERN_TYPE_PCRE2;\n \tdie(\"bad %s argument: %s\", opt, arg);\n }\n \n@@ -215,18 +210,6 @@ static void grep_set_pattern_type_option(enum grep_pattern_type pattern_type, st\n \t\topt->pcre2 = 1;\n #endif\n \t\tbreak;\n-\n-\tcase GREP_PATTERN_TYPE_PCRE1:\n-\t\topt->fixed = 0;\n-\t\topt->pcre1 = 1;\n-\t\topt->pcre2 = 0;\n-\t\tbreak;\n-\n-\tcase GREP_PATTERN_TYPE_PCRE2:\n-\t\topt->fixed = 0;\n-\t\topt->pcre1 = 0;\n-\t\topt->pcre2 = 1;\n-\t\tbreak;\n \t}\n }\n \ndiff --git a/grep.h b/grep.h\nindex dd16b5fd53..1b56d327e2 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -89,9 +89,7 @@ enum grep_pattern_type {\n \tGREP_PATTERN_TYPE_BRE,\n \tGREP_PATTERN_TYPE_ERE,\n \tGREP_PATTERN_TYPE_FIXED,\n-\tGREP_PATTERN_TYPE_PCRE,\n-\tGREP_PATTERN_TYPE_PCRE1,\n-\tGREP_PATTERN_TYPE_PCRE2\n+\tGREP_PATTERN_TYPE_PCRE\n };\n \n struct grep_expr {\ndiff --git a/t/README b/t/README\nindex 547b06e700..a90cb62583 100644\n--- a/t/README\n+++ b/t/README\n@@ -808,18 +808,6 @@ use these, and \"test_set_prereq\" for how to define your own.\n Git was compiled with support for PCRE. Wrap any tests\n that use git-grep --perl-regexp or git-grep -P in these.\n \n- - LIBPCRE1\n-\n- Git was compiled with PCRE v1 support via\n- USE_LIBPCRE=YesPlease. Wrap any PCRE using tests that for some\n- reason need v1 of the PCRE library instead of v2 in these.\n-\n- - LIBPCRE2\n-\n- Git was compiled with PCRE v2 support via\n- USE_LIBPCRE2=YesPlease. Wrap any PCRE using tests that for some\n- reason need v2 of the PCRE library instead of v1 in these.\n-\n - CASE_INSENSITIVE_FS\n \n Test is run on a case insensitive file system.\ndiff --git a/t/perf/p7820-grep-engines.sh b/t/perf/p7820-grep-engines.sh\nindex 70c5a5ca32..96d993ec7d 100755\n--- a/t/perf/p7820-grep-engines.sh\n+++ b/t/perf/p7820-grep-engines.sh\n@@ -7,7 +7,7 @@ test_description=\"Comparison of git-grep's regex engines\"\n test_perf_large_repo\n test_checkout_worktree\n \n-for engine in extended pcre1 pcre2\n+for engine in extended perl\n do\n \t# Patterns stolen from http://sljit.sourceforge.net/pcre.html\n \tfor pattern in \\\ndiff --git a/t/t7810-grep.sh b/t/t7810-grep.sh\nindex bae7e524b9..f5f1b61e02 100755\n--- a/t/t7810-grep.sh\n+++ b/t/t7810-grep.sh\n@@ -1063,16 +1063,6 @@ test_expect_success PCRE 'grep -P pattern' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE1 'grep libpcre1 pattern' '\n-\tgit -c grep.patternType=pcre1 grep \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n-\ttest_cmp expected actual\n-'\n-\n-test_expect_success LIBPCRE2 'grep libpcre2 pattern' '\n-\tgit -c grep.patternType=pcre2 grep \"\\p{Ps}.*?\\p{Pe}\" hello.c >actual &&\n-\ttest_cmp expected actual\n-'\n-\n test_expect_success 'grep pattern with grep.extendedRegexp=true' '\n \t>empty &&\n \ttest_must_fail git -c grep.extendedregexp=true \\\n@@ -1522,28 +1512,10 @@ test_expect_success 'grep does not report i-t-a and assume unchanged with -L' '\n \ttest_cmp expected actual\n '\n \n-test_expect_success PCRE \"grep with grep.patternType synonyms perl/pcre\" '\n+test_expect_success PCRE \"grep with grep.patternType synonyms perl\" '\n \techo \"#include \" >expected &&\n \tgit -c grep.patternType=perl grep -h --no-line-number \"st(?=dio)\" >actual &&\n-\ttest_cmp expected actual &&\n-\tgit -c grep.patternType=pcre grep -h --no-line-number \"st(?=dio)\" >actual &&\n \ttest_cmp expected actual\n '\n \n-test_expect_success LIBPCRE1 \"grep with grep.patternType=pcre1\" '\n-\techo \"#include \" >expected &&\n-\tgit -c grep.patternType=pcre1 grep -h --no-line-number \"st(?=dio)\" >actual &&\n-\ttest_cmp expected actual &&\n-\ttest_must_fail git -c grep.patternType=pcre1 grep \"foo(?+bar)\" 2>error &&\n-\ttest_i18ngrep -q \"digit expected after\" error\n-'\n-\n-test_expect_success LIBPCRE2 \"grep with grep.patternType=pcre2\" '\n-\techo \"#include \" >expected &&\n-\tgit -c grep.patternType=pcre2 grep -h --no-line-number \"st(?=dio)\" >actual &&\n-\ttest_cmp expected actual &&\n-\ttest_must_fail git -c grep.patternType=pcre2 grep \"foo(?+bar)\" 2>error &&\n-\ttest_i18ngrep -q \"digit expected after\" error\n-'\n-\n test_done\ndiff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh\nindex 7a8a48493b..ef658b7899 100755\n--- a/t/t7814-grep-recurse-submodules.sh\n+++ b/t/t7814-grep-recurse-submodules.sh\n@@ -358,19 +358,7 @@ test_expect_success 'grep --recurse-submodules should pass the pattern type alon\n \t\tEOF\n \t\ttest_cmp expect actual &&\n \t\tgit -c grep.patternType=perl grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n-\t\ttest_cmp expect actual &&\n-\t\tgit -c grep.patternType=pcre grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n-\t\ttest_cmp expect actual &&\n-\t\tif test_have_prereq LIBPCRE1\n-\t\tthen\n-\t\t\tgit -c grep.patternType=pcre1 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n-\t\t\ttest_cmp expect actual\n-\t\tfi &&\n-\t\tif test_have_prereq LIBPCRE2\n-\t\tthen\n-\t\t\tgit -c grep.patternType=pcre2 grep --recurse-submodules -e \"(.|.)[\\d]\" >actual &&\n-\t\t\ttest_cmp expect actual\n-\t\tfi\n+\t\ttest_cmp expect actual\n \tfi\n '\n \ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 6f873de3e7..969f931ebf 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1010,9 +1010,8 @@ esac\n ( COLUMNS=1 && test $COLUMNS = 1 ) && test_set_prereq COLUMNS_CAN_BE_1\n test -z \"$NO_PERL\" && test_set_prereq PERL\n test -z \"$NO_PYTHON\" && test_set_prereq PYTHON\n-test -n \"$USE_LIBPCRE1$USE_LIBPCRE2\" && test_set_prereq PCRE\n-test -n \"$USE_LIBPCRE1\" && test_set_prereq LIBPCRE1\n-test -n \"$USE_LIBPCRE2\" && test_set_prereq LIBPCRE2\n+test -n \"$USE_LIBPCRE1\" && test_set_prereq PCRE\n+test -n \"$USE_LIBPCRE2\" && test_set_prereq PCRE\n test -z \"$NO_GETTEXT\" && test_set_prereq GETTEXT\n \n # Can we rely on git's output in the C locale?\n-- \n2.11.0\n\n"},{"id":"317422","messageId":"20170420212345.7408-16-avarab@gmail.com","threadId":"45765","inReplyTo":"20170420212345.7408-1-avarab@gmail.com","subject":"[PATCH v3 15/18] grep: add support for the PCRE v1 JIT API","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-04-20T21:23:42Z","receivedAt":"2017-04-20T21:25:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Change the grep PCRE v1 code to use JIT when available. When PCRE\nsupport was initially added in commit 63e7e9d8b6 (\"git-grep: Learn\nPCRE\", 2011-05-09) PCRE had no JIT support, it was integrated into\n8.20 released on 2011-10-21.\n\nWhen JIT support is enabled the PCRE performance usually improves by\nmore than 50%. The pattern compilation times are relatively slower,\nbut those relative numbers are tiny, and are easily made back in all\nbut the most trivial cases of grep. Detailed benchhmarks are available\nat: http://sljit.sourceforge.net/pcre.html\n\nWith this change the difference in a t/perf/p7820-grep-engines.sh run\nis, shown with git --word-diff:\n\n 7820.1: extended with how.to [-0.28(1.23+0.44)-]{+0.28(1.18+0.39)+}\n 7820.2: extended with ^how to [-0.26(1.15+0.38)-]{+0.27(1.13+0.40)+}\n 7820.3: extended with \\w+our\\w* [-6.06(38.44+0.35)-]{+6.11(38.66+0.32)+}\n 7820.4: extended with -?-?-?-?-?-?-?-?-?-?-?-----------$ [-0.37(1.57+0.38)-]{+0.37(1.56+0.42)+}\n 7820.5: pcre1 with how.to [-0.26(1.15+0.37)-]{+0.19(0.39+0.55)+}\n 7820.6: pcre1 with ^how to [-0.46(2.66+0.31)-]{+0.22(0.67+0.44)+}\n 7820.7: pcre1 with \\w+our\\w* [-16.42(99.42+0.48)-]{+0.51(3.05+0.24)+}\n 7820.8: pcre1 with -?-?-?-?-?-?-?-?-?-?-?-----------$ [-81.52(275.37+0.41)-]{+5.16(19.31+0.33)+}\n\nThe conditional support for JIT is implemented as suggested in the\npcrejit(3) man page. E.g. defining PCRE_STUDY_JIT_COMPILE to 0 if it's\nnot present.\n\nThere's no graceful fallback if pcre_jit_stack_alloc() fails under\nPCRE_CONFIG_JIT, instead the program will abort. I don't think this is\nworth handling, it'll only fail in cases where malloc() doesn't work,\nin which case we're screwed anyway.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n grep.c | 27 ++++++++++++++++++++++++++-\n grep.h | 5 +++++\n 2 files changed, 31 insertions(+), 1 deletion(-)\n\ndiff --git a/grep.c b/grep.c\nindex d2c87ee2c3..eb68bdaa2a 100644\n--- a/grep.c\n+++ b/grep.c\n@@ -331,6 +331,9 @@ static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)\n \tconst char *error;\n \tint erroffset;\n \tint options = PCRE_MULTILINE;\n+#ifdef PCRE_CONFIG_JIT\n+\tint canjit;\n+#endif\n \n \tif (opt->ignore_case) {\n \t\tif (has_non_ascii(p->pattern))\n@@ -345,9 +348,19 @@ static void compile_pcre1_regexp(struct grep_pat *p, const struct grep_opt *opt)\n \tif (!p->pcre1_regexp)\n \t\tcompile_regexp_failed(p, error);\n \n-\tp->pcre1_extra_info = pcre_study(p->pcre1_regexp, 0, &error);\n+\tp->pcre1_extra_info = pcre_study(p->pcre1_regexp, PCRE_STUDY_JIT_COMPILE, &error);\n \tif (!p->pcre1_extra_info && error)\n \t\tdie(\"%s\", error);\n+\n+#ifdef PCRE_CONFIG_JIT\n+\tpcre_config(PCRE_CONFIG_JIT, &canjit);\n+\tif (canjit == 1) {\n+\t\tp->pcre1_jit_stack = pcre_jit_stack_alloc(1, 1024 * 1024);\n+\t\tif (!p->pcre1_jit_stack)\n+\t\t\tdie(\"BUG: Couldn't allocate PCRE JIT stack\");\n+\t\tpcre_assign_jit_stack(p->pcre1_extra_info, NULL, p->pcre1_jit_stack);\n+\t}\n+#endif\n }\n \n static int pcre1match(struct grep_pat *p, const char *line, const char *eol,\n@@ -358,8 +371,15 @@ static int pcre1match(struct grep_pat *p, const char *line, const char *eol,\n \tif (eflags & REG_NOTBOL)\n \t\tflags |= PCRE_NOTBOL;\n \n+#ifdef PCRE_CONFIG_JIT\n+\tret = pcre_jit_exec(p->pcre1_regexp, p->pcre1_extra_info, line, eol - line,\n+\t\t\t 0, flags, ovector, ARRAY_SIZE(ovector),\n+\t\t\t p->pcre1_jit_stack);\n+#else\n \tret = pcre_exec(p->pcre1_regexp, p->pcre1_extra_info, line, eol - line,\n \t\t\t0, flags, ovector, ARRAY_SIZE(ovector));\n+#endif\n+\n \tif (ret < 0 && ret != PCRE_ERROR_NOMATCH)\n \t\tdie(\"pcre_exec failed with error code %d\", ret);\n \tif (ret > 0) {\n@@ -374,7 +394,12 @@ static int pcre1match(struct grep_pat *p, const char *line, const char *eol,\n static void free_pcre1_regexp(struct grep_pat *p)\n {\n \tpcre_free(p->pcre1_regexp);\n+#ifdef PCRE_CONFIG_JIT\n+\tpcre_free_study(p->pcre1_extra_info);\n+\tpcre_jit_stack_free(p->pcre1_jit_stack);\n+#else\n \tpcre_free(p->pcre1_extra_info);\n+#endif\n \tpcre_free((void *)p->pcre1_tables);\n }\n #else /* !USE_LIBPCRE1 */\ndiff --git a/grep.h b/grep.h\nindex fa2ab9485f..29e20bf837 100644\n--- a/grep.h\n+++ b/grep.h\n@@ -3,9 +3,13 @@\n #include \"color.h\"\n #ifdef USE_LIBPCRE1\n #include \n+#ifndef PCRE_STUDY_JIT_COMPILE\n+#define PCRE_STUDY_JIT_COMPILE 0\n+#endif\n #else\n typedef int pcre;\n typedef int pcre_extra;\n+typedef int pcre_jit_stack;\n #endif\n #include \"kwset.h\"\n #include \"thread-utils.h\"\n@@ -48,6 +52,7 @@ struct grep_pat {\n \tregex_t regexp;\n \tpcre *pcre1_regexp;\n \tpcre_extra *pcre1_extra_info;\n+\tpcre_jit_stack *pcre1_jit_stack;\n \tconst unsigned char *pcre1_tables;\n \tkwset_t kws;\n \tunsigned fixed:1;\n-- \n2.11.0\n\n"},{"id":"317430","messageId":"20170420222016.GJ142567@google.com","threadId":"45765","inReplyTo":"20170420212345.7408-4-avarab@gmail.com","subject":"Re: [PATCH v3 03/18] grep: submodule-related case statements should die if new fields are added","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-04-20T22:20:16Z","receivedAt":"2017-04-20T22:20:24Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 04/20, Ævar Arnfjörð Bjarmason wrote:\n> Change two case statements added in commit 0281e487fd (\"grep:\n> optionally recurse into submodules\", 2016-12-16) so that they die if\n> new GREP_PATTERN_* enum fields are added without updating them.\n> \n> These case statements currently check for an exhaustive list of\n> fields, but if a new field is added it's easy to introduce a bug here\n> where the code will start subtly doing the wrong thing, e.g. if a new\n> pattern type is added we'll fall through to\n> GREP_PATTERN_TYPE_UNSPECIFIED, i.e. the \"basic\" POSIX regular\n> expressions.\n> \n> This should arguably be done for the switch(opt->binary)\n> case-statement as well, but isn't trivial to add since that code isn't\n> currently working with an exhaustive list.\n\nI was under the impression that the code wouldn't compile if there is a\nmissing enum field in the switch statement. Does it instead silently\nfall through? I would choose not compiling over a die statement that may\nnot be caught during the development of a new series.\n\n> \n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> builtin/grep.c | 4 ++++\n> 1 file changed, 4 insertions(+)\n> \n> diff --git a/builtin/grep.c b/builtin/grep.c\n> index 3ffb5b4e81..be3dbd6957 100644\n> --- a/builtin/grep.c\n> +++ b/builtin/grep.c\n> @@ -495,6 +495,8 @@ static void compile_submodule_options(const struct grep_opt *opt,\n> \t\tbreak;\n> \tcase GREP_PATTERN_TYPE_UNSPECIFIED:\n> \t\tbreak;\n> +\tdefault:\n> +\t\tdie(\"BUG: Added a new grep pattern type without updating switch statement\");\n> \t}\n> \n> \tfor (pattern = opt->pattern_list; pattern != NULL;\n> @@ -515,6 +517,8 @@ static void compile_submodule_options(const struct grep_opt *opt,\n> \t\tcase GREP_PATTERN_BODY:\n> \t\tcase GREP_PATTERN_HEAD:\n> \t\t\tbreak;\n> +\t\tdefault:\n> +\t\t\tdie(\"BUG: Added a new grep token type without updating case statement\");\n> \t\t}\n> \t}\n> \n> -- \n> 2.11.0\n> \n\n-- \nBrandon Williams\n"},{"id":"317432","messageId":"20170420222209.GK142567@google.com","threadId":"45765","inReplyTo":"20170420212345.7408-2-avarab@gmail.com","subject":"Re: [PATCH v3 01/18] grep: amend submodule recursion test in preparation for rx engine testing","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2017-04-20T22:22:09Z","receivedAt":"2017-04-20T22:22:17Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 04/20, Ævar Arnfjörð Bjarmason wrote:\n> Amend the submodule recursion test added in commit 0281e487fd (\"grep:\n> optionally recurse into submodules\", 2016-12-16) to prepare it for\n> subsequent tests of whether it passes along the grep.patternType to\n> the submodule greps.\n> \n> This is just the result of searching & replacing:\n> \n> foobar -> (1|2)d(3|4)\n> foo -> (1|2)\n> bar -> (3|4)\n> \n> Currently there's no tests for whether e.g. -P or -E is correctly\n> passed along, tests for that will be added in a follow-up change, but\n> first add content to the tests which will match differently under\n> different regex engines.\n> \n> Reuse the pattern established in an earlier commit of mine in this\n> series (\"log: add exhaustive tests for pattern style options &\n> config\", 2017-04-07). The pattern \"(.|.)[\\d]\" will match this content\n> differently under fixed/basic/extended & perl.\n> \n> Signed-off-by: Ævar Arnfjörð Bjarmason \n\nLooks good.\n\n-- \nBrandon Williams\n"},{"id":"317434","messageId":"20170420222747.ooo5z63tzi6d5ayl@sigill.intra.peff.net","threadId":"45765","inReplyTo":"20170420222016.GJ142567@google.com","subject":"Re: [PATCH v3 03/18] grep: submodule-related case statements should die if new fields are added","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-04-20T22:27:47Z","receivedAt":"2017-04-20T22:27:57Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Apr 20, 2017 at 03:20:16PM -0700, Brandon Williams wrote:\n\n> On 04/20, Ævar Arnfjörð Bjarmason wrote:\n> > Change two case statements added in commit 0281e487fd (\"grep:\n> > optionally recurse into submodules\", 2016-12-16) so that they die if\n> > new GREP_PATTERN_* enum fields are added without updating them.\n> > \n> > These case statements currently check for an exhaustive list of\n> > fields, but if a new field is added it's easy to introduce a bug here\n> > where the code will start subtly doing the wrong thing, e.g. if a new\n> > pattern type is added we'll fall through to\n> > GREP_PATTERN_TYPE_UNSPECIFIED, i.e. the \"basic\" POSIX regular\n> > expressions.\n> > \n> > This should arguably be done for the switch(opt->binary)\n> > case-statement as well, but isn't trivial to add since that code isn't\n> > currently working with an exhaustive list.\n> \n> I was under the impression that the code wouldn't compile if there is a\n> missing enum field in the switch statement. Does it instead silently\n> fall through? I would choose not compiling over a die statement that may\n> not be caught during the development of a new series.\n\nUsually -Wswitch would catch this, but the variable in question is\ndeclared as an int. The original pattern_type_arg variable is an int\nbecause we pass its address via OPT_SET_INT().\n\nWe could make the argument to compile_submodule_options() an enum, but\nthen we get an implicit cast when we pass the int (which could fail at\nruntime). Yech. I'm not sure if there's a good and easy solution.\n\n-Peff\n"}]}