threads / patch / 29732

patchgrep -P: Fix matching ^ and $

Subject: [PATCH] grep -P: Fix matching ^ and $

## tl;dr

6 messages between Feb 25, 2012 and Feb 27, 2012. Diffs are folded; open one to read it.

replies: 5people: 3as markdown or json

Michał Kiedrowicz· Feb 25, 2012, 09:24 UTC · lore

When `git-grep` is run with -P/--perl-regexp, it doesn't match ^ and $ at the beginning/end of the line. This is because PCRE normally matches ^ and $ at the beginning/end of the whole text, not for each line, and git-grep firstly passes a large chunk of text (possibly containing many lines) to pcre_exec() before it splits the text into lines. This makes `git-grep -P` behave differently from `git-grep -E` and also from `grep -P` and `pcregrep`:

	$ cat file
	a
	 b
	$ git --no-pager grep --no-index -P '^ ' file
	$ git --no-pager grep --no-index -E '^ ' file
	file: b
	$ grep -c -P '^ ' file
	 b
	$ pcregrep -c '^ ' file
	 b
Reported-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
---
 grep.c          |    2 +-
 t/t7810-grep.sh |   23 +++++++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletions(-)
Show changes to 2 files +24 −1

grep.c, t/t7810-grep.sh

diff --git a/grep.c b/grep.c
index 3821400..f492d26 100644
--- a/grep.c
+++ b/grep.c
@@ -79,7 +79,7 @@ static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)
 {
 	const char *error;
 	int erroffset;
-	int options = 0;
+	int options = PCRE_MULTILINE;
 
 	if (opt->ignore_case)
 		options |= PCRE_CASELESS;
diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh
index 75f4716..dd6e6d5 100755
--- a/t/t7810-grep.sh
+++ b/t/t7810-grep.sh
@@ -47,6 +47,13 @@ test_expect_success setup '
 	echo vvv >t/v &&
 	mkdir t/a &&
 	echo vvv >t/a/v &&
+	{
+		echo "line without leading space1"
+		echo " line with leading space1"
+		echo " line with leading space2"
+		echo " line with leading space3"
+		echo "line without leading space2"
+	} >space &&
 	git add . &&
 	test_tick &&
 	git commit -m initial
@@ -893,4 +900,20 @@ test_expect_success 'mimic ack-grep --group' '
 	test_cmp expected actual
 '
 
+cat >expected <<EOF
+space: line with leading space1
+space: line with leading space2
+space: line with leading space3
+EOF
+
+test_expect_success 'grep -E "^ "' '
+	git grep -E "^ " space >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success "grep -P '^ '" '
+	git grep -P "^ " space >actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
1.7.8.4
Michał Kiedrowicz· Feb 25, 2012, 09:30 UTC · re: Michał Kiedrowicz · lore

Re: [PATCH] grep -P: Fix matching ^ and $

Michał Kiedrowicz <michal.kiedrowicz@gmail.com> wrote:
Show 18 quoted lines
> When `git-grep` is run with -P/--perl-regexp, it doesn't match ^ and $ at
> the beginning/end of the line.  This is because PCRE normally matches ^
> and $ at the beginning/end of the whole text, not for each line, and git-grep
> firstly passes a large chunk of text (possibly containing many lines) to
> pcre_exec() before it splits the text into lines.  This makes `git-grep -P`
> behave differently from `git-grep -E` and also from `grep -P` and `pcregrep`:
> 
> 	$ cat file
> 	a
> 	 b
> 	$ git --no-pager grep --no-index -P '^ ' file
> 	$ git --no-pager grep --no-index -E '^ ' file
> 	file: b
> 	$ grep -c -P '^ ' file
> 	 b
> 	$ pcregrep -c '^ ' file
> 	 b
> 

Original report: http://permalink.gmane.org/gmane.comp.version-control.git/190830

Show 59 quoted lines
> Reported-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
> Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
> ---
>  grep.c          |    2 +-
>  t/t7810-grep.sh |   23 +++++++++++++++++++++++
>  2 files changed, 24 insertions(+), 1 deletions(-)
> 
> diff --git a/grep.c b/grep.c
> index 3821400..f492d26 100644
> --- a/grep.c
> +++ b/grep.c
> @@ -79,7 +79,7 @@ static void compile_pcre_regexp(struct grep_pat *p, const struct grep_opt *opt)
>  {
>  	const char *error;
>  	int erroffset;
> -	int options = 0;
> +	int options = PCRE_MULTILINE;
>  
>  	if (opt->ignore_case)
>  		options |= PCRE_CASELESS;
> diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh
> index 75f4716..dd6e6d5 100755
> --- a/t/t7810-grep.sh
> +++ b/t/t7810-grep.sh
> @@ -47,6 +47,13 @@ test_expect_success setup '
>  	echo vvv >t/v &&
>  	mkdir t/a &&
>  	echo vvv >t/a/v &&
> +	{
> +		echo "line without leading space1"
> +		echo " line with leading space1"
> +		echo " line with leading space2"
> +		echo " line with leading space3"
> +		echo "line without leading space2"
> +	} >space &&
>  	git add . &&
>  	test_tick &&
>  	git commit -m initial
> @@ -893,4 +900,20 @@ test_expect_success 'mimic ack-grep --group' '
>  	test_cmp expected actual
>  '
>  
> +cat >expected <<EOF
> +space: line with leading space1
> +space: line with leading space2
> +space: line with leading space3
> +EOF
> +
> +test_expect_success 'grep -E "^ "' '
> +	git grep -E "^ " space >actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success "grep -P '^ '" '
> +	git grep -P "^ " space >actual &&
> +	test_cmp expected actual
> +'
> +
>  test_done
Zbigniew Jędrzejewski-Szmek· Feb 25, 2012, 17:52 UTC · re: Michał Kiedrowicz · lore

Re: [PATCH] grep -P: Fix matching ^ and $

On 02/25/2012 10:30 AM, Michał Kiedrowicz wrote:
Show 8 quoted lines
> Michał Kiedrowicz<michal.kiedrowicz@gmail.com>  wrote:
>
>> When `git-grep` is run with -P/--perl-regexp, it doesn't match ^ and $ at
>> the beginning/end of the line.  This is because PCRE normally matches ^
>> and $ at the beginning/end of the whole text, not for each line, and git-grep
>> firstly passes a large chunk of text (possibly containing many lines) to
>> pcre_exec() before it splits the text into lines.  This makes `git-grep -P`
>> behave differently from `git-grep -E` and also from `grep -P` and `pcregrep`:
Thanks! I can confirm that I now get the expected output.
Zbyszek
> Original report:
> http://permalink.gmane.org/gmane.comp.version-control.git/190830
Junio C Hamano· Feb 26, 2012, 22:39 UTC · re: Michał Kiedrowicz · lore

Re: [PATCH] grep -P: Fix matching ^ and $

Michał Kiedrowicz <michal.kiedrowicz@gmail.com> writes:
Show 19 quoted lines
> @@ -893,4 +900,20 @@ test_expect_success 'mimic ack-grep --group' '
>  	test_cmp expected actual
>  '
>  
> +cat >expected <<EOF
> +space: line with leading space1
> +space: line with leading space2
> +space: line with leading space3
> +EOF
> +
> +test_expect_success 'grep -E "^ "' '
> +	git grep -E "^ " space >actual &&
> +	test_cmp expected actual
> +'
> +
> +test_expect_success "grep -P '^ '" '
> +	git grep -P "^ " space >actual &&
> +	test_cmp expected actual
> +'

This test does not pass for me as I do not usually build with pcre; shouldn't it be protected with some test prerequisite?

Otherwise the patch looks good; thanks.
Zbigniew Jędrzejewski-Szmek· Feb 27, 2012, 16:45 UTC · re: Junio C Hamano · lore

[PATCH] grep -P: add tests for matching ^ and $

From: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>

When a fix for matching ^ and $ with -P was commited to master in fba4f1 (grep -P: Fix matching ^ and $), the tests were missing the LIBPCRE prerequisite check and were dropped from the patch. Here are the tests guarded with LIBPCRE.

Signed-off-by: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
Signed-off-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>
---
 t/t7810-grep.sh |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
Show changes to t/t7810-grep.sh +23 −0
diff --git a/t/t7810-grep.sh b/t/t7810-grep.sh
index 75f4716..d9ad633 100755
--- a/t/t7810-grep.sh
+++ b/t/t7810-grep.sh
@@ -47,6 +47,13 @@ test_expect_success setup '
 	echo vvv >t/v &&
 	mkdir t/a &&
 	echo vvv >t/a/v &&
+	{
+		echo "line without leading space1"
+		echo " line with leading space1"
+		echo " line with leading space2"
+		echo " line with leading space3"
+		echo "line without leading space2"
+	} >space &&
 	git add . &&
 	test_tick &&
 	git commit -m initial
@@ -893,4 +900,20 @@ test_expect_success 'mimic ack-grep --group' '
 	test_cmp expected actual
 '
 
+cat >expected <<EOF
+space: line with leading space1
+space: line with leading space2
+space: line with leading space3
+EOF
+
+test_expect_success LIBPCRE 'grep -E "^ "' '
+	git grep -E "^ " space >actual &&
+	test_cmp expected actual
+'
+
+test_expect_success LIBPCRE 'grep -P "^ "' '
+	git grep -P "^ " space >actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
1.7.9.2.396.ga883d.dirty
Junio C Hamano· Feb 27, 2012, 20:21 UTC · re: Zbigniew Jędrzejewski-Szmek · lore

Re: [PATCH] grep -P: add tests for matching ^ and $

Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl> writes:
Show 6 quoted lines
> From: Michał Kiedrowicz <michal.kiedrowicz@gmail.com>
>
> When a fix for matching ^ and $ with -P was commited to master in
> fba4f1 (grep -P: Fix matching ^ and $), the tests were missing the
> LIBPCRE prerequisite check and were dropped from the patch. Here are
> the tests guarded with LIBPCRE.
Thanks.

The real reason I separated it was because I didn't want to worry about the test part when merging this down to older maintenance releases.

← back to recent threads