threads / patch / 27235

patchgrep: add known breakage of coloring when using extended patterns

Subject: [PATCH] grep: add known breakage of coloring when using extended patterns

## tl;dr

8 messages between May 2, 2011 and May 3, 2011. Diffs are folded; open one to read it.

replies: 7people: 4as markdown or json

Bert Wesarg· May 2, 2011, 11:35 UTC · lore

The coloring code does not respect the extended pattern format, expose this with an known breakage test.

Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
---
 t/t7812-grep-color.sh |   25 +++++++++++++++++++++++++
 1 files changed, 25 insertions(+), 0 deletions(-)
 create mode 100755 t/t7812-grep-color.sh
Show changes to t/t7812-grep-color.sh +25 −0
diff --git a/t/t7812-grep-color.sh b/t/t7812-grep-color.sh
new file mode 100755
index 0000000..2cffff3
--- /dev/null
+++ b/t/t7812-grep-color.sh
@@ -0,0 +1,25 @@
+#!/bin/sh
+
+test_description='git grep --color
+'
+
+. ./test-lib.sh
+
+test_expect_success 'setup' '
+	test_commit initial input "foo bar baz
+bar baz
+"
+'
+
+cat >expected <<EOF
+input<CYAN>:<RESET><BOLD;RED>foo<RESET> bar baz
+EOF
+
+
+test_expect_failure 'grep coloring does not honor boolean algebra' '
+	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
+		test_decode_color >actual
+    test_cmp expected actual
+'
+
+test_done

-- 
1.7.5.349.gfeb1a
Johannes Sixt· May 2, 2011, 11:41 UTC · re: Bert Wesarg · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

Am 5/2/2011 13:35, schrieb Bert Wesarg:
> +test_expect_failure 'grep coloring does not honor boolean algebra' '
I'm sure you meant to say 'grep coloring honors boolean algebra', no?
-- Hannes
Bert Wesarg· May 2, 2011, 11:48 UTC · re: Johannes Sixt · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

On Mon, May 2, 2011 at 13:41, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 5/2/2011 13:35, schrieb Bert Wesarg:
>> +test_expect_failure 'grep coloring does not honor boolean algebra' '
>
> I'm sure you meant to say 'grep coloring honors boolean algebra', no?
You're right, I'm not so good in double-negations.
Bert
>
> -- Hannes
>
René Scharfe· May 2, 2011, 17:14 UTC · re: Bert Wesarg · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

Am 02.05.2011 13:35, schrieb Bert Wesarg:
Show 40 quoted lines
> The coloring code does not respect the extended pattern format, expose
> this with an known breakage test.
>
> Signed-off-by: Bert Wesarg<bert.wesarg@googlemail.com>
> ---
>   t/t7812-grep-color.sh |   25 +++++++++++++++++++++++++
>   1 files changed, 25 insertions(+), 0 deletions(-)
>   create mode 100755 t/t7812-grep-color.sh
>
> diff --git a/t/t7812-grep-color.sh b/t/t7812-grep-color.sh
> new file mode 100755
> index 0000000..2cffff3
> --- /dev/null
> +++ b/t/t7812-grep-color.sh
> @@ -0,0 +1,25 @@
> +#!/bin/sh
> +
> +test_description='git grep --color
> +'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup' '
> +	test_commit initial input "foo bar baz
> +bar baz
> +"
> +'
> +
> +cat>expected<<EOF
> +input<CYAN>:<RESET><BOLD;RED>foo<RESET>  bar baz
> +EOF
> +
> +
> +test_expect_failure 'grep coloring does not honor boolean algebra' '
> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
> +		test_decode_color>actual
> +    test_cmp expected actual
> +'
> +
> +test_done

The current code highlights the given search terms ("atoms"). You expect the matching part of a line to be highlighted instead. Another example with differing results between the two ways is this:

	$ git grep -v -e foo --and -e bar

The first one was easy to implement and useful already. I wonder how to implement the second one (I couldn't think of an easy way back then) and also if we really prefer it to the current behaviour in the first place.

The twisted logic that convinced me to forget about the issue goes like this: Even by giving negative search terms, the user is expressing a certain value of interest in them; highlighting these atoms even in lines which were not matched by them is a courtesy.

René
Junio C Hamano· May 2, 2011, 17:37 UTC · re: René Scharfe · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
>> +	test_commit initial input "foo bar baz
...
>> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
...
> The current code highlights the given search terms ("atoms").

Hmm, I was probably not paying attention to "coloring the parts that matched" topic at all, but wouldn't it be easier and more efficient to paint only "foo" without painting "bar baz"? We know the first term "foo" matches, and the rest \(...\) that is --or'ed does not have to even be evaluated, no?

René Scharfe· May 2, 2011, 21:07 UTC · re: Junio C Hamano · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

Am 02.05.2011 19:37, schrieb Junio C Hamano:
Show 13 quoted lines
> René Scharfe<rene.scharfe@lsrfire.ath.cx>  writes:
>
>>> +	test_commit initial input "foo bar baz
> ...
>>> +	git grep --color -e foo --or \( -e bar --and --not -e baz \) |
> ...
>> The current code highlights the given search terms ("atoms").
>
> Hmm, I was probably not paying attention to "coloring the parts that
> matched" topic at all, but wouldn't it be easier and more efficient to
> paint only "foo" without painting "bar baz"?  We know the first term "foo"
> matches, and the rest \(...\) that is --or'ed does not have to even be
> evaluated, no?

Aggregating the set of matching characters and passing them back during expression evaluation can be more efficient, yes, as it would avoid calling regexec() on the printed lines again only to find out what to color. But I wouldn't call it easy. E.g. how to return the set of matching characters in the following case?

	$ git grep --color -e foo --and -e bar

I can only think of using lists or, even uglier, perhaps a bitmap to remember which characters to highlight.

Also GNU grep doesn't only color the first sufficient match. E.g. this will color both b and d:

	$ echo abcde | grep --color -e b -e d
And I think this makes sense and meets my expectations.

But the other mode, which only colors the exact characters that were used to determine that the shown line matches (or doesn't match, if -v is given) may be interesting as well. I'm curious to see an implementation and wonder if the results are really better than the ones of the current (cheating) approach. :)

René
Junio C Hamano· May 2, 2011, 22:04 UTC · re: René Scharfe · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
Show 7 quoted lines
> Aggregating the set of matching characters and passing them back
> during expression evaluation can be more efficient, yes, as it would
> avoid calling regexec() on the printed lines again only to find out
> what to color.  But I wouldn't call it easy.  E.g. how to return the
> set of matching characters in the following case?
>
> 	$ git grep --color -e foo --and -e bar

I would naïvely expect each of the "grep_expr" in the parsed grep tree to have not just a one-bit "hit", but rm_so/rm_eo pair, but as I said, I was asleep while "coloring the parts that matched" discussion was in progress ;-)

But
> Also GNU grep doesn't only color the first sufficient match.
> E.g. this will color both b and d:
>
> 	$ echo abcde | grep --color -e b -e d

that is probably a useful thing to have. Obviously, my suggestion should optimize away the match with 'd' to begin with, so keeping rm_so/rm_eo pair would not help painting d at all. Worse yet, I would expect that

	$ echo abcdeabcde | grep --color -e b -e d

would need to paint all occurrences for such a purpose, so obviously you would need an extra pass to find all potential hits. Ugly and makes me personally wonder if it is really worth it, but the user is explicitly asking for --color and is prepared to pay the price, so it probably is Ok.

René Scharfe· May 3, 2011, 17:36 UTC · re: Junio C Hamano · lore

Re: [PATCH] grep: add known breakage of coloring when using extended patterns

Am 03.05.2011 00:04, schrieb Junio C Hamano:
Show 14 quoted lines
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> 
>> Aggregating the set of matching characters and passing them back
>> during expression evaluation can be more efficient, yes, as it would
>> avoid calling regexec() on the printed lines again only to find out
>> what to color.  But I wouldn't call it easy.  E.g. how to return the
>> set of matching characters in the following case?
>>
>> 	$ git grep --color -e foo --and -e bar
> 
> I would naïvely expect each of the "grep_expr" in the parsed grep tree to
> have not just a one-bit "hit", but rm_so/rm_eo pair, but as I said, I was
> asleep while "coloring the parts that matched" discussion was in progress
> ;-)

That's true, but each one represents a single contiguous run of characters to color. But I meant to say that returning non-contiguous ranges, as you can get in the --and example above, is not that easy. To me.

Hmm, if one just wants to color the parts that triggered a match it gets a lot easier. Each pattern just needs to remember the last matching offsets and then show_line can walk the expression tree to find out what needs coloring. Not sure about the details, though. :)

Show 17 quoted lines
> But
> 
>> Also GNU grep doesn't only color the first sufficient match.
>> E.g. this will color both b and d:
>>
>> 	$ echo abcde | grep --color -e b -e d
> 
> that is probably a useful thing to have.  Obviously, my suggestion should
> optimize away the match with 'd' to begin with, so keeping rm_so/rm_eo
> pair would not help painting d at all.  Worse yet, I would expect that
> 
> 	$ echo abcdeabcde | grep --color -e b -e d
> 
> would need to paint all occurrences for such a purpose, so obviously you
> would need an extra pass to find all potential hits.  Ugly and makes me
> personally wonder if it is really worth it, but the user is explicitly
> asking for --color and is prepared to pay the price, so it probably is Ok.

The performance impact is low as long as there are lots more lines that don't match than matching ones. Which should be a given for most uses of grep.

I always miss hit coloring if I have to use an environment that doesn't provide , similarly to syntax highlighting, so I do think it's worth it, for me at least.

René

← back to recent threads