threads / patch / 21663

patchunset GREP_OPTIONS in test-lib.sh

Subject: [PATCH] unset GREP_OPTIONS in test-lib.sh

## tl;dr

12 messages between Nov 18, 2009 and Mar 7, 2010. Diffs are folded; open one to read it.

replies: 11people: 5as markdown or json

Bert Wesarg· Nov 18, 2009, 16:15 UTC · lore

I used to set GREP_OPTIONS to exclude *.orig and *.rej files. But with this the test t4252-am-options.sh fails because it calls grep with a .rej file:

    grep "@@ -1,3 +1,3 @@" file-2.rej
Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
---
 t/test-lib.sh |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
Show changes to t/test-lib.sh +2 −0
diff --git a/t/test-lib.sh b/t/test-lib.sh
index f2ca536..6ac8dc6 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -65,6 +65,8 @@ GIT_TEST_CMP=${GIT_TEST_CMP:-diff -u}
 # CDPATH into the environment
 unset CDPATH
 
+unset GREP_OPTIONS
+
 case $(echo $GIT_TRACE |tr "[A-Z]" "[a-z]") in
 	1|2|true)
 		echo "* warning: Some tests will not work if GIT_TRACE" \
-- 
tg: (785c58e..) bw/unset-GREP_OPTIONS (depends on: master)
Junio C Hamano· Nov 18, 2009, 22:05 UTC · re: Bert Wesarg · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

Bert Wesarg <bert.wesarg@googlemail.com> writes:
> I used to set GREP_OPTIONS to exclude *.orig and *.rej files. But with this
> the test t4252-am-options.sh fails because it calls grep with a .rej file:
Yuck.  Will apply.
That actually makes me worried about a different issue.

Do we kill that environment variable when we call out to external grep in grep.c? If not, we should. An alternative is to teach our internal one to also honor it, but I personally do not find it too attractive to mimic the design mistake of GREP_OPTIONS myself.

René Scharfe· Nov 22, 2009, 15:58 UTC · re: Junio C Hamano · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

Junio C Hamano schrieb:
> Do we kill that environment variable when we call out to external grep in
> grep.c?  If not, we should.  An alternative is to teach our internal one
> to also honor it, but I personally do not find it too attractive to mimic
> the design mistake of GREP_OPTIONS myself.

We don't. Here's a patch with a simple test case that makes git grep unset GREP_OPTIONS before it calls the external grep.

While we're at it, also unset GREP_COLOR and GREP_COLORS in case colouring is not enabled, to be on the safe side. The presence of these variables alone is not sufficient to trigger coloured output with GNU grep, but other implementations may behave differently.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
---
 builtin-grep.c  |    4 ++++
 t/t7002-grep.sh |    5 +++++
 2 files changed, 9 insertions(+), 0 deletions(-)
Show changes to 2 files +9 −0

builtin-grep.c, t/t7002-grep.sh

diff --git a/builtin-grep.c b/builtin-grep.c
index 01be9bf..9a9e3fc 100644
--- a/builtin-grep.c
+++ b/builtin-grep.c
@@ -433,7 +433,11 @@ static int external_grep(struct grep_opt *opt, const char **paths, int cached)
 
 		if (opt->color_external && strlen(opt->color_external) > 0)
 			push_arg(opt->color_external);
+	} else {
+		unsetenv("GREP_COLOR");
+		unsetenv("GREP_COLORS");
 	}
+	unsetenv("GREP_OPTIONS");
 
 	hit = 0;
 	argc = nr;
diff --git a/t/t7002-grep.sh b/t/t7002-grep.sh
index ae5290a..dd0da6c 100755
--- a/t/t7002-grep.sh
+++ b/t/t7002-grep.sh
@@ -213,6 +213,11 @@ test_expect_success 'grep -e A --and --not -e B' '
 	test_cmp expected actual
 '
 
+test_expect_success 'grep should ignore GREP_OPTIONS' '
+	GREP_OPTIONS=-v git grep " mmap bar\$" >actual &&
+	test_cmp expected actual
+'
+
 test_expect_success 'grep -f, non-existent file' '
 	test_must_fail git grep -f patterns
 '
-- 
1.6.5.3
Carlo Marcelo Arenas Belon· Nov 23, 2009, 11:22 UTC · re: René Scharfe · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

On Sun, Nov 22, 2009 at 04:58:09PM +0100, Ren? Scharfe wrote:
Show 13 quoted lines
> Junio C Hamano schrieb:
> > Do we kill that environment variable when we call out to external grep in
> > grep.c?  If not, we should.  An alternative is to teach our internal one
> > to also honor it, but I personally do not find it too attractive to mimic
> > the design mistake of GREP_OPTIONS myself.
> 
> We don't.  Here's a patch with a simple test case that makes git grep
> unset GREP_OPTIONS before it calls the external grep.
> 
> While we're at it, also unset GREP_COLOR and GREP_COLORS in case
> colouring is not enabled, to be on the safe side.  The presence of
> these variables alone is not sufficient to trigger coloured output with
> GNU grep, but other implementations may behave differently.
why not better to apply the proposed patch from Junio in :
  http://article.gmane.org/gmane.comp.version-control.git/127980/

it would IMHO correct all reported issues and serve as well as a catch all from other tools that could be introduced in the future and that will be similarly affected by this misfeature.

Carlo
Junio C Hamano· Nov 23, 2009, 18:27 UTC · re: Carlo Marcelo Arenas Belon · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

Carlo Marcelo Arenas Belon <carenas@sajinet.com.pe> writes:
Show 7 quoted lines
> why not better to apply the proposed patch from Junio in :
>
>   http://article.gmane.org/gmane.comp.version-control.git/127980/
>
> it would IMHO correct all reported issues and serve as well as a catch
> all from other tools that could be introduced in the future and that
> will be similarly affected by this misfeature.

I think René's patch is more sensible than $gmane/127980 because we have no business mucking with these environment variables when we are running things other than external grep. You could be using system's "grep" in your pre-commit hook to find some stuff, and your hook either may rely on your having a particular set of GREP_OPTIONS in your environment, or may be designed to work well with GREP_OPTIONS.

René Scharfe· Nov 23, 2009, 23:18 UTC · re: Junio C Hamano · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

Junio C Hamano schrieb:
Show 16 quoted lines
> Carlo Marcelo Arenas Belon <carenas@sajinet.com.pe> writes:
> 
>> why not better to apply the proposed patch from Junio in :
>>
>>   http://article.gmane.org/gmane.comp.version-control.git/127980/
>>
>> it would IMHO correct all reported issues and serve as well as a catch
>> all from other tools that could be introduced in the future and that
>> will be similarly affected by this misfeature.
> 
> I think René's patch is more sensible than $gmane/127980 because we have
> no business mucking with these environment variables when we are running
> things other than external grep.  You could be using system's "grep" in
> your pre-commit hook to find some stuff, and your hook either may rely
> on your having a particular set of GREP_OPTIONS in your environment, or
> may be designed to work well with GREP_OPTIONS.

Yes, but what about git commands that are implemented as shell scripts and use grep? Something like the following patch?

We'd need to run this from time to time to make sure no new grep calls creep in:

   git grep -L "unset GREP_OPTIONS" -- $(git grep -l "grep" git-*.sh)

-- 8< -- Unset GREP_OPTIONS at the top of git commands that are implemented as shell scripts and call grep, in order to avoid side effects caused by unexpected default options of users.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
---
 git-am.sh                  |    3 +++
 git-bisect.sh              |    3 +++
 git-filter-branch.sh       |    3 +++
 git-instaweb.sh            |    3 +++
 git-notes.sh               |    3 +++
 git-rebase--interactive.sh |    3 +++
 git-rebase.sh              |    3 +++
 git-submodule.sh           |    3 +++
 8 files changed, 24 insertions(+), 0 deletions(-)
Show changes to 8 files +24 −0

git-am.sh, git-bisect.sh, git-filter-branch.sh, git-instaweb.sh, git-notes.sh, git-rebase--interactive.sh, git-rebase.sh, git-submodule.sh

diff --git a/git-am.sh b/git-am.sh
index 151512a..1390eec 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -38,6 +38,9 @@ set_reflog_action am
 require_work_tree
 cd_to_toplevel
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 git var GIT_COMMITTER_IDENT >/dev/null ||
 	die "You need to set your committer info first"
 
diff --git a/git-bisect.sh b/git-bisect.sh
index a5ea843..fcf500f 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -30,6 +30,9 @@ OPTIONS_SPEC=
 . git-sh-setup
 require_work_tree
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
 _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
 
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 6b8b6a4..d3a8b3e 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -107,6 +107,9 @@ USAGE="[--env-filter <command>] [--tree-filter <command>]
 OPTIONS_SPEC=
 . git-sh-setup
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 if [ "$(is_bare_repository)" = false ]; then
 	git diff-files --ignore-submodules --quiet &&
 	git diff-index --cached --quiet HEAD -- ||
diff --git a/git-instaweb.sh b/git-instaweb.sh
index 622a5f0..86916e1 100755
--- a/git-instaweb.sh
+++ b/git-instaweb.sh
@@ -21,6 +21,9 @@ restart        restart the web server
 
 . git-sh-setup
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 fqgitdir="$GIT_DIR"
 local="$(git config --bool --get instaweb.local)"
 httpd="$(git config --get instaweb.httpd)"
diff --git a/git-notes.sh b/git-notes.sh
index e642e47..e5f0edf 100755
--- a/git-notes.sh
+++ b/git-notes.sh
@@ -3,6 +3,9 @@
 USAGE="(edit [-F <file> | -m <msg>] | show) [commit]"
 . git-sh-setup
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 test -z "$1" && usage
 ACTION="$1"; shift
 
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 27daaa9..d0bb8a3 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -33,6 +33,9 @@ root               rebase all reachable commmits up to the root(s)
 . git-sh-setup
 require_work_tree
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 DOTEST="$GIT_DIR/rebase-merge"
 TODO="$DOTEST"/git-rebase-todo
 DONE="$DOTEST"/done
diff --git a/git-rebase.sh b/git-rebase.sh
index 6830e16..18c680b 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -34,6 +34,9 @@ set_reflog_action rebase
 require_work_tree
 cd_to_toplevel
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 OK_TO_SKIP_PRE_REBASE=
 RESOLVEMSG="
 When you have resolved this problem run \"git rebase --continue\".
diff --git a/git-submodule.sh b/git-submodule.sh
index 850d423..e557aca 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -17,6 +17,9 @@ OPTIONS_SPEC=
 . git-parse-remote
 require_work_tree
 
+# Make sure we're in full control when calling grep in this script.
+unset GREP_OPTIONS
+
 command=
 branch=
 reference=
-- 
1.6.5
René Scharfe· Nov 23, 2009, 23:29 UTC · re: René Scharfe · lore

[PATCH] mergetool--lib: simplify guess_merge_tool()

Use a case statement instead of calling grep to find out if the editor's name contains the string "vim". Remove the check for emacs, as this branch did the same as the default one anyway.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
---
This removes all grep calls from this script.
 git-mergetool--lib.sh |   13 ++++++-------
 1 files changed, 6 insertions(+), 7 deletions(-)
Show changes to git-mergetool--lib.sh +6 −7
diff --git a/git-mergetool--lib.sh b/git-mergetool--lib.sh
index f7c571e..5b62785 100644
--- a/git-mergetool--lib.sh
+++ b/git-mergetool--lib.sh
@@ -338,15 +338,14 @@ guess_merge_tool () {
 		fi
 		tools="$tools gvimdiff diffuse ecmerge p4merge araxis"
 	fi
-	if echo "${VISUAL:-$EDITOR}" | grep emacs > /dev/null 2>&1; then
-		# $EDITOR is emacs so add emerge as a candidate
-		tools="$tools emerge vimdiff"
-	elif echo "${VISUAL:-$EDITOR}" | grep vim > /dev/null 2>&1; then
-		# $EDITOR is vim so add vimdiff as a candidate
+	case "${VISUAL:-$EDITOR}" in
+	*vim*)
 		tools="$tools vimdiff emerge"
-	else
+		;;
+	*)
 		tools="$tools emerge vimdiff"
-	fi
+		;;
+	esac
 	echo >&2 "merge tool candidates: $tools"
 
 	# Loop over each candidate and stop when a valid merge tool is found.
-- 
1.6.5
David Aguilar· Nov 27, 2009, 03:22 UTC · re: René Scharfe · lore

Re: [PATCH] mergetool--lib: simplify guess_merge_tool()

On Tue, Nov 24, 2009 at 12:29:17AM +0100, René Scharfe wrote:
Show 7 quoted lines
> Use a case statement instead of calling grep to find out if the editor's
> name contains the string "vim".  Remove the check for emacs, as this
> branch did the same as the default one anyway.
> 
> Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
> ---
> This removes all grep calls from this script.

Very nice. Thanks,

-- 
		David
Junio C Hamano· Nov 23, 2009, 23:52 UTC · re: René Scharfe · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
Show 7 quoted lines
> Yes, but what about git commands that are implemented as shell scripts
> and use grep?  Something like the following patch?
>
> We'd need to run this from time to time to make sure no new grep calls
> creep in:
>
>    git grep -L "unset GREP_OPTIONS" -- $(git grep -l "grep" git-*.sh)

Hmm, but "bisect run" runs user's script and it may want to see GREP_OPTIONS from the environment, no? Same for any of the hooks that am and rebase might want to run.

 git-sh-setup.sh            |   14 ++++++++++++++
 git-am.sh                  |    4 ++--
 git-bisect.sh              |    4 ++--
 git-filter-branch.sh       |    2 +-
 git-instaweb.sh            |    8 ++++----
 git-rebase--interactive.sh |   10 +++++-----
 git-rebase.sh              |    2 +-
 git-submodule.sh           |    6 +++---
 8 files changed, 32 insertions(+), 18 deletions(-)
Show changes to 8 files +32 −18

git-sh-setup.sh, git-am.sh, git-bisect.sh, git-filter-branch.sh, git-instaweb.sh, git-rebase--interactive.sh, git-rebase.sh, git-submodule.sh

diff --git a/git-sh-setup.sh b/git-sh-setup.sh
index c41c2f7..2b2afa6 100755
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -114,6 +114,20 @@ git_editor() {
 	eval "${GIT_EDITOR:=vi}" '"$@"'
 }
 
+sane_grep () {
+	GREP_OPTIONS= \
+	GREP_COLOR= \
+	GREP_COLORS= \
+	LC_ALL=C grep "$@"
+}
+
+sane_egrep () {
+	GREP_OPTIONS= \
+	GREP_COLOR= \
+	GREP_COLORS= \
+	LC_ALL=C egrep "$@"
+}
+
 is_bare_repository () {
 	git rev-parse --is-bare-repository
 }
diff --git a/git-am.sh b/git-am.sh
index c132f50..b49f26a 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -205,7 +205,7 @@ check_patch_format () {
 			# and see if it looks like that they all begin with the
 			# header field names...
 			sed -n -e '/^$/q' -e '/^[ 	]/d' -e p "$1" |
-			LC_ALL=C egrep -v '^[!-9;-~]+:' >/dev/null ||
+			sane_egrep -v '^[!-9;-~]+:' >/dev/null ||
 			patch_format=mbox
 		fi
 	} < "$1" || clean_abort
@@ -554,7 +554,7 @@ do
 			stop_here $this
 
 		# skip pine's internal folder data
-		grep '^Author: Mail System Internal Data$' \
+		sane_grep '^Author: Mail System Internal Data$' \
 			<"$dotest"/info >/dev/null &&
 			go_next && continue
 
diff --git a/git-bisect.sh b/git-bisect.sh
index 6f6f039..0c422d5 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -393,7 +393,7 @@ bisect_run () {
 
       cat "$GIT_DIR/BISECT_RUN"
 
-      if grep "first bad commit could be any of" "$GIT_DIR/BISECT_RUN" \
+      if sane_grep "first bad commit could be any of" "$GIT_DIR/BISECT_RUN" \
 		> /dev/null; then
 	  echo >&2 "bisect run cannot continue any more"
 	  exit $res
@@ -405,7 +405,7 @@ bisect_run () {
 	  exit $res
       fi
 
-      if grep "is the first bad commit" "$GIT_DIR/BISECT_RUN" > /dev/null; then
+      if sane_grep "is the first bad commit" "$GIT_DIR/BISECT_RUN" > /dev/null; then
 	  echo "bisect run success"
 	  exit 0;
       fi
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index a480d6f..8ef1bde 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -457,7 +457,7 @@ if [ "$filter_tag_name" ]; then
 				git mktag) ||
 				die "Could not create new tag object for $ref"
 			if git cat-file tag "$ref" | \
-			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			   sane_grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
 			then
 				warn "gpg signature stripped from tag object $sha1t"
 			fi
diff --git a/git-instaweb.sh b/git-instaweb.sh
index d96eddb..84805c6 100755
--- a/git-instaweb.sh
+++ b/git-instaweb.sh
@@ -41,7 +41,7 @@ resolve_full_httpd () {
 	case "$httpd" in
 	*apache2*|*lighttpd*)
 		# ensure that the apache2/lighttpd command ends with "-f"
-		if ! echo "$httpd" | grep -- '-f *$' >/dev/null 2>&1
+		if ! echo "$httpd" | sane_grep -- '-f *$' >/dev/null 2>&1
 		then
 			httpd="$httpd -f"
 		fi
@@ -297,8 +297,8 @@ EOF
 
 	# check to see if Dennis Stosberg's mod_perl compatibility patch
 	# (<20060621130708.Gcbc6e5c@leonov.stosberg.net>) has been applied
-	if test -f "$module_path/mod_perl.so" && grep 'MOD_PERL' \
-				"$GIT_DIR/gitweb/gitweb.cgi" >/dev/null
+	if test -f "$module_path/mod_perl.so" &&
+	   sane_grep 'MOD_PERL' "$GIT_DIR/gitweb/gitweb.cgi" >/dev/null
 	then
 		# favor mod_perl if available
 		cat >> "$conf" <<EOF
@@ -316,7 +316,7 @@ EOF
 		# plain-old CGI
 		resolve_full_httpd
 		list_mods=$(echo "$full_httpd" | sed "s/-f$/-l/")
-		$list_mods | grep 'mod_cgi\.c' >/dev/null 2>&1 || \
+		$list_mods | sane_grep 'mod_cgi\.c' >/dev/null 2>&1 || \
 		echo "LoadModule cgi_module $module_path/mod_cgi.so" >> "$conf"
 		cat >> "$conf" <<EOF
 AddHandler cgi-script .cgi
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 23ded48..6268e76 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -106,8 +106,8 @@ mark_action_done () {
 	sed -e 1q < "$TODO" >> "$DONE"
 	sed -e 1d < "$TODO" >> "$TODO".new
 	mv -f "$TODO".new "$TODO"
-	count=$(grep -c '^[^#]' < "$DONE")
-	total=$(($count+$(grep -c '^[^#]' < "$TODO")))
+	count=$(sane_grep -c '^[^#]' < "$DONE")
+	total=$(($count+$(sane_grep -c '^[^#]' < "$TODO")))
 	if test "$last_count" != "$count"
 	then
 		last_count=$count
@@ -147,7 +147,7 @@ die_abort () {
 }
 
 has_action () {
-	grep '^[^#]' "$1" >/dev/null
+	sane_grep '^[^#]' "$1" >/dev/null
 }
 
 pick_one () {
@@ -731,7 +731,7 @@ first and then run 'git rebase --continue' again."
 			git rev-list $REVISIONS |
 			while read rev
 			do
-				if test -f "$REWRITTEN"/$rev -a "$(grep "$rev" "$DOTEST"/not-cherry-picks)" = ""
+				if test -f "$REWRITTEN"/$rev -a "$(sane_grep "$rev" "$DOTEST"/not-cherry-picks)" = ""
 				then
 					# Use -f2 because if rev-list is telling us this commit is
 					# not worthwhile, we don't want to track its multiple heads,
@@ -739,7 +739,7 @@ first and then run 'git rebase --continue' again."
 					# be rebasing on top of it
 					git rev-list --parents -1 $rev | cut -d' ' -s -f2 > "$DROPPED"/$rev
 					short=$(git rev-list -1 --abbrev-commit --abbrev=7 $rev)
-					grep -v "^[a-z][a-z]* $short" <"$TODO" > "${TODO}2" ; mv "${TODO}2" "$TODO"
+					sane_grep -v "^[a-z][a-z]* $short" <"$TODO" > "${TODO}2" ; mv "${TODO}2" "$TODO"
 					rm "$REWRITTEN"/$rev
 				fi
 			done
diff --git a/git-rebase.sh b/git-rebase.sh
index 6ec155c..0ec4355 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -467,7 +467,7 @@ orig_head=$branch
 mb=$(git merge-base "$onto" "$branch")
 if test "$upstream" = "$onto" && test "$mb" = "$onto" &&
 	# linear history?
-	! (git rev-list --parents "$onto".."$branch" | grep " .* ") > /dev/null
+	! (git rev-list --parents "$onto".."$branch" | sane_grep " .* ") > /dev/null
 then
 	if test -z "$force_rebase"
 	then
diff --git a/git-submodule.sh b/git-submodule.sh
index 0462e52..b7ccd12 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -57,7 +57,7 @@ resolve_relative_url ()
 #
 module_list()
 {
-	git ls-files --error-unmatch --stage -- "$@" | grep '^160000 '
+	git ls-files --error-unmatch --stage -- "$@" | sane_grep '^160000 '
 }
 
 #
@@ -567,7 +567,7 @@ cmd_summary() {
 	cd_to_toplevel
 	# Get modified modules cared by user
 	modules=$(git $diff_cmd $cached --raw $head -- "$@" |
-		egrep '^:([0-7]* )?160000' |
+		sane_egrep '^:([0-7]* )?160000' |
 		while read mod_src mod_dst sha1_src sha1_dst status name
 		do
 			# Always show modules deleted or type-changed (blob<->module)
@@ -581,7 +581,7 @@ cmd_summary() {
 	test -z "$modules" && return
 
 	git $diff_cmd $cached --raw $head -- $modules |
-	egrep '^:([0-7]* )?160000' |
+	sane_egrep '^:([0-7]* )?160000' |
 	cut -c2- |
 	while read mod_src mod_dst sha1_src sha1_dst status name
 	do
René Scharfe· Nov 23, 2009, 23:59 UTC · re: Junio C Hamano · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

Junio C Hamano schrieb:
Show 48 quoted lines
> René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
> 
>> Yes, but what about git commands that are implemented as shell scripts
>> and use grep?  Something like the following patch?
>>
>> We'd need to run this from time to time to make sure no new grep calls
>> creep in:
>>
>>    git grep -L "unset GREP_OPTIONS" -- $(git grep -l "grep" git-*.sh)
> 
> Hmm, but "bisect run" runs user's script and it may want to see
> GREP_OPTIONS from the environment, no?  Same for any of the hooks that am
> and rebase might want to run.
> 
> 
> 
>  git-sh-setup.sh            |   14 ++++++++++++++
>  git-am.sh                  |    4 ++--
>  git-bisect.sh              |    4 ++--
>  git-filter-branch.sh       |    2 +-
>  git-instaweb.sh            |    8 ++++----
>  git-rebase--interactive.sh |   10 +++++-----
>  git-rebase.sh              |    2 +-
>  git-submodule.sh           |    6 +++---
>  8 files changed, 32 insertions(+), 18 deletions(-)
> 
> diff --git a/git-sh-setup.sh b/git-sh-setup.sh
> index c41c2f7..2b2afa6 100755
> --- a/git-sh-setup.sh
> +++ b/git-sh-setup.sh
> @@ -114,6 +114,20 @@ git_editor() {
>  	eval "${GIT_EDITOR:=vi}" '"$@"'
>  }
>  
> +sane_grep () {
> +	GREP_OPTIONS= \
> +	GREP_COLOR= \
> +	GREP_COLORS= \
> +	LC_ALL=C grep "$@"
> +}
> +
> +sane_egrep () {
> +	GREP_OPTIONS= \
> +	GREP_COLOR= \
> +	GREP_COLORS= \
> +	LC_ALL=C egrep "$@"
> +}
> +
Ah, yes, much nicer.
René
Junio C Hamano· Nov 24, 2009, 00:35 UTC · re: René Scharfe · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

René Scharfe <rene.scharfe@lsrfire.ath.cx> writes:
Show 24 quoted lines
>> diff --git a/git-sh-setup.sh b/git-sh-setup.sh
>> index c41c2f7..2b2afa6 100755
>> --- a/git-sh-setup.sh
>> +++ b/git-sh-setup.sh
>> @@ -114,6 +114,20 @@ git_editor() {
>>  	eval "${GIT_EDITOR:=vi}" '"$@"'
>>  }
>>  
>> +sane_grep () {
>> +	GREP_OPTIONS= \
>> +	GREP_COLOR= \
>> +	GREP_COLORS= \
>> +	LC_ALL=C grep "$@"
>> +}
>> +
>> +sane_egrep () {
>> +	GREP_OPTIONS= \
>> +	GREP_COLOR= \
>> +	GREP_COLORS= \
>> +	LC_ALL=C egrep "$@"
>> +}
>> +
>
> Ah, yes, much nicer.

Actually I am having a second thought after spending some time trying to come up with a commit log message. This leaves the door open for user scripts to honor the environment variables, but it also means that everybody needs to be aware of the insanity. It is really tempting to treat these exactly like CDPATH and unconditionally unset it.

Oh, and unsetting GREP_COLORS/GREP_COLOR was a mistake, I think. As long as we do not pass --color (and unset GREP_OPTIONS to make sure it is not given), their settings should not matter to us.

-- >8 --
Subject: [PATCH] Protect scripted Porcelains from GREP_OPTIONS insanity

If the user has exported the GREP_OPTIONS environment variable, the output from "grep" and "egrep" in scripted Porcelains may be different from what they expect. For example, we may want to count number of matching lines, by "grep" piped to "wc -l", and GREP_OPTIONS=-C3 will break such use.

The approach taken by this change to address this issue is to protect only our own use of grep/egrep. Because we do not unset it at the beginning of our scripts, hook scripts run from the scripted Porcelains are exposed to the same insanity this environment variable causes when grep/egrep is used to implement logic (e.g. "grep | wc -l"), and it is entirely up to the hook scripts to protect themselves.

On the other hand, applypatch-msg hook may want to show offending words in the proposed commit log message using grep to the end user, and the user might want to set GREP_OPTIONS=--color to paint the match more visibly. The approach to protect only our own use without unsetting the environment variable globally will allow this use case.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 git-sh-setup.sh            |    8 ++++++++
 git-am.sh                  |    4 ++--
 git-bisect.sh              |    4 ++--
 git-filter-branch.sh       |    2 +-
 git-instaweb.sh            |    8 ++++----
 git-rebase--interactive.sh |   10 +++++-----
 git-rebase.sh              |    2 +-
 git-submodule.sh           |    6 +++---
 8 files changed, 26 insertions(+), 18 deletions(-)
Show changes to 8 files +26 −18

git-sh-setup.sh, git-am.sh, git-bisect.sh, git-filter-branch.sh, git-instaweb.sh, git-rebase--interactive.sh, git-rebase.sh, git-submodule.sh

diff --git a/git-sh-setup.sh b/git-sh-setup.sh
index c41c2f7..aa07cc3 100755
--- a/git-sh-setup.sh
+++ b/git-sh-setup.sh
@@ -114,6 +114,14 @@ git_editor() {
 	eval "${GIT_EDITOR:=vi}" '"$@"'
 }
 
+sane_grep () {
+	GREP_OPTIONS= LC_ALL=C grep "$@"
+}
+
+sane_egrep () {
+	GREP_OPTIONS= LC_ALL=C egrep "$@"
+}
+
 is_bare_repository () {
 	git rev-parse --is-bare-repository
 }
diff --git a/git-am.sh b/git-am.sh
index c132f50..b49f26a 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -205,7 +205,7 @@ check_patch_format () {
 			# and see if it looks like that they all begin with the
 			# header field names...
 			sed -n -e '/^$/q' -e '/^[ 	]/d' -e p "$1" |
-			LC_ALL=C egrep -v '^[!-9;-~]+:' >/dev/null ||
+			sane_egrep -v '^[!-9;-~]+:' >/dev/null ||
 			patch_format=mbox
 		fi
 	} < "$1" || clean_abort
@@ -554,7 +554,7 @@ do
 			stop_here $this
 
 		# skip pine's internal folder data
-		grep '^Author: Mail System Internal Data$' \
+		sane_grep '^Author: Mail System Internal Data$' \
 			<"$dotest"/info >/dev/null &&
 			go_next && continue
 
diff --git a/git-bisect.sh b/git-bisect.sh
index 6f6f039..0c422d5 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -393,7 +393,7 @@ bisect_run () {
 
       cat "$GIT_DIR/BISECT_RUN"
 
-      if grep "first bad commit could be any of" "$GIT_DIR/BISECT_RUN" \
+      if sane_grep "first bad commit could be any of" "$GIT_DIR/BISECT_RUN" \
 		> /dev/null; then
 	  echo >&2 "bisect run cannot continue any more"
 	  exit $res
@@ -405,7 +405,7 @@ bisect_run () {
 	  exit $res
       fi
 
-      if grep "is the first bad commit" "$GIT_DIR/BISECT_RUN" > /dev/null; then
+      if sane_grep "is the first bad commit" "$GIT_DIR/BISECT_RUN" > /dev/null; then
 	  echo "bisect run success"
 	  exit 0;
       fi
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index a480d6f..8ef1bde 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -457,7 +457,7 @@ if [ "$filter_tag_name" ]; then
 				git mktag) ||
 				die "Could not create new tag object for $ref"
 			if git cat-file tag "$ref" | \
-			   grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
+			   sane_grep '^-----BEGIN PGP SIGNATURE-----' >/dev/null 2>&1
 			then
 				warn "gpg signature stripped from tag object $sha1t"
 			fi
diff --git a/git-instaweb.sh b/git-instaweb.sh
index d96eddb..84805c6 100755
--- a/git-instaweb.sh
+++ b/git-instaweb.sh
@@ -41,7 +41,7 @@ resolve_full_httpd () {
 	case "$httpd" in
 	*apache2*|*lighttpd*)
 		# ensure that the apache2/lighttpd command ends with "-f"
-		if ! echo "$httpd" | grep -- '-f *$' >/dev/null 2>&1
+		if ! echo "$httpd" | sane_grep -- '-f *$' >/dev/null 2>&1
 		then
 			httpd="$httpd -f"
 		fi
@@ -297,8 +297,8 @@ EOF
 
 	# check to see if Dennis Stosberg's mod_perl compatibility patch
 	# (<20060621130708.Gcbc6e5c@leonov.stosberg.net>) has been applied
-	if test -f "$module_path/mod_perl.so" && grep 'MOD_PERL' \
-				"$GIT_DIR/gitweb/gitweb.cgi" >/dev/null
+	if test -f "$module_path/mod_perl.so" &&
+	   sane_grep 'MOD_PERL' "$GIT_DIR/gitweb/gitweb.cgi" >/dev/null
 	then
 		# favor mod_perl if available
 		cat >> "$conf" <<EOF
@@ -316,7 +316,7 @@ EOF
 		# plain-old CGI
 		resolve_full_httpd
 		list_mods=$(echo "$full_httpd" | sed "s/-f$/-l/")
-		$list_mods | grep 'mod_cgi\.c' >/dev/null 2>&1 || \
+		$list_mods | sane_grep 'mod_cgi\.c' >/dev/null 2>&1 || \
 		echo "LoadModule cgi_module $module_path/mod_cgi.so" >> "$conf"
 		cat >> "$conf" <<EOF
 AddHandler cgi-script .cgi
diff --git a/git-rebase--interactive.sh b/git-rebase--interactive.sh
index 23ded48..6268e76 100755
--- a/git-rebase--interactive.sh
+++ b/git-rebase--interactive.sh
@@ -106,8 +106,8 @@ mark_action_done () {
 	sed -e 1q < "$TODO" >> "$DONE"
 	sed -e 1d < "$TODO" >> "$TODO".new
 	mv -f "$TODO".new "$TODO"
-	count=$(grep -c '^[^#]' < "$DONE")
-	total=$(($count+$(grep -c '^[^#]' < "$TODO")))
+	count=$(sane_grep -c '^[^#]' < "$DONE")
+	total=$(($count+$(sane_grep -c '^[^#]' < "$TODO")))
 	if test "$last_count" != "$count"
 	then
 		last_count=$count
@@ -147,7 +147,7 @@ die_abort () {
 }
 
 has_action () {
-	grep '^[^#]' "$1" >/dev/null
+	sane_grep '^[^#]' "$1" >/dev/null
 }
 
 pick_one () {
@@ -731,7 +731,7 @@ first and then run 'git rebase --continue' again."
 			git rev-list $REVISIONS |
 			while read rev
 			do
-				if test -f "$REWRITTEN"/$rev -a "$(grep "$rev" "$DOTEST"/not-cherry-picks)" = ""
+				if test -f "$REWRITTEN"/$rev -a "$(sane_grep "$rev" "$DOTEST"/not-cherry-picks)" = ""
 				then
 					# Use -f2 because if rev-list is telling us this commit is
 					# not worthwhile, we don't want to track its multiple heads,
@@ -739,7 +739,7 @@ first and then run 'git rebase --continue' again."
 					# be rebasing on top of it
 					git rev-list --parents -1 $rev | cut -d' ' -s -f2 > "$DROPPED"/$rev
 					short=$(git rev-list -1 --abbrev-commit --abbrev=7 $rev)
-					grep -v "^[a-z][a-z]* $short" <"$TODO" > "${TODO}2" ; mv "${TODO}2" "$TODO"
+					sane_grep -v "^[a-z][a-z]* $short" <"$TODO" > "${TODO}2" ; mv "${TODO}2" "$TODO"
 					rm "$REWRITTEN"/$rev
 				fi
 			done
diff --git a/git-rebase.sh b/git-rebase.sh
index 6ec155c..0ec4355 100755
--- a/git-rebase.sh
+++ b/git-rebase.sh
@@ -467,7 +467,7 @@ orig_head=$branch
 mb=$(git merge-base "$onto" "$branch")
 if test "$upstream" = "$onto" && test "$mb" = "$onto" &&
 	# linear history?
-	! (git rev-list --parents "$onto".."$branch" | grep " .* ") > /dev/null
+	! (git rev-list --parents "$onto".."$branch" | sane_grep " .* ") > /dev/null
 then
 	if test -z "$force_rebase"
 	then
diff --git a/git-submodule.sh b/git-submodule.sh
index 0462e52..b7ccd12 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -57,7 +57,7 @@ resolve_relative_url ()
 #
 module_list()
 {
-	git ls-files --error-unmatch --stage -- "$@" | grep '^160000 '
+	git ls-files --error-unmatch --stage -- "$@" | sane_grep '^160000 '
 }
 
 #
@@ -567,7 +567,7 @@ cmd_summary() {
 	cd_to_toplevel
 	# Get modified modules cared by user
 	modules=$(git $diff_cmd $cached --raw $head -- "$@" |
-		egrep '^:([0-7]* )?160000' |
+		sane_egrep '^:([0-7]* )?160000' |
 		while read mod_src mod_dst sha1_src sha1_dst status name
 		do
 			# Always show modules deleted or type-changed (blob<->module)
@@ -581,7 +581,7 @@ cmd_summary() {
 	test -z "$modules" && return
 
 	git $diff_cmd $cached --raw $head -- $modules |
-	egrep '^:([0-7]* )?160000' |
+	sane_egrep '^:([0-7]* )?160000' |
 	cut -c2- |
 	while read mod_src mod_dst sha1_src sha1_dst status name
 	do
-- 
1.6.6.rc0.15.g4fa80.dirty
Bert Wesarg· Mar 7, 2010, 10:30 UTC · re: Junio C Hamano · lore

Re: [PATCH] unset GREP_OPTIONS in test-lib.sh

On Wed, Nov 18, 2009 at 23:05, Junio C Hamano <gitster@pobox.com> wrote:
Show 6 quoted lines
> Bert Wesarg <bert.wesarg@googlemail.com> writes:
>
>> I used to set GREP_OPTIONS to exclude *.orig and *.rej files. But with this
>> the test t4252-am-options.sh fails because it calls grep with a .rej file:
>
> Yuck.  Will apply.
Ping.

Regards, Bert

← back to recent threads