on broken command chains in tests (was: Re: [PATCH] status: show commit sha1 in "You are currently)

From: SZEDER Gábor <szeder@ira.uka.de>
Date: Oct 15, 2013, 13:35 UTC
Message-ID: <20131015133516.GC26156@goldbirke>
In-Reply-To: <20131011174210.GS9464@google.com>

Hi,

On Fri, Oct 11, 2013 at 10:42:10AM -0700, Jonathan Nieder wrote:

Show 6 quoted lines

> -- >8 --
> Subject: status test: add missing && to <<EOF blocks
> 
> When a test forgets to include && after each command, it is possible
> for an early command to succeed but the test to fail, which can hide
> bugs.

Surely you meant "succeed" and "fail" the other way around :)

Show 22 quoted lines

> Checked using the following patch to the test harness:
> 
> 	--- a/t/test-lib.sh
> 	+++ b/t/test-lib.sh
> 	@@ -425,7 +425,17 @@ test_eval_ () {
> 		eval </dev/null >&3 2>&4 "$*"
> 	 }
> 
> 	+check_command_chaining_ () {
> 	+	eval >&3 2>&4 "(exit 189) && $*"
> 	+	eval_chain_ret=$?
> 	+	if test "$eval_chain_ret" != 189
> 	+	then
> 	+		error 'bug in test script: missing "&&" in test commands'
> 	+	fi
> 	+}
> 	+
> 	 test_run_ () {
> 	+	check_command_chaining_ "$1"
> 		test_cleanup=:
> 		expecting_failure=$2
> 		setup_malloc_check

Clever.

If I do a

  -               error 'bug in test script: missing "&&" in test commands'
  +               say_color error 'error: bug in test script: missing "&&" in test commands'

to avoid erroring out and skipping the rest of the test script on the first broken command chain, then we can see that we have a lot of broken command chains in the test suite:

  $ for t in t[0-9][0-9][0-9][0-9]*.sh ; do ./$t ; done |grep -c '^error:.*missing "&&" in test commands$'
  345

After a cursory look most of them seem to be the simple "missing &&" type, but there are some funny ones, too.

Gábor