threads / discuss / 13155

[BUGS] test failure in t4014-format-patch.sh with master

Subject: [BUGS] test failure in t4014-format-patch.sh with master

## tl;dr

12 messages between Apr 17, 2008 and Apr 19, 2008.

replies: 11people: 5as markdown or json

Remi Vanicat· Apr 17, 2008, 09:30 UTC · lore
Hello,

While trying to wrote a test for git, I discover that there is a test failure in master on my computer:

$ ./t4014-format-patch.sh
[....]
* FAIL 8: replay did not screw up the log message
 git cat-file commit rebuild-1 | grep "^Side .* with .* backslash-n"
It seem that now, somewhere a \n has been interpreted as a newline.

Bisecting point the problem to the 2e6e3e82 commit "git-am: cope better with an empty Subject: line"

-- 
Rémi Vanicat
Linus Torvalds· Apr 17, 2008, 15:11 UTC · re: Remi Vanicat · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On Thu, 17 Apr 2008, Remi Vanicat wrote:
> 
> While trying to wrote a test for git, I discover that there is a test
> failure in master on my computer:
Hmm. Doesn't fail for me.
Show 7 quoted lines
> $ ./t4014-format-patch.sh
> [....]
> * FAIL 8: replay did not screw up the log message
> 
>  git cat-file commit rebuild-1 | grep "^Side .* with .* backslash-n"
> 
> It seem that now, somewhere a \n has been interpreted as a newline.

I wonder if it is some shell-specific variable expansion issue. What's your /bin/sh? (I assume, from your email address, that you're using Debian, and I thought Debian used bash, but maybe that's not true).

What does
	a="hello\nhi"
	echo $a
result in?
		Linus
Björn Steinbrink· Apr 17, 2008, 15:34 UTC · re: Linus Torvalds · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On 2008.04.17 08:11:52 -0700, Linus Torvalds wrote:
Show 27 quoted lines
> 
> 
> On Thu, 17 Apr 2008, Remi Vanicat wrote:
> > 
> > While trying to wrote a test for git, I discover that there is a test
> > failure in master on my computer:
> 
> Hmm. Doesn't fail for me.
> 
> > $ ./t4014-format-patch.sh
> > [....]
> > * FAIL 8: replay did not screw up the log message
> > 
> >  git cat-file commit rebuild-1 | grep "^Side .* with .* backslash-n"
> > 
> > It seem that now, somewhere a \n has been interpreted as a newline.
> 
> I wonder if it is some shell-specific variable expansion issue. What's 
> your /bin/sh? (I assume, from your email address, that you're using 
> Debian, and I thought Debian used bash, but maybe that's not true).
> 
> What does
> 
> 	a="hello\nhi"
> 	echo $a
> 
> result in?

Debian has its own dash which is supposed to be a lightweight alternative to bash and "for checking POSIX compliance of scripts" (quote from the packages description). I don't happen to know off-hand whether POSIX says that echo should default to -e, but dash seems to do that:

$ dash $ a="hello\nhi" $ echo $a hello hi

Björn
Linus Torvalds· Apr 17, 2008, 15:44 UTC · re: Björn Steinbrink · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On Thu, 17 Apr 2008, Björn Steinbrink wrote:
Show 12 quoted lines
> 
> Debian has its own dash which is supposed to be a lightweight
> alternative to bash and "for checking POSIX compliance of scripts"
> (quote from the packages description). I don't happen to know off-hand
> whether POSIX says that echo should default to -e, but dash seems to do
> that:
> 
> $ dash
> $ a="hello\nhi"
> $ echo $a
> hello
> hi

Ahh. So that "echo" should just be replaced with a 'printf "%s\n"' instead?

We have a _lot_ of "echo"s though. I suspect the only ones we'd ever catch are the ones explicitly tested for. I suspect that the dash echo is just broken.

		Linus
Björn Steinbrink· Apr 17, 2008, 16:03 UTC · re: Linus Torvalds · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On 2008.04.17 08:44:07 -0700, Linus Torvalds wrote:
Show 22 quoted lines
> 
> 
> On Thu, 17 Apr 2008, Björn Steinbrink wrote:
> > 
> > Debian has its own dash which is supposed to be a lightweight
> > alternative to bash and "for checking POSIX compliance of scripts"
> > (quote from the packages description). I don't happen to know off-hand
> > whether POSIX says that echo should default to -e, but dash seems to do
> > that:
> > 
> > $ dash
> > $ a="hello\nhi"
> > $ echo $a
> > hello
> > hi
> 
> Ahh. So that "echo" should just be replaced with a 'printf "%s\n"' 
> instead? 
> 
> We have a _lot_ of "echo"s though. I suspect the only ones we'd ever catch 
> are the ones explicitly tested for. I suspect that the dash echo is just 
> broken.

Wow, seems that "echo" in itself is "broken". According to POSIX[1] backslashes induce implementation defined behaviour, and there seem to be two historic versions of echo, one that knows about -n and one that interprets escape sequences. So echo is totally non-portable unless you avoid -n as well as backslashes. Ouch.

The POSIX docs, as well as random Google results, indeed recommend to use printf instead. But that switch seems painful as well...

Björn
[1] http://www.opengroup.org/onlinepubs/009695399/utilities/echo.html
Jeff King· Apr 17, 2008, 16:24 UTC · re: Linus Torvalds · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On Thu, Apr 17, 2008 at 08:44:07AM -0700, Linus Torvalds wrote:
Show 6 quoted lines
> Ahh. So that "echo" should just be replaced with a 'printf "%s\n"' 
> instead? 
> 
> We have a _lot_ of "echo"s though. I suspect the only ones we'd ever catch 
> are the ones explicitly tested for. I suspect that the dash echo is just 
> broken.

As Bjorn mentioned, it's POSIX. Junio and I already hunted down quite a few of these, so I think we are OK in general, and this is just a new one that got introduced.

-Peff
Jeff King· Apr 17, 2008, 16:27 UTC · re: Jeff King · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On Thu, Apr 17, 2008 at 12:24:22PM -0400, Jeff King wrote:
> As Bjorn mentioned, it's POSIX.  Junio and I already hunted down quite a
> few of these, so I think we are OK in general, and this is just a new
> one that got introduced.
Oops, I meant to mention this thread:
  http://article.gmane.org/gmane.comp.version-control.git/48419
-Peff
Rémi Vanicat· Apr 17, 2008, 15:44 UTC · re: Linus Torvalds · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

2008/4/17, Linus Torvalds <torvalds@linux-foundation.org>:
>
>
>  On Thu, 17 Apr 2008, Remi Vanicat wrote:
[...]
Show 5 quoted lines
>  > $ ./t4014-format-patch.sh
>  > [....]
>  > * FAIL 8: replay did not screw up the log message
>
>  Hmm. Doesn't fail for me.
[...]
>  I wonder if it is some shell-specific variable expansion issue. What's
>  your /bin/sh? (I assume, from your email address, that you're using
>  Debian, and I thought Debian used bash, but maybe that's not true).
Debian used bash, but I use dash as /bin/sh
Show 6 quoted lines
>  What does
>
>         a="hello\nhi"
>         echo $a
>
>  result in?
here is the problem:

$/bin/sh $ a="hello\nhi" $ echo $a hello hi

Mmm, I Wonder what posix said about this, but it look like a bug with dash
Junio C Hamano· Apr 18, 2008, 18:30 UTC · re: Rémi Vanicat · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

"Rémi Vanicat" <vanicat@debian.org> writes:
Show 24 quoted lines
> 2008/4/17, Linus Torvalds <torvalds@linux-foundation.org>:
>>
>>  On Thu, 17 Apr 2008, Remi Vanicat wrote:
> [...]
>>  > $ ./t4014-format-patch.sh
>>  > [....]
>>  > * FAIL 8: replay did not screw up the log message
>>
>>  Hmm. Doesn't fail for me.
>
> [...]
>
>>  I wonder if it is some shell-specific variable expansion issue. What's
>>  your /bin/sh? (I assume, from your email address, that you're using
>>  Debian, and I thought Debian used bash, but maybe that's not true).
>
> Debian used bash, but I use dash as /bin/sh
>
>>  What does
>>
>>         a="hello\nhi"
>>         echo $a
>>
>>  result in?
So perhaps this one is the troublemaker?
 git-am.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/git-am.sh b/git-am.sh
index 646dc45..5b20bbd 100755
--- a/git-am.sh
+++ b/git-am.sh
@@ -330,7 +330,7 @@ do
 		SUBJECT="$(sed -n '/^Subject/ s/Subject: //p' "$dotest/info")"
 		case "$keep_subject" in -k)  SUBJECT="[PATCH] $SUBJECT" ;; esac
 
-		(echo "$SUBJECT" ; echo ; cat "$dotest/msg") |
+		(printf '%s\n' "$SUBJECT" ; echo ; cat "$dotest/msg") |
 			git stripspace > "$dotest/msg-clean"
 		;;
 	esac
Linus Torvalds· Apr 18, 2008, 18:37 UTC · re: Junio C Hamano · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

On Fri, 18 Apr 2008, Junio C Hamano wrote:
>
> -		(echo "$SUBJECT" ; echo ; cat "$dotest/msg") |
> +		(printf '%s\n' "$SUBJECT" ; echo ; cat "$dotest/msg") |

Well, get rid of the superfluous second 'echo' then too, replacing it with another '\n'. IOW:

	( printf '%s\n\n' "$SUBJECT" ; cat "$dotest/msg" ) | 
instead.
		Linus
Rémi Vanicat· Apr 18, 2008, 22:13 UTC · re: Linus Torvalds · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

2008/4/18, Linus Torvalds <torvalds@linux-foundation.org>:
Show 14 quoted lines
>
>
>  On Fri, 18 Apr 2008, Junio C Hamano wrote:
>  >
>  > -             (echo "$SUBJECT" ; echo ; cat "$dotest/msg") |
>  > +             (printf '%s\n' "$SUBJECT" ; echo ; cat "$dotest/msg") |
>
>
> Well, get rid of the superfluous second 'echo' then too, replacing it
>  with another '\n'. IOW:
>
>         ( printf '%s\n\n' "$SUBJECT" ; cat "$dotest/msg" ) |
>
>  instead.
This work for me: no more test failure on my computer
Junio C Hamano· Apr 19, 2008, 06:51 UTC · re: Rémi Vanicat · lore

Re: [BUGS] test failure in t4014-format-patch.sh with master

"Rémi Vanicat" <vanicat@debian.org> writes:
Show 17 quoted lines
> 2008/4/18, Linus Torvalds <torvalds@linux-foundation.org>:
>>
>>
>>  On Fri, 18 Apr 2008, Junio C Hamano wrote:
>>  >
>>  > -             (echo "$SUBJECT" ; echo ; cat "$dotest/msg") |
>>  > +             (printf '%s\n' "$SUBJECT" ; echo ; cat "$dotest/msg") |
>>
>>
>> Well, get rid of the superfluous second 'echo' then too, replacing it
>>  with another '\n'. IOW:
>>
>>         ( printf '%s\n\n' "$SUBJECT" ; cat "$dotest/msg" ) |
>>
>>  instead.
>
> This work for me: no more test failure on my computer

Thanks for confirmation. I reproduced the breakage and the fix myself with "make SHELL_PATH=/bin/dash" myself, too.

← back to recent threads