# [PATCH] test-lib.sh: use printf instead of echo 14 messages from 2014-03-14 to 2014-03-20. Participants: Uwe Storbeck, Jonathan Nieder, Johannes Sixt, Junio C Hamano, David Kastrup. Thread: https://gitlist.dev/t/36163 ## Uwe Storbeck, 2014-03-14 23:57 Subject: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <20140314235735.GA6959@ibr.ch> URL: https://gitlist.dev/e/20140314235735.GA6959%40ibr.ch ``` when variables may contain backslash sequences. Backslash sequences are interpreted as control characters by the echo command of some shells (e.g. dash). Signed-off-by: Uwe Storbeck --- t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 1531c24..8209204 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -277,7 +277,7 @@ error "Test script did not set test_description." if test "$help" = "t" then - echo "$test_description" + printf '%s\n' "$test_description" exit 0 fi @@ -328,7 +328,7 @@ test_failure_ () { test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift - echo "$@" | sed -e 's/^/# /' + printf '%s\n' "$@" | sed -e 's/^/# /' test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } } -- 1.9.0 ``` ## Jonathan Nieder, 2014-03-15 00:18 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <20140315001855.GK15625@google.com> URL: https://gitlist.dev/e/20140315001855.GK15625%40google.com In-Reply-To: <20140314235735.GA6959@ibr.ch> ``` Uwe Storbeck wrote: > Backslash sequences are interpreted as control characters > by the echo command of some shells (e.g. dash). This has bothered me for a while but never enough to do anything about it. Thanks for fixing it. > Signed-off-by: Uwe Storbeck Reviewed-by: Jonathan Nieder (patch left unsnipped for reference) > --- > t/test-lib.sh | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/t/test-lib.sh b/t/test-lib.sh > index 1531c24..8209204 100644 > --- a/t/test-lib.sh > +++ b/t/test-lib.sh > @@ -277,7 +277,7 @@ error "Test script did not set test_description." > > if test "$help" = "t" > then > - echo "$test_description" > + printf '%s\n' "$test_description" > exit 0 > fi > > @@ -328,7 +328,7 @@ test_failure_ () { > test_failure=$(($test_failure + 1)) > say_color error "not ok $test_count - $1" > shift > - echo "$@" | sed -e 's/^/# /' > + printf '%s\n' "$@" | sed -e 's/^/# /' > test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } > } > ``` ## Johannes Sixt, 2014-03-15 08:59 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <53241688.2050604@kdbg.org> URL: https://gitlist.dev/e/53241688.2050604%40kdbg.org In-Reply-To: <20140314235735.GA6959@ibr.ch> ``` Am 15.03.2014 00:57, schrieb Uwe Storbeck: > when variables may contain backslash sequences. > > Backslash sequences are interpreted as control characters > by the echo command of some shells (e.g. dash). > > Signed-off-by: Uwe Storbeck > --- > t/test-lib.sh | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/t/test-lib.sh b/t/test-lib.sh > index 1531c24..8209204 100644 > --- a/t/test-lib.sh > +++ b/t/test-lib.sh > @@ -277,7 +277,7 @@ error "Test script did not set test_description." > > if test "$help" = "t" > then > - echo "$test_description" > + printf '%s\n' "$test_description" > exit 0 > fi > > @@ -328,7 +328,7 @@ test_failure_ () { > test_failure=$(($test_failure + 1)) > say_color error "not ok $test_count - $1" > shift > - echo "$@" | sed -e 's/^/# /' > + printf '%s\n' "$@" | sed -e 's/^/# /' This should be printf '%s\n' "$*" | sed -e 's/^/# /' > test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } > } > > ``` ## Uwe Storbeck, 2014-03-17 18:38 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <20140317183807.GB1971@ibr.ch> URL: https://gitlist.dev/e/20140317183807.GB1971%40ibr.ch In-Reply-To: <53241688.2050604@kdbg.org> ``` On Mar 15, Johannes Sixt wrote: > > - echo "$@" | sed -e 's/^/# /' > > + printf '%s\n' "$@" | sed -e 's/^/# /' > > This should be > > printf '%s\n' "$*" | sed -e 's/^/# /' Right, that should be $* to always be one argument for the format pattern. Thanks Uwe ``` ## Uwe Storbeck, 2014-03-17 18:39 Subject: [PATCH v2] test-lib.sh: use printf instead of echo Message-ID: <20140317183917.GA28675@ibr.ch> URL: https://gitlist.dev/e/20140317183917.GA28675%40ibr.ch In-Reply-To: <20140314235735.GA6959@ibr.ch> ``` when variables may contain backslash sequences. Backslash sequences are interpreted as control characters by the echo command of some shells (e.g. dash). Signed-off-by: Uwe Storbeck --- Changed $@ to $* in printf. Thanks to Johannes Sixt to point that out. t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 1531c24..3c7cb1d 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -277,7 +277,7 @@ error "Test script did not set test_description." if test "$help" = "t" then - echo "$test_description" + printf '%s\n' "$test_description" exit 0 fi @@ -328,7 +328,7 @@ test_failure_ () { test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift - echo "$@" | sed -e 's/^/# /' + printf '%s\n' "$*" | sed -e 's/^/# /' test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } } -- 1.9.0 ``` ## Junio C Hamano, 2014-03-17 19:26 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: URL: https://gitlist.dev/e/xmqq61nceidp.fsf%40gitster.dls.corp.google.com In-Reply-To: <20140315001855.GK15625@google.com> ``` Jonathan Nieder writes: > Uwe Storbeck wrote: > >> Backslash sequences are interpreted as control characters >> by the echo command of some shells (e.g. dash). > > This has bothered me for a while but never enough to do anything about > it. Thanks for fixing it. > >> Signed-off-by: Uwe Storbeck > > Reviewed-by: Jonathan Nieder > > (patch left unsnipped for reference) >> --- >> t/test-lib.sh | 4 ++-- >> 1 file changed, 2 insertions(+), 2 deletions(-) >> >> diff --git a/t/test-lib.sh b/t/test-lib.sh >> index 1531c24..8209204 100644 >> --- a/t/test-lib.sh >> +++ b/t/test-lib.sh >> @@ -277,7 +277,7 @@ error "Test script did not set test_description." >> >> if test "$help" = "t" >> then >> - echo "$test_description" >> + printf '%s\n' "$test_description" >> exit 0 >> fi >> >> @@ -328,7 +328,7 @@ test_failure_ () { >> test_failure=$(($test_failure + 1)) >> say_color error "not ok $test_count - $1" >> shift >> - echo "$@" | sed -e 's/^/# /' >> + printf '%s\n' "$@" | sed -e 's/^/# /' This is wrong, isn't it? Why do we want one line per item here? >> test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } >> } >> ``` ## Uwe Storbeck, 2014-03-18 00:14 Subject: [PATCH v3] test-lib.sh: do not "echo" externally supplied strings Message-ID: <20140318001410.GA4813@ibr.ch> URL: https://gitlist.dev/e/20140318001410.GA4813%40ibr.ch In-Reply-To: <20140314235735.GA6959@ibr.ch> ``` In some places we "echo" a string that is supplied by the calling test script and may contain backslash sequences. The echo command of some shells, most notably "dash", interprets these backslash sequences (POSIX.1 allows this) which may scramble the test output. Signed-off-by: Uwe Storbeck --- Commit message rewritten to avoid title continuation in the body. Thanks Junio C Hamano for the hint. t/test-lib.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/t/test-lib.sh b/t/test-lib.sh index 1531c24..3c7cb1d 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -277,7 +277,7 @@ error "Test script did not set test_description." if test "$help" = "t" then - echo "$test_description" + printf '%s\n' "$test_description" exit 0 fi @@ -328,7 +328,7 @@ test_failure_ () { test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift - echo "$@" | sed -e 's/^/# /' + printf '%s\n' "$*" | sed -e 's/^/# /' test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } } -- 1.9.0 ``` ## Junio C Hamano, 2014-03-18 18:47 Subject: Re: [PATCH v3] test-lib.sh: do not "echo" externally supplied strings Message-ID: URL: https://gitlist.dev/e/xmqqvbvb8ht7.fsf%40gitster.dls.corp.google.com In-Reply-To: <20140318001410.GA4813@ibr.ch> ``` Uwe Storbeck writes: > In some places we "echo" a string that is supplied by the calling > test script and may contain backslash sequences. The echo command > of some shells, most notably "dash", interprets these backslash > sequences (POSIX.1 allows this) which may scramble the test > output. > > Signed-off-by: Uwe Storbeck > --- > > Commit message rewritten to avoid title continuation in the body. > Thanks Junio C Hamano for the hint. Here is what I queued yesterday. I was wrong to say "control characters"; a backslash sequence is not necessarily a control character (e.g. \c at the end that suppresses the final LF), so I'll replace it with your version. The test titles are not externally supplied but under our control, so it is less problematic than the "rebase -i" case where we do get bitten by user supplied commit title string. Still it is a good idea to apply this change to protect ourselves. Thanks. commit 215cd588caebe86fe77115bdda97930b4659aecd Author: Uwe Storbeck Date: Sat Mar 15 00:57:36 2014 +0100 test-lib.sh: do not "echo" test titles The test title could be a string with backslash in it, and can be interpreted as control characters by the echo command of some shells (e.g. dash). Signed-off-by: Uwe Storbeck Signed-off-by: Junio C Hamano diff --git a/t/test-lib.sh b/t/test-lib.sh index 1531c24..3c7cb1d 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -277,7 +277,7 @@ error "Test script did not set test_description." if test "$help" = "t" then - echo "$test_description" + printf '%s\n' "$test_description" exit 0 fi @@ -328,7 +328,7 @@ test_failure_ () { test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift - echo "$@" | sed -e 's/^/# /' + printf '%s\n' "$*" | sed -e 's/^/# /' test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; } } ``` ## Jonathan Nieder, 2014-03-18 22:18 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <20140318221844.GA828@google.com> URL: https://gitlist.dev/e/20140318221844.GA828%40google.com In-Reply-To: ``` Junio C Hamano wrote: >> Uwe Storbeck wrote: >>> + printf '%s\n' "$@" | sed -e 's/^/# /' > > This is wrong, isn't it? Why do we want one line per item here? Yes, Hannes caught the same, too. Sorry for the sloppiness. We currently use "echo" all over the place (e.g., 'echo "$path"' in git-sh-setup), and every time we fix it there is a chance of making mistakes. I wonder if it would make sense to add a helper to make the echo calls easier to replace: -- >8 -- Subject: git-sh-setup: introduce sane_echo helper Using 'echo' with arguments that might contain backslashes or "-e" or "-n" can produce confusing results that vary from platform to platform (e.g., dash always interprets \ escape sequences and echoes "-e" verbatim, whereas bash does not interpret \ escapes unless "-e" is passed as an argument to echo and suppresses the "-e" from its output). Instead, we should use printf, which is more predictable: printf '%s\n' "$foo"; # Just prints $foo, on all platforms. Blindly replacing echo with "printf '%s\n'" would not be good enough because that printf prints each argument on its own line. Provide a sane_echo helper that prints its arguments, space-delimited, on a single line, to make this easier to remember, and tweak 'say' and 'die_with_status' to illustrate how it is used. No functional change intended. Signed-off-by: Jonathan Nieder --- git-sh-setup.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git i/git-sh-setup.sh w/git-sh-setup.sh index 256c89a..f35b5b9 100644 --- i/git-sh-setup.sh +++ w/git-sh-setup.sh @@ -43,6 +43,10 @@ git_broken_path_fix () { # @@BROKEN_PATH_FIX@@ +sane_echo () { + printf '%s\n' "$*" +} + die () { die_with_status 1 "$@" } @@ -50,7 +54,7 @@ die () { die_with_status () { status=$1 shift - printf >&2 '%s\n' "$*" + sane_echo >&2 "$*" exit "$status" } @@ -59,7 +63,7 @@ GIT_QUIET= say () { if test -z "$GIT_QUIET" then - printf '%s\n' "$*" + sane_echo "$*" fi } ``` ## Junio C Hamano, 2014-03-19 17:17 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: URL: https://gitlist.dev/e/xmqqfvme5cql.fsf%40gitster.dls.corp.google.com In-Reply-To: <20140318221844.GA828@google.com> ``` Jonathan Nieder writes: > Junio C Hamano wrote: >>> Uwe Storbeck wrote: > >>>> + printf '%s\n' "$@" | sed -e 's/^/# /' >> >> This is wrong, isn't it? Why do we want one line per item here? > > Yes, Hannes caught the same, too. Sorry for the sloppiness. > > We currently use "echo" all over the place (e.g., 'echo "$path"' in > git-sh-setup), and every time we fix it there is a chance of making > mistakes. I wonder if it would make sense to add a helper to make the > echo calls easier to replace: I agree that we would benefit from having a helper to print a single line, which we very often do, without having to worry about the boilerplate '%s\n' of printf or the portability gotcha of echo. I am a bit reluctant to name the helper "sane_echo" to declare "echo that interprets backslashes in the string is insane", though. For these "print a single line" uses, we are only interested in using a subset of the features offered by 'echo', but that does not mean the other features we do not want to trigger in our use is of no use to any sane person. It very different from "sane_unset" that works around "unset" on an unset variable that can trigger an error when nobody sane is interested in that error condition. If somebody is interested if a variable is not yet set and behave differently, there are more direct ways to see the "set-ness" of a variable, and asking "unset" for that information is insane, hence I think the name "sane_unset" is justified. I do not feel the same way for "sane_echo". I would have called it "say" if the name weren't taken. > -- >8 -- > Subject: git-sh-setup: introduce sane_echo helper > > Using 'echo' with arguments that might contain backslashes or "-e" or > "-n" can produce confusing results that vary from platform to platform > (e.g., dash always interprets \ escape sequences and echoes "-e" > verbatim, whereas bash does not interpret \ escapes unless "-e" is > passed as an argument to echo and suppresses the "-e" from its > output). > > Instead, we should use printf, which is more predictable: > > printf '%s\n' "$foo"; # Just prints $foo, on all platforms. > > Blindly replacing echo with "printf '%s\n'" would not be good enough > because that printf prints each argument on its own line. Provide a > sane_echo helper that prints its arguments, space-delimited, on a > single line, to make this easier to remember, and tweak 'say' > and 'die_with_status' to illustrate how it is used. > > No functional change intended. > > Signed-off-by: Jonathan Nieder > --- > git-sh-setup.sh | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git i/git-sh-setup.sh w/git-sh-setup.sh > index 256c89a..f35b5b9 100644 > --- i/git-sh-setup.sh > +++ w/git-sh-setup.sh > @@ -43,6 +43,10 @@ git_broken_path_fix () { > > # @@BROKEN_PATH_FIX@@ > > +sane_echo () { > + printf '%s\n' "$*" > +} > + > die () { > die_with_status 1 "$@" > } > @@ -50,7 +54,7 @@ die () { > die_with_status () { > status=$1 > shift > - printf >&2 '%s\n' "$*" > + sane_echo >&2 "$*" > exit "$status" > } > > @@ -59,7 +63,7 @@ GIT_QUIET= > say () { > if test -z "$GIT_QUIET" > then > - printf '%s\n' "$*" > + sane_echo "$*" > fi > } > ``` ## David Kastrup, 2014-03-19 17:22 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <871txygl27.fsf@fencepost.gnu.org> URL: https://gitlist.dev/e/871txygl27.fsf%40fencepost.gnu.org In-Reply-To: ``` Junio C Hamano writes: > Jonathan Nieder writes: > >> Junio C Hamano wrote: >>>> Uwe Storbeck wrote: >> >>>>> + printf '%s\n' "$@" | sed -e 's/^/# /' >>> >>> This is wrong, isn't it? Why do we want one line per item here? >> >> Yes, Hannes caught the same, too. Sorry for the sloppiness. >> >> We currently use "echo" all over the place (e.g., 'echo "$path"' in >> git-sh-setup), and every time we fix it there is a chance of making >> mistakes. I wonder if it would make sense to add a helper to make the >> echo calls easier to replace: > > I agree that we would benefit from having a helper to print a single > line, which we very often do, without having to worry about the > boilerplate '%s\n' of printf or the portability gotcha of echo. > > I am a bit reluctant to name the helper "sane_echo" to declare "echo > that interprets backslashes in the string is insane", though. raw_echo -- David Kastrup ``` ## Junio C Hamano, 2014-03-19 19:24 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: URL: https://gitlist.dev/e/xmqqob122dq0.fsf%40gitster.dls.corp.google.com In-Reply-To: <871txygl27.fsf@fencepost.gnu.org> ``` David Kastrup writes: > Junio C Hamano writes: > >> Jonathan Nieder writes: >> >>> Junio C Hamano wrote: >>>>> Uwe Storbeck wrote: >>> >>>>>> + printf '%s\n' "$@" | sed -e 's/^/# /' >>>> >>>> This is wrong, isn't it? Why do we want one line per item here? >>> >>> Yes, Hannes caught the same, too. Sorry for the sloppiness. >>> >>> We currently use "echo" all over the place (e.g., 'echo "$path"' in >>> git-sh-setup), and every time we fix it there is a chance of making >>> mistakes. I wonder if it would make sense to add a helper to make the >>> echo calls easier to replace: >> >> I agree that we would benefit from having a helper to print a single >> line, which we very often do, without having to worry about the >> boilerplate '%s\n' of printf or the portability gotcha of echo. >> >> I am a bit reluctant to name the helper "sane_echo" to declare "echo >> that interprets backslashes in the string is insane", though. > > raw_echo Yeah, but the thing is, this is not even "raw" if you view it from the direction of knowing what "echo" does. That is why I repeated "helper to print a single line", which is a viewpoint from the user side. "We do not care how it is implemented, we just want a single line printed" is what we want to express, which "say" is perfectly in line with. "We use a subset semantics of 'echo' to implement it" is of secondary concern. ``` ## Jonathan Nieder, 2014-03-20 00:17 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: <20140320001718.GM15625@google.com> URL: https://gitlist.dev/e/20140320001718.GM15625%40google.com In-Reply-To: ``` Junio C Hamano wrote: > Jonathan Nieder writes: >> We currently use "echo" all over the place (e.g., 'echo "$path"' in >> git-sh-setup), and every time we fix it there is a chance of making >> mistakes. I wonder if it would make sense to add a helper to make the >> echo calls easier to replace: > > I agree that we would benefit from having a helper to print a single > line, which we very often do, without having to worry about the > boilerplate '%s\n' of printf or the portability gotcha of echo. > > I am a bit reluctant to name the helper "sane_echo" to declare "echo > that interprets backslashes in the string is insane", though. For > these "print a single line" uses, we are only interested in using a > subset of the features offered by 'echo', but that does not mean the > other features we do not want to trigger in our use is of no use to > any sane person. In a portable script, uncareful use of 'echo' is always insane. In a script tailored to an environment where echo behaves consistently it is perfectly reasonable to use 'echo', but that's a different story. In the context of git, saying "Here is the thing you should always use instead of echo" is a good thing, in my opinion. Hoping that clarifies, Jonathan ``` ## Junio C Hamano, 2014-03-20 16:42 Subject: Re: [PATCH] test-lib.sh: use printf instead of echo Message-ID: URL: https://gitlist.dev/e/xmqq4n2s255z.fsf%40gitster.dls.corp.google.com In-Reply-To: <20140320001718.GM15625@google.com> ``` Jonathan Nieder writes: > Junio C Hamano wrote: > ... >> I am a bit reluctant to name the helper "sane_echo" to declare "echo >> that interprets backslashes in the string is insane", though. For >> these "print a single line" uses, we are only interested in using a >> subset of the features offered by 'echo', but that does not mean the >> other features we do not want to trigger in our use is of no use to >> any sane person. > > In a portable script, uncareful use of 'echo' is always insane. I agree that makes sense and I actually think that it is a bit stronger than that. If a script is meant to be portable, there is no way to use "echo" on a string whose contents is unknown sanely. There is no "careful use is OK". > In a script tailored to an environment where echo behaves consistently > it is perfectly reasonable to use 'echo', but that's a different > story. In the context of git, saying "Here is the thing you should > always use instead of echo" is a good thing, in my opinion. That is true in my opinion, but that thing is also what you should always use instead of "printf '%s\n'". A guideline more useful for the users is "Here is the thing you should always use when literally emitting a single line.", isn't it? ```