# [PATCH] t9001: avoid not portable '\n' with sed 8 messages from 2014-06-04 to 2014-06-04. Participants: Torsten Bögershausen, Junio C Hamano, John Keeping, David Kastrup, Andreas Schwab. Thread: https://gitlist.dev/t/36833 ## Torsten Bögershausen, 2014-06-04 08:20 Subject: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: <538ED6DF.5020505@web.de> URL: https://gitlist.dev/e/538ED6DF.5020505%40web.de ``` t9001 used a '\n' in a sed expression to split one line into two lines. Some versions of sed simply ignore the '\' before the 'n', treating '\n' as 'n'. As the test already requires perl as a prerequisite, use perl instead of sed. Signed-off-by: Torsten Bögershausen --- t/t9001-send-email.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh index 64d9434..2bf48d1 100755 --- a/t/t9001-send-email.sh +++ b/t/t9001-send-email.sh @@ -1342,7 +1342,7 @@ test_cover_addresses () { git format-patch --cover-letter -2 -o outdir && cover=`echo outdir/0000-*.patch` && mv $cover cover-to-edit.patch && - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && + "$PERL_PATH" -pe "s/^From:/$header: extra\@address.com\nFrom:/" cover-to-edit.patch | tr Q "$LF" >"$cover" && git send-email \ --force \ --from="Example " \ -- 2.0.0.553.ged01b91 ``` ## Junio C Hamano, 2014-06-04 17:42 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: URL: https://gitlist.dev/e/xmqqr434vaeh.fsf%40gitster.dls.corp.google.com In-Reply-To: <538ED6DF.5020505@web.de> ``` Torsten Bögershausen writes: > t9001 used a '\n' in a sed expression to split one line into two lines. > Some versions of sed simply ignore the '\' before the 'n', treating > '\n' as 'n'. > > As the test already requires perl as a prerequisite, use perl instead of sed. > > Signed-off-by: Torsten Bögershausen > --- Hmph. I read this in pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html The escape sequence '\n' shall match a embedded in the pattern space. so it may be better to be a bit more explicit in the log message to say whose implementation has this issue to warn people. > t/t9001-send-email.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh > index 64d9434..2bf48d1 100755 > --- a/t/t9001-send-email.sh > +++ b/t/t9001-send-email.sh > @@ -1342,7 +1342,7 @@ test_cover_addresses () { > git format-patch --cover-letter -2 -o outdir && > cover=`echo outdir/0000-*.patch` && > mv $cover cover-to-edit.patch && > - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && > + "$PERL_PATH" -pe "s/^From:/$header: extra\@address.com\nFrom:/" cover-to-edit.patch | tr Q "$LF" >"$cover" && We have a shell function "perl" in test-lib-function.sh these days so that you do not have to write "$PERL_PATH" yourself in tests ;-) > git send-email \ > --force \ > --from="Example " \ Thanks. ``` ## Junio C Hamano, 2014-06-04 18:13 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: URL: https://gitlist.dev/e/xmqqd2eov8ys.fsf%40gitster.dls.corp.google.com In-Reply-To: ``` Junio C Hamano writes: > Torsten Bögershausen writes: > >> t9001 used a '\n' in a sed expression to split one line into two lines. >> Some versions of sed simply ignore the '\' before the 'n', treating >> '\n' as 'n'. >> >> As the test already requires perl as a prerequisite, use perl instead of sed. >> >> Signed-off-by: Torsten Bögershausen >> --- > > Hmph. I read this in pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html > > The escape sequence '\n' shall match a embedded in the > pattern space. > > so it may be better to be a bit more explicit in the log message to > say whose implementation has this issue to warn people. > >> t/t9001-send-email.sh | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh >> index 64d9434..2bf48d1 100755 >> --- a/t/t9001-send-email.sh >> +++ b/t/t9001-send-email.sh >> @@ -1342,7 +1342,7 @@ test_cover_addresses () { >> git format-patch --cover-letter -2 -o outdir && >> cover=`echo outdir/0000-*.patch` && >> mv $cover cover-to-edit.patch && >> - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && >> + "$PERL_PATH" -pe "s/^From:/$header: extra\@address.com\nFrom:/" cover-to-edit.patch | tr Q "$LF" >"$cover" && > > We have a shell function "perl" in test-lib-function.sh these days > so that you do not have to write "$PERL_PATH" yourself in tests ;-) Also, piping output from perl to tr feels somewhat suboptimal. I do not see where in the test material we use "Q to LF", and we may want to remove that altogether, but without that removal, an updated patch may look like this. t/t9001-send-email.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh index 64d9434..9f06b8c 100755 --- a/t/t9001-send-email.sh +++ b/t/t9001-send-email.sh @@ -1342,7 +1342,10 @@ test_cover_addresses () { git format-patch --cover-letter -2 -o outdir && cover=`echo outdir/0000-*.patch` && mv $cover cover-to-edit.patch && - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && + perl -pe " + s/^From:/$header: extra\@address.com\nFrom:/; + y/Q/\n/; + " cover-to-edit.patch >"$cover" && git send-email \ --force \ --from="Example " \ ``` ## Torsten Bögershausen, 2014-06-04 18:42 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: <538F689B.1020804@web.de> URL: https://gitlist.dev/e/538F689B.1020804%40web.de In-Reply-To: ``` On 2014-06-04 20.13, Junio C Hamano wrote: > Junio C Hamano writes: > >> Torsten Bögershausen writes: >> >>> t9001 used a '\n' in a sed expression to split one line into two lines. >>> Some versions of sed simply ignore the '\' before the 'n', treating >>> '\n' as 'n'. >>> >>> As the test already requires perl as a prerequisite, use perl instead of sed. >>> >>> Signed-off-by: Torsten Bögershausen >>> --- >> >> Hmph. I read this in pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html >> >> The escape sequence '\n' shall match a embedded in the >> pattern space. >> >> so it may be better to be a bit more explicit in the log message to >> say whose implementation has this issue to warn people. >> >>> t/t9001-send-email.sh | 2 +- >>> 1 file changed, 1 insertion(+), 1 deletion(-) >>> >>> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh >>> index 64d9434..2bf48d1 100755 >>> --- a/t/t9001-send-email.sh >>> +++ b/t/t9001-send-email.sh >>> @@ -1342,7 +1342,7 @@ test_cover_addresses () { >>> git format-patch --cover-letter -2 -o outdir && >>> cover=`echo outdir/0000-*.patch` && >>> mv $cover cover-to-edit.patch && >>> - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && >>> + "$PERL_PATH" -pe "s/^From:/$header: extra\@address.com\nFrom:/" cover-to-edit.patch | tr Q "$LF" >"$cover" && >> >> We have a shell function "perl" in test-lib-function.sh these days >> so that you do not have to write "$PERL_PATH" yourself in tests ;-) > > Also, piping output from perl to tr feels somewhat suboptimal. I do > not see where in the test material we use "Q to LF", and we may want > to remove that altogether, but without that removal, an updated > patch may look like this. > > t/t9001-send-email.sh | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh > index 64d9434..9f06b8c 100755 > --- a/t/t9001-send-email.sh > +++ b/t/t9001-send-email.sh > @@ -1342,7 +1342,10 @@ test_cover_addresses () { > git format-patch --cover-letter -2 -o outdir && > cover=`echo outdir/0000-*.patch` && > mv $cover cover-to-edit.patch && > - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && > + perl -pe " > + s/^From:/$header: extra\@address.com\nFrom:/; > + y/Q/\n/; > + " cover-to-edit.patch >"$cover" && > git send-email \ > --force \ > --from="Example " \ Good catch, the "tr" should had been removed. My first version used sed "s/^From:/$header: extra@address.comQFrom:/" and the Q was replaced by tr with a literal LF. So I think the 'Q' -> '\n' conversion should be removed completely :-) The sed in question is /usr/bin/sed under Mac OS X. Then we have the question: What exactly is the pattern space? >In default operation, sed cyclically shall append a line of input, less its terminating >character, into the pattern space.... Isn't that the stuff from the input? But that doesn't make too much sence to me, since "input lines" are terminated by \n. So pattern space seems to mean output when they talk about the \n Anyway, the \n (to insert a newline into the output) works under Linux, but not Mac OS. ``` ## John Keeping, 2014-06-04 18:46 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: <20140604184604.GC14457@serenity.lan> URL: https://gitlist.dev/e/20140604184604.GC14457%40serenity.lan In-Reply-To: ``` On Wed, Jun 04, 2014 at 10:42:46AM -0700, Junio C Hamano wrote: > Torsten Bögershausen writes: > > > t9001 used a '\n' in a sed expression to split one line into two lines. > > Some versions of sed simply ignore the '\' before the 'n', treating > > '\n' as 'n'. > > > > As the test already requires perl as a prerequisite, use perl instead of sed. > > > > Signed-off-by: Torsten Bögershausen > > --- > > Hmph. I read this in pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html > > The escape sequence '\n' shall match a embedded in the > pattern space. > > so it may be better to be a bit more explicit in the log message to > say whose implementation has this issue to warn people. > > > - sed "s/^From:/$header: extra@address.com\nFrom:/" cover-to-edit.patch >"$cover" && > > + "$PERL_PATH" -pe "s/^From:/$header: extra\@address.com\nFrom:/" cover-to-edit.patch | tr Q "$LF" >"$cover" && Note that quoted section of POSIX says "embedded in the pattern space"; under the description of the "s" command, it says: The replacement string shall be scanned from beginning to end. [...] The meaning of a immediately followed by any character other than '&', , a digit, or the delimiter character used for this command, is unspecified. A line can be split by substituting a into it. The application shall escape the in the replacement by preceding it by a . So the portable way to do it is: sed "s/^From:/$header: extra@address.com\ From:/" cover-to-edit.patch >"$cover" && but that requires the continuation to start in column 0, so the Perl variant is probably neater. ``` ## David Kastrup, 2014-06-04 18:47 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: <878upc7bre.fsf@fencepost.gnu.org> URL: https://gitlist.dev/e/878upc7bre.fsf%40fencepost.gnu.org In-Reply-To: ``` Junio C Hamano writes: > Torsten Bögershausen writes: > >> t9001 used a '\n' in a sed expression to split one line into two lines. >> Some versions of sed simply ignore the '\' before the 'n', treating >> '\n' as 'n'. >> >> As the test already requires perl as a prerequisite, use perl instead of sed. >> >> Signed-off-by: Torsten Bögershausen >> --- > > Hmph. I read this in pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html > > The escape sequence '\n' shall match a embedded in the > pattern space. > > so it may be better to be a bit more explicit in the log message to > say whose implementation has this issue to warn people. "shall match" talks about the match expression, not the replacement. -- David Kastrup ``` ## Junio C Hamano, 2014-06-04 19:01 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: URL: https://gitlist.dev/e/xmqq1tv4v6qg.fsf%40gitster.dls.corp.google.com In-Reply-To: <20140604184604.GC14457@serenity.lan> ``` John Keeping writes: > Note that quoted section of POSIX says "embedded in the pattern space"; > under the description of the "s" command, it says: > > The replacement string shall be scanned from beginning to end. > [...] > The meaning of a immediately followed by any > character other than '&', , a digit, or the delimiter > character used for this command, is unspecified. Very true. ``` ## Andreas Schwab, 2014-06-04 19:16 Subject: Re: [PATCH] t9001: avoid not portable '\n' with sed Message-ID: <87wqcw1o5o.fsf@igel.home> URL: https://gitlist.dev/e/87wqcw1o5o.fsf%40igel.home In-Reply-To: <20140604184604.GC14457@serenity.lan> ``` John Keeping writes: > So the portable way to do it is: > > sed "s/^From:/$header: extra@address.com\ > From:/" cover-to-edit.patch >"$cover" && That wouldn't work as \ is removed in double quotes. You either need to double the backslash or put it in single quotes. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ```