git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Avoid difference in tr semantics between System V and BSD

From
Ben Walton <bdwalton@gmail.com>
Date
Oct 28, 2013, 21:43 UTC
Message-ID
<1382996580-19031-1-git-send-email-bdwalton@gmail.com>
In-Reply-To
<xmqqiowhdqx8.fsf@gitster.dls.corp.google.com>

Solaris' tr (both /usr/bin/ and /usr/xpg4/bin) uses the System V semantics for tr whereby string1's length is truncated to the length of string2 if string2 is shorter. The BSD semantics, as used by GNU tr see string2 padded to the length of string1 using the final character in string2. POSIX explicitly doesn't specify the correct behavior here, making both equally valid.

This difference means that Solaris' native tr implementations produce different results for tr ":\t\n" "\0" than GNU tr. This breaks a few tests in t0008-ignores.sh.

Possible fixes for this are to make string2 be "\0\0\0" or "[\0*]".

Instead, use perl to perform these transliterations which means we don't need to worry about the difference at all. Since we're replacing tr with perl, we also use perl to replace the sed invocations used to transform the files.

Replace four identical transforms with a function named broken_c_unquote. Replace the other two identical transforms with a fuction named broken_c_unquote_verbose.

Signed-off-by: Ben Walton <bdwalton@gmail.com>
---
...Forgot to quote PERL_PATH in the previous iteration.
 t/t0008-ignores.sh | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh
index 181513a..b4d98e6 100755
--- a/t/t0008-ignores.sh
+++ b/t/t0008-ignores.sh
@@ -37,6 +37,14 @@ test_stderr () {
 	test_cmp "$HOME/expected-stderr" "$HOME/stderr"
 }
 
+broken_c_unquote () {
+	"$PERL_PATH" -pe 's/^"//; s/\\//; s/"$//; tr/\n/\0/' "$@"
+}
+
+broken_c_unquote_verbose () {
+	"$PERL_PATH" -pe 's/	"/	/; s/\\//; s/"$//; tr/:\t\n/\0/' "$@"
+}
+
 stderr_contains () {
 	regexp="$1"
 	if grep "$regexp" "$HOME/stderr"
@@ -606,12 +614,11 @@ cat <<-EOF >expected-verbose
 	$global_excludes:2:!globaltwo	b/globaltwo
 EOF
 
-sed -e 's/^"//' -e 's/\\//' -e 's/"$//' stdin | \
-	tr "\n" "\0" >stdin0
-sed -e 's/^"//' -e 's/\\//' -e 's/"$//' expected-default | \
-	tr "\n" "\0" >expected-default0
-sed -e 's/	"/	/' -e 's/\\//' -e 's/"$//' expected-verbose | \
-	tr ":\t\n" "\0" >expected-verbose0
+broken_c_unquote stdin >stdin0
+
+broken_c_unquote expected-default >expected-default0
+
+broken_c_unquote_verbose expected-verbose >expected-verbose0
 
 test_expect_success '--stdin' '
 	expect_from_stdin <expected-default &&
@@ -692,12 +699,11 @@ EOF
 grep -v '^::	' expected-all >expected-verbose
 sed -e 's/.*	//' expected-verbose >expected-default
 
-sed -e 's/^"//' -e 's/\\//' -e 's/"$//' stdin | \
-	tr "\n" "\0" >stdin0
-sed -e 's/^"//' -e 's/\\//' -e 's/"$//' expected-default | \
-	tr "\n" "\0" >expected-default0
-sed -e 's/	"/	/' -e 's/\\//' -e 's/"$//' expected-verbose | \
-	tr ":\t\n" "\0" >expected-verbose0
+broken_c_unquote stdin >stdin0
+
+broken_c_unquote expected-default >expected-default0
+
+broken_c_unquote_verbose expected-verbose >expected-verbose0
 
 test_expect_success '--stdin from subdirectory' '
 	expect_from_stdin <expected-default &&
-- 
1.8.1.2
Previous: Ben WaltonNext: Junio C Hamano
Message 23 of 24 in “Avoid broken Solaris tr”
  1. Avoid broken Solaris trBen Walton, Jun 18, 2013
  2. Junio C HamanoJun 18, 2013
  3. Ben WaltonOct 28, 2013
  4. Avoid difference in tr semantics between System V and BSDBen Walton, Oct 28, 2013
  5. Johannes SixtOct 28, 2013
  6. Jonathan NiederOct 28, 2013
  7. Junio C HamanoOct 28, 2013
  8. t/README: tests can use perl even with NO_PERLJonathan Nieder, Oct 28, 2013
  9. Johannes SixtOct 28, 2013
  10. Jeff KingOct 28, 2013
  11. Jonathan NiederOct 28, 2013
  12. Ben WaltonOct 28, 2013
  13. 0/3 perlJeff King, Oct 29, 2013
  14. 1/3 use @@PERL@@ in built scriptsJeff King, Oct 29, 2013
  15. Junio C HamanoOct 29, 2013
  16. 2/3 t: provide a perl() function which uses $PERL_PATHJeff King, Oct 29, 2013
  17. 3/3 t: use perl instead of "$PERL_PATH" where applicableJeff King, Oct 29, 2013
  18. Ben WaltonOct 28, 2013
  19. Ben WaltonOct 28, 2013
  20. Junio C HamanoOct 28, 2013
  21. Avoid difference in tr semantics between System V and BSDBen Walton, Oct 28, 2013
  22. Ben WaltonOct 28, 2013
  23. Avoid difference in tr semantics between System V and BSDBen Walton, Oct 28, 2013
  24. Junio C HamanoOct 30, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.