{"thread":{"id":"26117","subject":"t0050-filesystem.sh unicode tests borked on dash shell","startedAt":"2010-12-21T19:53:25Z","lastAt":"2010-12-28T18:05:56Z","messageCount":6,"participants":["Ramsay Jones","Jonathan Nieder","Thomas Rast","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"158453","messageId":"4D1105B5.5070703@ramsay1.demon.co.uk","threadId":"26117","inReplyTo":null,"subject":"t0050-filesystem.sh unicode tests borked on dash shell","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2010-12-21T19:53:25Z","receivedAt":"2010-12-21T19:53:25Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\nI noticed recently that the unicode tests, when run by the dash shell,\nhave not been working as designed. (The tests *pass*, but they are\n*not* testing what was intended)\n\nIn order to demonstrate, I added an \"false &&\" line after the touch in\ntest #8, so that (on Ubuntu):\n\n    $ ./t0050-filesystem -i\n    ok 1 - see what we expect\n    ok 2 - detection of case insensitive filesystem during repo init\n    ok 3 - detection of filesystem w/o symlink support during repo init\n    ok 4 - setup case tests\n    ok 5 - rename (case change)\n    ok 6 - merge (case change)\n    not ok 7 - add (with different case) # TODO known breakage\n    not ok - 8 setup unicode normalization tests\n    #\t\n    #\t\n    #\t  test_create_repo unicode &&\n    #\t  cd unicode &&\n    #\t  touch \"$aumlcdiar\" &&\n    #\tfalse &&\n    #\t  git add \"$aumlcdiar\" &&\n    #\t  git commit -m initial &&\n    #\t  git tag initial &&\n    #\t  git checkout -b topic &&\n    #\t  git mv $aumlcdiar tmp &&\n    #\t  git mv tmp \"$auml\" &&\n    #\t  git commit -m rename &&\n    #\t  git checkout -f master\n    #\t\n    #\t\n    $ ls trash\\ directory.t0050-filesystem/unicode/\n    \\x61\\xcc\\x88\n\n    $ bash t0050-filesystem -i\n    ok 1 - see what we expect\n    ok 2 - detection of case insensitive filesystem during repo init\n    ok 3 - detection of filesystem w/o symlink support during repo init\n    ok 4 - setup case tests\n    ok 5 - rename (case change)\n    ok 6 - merge (case change)\n    not ok 7 - add (with different case) # TODO known breakage\n    not ok - 8 setup unicode normalization tests\n    #\t\n    #\t\n    #\t  test_create_repo unicode &&\n    #\t  cd unicode &&\n    #\t  touch \"$aumlcdiar\" &&\n    #\tfalse &&\n    #\t  git add \"$aumlcdiar\" &&\n    #\t  git commit -m initial &&\n    #\t  git tag initial &&\n    #\t  git checkout -b topic &&\n    #\t  git mv $aumlcdiar tmp &&\n    #\t  git mv tmp \"$auml\" &&\n    #\t  git commit -m rename &&\n    #\t  git checkout -f master\n    #\t\n    #\t\n    $ ls trash\\ directory.t0050-filesystem/unicode/ | od -x\n    0000000 cc61 0a88\n    0000004\n\nSo bash works fine and I can avoid the problem by running the tests, thus:\n\n    $ SHELL_PATH=/bin/bash make NO_SVN_TESTS=1 test\n\nSince I have an older dash, I compiled dash from source (my dash git repo\nclaims:\n\n    $ git describe --tags\n    v0.5.6-24-gb61ab0b\n\n), but the result was exactly the same.\n\nI afraid I don't have time to investigate this further at the moment ...\n\nATB,\nRamsay Jones\n"},{"id":"158456","messageId":"20101221202755.GA27214@burratino","threadId":"26117","inReplyTo":"4D1105B5.5070703@ramsay1.demon.co.uk","subject":"[PATCH] t0050: fix printf format strings for portability","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-12-21T20:27:55Z","receivedAt":"2010-12-21T20:27:55Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Unlike bash and ksh, dash passes through hexadecimal \\xcc escapes.\nSo when run with dash, these tests *pass* (since '\\xcc' is a perfectly\nreasonable filename) but they are not testing what was intended.\n\nUse octal escapes instead, in the spirit of v1.6.1-rc1~55^2\n(2008-11-09).\n\nReported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nRamsay Jones wrote:\n\n> I noticed recently that the unicode tests, when run by the dash shell,\n> have not been working as designed. (The tests *pass*, but they are\n> *not* testing what was intended)\n> \n> In order to demonstrate, I added an \"false &&\" line after the touch in\n> test #8, so that (on Ubuntu):\n> \n>     $ ./t0050-filesystem -i\n[...]\n>     $ ls trash\\ directory.t0050-filesystem/unicode/\n>     \\x61\\xcc\\x88\n\nGood point.  POSIX printf is not required to support \\x escape\nsequences.\n\n t/t0050-filesystem.sh |    4 ++--\n 1 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\nindex 057c97c..1542cf6 100755\n--- a/t/t0050-filesystem.sh\n+++ b/t/t0050-filesystem.sh\n@@ -4,8 +4,8 @@ test_description='Various filesystem issues'\n \n . ./test-lib.sh\n \n-auml=`printf '\\xc3\\xa4'`\n-aumlcdiar=`printf '\\x61\\xcc\\x88'`\n+auml=$(printf '\\303\\244')\n+aumlcdiar=$(printf '\\141\\314\\210')\n \n case_insensitive=\n unibad=\n-- \n1.7.2.3.554.gc9b5c.dirty\n"},{"id":"158457","messageId":"201012212129.39141.trast@student.ethz.ch","threadId":"26117","inReplyTo":"4D1105B5.5070703@ramsay1.demon.co.uk","subject":"Re: t0050-filesystem.sh unicode tests borked on dash shell","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-12-21T20:29:38Z","receivedAt":"2010-12-21T20:29:38Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Ramsay Jones wrote:\n>     $ ls trash\\ directory.t0050-filesystem/unicode/\n>     \\x61\\xcc\\x88\n\nThe printf at the top evidently does not interpolate \\xAA sequences.\nSince my 'man 1p printf' POSIX manpage only mandates \\AAA octal\nsequences, maybe we should use that instead.  Can you verify that the\npatch below works for you?\n\nJudging from\n\n  git grep '\\\\x[0-9a-f][0-9a-f]' t\n\nthis is the only instance of this problem, the rest are in Perl code.\n\n--- 8< ---\nSubject: t0050: replace \\xAA by \\AAA in printf\n\nPOSIX does not mandate the hex escape sequences, and thus dash's\nbuilt-in printf does not expand them.  Use octal escapes instead.\n\ndiff --git c/t/t0050-filesystem.sh i/t/t0050-filesystem.sh\nindex 057c97c..87bf1ff 100755\n--- c/t/t0050-filesystem.sh\n+++ i/t/t0050-filesystem.sh\n@@ -4,8 +4,8 @@ test_description='Various filesystem issues'\n \n . ./test-lib.sh\n \n-auml=`printf '\\xc3\\xa4'`\n-aumlcdiar=`printf '\\x61\\xcc\\x88'`\n+auml=`printf '\\303\\244'`\n+aumlcdiar=`printf '\\141\\314\\210'`\n \n case_insensitive=\n unibad=\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"158460","messageId":"7v8vzi97rv.fsf@alter.siamese.dyndns.org","threadId":"26117","inReplyTo":"4D1105B5.5070703@ramsay1.demon.co.uk","subject":"Re: t0050-filesystem.sh unicode tests borked on dash shell","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-12-21T20:43:16Z","receivedAt":"2010-12-21T20:43:16Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ramsay Jones <ramsay@ramsay1.demon.co.uk> writes:\n\n>     $ ls trash\\ directory.t0050-filesystem/unicode/\n>     \\x61\\xcc\\x88\n\nThe built-in printf in dash seems to lack understanding of '\\xXX'.\nIt is tempting to patch it by using /usr/bin/printf but it is unclear how\nportable it would be.\n\n t/t0050-filesystem.sh |    4 ++--\n 1 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\nindex 41df6bc..8ad102b 100755\n--- a/t/t0050-filesystem.sh\n+++ b/t/t0050-filesystem.sh\n@@ -4,8 +4,8 @@ test_description='Various filesystem issues'\n \n . ./test-lib.sh\n \n-auml=`printf '\\xc3\\xa4'`\n-aumlcdiar=`printf '\\x61\\xcc\\x88'`\n+auml=$(/usr/bin/printf '\\xc3\\xa4')\n+aumlcdiar=$(/usr/bin/printf '\\x61\\xcc\\x88')\n \n case_insensitive=\n unibad=\n"},{"id":"158462","messageId":"7vvd2m7r6y.fsf@alter.siamese.dyndns.org","threadId":"26117","inReplyTo":"20101221202755.GA27214@burratino","subject":"Re: [PATCH] t0050: fix printf format strings for portability","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-12-21T21:26:45Z","receivedAt":"2010-12-21T21:26:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> Unlike bash and ksh, dash passes through hexadecimal \\xcc escapes.\n> So when run with dash, these tests *pass* (since '\\xcc' is a perfectly\n> reasonable filename) but they are not testing what was intended.\n>\n> Use octal escapes instead, in the spirit of v1.6.1-rc1~55^2\n> (2008-11-09).\n\nThanks.\n"},{"id":"158681","messageId":"4D1A2704.8020500@ramsay1.demon.co.uk","threadId":"26117","inReplyTo":"20101221202755.GA27214@burratino","subject":"Re: [PATCH] t0050: fix printf format strings for portability","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2010-12-28T18:05:56Z","receivedAt":"2010-12-28T18:05:56Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Jonathan Nieder wrote:\n> diff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\n> index 057c97c..1542cf6 100755\n> --- a/t/t0050-filesystem.sh\n> +++ b/t/t0050-filesystem.sh\n> @@ -4,8 +4,8 @@ test_description='Various filesystem issues'\n>  \n>  . ./test-lib.sh\n>  \n> -auml=`printf '\\xc3\\xa4'`\n> -aumlcdiar=`printf '\\x61\\xcc\\x88'`\n> +auml=$(printf '\\303\\244')\n> +aumlcdiar=$(printf '\\141\\314\\210')\n>  \n>  case_insensitive=\n>  unibad=\n\nThanks everyone (Jonathan, Junio and Thomas) for the quick reply and fix!\n[I should have figured it out myself, but I just couldn't imagine printf\nnot supporting hex escapes! :-P ]\n\nATB,\nRamsay Jones\n"}]}