# [PATCH] t1402: work around shell quoting issue on NetBSD 11 messages from 2013-01-08 to 2013-01-10. Participants: René Scharfe, Junio C Hamano, Greg Troxel, Adam Spiers. Thread: https://gitlist.dev/t/32563 ## René Scharfe, 2013-01-08 20:23 Subject: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: <50EC8025.8000707@lsrfire.ath.cx> URL: https://gitlist.dev/e/50EC8025.8000707%40lsrfire.ath.cx ``` The test fails for me on NetBSD 6.0.1 and reports: ok 1 - ref name '' is invalid ok 2 - ref name '/' is invalid ok 3 - ref name '/' is invalid with options --allow-onelevel ok 4 - ref name '/' is invalid with options --normalize error: bug in the test script: not 2 or 3 parameters to test-expect-success The alleged bug is in this line: invalid_ref NOT_MINGW '/' '--allow-onelevel --normalize' invalid_ref() constructs a test case description using its last argument, but the shell seems to split it up into two pieces if it contains a space. Minimal test case: # on NetBSD with /bin/sh $ a() { echo $#-$1-$2; } $ t="x"; a "${t:+$t}" 1-x- $ t="x y"; a "${t:+$t}" 2-x-y $ t="x y"; a "${t:+x y}" 1-x y- # and with bash $ t="x y"; a "${t:+$t}" 1-x y- $ t="x y"; a "${t:+x y}" 1-x y- This may be a bug in the shell, but here's a simple workaround: Construct the description string first and store it in a variable, and then use that to call test_expect_success(). Signed-off-by: Rene Scharfe --- t/t1402-check-ref-format.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh index 1ae4d87..1a5a5f3 100755 --- a/t/t1402-check-ref-format.sh +++ b/t/t1402-check-ref-format.sh @@ -11,7 +11,8 @@ valid_ref() { prereq=$1 shift esac - test_expect_success $prereq "ref name '$1' is valid${2:+ with options $2}" " + desc="ref name '$1' is valid${2:+ with options $2}" + test_expect_success $prereq "$desc" " git check-ref-format $2 '$1' " } @@ -22,7 +23,8 @@ invalid_ref() { prereq=$1 shift esac - test_expect_success $prereq "ref name '$1' is invalid${2:+ with options $2}" " + desc="ref name '$1' is invalid${2:+ with options $2}" + test_expect_success $prereq "$desc" " test_must_fail git check-ref-format $2 '$1' " } -- 1.7.12 ``` ## Junio C Hamano, 2013-01-08 20:39 Subject: Re: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: <7vr4lvcstt.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vr4lvcstt.fsf%40alter.siamese.dyndns.org In-Reply-To: <50EC8025.8000707@lsrfire.ath.cx> ``` René Scharfe writes: > The test fails for me on NetBSD 6.0.1 and reports: > > ok 1 - ref name '' is invalid > ok 2 - ref name '/' is invalid > ok 3 - ref name '/' is invalid with options --allow-onelevel > ok 4 - ref name '/' is invalid with options --normalize > error: bug in the test script: not 2 or 3 parameters to test-expect-success > > The alleged bug is in this line: > > invalid_ref NOT_MINGW '/' '--allow-onelevel --normalize' > > invalid_ref() constructs a test case description using its last argument, > but the shell seems to split it up into two pieces if it contains a > space. Minimal test case: > > # on NetBSD with /bin/sh > $ a() { echo $#-$1-$2; } > $ t="x"; a "${t:+$t}" > 1-x- > $ t="x y"; a "${t:+$t}" > 2-x-y > $ t="x y"; a "${t:+x y}" > 1-x y- > > # and with bash > $ t="x y"; a "${t:+$t}" > 1-x y- > $ t="x y"; a "${t:+x y}" > 1-x y- > > This may be a bug in the shell, but here's a simple workaround: Construct > the description string first and store it in a variable, and then use > that to call test_expect_success(). Interesting. I notice that t0008 added recently to 'pu' has the same construct. > > Signed-off-by: Rene Scharfe > --- > t/t1402-check-ref-format.sh | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/t/t1402-check-ref-format.sh b/t/t1402-check-ref-format.sh > index 1ae4d87..1a5a5f3 100755 > --- a/t/t1402-check-ref-format.sh > +++ b/t/t1402-check-ref-format.sh > @@ -11,7 +11,8 @@ valid_ref() { > prereq=$1 > shift > esac > - test_expect_success $prereq "ref name '$1' is valid${2:+ with options $2}" " > + desc="ref name '$1' is valid${2:+ with options $2}" > + test_expect_success $prereq "$desc" " > git check-ref-format $2 '$1' > " > } > @@ -22,7 +23,8 @@ invalid_ref() { > prereq=$1 > shift > esac > - test_expect_success $prereq "ref name '$1' is invalid${2:+ with options $2}" " > + desc="ref name '$1' is invalid${2:+ with options $2}" > + test_expect_success $prereq "$desc" " > test_must_fail git check-ref-format $2 '$1' > " > } ``` ## René Scharfe, 2013-01-08 21:13 Subject: Re: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: <50EC8BE7.2010508@lsrfire.ath.cx> URL: https://gitlist.dev/e/50EC8BE7.2010508%40lsrfire.ath.cx In-Reply-To: <7vr4lvcstt.fsf@alter.siamese.dyndns.org> ``` Am 08.01.2013 21:39, schrieb Junio C Hamano: > René Scharfe writes: >> # on NetBSD with /bin/sh >> $ a() { echo $#-$1-$2; } >> $ t="x"; a "${t:+$t}" >> 1-x- >> $ t="x y"; a "${t:+$t}" >> 2-x-y >> $ t="x y"; a "${t:+x y}" >> 1-x y- >> >> # and with bash >> $ t="x y"; a "${t:+$t}" >> 1-x y- >> $ t="x y"; a "${t:+x y}" >> 1-x y- >> >> This may be a bug in the shell, but here's a simple workaround: Construct >> the description string first and store it in a variable, and then use >> that to call test_expect_success(). > > Interesting. I notice that t0008 added recently to 'pu' has the > same construct. A quick check shows that subtests 64-68 and 89-93 of t0008 fail for me on Debian (10 in total) and subtests 64 and 89 fail on NetBSD (2 in total). Unlike t1402 they don't report "bug in the test script". t0008 only uses ${:+} substitution on variables that don't contain spaces. With the test changed to store the description in a variable first I still get the same 2 failures. There must be something else going on here. The different results are interesting, especially the higher number of failures on Debian. René ``` ## Junio C Hamano, 2013-01-08 21:37 Subject: Re: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: <7vboczcq5a.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vboczcq5a.fsf%40alter.siamese.dyndns.org In-Reply-To: <50EC8BE7.2010508@lsrfire.ath.cx> ``` René Scharfe writes: > A quick check shows that subtests 64-68 and 89-93 of t0008 fail for me > on Debian (10 in total) and subtests 64 and 89 fail on NetBSD (2 in > total). Unlike t1402 they don't report "bug in the test script". > > t0008 only uses ${:+} substitution on variables that don't contain > spaces. With the test changed to store the description in a variable > first I still get the same 2 failures. > > There must be something else going on here. The different results are > interesting, especially the higher number of failures on Debian. I forgot to mention that some of them seem to be broken under dash but pass under bash. ``` ## Greg Troxel, 2013-01-09 01:27 Subject: Re: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: URL: https://gitlist.dev/e/rmihamrjgcj.fsf%40fnord.ir.bbn.com In-Reply-To: <50EC8025.8000707@lsrfire.ath.cx> ``` René Scharfe writes: > invalid_ref() constructs a test case description using its last argument, > but the shell seems to split it up into two pieces if it contains a > space. Minimal test case: This is indeed a bug in NetBSD's shell, which I reported after finding this test case problem, and I think someone is working on a fix. But because git does not intend to be a shell torture test, if it's possible to avoid bugs in a reasonable way, I think it's nice to do so. ``` ## Greg Troxel, 2013-01-09 02:07 Subject: Re: [PATCH] t1402: work around shell quoting issue on NetBSD Message-ID: URL: https://gitlist.dev/e/rmitxqrhzwl.fsf%40fnord.ir.bbn.com In-Reply-To: <50EC8025.8000707@lsrfire.ath.cx> ``` René Scharfe writes: > The test fails for me on NetBSD 6.0.1 and reports: > > ok 1 - ref name '' is invalid > ok 2 - ref name '/' is invalid > ok 3 - ref name '/' is invalid with options --allow-onelevel > ok 4 - ref name '/' is invalid with options --normalize > error: bug in the test script: not 2 or 3 parameters to test-expect-success > > The alleged bug is in this line: > > invalid_ref NOT_MINGW '/' '--allow-onelevel --normalize' The bug in NetBSD's sh has been fixed in -current: http://gnats.netbsd.org/47361 and the change will almost certainly make it to the -6 and -5 release branches. With the fixed sh: 414c78c (with the workaround): t1402 passes 69637e5 (without the workaround): t1402 passes With the buggy sh, 414c78c (with the workaround): t1402 passes 69637e5 (without workaround): t1402 fails so I can confirm that the workaround is successful on NetBSD 5. Thanks for addressing this, and sorry I didn't mention it on this list. Greg ``` ## René Scharfe, 2013-01-09 23:49 Subject: [PATCH] t0008: avoid brace expansion Message-ID: <50EE01F8.1070109@lsrfire.ath.cx> URL: https://gitlist.dev/e/50EE01F8.1070109%40lsrfire.ath.cx In-Reply-To: <7vboczcq5a.fsf@alter.siamese.dyndns.org> ``` Brace expansion is not required by POSIX and not supported by dash nor NetBSD's sh. Explicitly list all combinations instead. Signed-off-by: Rene Scharfe --- t/t0008-ignores.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh index 9b0fcd6..0273680 100755 --- a/t/t0008-ignores.sh +++ b/t/t0008-ignores.sh @@ -129,8 +129,9 @@ test_expect_success 'setup' ' one ignored-* EOF - touch {,a/}{not-ignored,ignored-{and-untracked,but-in-index}} && - git add -f {,a/}ignored-but-in-index + touch not-ignored ignored-and-untracked ignored-but-in-index && + touch a/not-ignored a/ignored-and-untracked a/ignored-but-in-index && + git add -f ignored-but-in-index a/ignored-but-in-index && cat <<-\EOF >a/.gitignore && two* *three -- 1.8.0 ``` ## Adam Spiers, 2013-01-09 23:56 Subject: Re: [PATCH] t0008: avoid brace expansion Message-ID: URL: https://gitlist.dev/e/CAOkDyE_EuuV04KxkkLuHMV%2BVbDWsDMN1q3YShLtKaimaXH40Sg%40mail.gmail.com In-Reply-To: <50EE01F8.1070109@lsrfire.ath.cx> ``` On Wed, Jan 9, 2013 at 11:49 PM, René Scharfe wrote: > Brace expansion is not required by POSIX and not supported by dash nor > NetBSD's sh. Explicitly list all combinations instead. Good catch, thanks! ``` ## Junio C Hamano, 2013-01-10 00:18 Subject: Re: [PATCH] t0008: avoid brace expansion Message-ID: <7vsj693n6o.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vsj693n6o.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` Adam Spiers writes: > On Wed, Jan 9, 2013 at 11:49 PM, René Scharfe > wrote: >> Brace expansion is not required by POSIX and not supported by dash nor >> NetBSD's sh. Explicitly list all combinations instead. > > Good catch, thanks! Yeah; thanks. It would also be nice to avoid touch while we are at it, by the way. ``` ## Adam Spiers, 2013-01-10 00:22 Subject: Re: [PATCH] t0008: avoid brace expansion Message-ID: URL: https://gitlist.dev/e/CAOkDyE_gSn488N9q1_PD40s%2BB9sRY32qMXddoC3fNUFD1jtbhQ%40mail.gmail.com In-Reply-To: <7vsj693n6o.fsf@alter.siamese.dyndns.org> ``` On Thu, Jan 10, 2013 at 12:18 AM, Junio C Hamano wrote: > Adam Spiers writes: > >> On Wed, Jan 9, 2013 at 11:49 PM, René Scharfe >> wrote: >>> Brace expansion is not required by POSIX and not supported by dash nor >>> NetBSD's sh. Explicitly list all combinations instead. >> >> Good catch, thanks! > > Yeah; thanks. > > It would also be nice to avoid touch while we are at it, by the way. Noted. ``` ## René Scharfe, 2013-01-10 21:24 Subject: Re: [PATCH] t0008: avoid brace expansion Message-ID: <50EF31A4.3000205@lsrfire.ath.cx> URL: https://gitlist.dev/e/50EF31A4.3000205%40lsrfire.ath.cx In-Reply-To: <7vsj693n6o.fsf@alter.siamese.dyndns.org> ``` Am 10.01.2013 01:18, schrieb Junio C Hamano: > Adam Spiers writes: > >> On Wed, Jan 9, 2013 at 11:49 PM, René Scharfe >> wrote: >>> Brace expansion is not required by POSIX and not supported by dash nor >>> NetBSD's sh. Explicitly list all combinations instead. >> >> Good catch, thanks! > > Yeah; thanks. > > It would also be nice to avoid touch while we are at it, by the way. Good idea! Replacement patch: --- >8 --- Brace expansion is a shell feature that's not required by POSIX and not supported by dash nor NetBSD's sh. Explicitly list all combinations instead. Also avoid calling touch by creating the test files with a redirection instead, as suggested by Junio. Signed-off-by: Rene Scharfe --- t/t0008-ignores.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/t/t0008-ignores.sh b/t/t0008-ignores.sh index 9b0fcd6..d7df719 100755 --- a/t/t0008-ignores.sh +++ b/t/t0008-ignores.sh @@ -129,8 +129,13 @@ test_expect_success 'setup' ' one ignored-* EOF - touch {,a/}{not-ignored,ignored-{and-untracked,but-in-index}} && - git add -f {,a/}ignored-but-in-index + for dir in . a + do + : >$dir/not-ignored && + : >$dir/ignored-and-untracked && + : >$dir/ignored-but-in-index + done && + git add -f ignored-but-in-index a/ignored-but-in-index && cat <<-\EOF >a/.gitignore && two* *three -- 1.8.0 ```