# Git build on antique PowerMac 9 messages from 2019-05-05 to 2019-05-11. Participants: Jeffrey Walton, SZEDER Gábor, Eric Sunshine, Ævar Arnfjörð Bjarmason. Thread: https://gitlist.dev/t/51039 ## Jeffrey Walton, 2019-05-05 19:42 Subject: Git build on antique PowerMac Message-ID: URL: https://gitlist.dev/e/CAH8yC8nTyq5Fjc-6fUNn1iwkCO9i1Vrc0vFqxEVg4Eor3h-0Lw%40mail.gmail.com ``` Hi Everyone, I have a PowerMac I use for testing. It provides several testing differentiators, like OS X 10.5, Bash 3.2, GCC 4.0.1, Apple cc-tools linker, and big-endian PowerPC. (I think Gentoo provides a Linux image for the hardware, but I don't use it). The Git libraries and programs build fine out of the box. They also seem to work as expected once installed. The pain point is the self self tests, which I have never been able to build (or execute). I'd like to close this gap. make -C templates SHELL_PATH='/bin/sh' PERL_PATH='/usr/bin/perl' : no custom templates yet make -C t/ all rm -f -r 'test-results' readline() on unopened filehandle test_must_fail run_sub_test_lib_test at check-non-portable-shell.pl line 34. Modification of a read-only value attempted at check-non-portable-shell.pl line 34. make[1]: *** [test-lint-shell-syntax] Error 255 Does anyone want to take a shot at this antique system? I can provide SSH access with root. Jeff ``` ## SZEDER Gábor, 2019-05-05 20:14 Subject: Re: Git build on antique PowerMac Message-ID: <20190505201452.GL14763@szeder.dev> URL: https://gitlist.dev/e/20190505201452.GL14763%40szeder.dev In-Reply-To: ``` On Sun, May 05, 2019 at 03:42:45PM -0400, Jeffrey Walton wrote: > I have a PowerMac I use for testing. It provides several testing > differentiators, like OS X 10.5, Bash 3.2, GCC 4.0.1, Apple cc-tools > linker, and big-endian PowerPC. (I think Gentoo provides a Linux image > for the hardware, but I don't use it). > > The Git libraries and programs build fine out of the box. They also > seem to work as expected once installed. > > The pain point is the self self tests, which I have never been able to > build (or execute). I'd like to close this gap. > > make -C templates SHELL_PATH='/bin/sh' PERL_PATH='/usr/bin/perl' > : no custom templates yet > make -C t/ all > rm -f -r 'test-results' > readline() on unopened filehandle test_must_fail > run_sub_test_lib_test at check-non-portable-shell.pl line 34. > Modification of a read-only value attempted at > check-non-portable-shell.pl line 34. > make[1]: *** [test-lint-shell-syntax] Error 255 You could try: make -C t all TEST_LINT= That error comes from one of the "lint" targets that are supposed to catch some common errors that we repeatedly made in our tests. This linter is enabled by default, but this command disables it. Yeah, it's not the proper solution, but I would think that you don't need them on that antique PowerMac anyway. ``` ## Eric Sunshine, 2019-05-05 20:47 Subject: Re: Git build on antique PowerMac Message-ID: URL: https://gitlist.dev/e/CAPig%2BcSzsmeHp1S5WG3TVXEyZKt9eVrsCsWEkpwbNUZK_SJ53g%40mail.gmail.com In-Reply-To: ``` On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton wrote: > I have a PowerMac I use for testing. [...] > make -C t/ all > readline() on unopened filehandle test_must_fail > run_sub_test_lib_test at check-non-portable-shell.pl line 34. > Modification of a read-only value attempted at > check-non-portable-shell.pl line 34. > make[1]: *** [test-lint-shell-syntax] Error 255 > > Does anyone want to take a shot at this antique system? I can provide > SSH access with root. As the author of the code triggering that problem, I can take a look at it. (You already have my public SSH key on your Solaris box.) Given [1], I can see (I guess) why it's complaining about modification of a read-only variable. However, the unopened filehandle complaint looks odd. [1]: https://www.perlmonks.org/?node_id=570712#default_unlocalized ``` ## Ævar Arnfjörð Bjarmason, 2019-05-08 22:27 Subject: Re: Git build on antique PowerMac Message-ID: <87mujwh98z.fsf@evledraar.gmail.com> URL: https://gitlist.dev/e/87mujwh98z.fsf%40evledraar.gmail.com In-Reply-To: ``` On Sun, May 05 2019, Eric Sunshine wrote: > On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton wrote: >> I have a PowerMac I use for testing. [...] >> make -C t/ all >> readline() on unopened filehandle test_must_fail >> run_sub_test_lib_test at check-non-portable-shell.pl line 34. >> Modification of a read-only value attempted at >> check-non-portable-shell.pl line 34. >> make[1]: *** [test-lint-shell-syntax] Error 255 >> >> Does anyone want to take a shot at this antique system? I can provide >> SSH access with root. > > As the author of the code triggering that problem, I can take a look > at it. (You already have my public SSH key on your Solaris box.) > > Given [1], I can see (I guess) why it's complaining about modification > of a read-only variable. However, the unopened filehandle complaint > looks odd. > > [1]: https://www.perlmonks.org/?node_id=570712#default_unlocalized It's because "while (<>) { readline }" didn't use to be supported until 5.10, and he's on 5.8. $ seq 1 4 | ./miniperl -we 'print $]; while (<>) { chomp; print "$.: $_ -> " . readline }; print "\n"' readline() on unopened filehandle 1: 1 -> at -e line 1. Use of uninitialized value in concatenation (.) or string at -e line 1. readline() on unopened filehandle 2: 2 -> at -e line 1. readline() on unopened filehandle 3: 3 -> at -e line 1. readline() on unopened filehandle 4: 4 -> at -e line 1. 5.009004 $ seq 1 4 | perl -we 'print $]; while (<>) { chomp; print "$.: $_ -> " . readline }; print "\n"' 5.0280011: 1 -> 2 3: 3 -> 4 ``` ## Eric Sunshine, 2019-05-09 09:10 Subject: Re: Git build on antique PowerMac Message-ID: URL: https://gitlist.dev/e/CAPig%2BcTM-ObsvvNy8uDmO1mS_KfcjppeRBMe%3DpnM5GkvS5ZVgQ%40mail.gmail.com In-Reply-To: <87mujwh98z.fsf@evledraar.gmail.com> ``` On Wed, May 8, 2019 at 6:28 PM Ævar Arnfjörð Bjarmason wrote: > On Sun, May 05 2019, Eric Sunshine wrote: > > On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton wrote: > >> readline() on unopened filehandle test_must_fail > > > > Given [1], I can see (I guess) why it's complaining about modification > > of a read-only variable. However, the unopened filehandle complaint > > looks odd. > > It's because "while (<>) { readline }" didn't use to be supported until > 5.10, and he's on 5.8. Thanks for researching this. Now it makes sense. ``` ## Eric Sunshine, 2019-05-09 10:20 Subject: [PATCH] check-non-portable-shell: support Perl versions older than 5.10 Message-ID: <20190509102037.27044-1-sunshine@sunshineco.com> URL: https://gitlist.dev/e/20190509102037.27044-1-sunshine%40sunshineco.com In-Reply-To: ``` For thoroughness when checking for one-shot environment variable assignments at shell function call sites, check-non-portable-shell stitches together incomplete lines (those ending with backslash). This allows it to correctly flag such undesirable usage even when the variable assignment and function call are split across lines, for example: FOO=bar \ func where 'func' is a shell function. The stitching is accomplished like this: while (<>) { chomp; # stitch together incomplete lines (those ending with "\") while (s/\\$//) { $_ .= readline; chomp; } # detect unportable/undesirable shell constructs ... } Although this implementation is well supported in reasonably modern Perl versions (5.10 and later), it fails in a couple ways with older versions (such as Perl 5.8 shipped with ancient Mac OS 10.5). In particular, in older Perl versions, 'readline' is not connected to the file handle associated with the "magic" while (<>) {...} construct, so 'readline' throws a "readline() on unopened filehandle" error. Furthermore, $_ assigned by the outer while-loop is read-only, so the attempt to modify it via "$_ .= readline" in the inner while-loop fails with a "Modification of a read-only value" error. Avoid both problems by collecting the stitched-together line in a variable other than $_ and dropping the inner loop entirely. Signed-off-by: Eric Sunshine --- t/check-non-portable-shell.pl | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl index 166d64d4a2..60e607ba42 100755 --- a/t/check-non-portable-shell.pl +++ b/t/check-non-portable-shell.pl @@ -27,14 +27,14 @@ sub err { close $f; } +my $line = ''; while (<>) { chomp; + $line .= $_; # stitch together incomplete lines (those ending with "\") - while (s/\\$//) { - $_ .= readline; - chomp; - } + next if $line =~ s/\\$//; + local $_ = $line; /\bcp\s+-a/ and err 'cp -a is not portable'; /\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)'; /\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)'; @@ -48,6 +48,7 @@ sub err { /\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)'; /^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and err '"FOO=bar shell_func" assignment extends beyond "shell_func"'; + $line = ''; # this resets our $. for each file close ARGV if eof; } -- 2.21.0.1084.g81c186ecd2 ``` ## Ævar Arnfjörð Bjarmason, 2019-05-09 12:33 Subject: Re: [PATCH] check-non-portable-shell: support Perl versions older than 5.10 Message-ID: <87ftpnhknn.fsf@evledraar.gmail.com> URL: https://gitlist.dev/e/87ftpnhknn.fsf%40evledraar.gmail.com In-Reply-To: <20190509102037.27044-1-sunshine@sunshineco.com> ``` On Thu, May 09 2019, Eric Sunshine wrote: > For thoroughness when checking for one-shot environment variable > assignments at shell function call sites, check-non-portable-shell > stitches together incomplete lines (those ending with backslash). This > allows it to correctly flag such undesirable usage even when the > variable assignment and function call are split across lines, for > example: > > FOO=bar \ > func > > where 'func' is a shell function. > > The stitching is accomplished like this: > > while (<>) { > chomp; > # stitch together incomplete lines (those ending with "\") > while (s/\\$//) { > $_ .= readline; > chomp; > } > # detect unportable/undesirable shell constructs > ... > } > > Although this implementation is well supported in reasonably modern Perl > versions (5.10 and later), it fails in a couple ways with older versions > (such as Perl 5.8 shipped with ancient Mac OS 10.5). > > In particular, in older Perl versions, 'readline' is not connected to > the file handle associated with the "magic" while (<>) {...} construct, > so 'readline' throws a "readline() on unopened filehandle" error. > Furthermore, $_ assigned by the outer while-loop is read-only, so the > attempt to modify it via "$_ .= readline" in the inner while-loop fails > with a "Modification of a read-only value" error. > > Avoid both problems by collecting the stitched-together line in a > variable other than $_ and dropping the inner loop entirely. > > Signed-off-by: Eric Sunshine > --- > t/check-non-portable-shell.pl | 9 +++++---- > 1 file changed, 5 insertions(+), 4 deletions(-) > > diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl > index 166d64d4a2..60e607ba42 100755 > --- a/t/check-non-portable-shell.pl > +++ b/t/check-non-portable-shell.pl > @@ -27,14 +27,14 @@ sub err { > close $f; > } > > +my $line = ''; > while (<>) { > chomp; > + $line .= $_; > # stitch together incomplete lines (those ending with "\") > - while (s/\\$//) { > - $_ .= readline; > - chomp; > - } > + next if $line =~ s/\\$//; > > + local $_ = $line; > /\bcp\s+-a/ and err 'cp -a is not portable'; > /\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)'; > /\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)'; > @@ -48,6 +48,7 @@ sub err { > /\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)'; > /^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and > err '"FOO=bar shell_func" assignment extends beyond "shell_func"'; > + $line = ''; > # this resets our $. for each file > close ARGV if eof; > } This fix is fine, but just for the record: There's no problem with assigning to $_, it just throws an error about $_ *because* of the readline() issue, i.e. it'll fail, clobber $_ to a read-only value, and off we go. So just assigning to $_ is fine, and you don't need to localize it. Anyway, I tested this on 5.8, it works, then looked at the output and wondered if I could improve it, came up with this: diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl index 60e607ba42..d5fd0a3050 100755 --- a/t/check-non-portable-shell.pl +++ b/t/check-non-portable-shell.pl @@ -8,13 +8,26 @@ my $exit_code=0; my %func; +my $start_nr = 0; +my $line = ''; sub err { my $msg = shift; - s/^\s+//; - s/\s+$//; - s/\s+/ /g; - print "$ARGV:$.: error: $msg: $_\n"; + if (/\n/) { + $. = $start_nr; + my ($ws) = $_ =~ /^(\s+)/; + for (split /^/) { + s/^\Q$ws\E//; + print "$ARGV:$.: error: $msg: $_"; + $.++; + } + print "\n"; + } else { + s/^\s+//; + s/\s+$//; + s/\s+/ /g; + print "$ARGV:$.: error: $msg: $_\n"; + } $exit_code = 1; } @@ -27,14 +40,16 @@ sub err { close $f; } -my $line = ''; while (<>) { chomp; - $line .= $_; # stitch together incomplete lines (those ending with "\") - next if $line =~ s/\\$//; - - local $_ = $line; + if (s/\\$//) { + $start_nr ||= $.; + $line .= "$_\n"; + next; + } else { + $_ = $line . $_; + } /\bcp\s+-a/ and err 'cp -a is not portable'; /\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)'; /\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)'; @@ -48,7 +63,11 @@ sub err { /\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)'; /^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and err '"FOO=bar shell_func" assignment extends beyond "shell_func"'; + + # No longer spanning lines + $start_nr = 0; $line = ''; + # this resets our $. for each file close ARGV if eof; } diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh index c03054c538..b4af7032ad 100755 --- a/t/t0000-basic.sh +++ b/t/t0000-basic.sh @@ -156,8 +156,11 @@ test_expect_success 'pretend we have a fully passing test suite' " " test_expect_success 'pretend we have a partially passing test suite' " - test_must_fail run_sub_test_lib_test \ - partial-pass '2/3 tests passing' <<-\\EOF && + test_must_fail penis run_sub_test_lib_test \ + partial-pass '2/3 tests passing' <<-\\EOF \ + partial-pass '2/3 tests passing' <<-\\EOF \ + partial-pass '2/3 tests passing' <<-\\EOF \ + cp -a hi there && test_expect_success 'passing test #1' 'true' test_expect_success 'failing test #2' 'false' test_expect_success 'passing test #3' 'true' diff --git a/t/t0001-init.sh b/t/t0001-init.sh index 1f462204ea..a25ac208e5 100755 --- a/t/t0001-init.sh +++ b/t/t0001-init.sh @@ -122,6 +122,7 @@ test_expect_success 'plain bare with GIT_WORK_TREE' ' test_expect_success 'GIT_DIR bare' ' mkdir git-dir-bare.git && + cp -a foo bar && GIT_DIR=git-dir-bare.git git init && check_config git-dir-bare.git true unset ' I.e. now for these multi-line issues we'll print the whole offending multi-line invocation $ ~/g/perl/miniperl -I ~/g/perl/lib check-non-portable-shell.pl t[0-9]*.sh t0000-basic.sh:159: error: cp -a is not portable: test_must_fail penis run_sub_test_lib_test t0000-basic.sh:160: error: cp -a is not portable: partial-pass '2/3 tests passing' <<-\\EOF t0000-basic.sh:161: error: cp -a is not portable: partial-pass '2/3 tests passing' <<-\\EOF t0000-basic.sh:162: error: cp -a is not portable: partial-pass '2/3 tests passing' <<-\\EOF t0000-basic.sh:163: error: cp -a is not portable: cp -a hi there && t0001-init.sh:125: error: cp -a is not portable: cp -a foo bar && I figured it was better than the current output just squashing such a long line together, i.e. it'll print this now (before/after this patch): $ ~/g/perl/miniperl -I ~/g/perl/lib check-non-portable-shell.pl t[0-9]*.sh t0000-basic.sh:163: error: cp -a is not portable: test_must_fail penis run_sub_test_lib_test partial-pass '2/3 tests passing' <<-\\EOF partial-pass '2/3 tests passing' <<-\\EOF partial-pass '2/3 tests passing' <<-\\EOF cp -a hi there && t0001-init.sh:125: error: cp -a is not portable: cp -a foo bar && There's ways to make that WIP patch shorter etc. I wasn't trying to golf it, also I think we can get rid of that s/\s+$// if we split up lines like this. ``` ## Eric Sunshine, 2019-05-10 20:39 Subject: Re: [PATCH] check-non-portable-shell: support Perl versions older than 5.10 Message-ID: URL: https://gitlist.dev/e/CAPig%2BcR30dnurYq%3DG_Ki2Nf7vFCkebcR%2BegvhbGE5w8XSBKQBA%40mail.gmail.com In-Reply-To: <87ftpnhknn.fsf@evledraar.gmail.com> ``` On Thu, May 9, 2019 at 8:33 AM Ævar Arnfjörð Bjarmason wrote: > On Thu, May 09 2019, Eric Sunshine wrote: > > Although this implementation is well supported in reasonably modern Perl > > versions (5.10 and later), it fails in a couple ways with older versions > > (such as Perl 5.8 shipped with ancient Mac OS 10.5). > > [...] > > Signed-off-by: Eric Sunshine > > This fix is fine, but just for the record: There's no problem with > assigning to $_, it just throws an error about $_ *because* of the > readline() issue, i.e. it'll fail, clobber $_ to a read-only value, and > off we go. > So just assigning to $_ is fine, and you don't need to localize it. Thanks for the clarification. I'll drop the misleading discussion of $_ from the commit message and eliminate $_ localization from the patch proper. > Anyway, I tested this on 5.8, it works, then looked at the output and > wondered if I could improve it, came up with this: > [...large WIP patch snipped...] > I figured it was better than the current output just squashing such a > long line together [...] That could be done, though it is outside the scope of this portability-fix patch. Also, the existing output with its sufficiently clear problem explanations is probably "good enough" to clue in the developer of new test code as to what needs fixing, so I'm also not sure that it's worth that much effort to prettify what should be a relatively rare warning message from code which has not yet made it into the project. I wouldn't be opposed to someone working on it, but I don't plan on tackling it myself. ``` ## Eric Sunshine, 2019-05-11 00:18 Subject: [PATCH v2] check-non-portable-shell: support Perl versions older than 5.10 Message-ID: <20190511001853.23011-1-sunshine@sunshineco.com> URL: https://gitlist.dev/e/20190511001853.23011-1-sunshine%40sunshineco.com In-Reply-To: <20190509102037.27044-1-sunshine@sunshineco.com> ``` For thoroughness when checking for one-shot environment variable assignments at shell function call sites, check-non-portable-shell stitches together incomplete lines (those ending with backslash). This allows it to correctly flag such undesirable usage even when the variable assignment and function call are split across lines, for example: FOO=bar \ func where 'func' is a shell function. The stitching is accomplished like this: while (<>) { chomp; # stitch together incomplete lines (those ending with "\") while (s/\\$//) { $_ .= readline; chomp; } # detect unportable/undesirable shell constructs ... } Although this implementation is well supported in reasonably modern Perl versions (5.10 and later), it fails with older versions (such as Perl 5.8 shipped with ancient Mac OS 10.5). In particular, in older Perl versions, 'readline' is not connected to the file handle associated with the "magic" while (<>) {...} construct, so 'readline' throws a "readline() on unopened filehandle" error. Work around this problem by dropping readline() and instead incorporating the stitching of incomplete lines directly into the existing while (<>) {...} loop. Helped-by: Ævar Arnfjörð Bjarmason Signed-off-by: Eric Sunshine --- This is a re-roll of [1]. The only change since v1 is to drop localization of $_ as suggested by Ævar in [2], and rewriting the commit message to avoid misleadingly talking about $_ as being problematic. [1]: http://public-inbox.org/git/20190509102037.27044-1-sunshine@sunshineco.com/ [2]: http://public-inbox.org/git/87ftpnhknn.fsf@evledraar.gmail.com/ t/check-non-portable-shell.pl | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl index 166d64d4a2..38bfeebd88 100755 --- a/t/check-non-portable-shell.pl +++ b/t/check-non-portable-shell.pl @@ -27,14 +27,14 @@ sub err { close $f; } +my $line = ''; while (<>) { chomp; + $line .= $_; # stitch together incomplete lines (those ending with "\") - while (s/\\$//) { - $_ .= readline; - chomp; - } + next if $line =~ s/\\$//; + $_ = $line; /\bcp\s+-a/ and err 'cp -a is not portable'; /\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)'; /\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)'; @@ -48,6 +48,7 @@ sub err { /\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)'; /^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and err '"FOO=bar shell_func" assignment extends beyond "shell_func"'; + $line = ''; # this resets our $. for each file close ARGV if eof; } -- 2.21.0.1090.g8b4b5564af ```