{"thread":{"id":"50711","subject":"Solaris and sed: Too many commands, last: s/\\n//","startedAt":"2019-03-11T20:32:27Z","lastAt":"2019-03-12T09:08:34Z","messageCount":13,"participants":["Jeffrey Walton","Eric Sunshine","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"371191","messageId":"CAH8yC8nnFYt0raL+twrG_v8XZn_FDJWyM=tH=QL13Z70_bHRiQ@mail.gmail.com","threadId":"50711","inReplyTo":null,"subject":"Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-11T20:31:55Z","receivedAt":"2019-03-11T20:32:27Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"Hi Everyone,\n\nI enabled self tests for Solaris. Solaris has some anemic utilities so\nI put /usr/gnu/bin first on-path.\n\nmake test is resulting in a lot of:\n\ngmake -C templates SHELL_PATH='/bin/bash' PERL_PATH='/usr/bin/perl'\ngmake[1]: Entering directory\n`/export/home/jwalton/Build-Scripts/git-2.21.0/templates'\n: no custom templates yet\ngmake[1]: Leaving directory\n`/export/home/jwalton/Build-Scripts/git-2.21.0/templates'\ngmake -C t/ all\ngmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/git-2.21.0/t'\nrm -f -r 'test-results'\nsed: Too many commands, last: s/\\n//\n--- chainlint/arithmetic-expansion.expect 2019-02-24\n11:31:46.000000000 -0500\n+++ chainlinttmp/arithmetic-expansion.actual 2019-03-11\n12:20:16.880610011 -0400\n@@ -1,9 +0,0 @@\n-(\n- foo &&\n- bar=$((42 + 1)) &&\n- baz\n->) &&\n-(\n-?!AMP?! bar=$((42 + 1))\n- baz\n->)\nsed: Too many commands, last: s/\\n//\n--- chainlint/bash-array.expect 2019-02-24 11:31:46.000000000 -0500\n+++ chainlinttmp/bash-array.actual 2019-03-11 12:20:16.902186323 -0400\n@@ -1,10 +0,0 @@\n-(\n- foo &&\n- bar=(gumbo stumbo wumbo) &&\n- baz\n->) &&\n-(\n- foo &&\n- bar=${#bar[@]} &&\n- baz\n->)\nsed: Too many commands, last: s/\\n//\n\nA more complete output is available at https://pastebin.com/gpZMUVmQ,\nbut it is more of the same.\n\nSolaris in a VM sucks. I can provide SSH access to the hardware if\nanyone is interested. It is just an Solaris i86pc on an older Ivy\nBridge.\n\nJeff\n"},{"id":"371193","messageId":"CAPig+cTCOfDCv=L3EWUqgVZx1VCdqt5ZjmSQMwHLX=R4C4hgTA@mail.gmail.com","threadId":"50711","inReplyTo":"CAH8yC8nnFYt0raL+twrG_v8XZn_FDJWyM=tH=QL13Z70_bHRiQ@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-03-11T21:15:01Z","receivedAt":"2019-03-11T21:15:15Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"[cc:+Ævar]\n\nOn Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> I enabled self tests for Solaris. Solaris has some anemic utilities so\n> I put /usr/gnu/bin first on-path.\n\nThe first question is if you are really running GNU 'sed'? My guess is\n\"no, it's still picking up Solaris's 'sed'\".\n\n> gmake -C templates SHELL_PATH='/bin/bash' PERL_PATH='/usr/bin/perl'\n> sed: Too many commands, last: s/\\n//\n> --- chainlint/arithmetic-expansion.expect 2019-02-24\n\nÆvar had taken a stab at getting this to work on Solaris back in\nSeptember 2018. He did fix one or two issues, but the one you report\nhere was not addressed. He ultimately abandoned the attempt, I think,\nafter Junio's reminder that the project had long ago declared\nSolaris's tools as a lost cause. See the relevant messages:\n\nhttp://public-inbox.org/git/87bm9cs5y6.fsf@evledraar.gmail.com/\nhttp://public-inbox.org/git/CAPig+cRrwW662Hzp8V1sH51x8qYxPyqG=VHpOkfd-=RwUSSZbw@mail.gmail.com\nhttp://public-inbox.org/git/877ek0rymz.fsf@evledraar.gmail.com\nhttp://public-inbox.org/git/xmqqzhwv51wy.fsf@gitster-ct.c.googlers.com\n\n> Solaris in a VM sucks. I can provide SSH access to the hardware if\n> anyone is interested. It is just an Solaris i86pc on an older Ivy\n> Bridge.\n\nI wouldn't mind taking a look at it, though I don't promise anything,\nand I suspect the only way forward is by ensuring that the GNU or XPG\ntools are used instead of the Solaris ones.\n"},{"id":"371196","messageId":"CAH8yC8=jtBURyr-mFJSe_cd4zNJfJDt672jZ+izo=z75ZBheug@mail.gmail.com","threadId":"50711","inReplyTo":"CAPig+cTCOfDCv=L3EWUqgVZx1VCdqt5ZjmSQMwHLX=R4C4hgTA@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-11T21:43:55Z","receivedAt":"2019-03-11T21:44:28Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n>\n> [cc:+Ævar]\n>\n> On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > I put /usr/gnu/bin first on-path.\n>\n> The first question is if you are really running GNU 'sed'? My guess is\n> \"no, it's still picking up Solaris's 'sed'\".\n\nI believe so. After modifying PATH, command -v returns:\n\n Solaris tools:\n sed: /usr/gnu/bin/sed\n awk: /usr/gnu/bin/awk\n grep: /usr/gnu/bin/grep\n\n(This was added to my scripts to confirm).\n\nMaybe Git would benefit from SED, AWK and GREP variables like PERL.\n\n> ...\n> > Solaris in a VM sucks. I can provide SSH access to the hardware if\n> > anyone is interested. It is just an Solaris i86pc on an older Ivy\n> > Bridge.\n>\n> I wouldn't mind taking a look at it, though I don't promise anything,\n> and I suspect the only way forward is by ensuring that the GNU or XPG\n> tools are used instead of the Solaris ones.\n\nSend over your authorized_keys. You will connect with:\n\n esunshine@151.196.22.177 -p 1523\n\nI'm in a dynamic IP address block. You will have to ping me on\noccasion to get the updated IP address.\n\nSome other machines you may be interested in:\n\n * PowerMac G5, PPC big-endian with OS X 10.5 (port 1522)\n * MacBook late 2012, x86_64 with OS X 10.9 (port 1524)\n * Intel Goldmont with SHA extensions (port 1526)\n\nAndy Polyakov uses the PowerMac for tuning his ASM used in OpenSSL.\n\nIf you want to speedup SHA-1 (re: unaligned accesses) then try the\nGoldmont machine. SHA-1 runs at 1.8 cycles per byte on Goldmont.\nHere's the compression function ready for copy/paste:\nhttps://github.com/noloader/SHA-Intrinsics\n\nJeff\n"},{"id":"371201","messageId":"20190311222805.GA6215@flurp.local","threadId":"50711","inReplyTo":"CAH8yC8=jtBURyr-mFJSe_cd4zNJfJDt672jZ+izo=z75ZBheug@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-03-11T22:28:05Z","receivedAt":"2019-03-11T22:28:14Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Mar 11, 2019 at 05:43:55PM -0400, Jeffrey Walton wrote:\n> On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > I put /usr/gnu/bin first on-path.\n> >\n> > The first question is if you are really running GNU 'sed'? My guess is\n> > \"no, it's still picking up Solaris's 'sed'\".\n> \n> I believe so. After modifying PATH, command -v returns:\n> \n> Solaris tools:\n> sed: /usr/gnu/bin/sed\n> awk: /usr/gnu/bin/awk\n> grep: /usr/gnu/bin/grep\n> \n> (This was added to my scripts to confirm).\n\nCan you doubly confirm by having t/Makefile tell us itself? I ask because that error message in your original email is exactly the error spit out by Solaris 'sed'. Perhaps apply this patch:\n\ndiff --git a/t/Makefile b/t/Makefile\nindex c83fd18861..d22eff4463 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -70,6 +70,7 @@ clean-chainlint:\n \n check-chainlint:\n \t@mkdir -p '$(CHAINLINTTMP_SQ)' && \\\n+\tprintf 'SED VERSION: ' && sed --version && \\\n \terr=0 && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n \t\t$(CHAINLINT) Maybe Git would benefit from SED, AWK and GREP variables like PERL.\n\nVery possibly.\n"},{"id":"371205","messageId":"CAH8yC8=YrZR=j9R8Ae+O2UzfxB_Hc1Osu68T-M1mczHiEasfJQ@mail.gmail.com","threadId":"50711","inReplyTo":"CAPig+cTCOfDCv=L3EWUqgVZx1VCdqt5ZjmSQMwHLX=R4C4hgTA@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-11T23:52:54Z","receivedAt":"2019-03-11T23:53:25Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n>\n> [cc:+Ævar]\n>\n> On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > I put /usr/gnu/bin first on-path.\n>\n> The first question is if you are really running GNU 'sed'? My guess is\n> \"no, it's still picking up Solaris's 'sed'\".\n\nYeah, you're right:\n\n$ make test V=1\n...\n\ngmake -C t/ all\ngmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/git-2.21.0/t'\nrm -f -r 'test-results'\nSED VERSION: sed: illegal option -- version\n\nI wonder why ... Let me check if I am exporting PATH.\n\nJeff\n"},{"id":"371206","messageId":"CAH8yC8kkzOwCqN_qruQgAWbTMKzZ_CenrD_etFwZNm=JgJZ7oA@mail.gmail.com","threadId":"50711","inReplyTo":"CAH8yC8=YrZR=j9R8Ae+O2UzfxB_Hc1Osu68T-M1mczHiEasfJQ@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T00:18:26Z","receivedAt":"2019-03-12T00:18:59Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 7:52 PM Jeffrey Walton wrote:\n>\n> On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> >\n> > [cc:+Ævar]\n> >\n> > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > I put /usr/gnu/bin first on-path.\n> >\n> > The first question is if you are really running GNU 'sed'? My guess is\n> > \"no, it's still picking up Solaris's 'sed'\".\n>\n> Yeah, you're right:\n>\n> $ make test V=1\n> ...\n>\n> gmake -C t/ all\n> gmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/git-2.21.0/t'\n> rm -f -r 'test-results'\n> SED VERSION: sed: illegal option -- version\n>\n> I wonder why ... Let me check if I am exporting PATH.\n\nI was not exporting PATH, so that was my mistake.\n\nHowever, PATH is now exported but I get the same result.\n\nIs the build system clearing the environment?\n\nJeff\n"},{"id":"371209","messageId":"CAH8yC8kHhhWA+ONv17LdLpvy6ubH3CTvJ6=ehP5MQLi_=n1=bQ@mail.gmail.com","threadId":"50711","inReplyTo":"20190311222805.GA6215@flurp.local","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T01:07:44Z","receivedAt":"2019-03-12T01:08:18Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 6:28 PM Eric Sunshine wrote:\n>\n> On Mon, Mar 11, 2019 at 05:43:55PM -0400, Jeffrey Walton wrote:\n> > On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> > > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > > I put /usr/gnu/bin first on-path.\n> > >\n> > > The first question is if you are really running GNU 'sed'? My guess is\n> > > \"no, it's still picking up Solaris's 'sed'\".\n> >\n> > I believe so. After modifying PATH, command -v returns:\n> >\n> > Solaris tools:\n> > sed: /usr/gnu/bin/sed\n> > awk: /usr/gnu/bin/awk\n> > grep: /usr/gnu/bin/grep\n> >\n> > (This was added to my scripts to confirm).\n> ...\n>\n> > Maybe Git would benefit from SED, AWK and GREP variables like PERL.\n>\n> Very possibly.\n\nAnother potential workaround is to use Perl. Perl is already a\nprerequisite, it get passed to the test gear through PERL_PATH, and it\navoids Solaris' anemic tools.\n\nSadly my Perl sucks. You would be disappointed if I tried to whip up a patch.\n\nJeff\n"},{"id":"371211","messageId":"CAH8yC8=PKro1_WW=wJQoVHC88cziJPfgkHh6jNKOUxeFn3KVsQ@mail.gmail.com","threadId":"50711","inReplyTo":"CAH8yC8kHhhWA+ONv17LdLpvy6ubH3CTvJ6=ehP5MQLi_=n1=bQ@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T01:55:23Z","receivedAt":"2019-03-12T01:55:58Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 9:07 PM Jeffrey Walton wrote:\n>\n> On Mon, Mar 11, 2019 at 6:28 PM Eric Sunshine wrote:\n> >\n> > On Mon, Mar 11, 2019 at 05:43:55PM -0400, Jeffrey Walton wrote:\n> > > On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> > > > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > > > I put /usr/gnu/bin first on-path.\n> > > >\n> > > > The first question is if you are really running GNU 'sed'? My guess is\n> > > > \"no, it's still picking up Solaris's 'sed'\".\n> > >\n> > > I believe so. After modifying PATH, command -v returns:\n> > >\n> > > Solaris tools:\n> > > sed: /usr/gnu/bin/sed\n> > > awk: /usr/gnu/bin/awk\n> > > grep: /usr/gnu/bin/grep\n> > >\n> > > (This was added to my scripts to confirm).\n> > ...\n> >\n> > > Maybe Git would benefit from SED, AWK and GREP variables like PERL.\n> >\n> > Very possibly.\n>\n> Another potential workaround is to use Perl. Perl is already a\n> prerequisite, it get passed to the test gear through PERL_PATH, and it\n> avoids Solaris' anemic tools.\n>\n> Sadly my Perl sucks. You would be disappointed if I tried to whip up a patch.\n\nOh man, you're using GNU make. I thought Git was using that anemic\nPosix Make. See attached.\n\nI think Solaris provides an older gawk. Is this an easier problem:\n\n awk: chainlint.sed:88: :squash\n awk: chainlint.sed:88: ^ syntax error\n awk: chainlint.sed:91: s/\\\\\\n//\n awk: chainlint.sed:91: ^ backslash not last character on line\n Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\n Usage: awk [POSIX or GNU style options] [--] 'program' file ...\n\nJeff\n\n\n--- t/Makefile\n+++ t/Makefile\n@@ -11,11 +11,25 @@\n TEST_SHELL_PATH ?= $(SHELL_PATH)\n PERL_PATH ?= /usr/bin/perl\n TAR ?= $(TAR)\n+AWK ?= $(AWK)\n+SED ?= $(SED)\n+GREP ?= $(GREP)\n RM ?= rm -f\n PROVE ?= prove\n DEFAULT_TEST_TARGET ?= test\n TEST_LINT ?= test-lint\n \n+# Fix Solaris tools. GNU tools at /usr/gnu/bin. Posix tools /usr/xpg4/bin.\n+ifneq ($(wildcard /usr/gnu/bin/grep),)\n+ GREP := /usr/gnu/bin/grep\n+endif\n+ifneq ($(wildcard /usr/gnu/bin/sed),)\n+ SED := /usr/gnu/bin/sed\n+endif\n+ifneq ($(wildcard /usr/gnu/bin/awk),)\n+ SED := /usr/gnu/bin/awk\n+endif\n+\n ifdef TEST_OUTPUT_DIRECTORY\n TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results\n CHAINLINTTMP = $(TEST_OUTPUT_DIRECTORY)/chainlinttmp\n@@ -35,7 +49,7 @@\n TGITWEB = $(sort $(wildcard t95[0-9][0-9]-*.sh))\n THELPERS = $(sort $(filter-out $(T),$(wildcard *.sh)))\n CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test)))\n-CHAINLINT = sed -f chainlint.sed\n+CHAINLINT = $(SED) -f chainlint.sed\n \n all: $(DEFAULT_TEST_TARGET)\n \n@@ -44,8 +58,8 @@\n \n failed:\n \t@failed=$$(cd '$(TEST_RESULTS_DIRECTORY_SQ)' && \\\n-\t\tgrep -l '^failed [1-9]' *.counts | \\\n-\t\tsed -n 's/\\.counts$$/.sh/p') && \\\n+\t\t$(GREP) -l '^failed [1-9]' *.counts | \\\n+\t\t$(SED) -n 's/\\.counts$$/.sh/p') && \\\n \ttest -z \"$$failed\" || $(MAKE) $$failed\n \n prove: pre-clean check-chainlint $(TEST_LINT)\n@@ -73,7 +87,7 @@\n \terr=0 && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n \t\t$(CHAINLINT) '$(CHAINLINTTMP_SQ)'/$$i.actual && \\\n+\t\t$(SED) -e '/^# LINT: /d' >'$(CHAINLINTTMP_SQ)'/$$i.actual && \\\n \t\tdiff -u chainlint/$$i.expect '$(CHAINLINTTMP_SQ)'/$$i.actual || err=1; \\\n \tdone && exit $$err\n \n@@ -81,7 +95,7 @@\n \ttest-lint-filenames\n \n test-lint-duplicates:\n-\t@dups=`echo $(T) | tr ' ' '\\n' | sed 's/-.*//' | sort | uniq -d` && \\\n+\t@dups=`echo $(T) | tr ' ' '\\n' | $(SED) 's/-.*//' | sort | uniq -d` && \\\n \t\ttest -z \"$$dups\" || { \\\n \t\techo >&2 \"duplicate test numbers:\" $$dups; exit 1; }\n \n@@ -97,7 +111,7 @@\n \t@# We do *not* pass a glob to ls-files but use grep instead, to catch\n \t@# non-ASCII characters (which are quoted within double-quotes)\n \t@bad=\"$$(git -c core.quotepath=true ls-files 2>/dev/null | \\\n-\t\t\tgrep '[\"*:<>?\\\\|]')\"; \\\n+\t\t\t$(GREP) '[\"*:<>?\\\\|]')\"; \\\n \t\ttest -z \"$$bad\" || { \\\n \t\techo >&2 \"non-portable file name(s): $$bad\"; exit 1; }\n \n"},{"id":"371212","messageId":"CAH8yC8kD3XkYP2o6k3ioCcNB_+wwQ=H2=V03z7Han1K8t2aUsg@mail.gmail.com","threadId":"50711","inReplyTo":"CAH8yC8=PKro1_WW=wJQoVHC88cziJPfgkHh6jNKOUxeFn3KVsQ@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T02:11:41Z","receivedAt":"2019-03-12T02:12:15Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 9:55 PM Jeffrey Walton wrote:\n>\n> On Mon, Mar 11, 2019 at 9:07 PM Jeffrey Walton wrote:\n> >\n> > On Mon, Mar 11, 2019 at 6:28 PM Eric Sunshine wrote:\n> > >\n> > > On Mon, Mar 11, 2019 at 05:43:55PM -0400, Jeffrey Walton wrote:\n> > > > On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> > > > > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > > > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > > > > I put /usr/gnu/bin first on-path.\n> > > > >\n> > > > > The first question is if you are really running GNU 'sed'? My guess is\n> > > > > \"no, it's still picking up Solaris's 'sed'\".\n> > > >\n> > > > I believe so. After modifying PATH, command -v returns:\n> > > >\n> > > > Solaris tools:\n> > > > sed: /usr/gnu/bin/sed\n> > > > awk: /usr/gnu/bin/awk\n> > > > grep: /usr/gnu/bin/grep\n> > > >\n> > > > (This was added to my scripts to confirm).\n> > > ...\n> > >\n> > > > Maybe Git would benefit from SED, AWK and GREP variables like PERL.\n> > >\n> > > Very possibly.\n> >\n> > Another potential workaround is to use Perl. Perl is already a\n> > prerequisite, it get passed to the test gear through PERL_PATH, and it\n> > avoids Solaris' anemic tools.\n> >\n> > Sadly my Perl sucks. You would be disappointed if I tried to whip up a patch.\n>\n> Oh man, you're using GNU make. I thought Git was using that anemic\n> Posix Make. See attached.\n>\n> I think Solaris provides an older gawk. Is this an easier problem:\n>\n> awk: chainlint.sed:88: :squash\n> awk: chainlint.sed:88: ^ syntax error\n> awk: chainlint.sed:91: s/\\\\\\n//\n> awk: chainlint.sed:91: ^ backslash not last character on line\n> Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\n> Usage: awk [POSIX or GNU style options] [--] 'program' file ...\n\nMy bad , there was a typo... 'awk' got assigned to SED variable.\n\nThis patch works as expected.\n\nJeff\n\n\ndiff --git a/t/Makefile b/t/Makefile\nindex c83fd18861..10abb1c34c 100644\n--- a/t/Makefile\n+++ b/t/Makefile\n@@ -11,11 +11,25 @@ SHELL_PATH ?= $(SHELL)\n TEST_SHELL_PATH ?= $(SHELL_PATH)\n PERL_PATH ?= /usr/bin/perl\n TAR ?= $(TAR)\n+AWK ?= $(AWK)\n+SED ?= $(SED)\n+GREP ?= $(GREP)\n RM ?= rm -f\n PROVE ?= prove\n DEFAULT_TEST_TARGET ?= test\n TEST_LINT ?= test-lint\n \n+# Fix Solaris tools. These are Posix. GNU tools located at /usr/gnu/bin.\n+ifneq ($(wildcard /usr/gnu/bin/grep),)\n+ GREP := /usr/gnu/bin/grep\n+endif\n+ifneq ($(wildcard /usr/gnu/bin/sed),)\n+ SED := /usr/gnu/bin/sed\n+endif\n+ifneq ($(wildcard /usr/gnu/bin/awk),)\n+ SED := /usr/gnu/bin/awk\n+endif\n+\n ifdef TEST_OUTPUT_DIRECTORY\n TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results\n CHAINLINTTMP = $(TEST_OUTPUT_DIRECTORY)/chainlinttmp\n@@ -35,7 +49,7 @@ T = $(sort $(wildcard t[0-9][0-9][0-9][0-9]-*.sh))\n TGITWEB = $(sort $(wildcard t95[0-9][0-9]-*.sh))\n THELPERS = $(sort $(filter-out $(T),$(wildcard *.sh)))\n CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.test)))\n-CHAINLINT = sed -f chainlint.sed\n+CHAINLINT = $(SED) -f chainlint.sed\n \n all: $(DEFAULT_TEST_TARGET)\n \n@@ -44,8 +58,8 @@ test: pre-clean check-chainlint $(TEST_LINT)\n \n failed:\n \t@failed=$$(cd '$(TEST_RESULTS_DIRECTORY_SQ)' && \\\n-\t\tgrep -l '^failed [1-9]' *.counts | \\\n-\t\tsed -n 's/\\.counts$$/.sh/p') && \\\n+\t\t$(GREP) -l '^failed [1-9]' *.counts | \\\n+\t\t$(SED) -n 's/\\.counts$$/.sh/p') && \\\n \ttest -z \"$$failed\" || $(MAKE) $$failed\n \n prove: pre-clean check-chainlint $(TEST_LINT)\n@@ -73,7 +87,7 @@ check-chainlint:\n \terr=0 && \\\n \tfor i in $(CHAINLINTTESTS); do \\\n \t\t$(CHAINLINT) '$(CHAINLINTTMP_SQ)'/$$i.actual && \\\n+\t\t$(SED) -e '/^# LINT: /d' >'$(CHAINLINTTMP_SQ)'/$$i.actual && \\\n \t\tdiff -u chainlint/$$i.expect '$(CHAINLINTTMP_SQ)'/$$i.actual || err=1; \\\n \tdone && exit $$err\n \n@@ -81,7 +95,7 @@ test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax \\\n \ttest-lint-filenames\n \n test-lint-duplicates:\n-\t@dups=`echo $(T) | tr ' ' '\\n' | sed 's/-.*//' | sort | uniq -d` && \\\n+\t@dups=`echo $(T) | tr ' ' '\\n' | $(SED) 's/-.*//' | sort | uniq -d` && \\\n \t\ttest -z \"$$dups\" || { \\\n \t\techo >&2 \"duplicate test numbers:\" $$dups; exit 1; }\n \n@@ -97,7 +111,7 @@ test-lint-filenames:\n \t@# We do *not* pass a glob to ls-files but use grep instead, to catch\n \t@# non-ASCII characters (which are quoted within double-quotes)\n \t@bad=\"$$(git -c core.quotepath=true ls-files 2>/dev/null | \\\n-\t\t\tgrep '[\"*:<>?\\\\|]')\"; \\\n+\t\t\t$(GREP) '[\"*:<>?\\\\|]')\"; \\\n \t\ttest -z \"$$bad\" || { \\\n \t\techo >&2 \"non-portable file name(s): $$bad\"; exit 1; }\n \n"},{"id":"371213","messageId":"CAH8yC8k8Y96K6qL4DU5QFuXLYpe+xOGoAA98Ahg6M9Ys0ERD9A@mail.gmail.com","threadId":"50711","inReplyTo":"CAH8yC8kD3XkYP2o6k3ioCcNB_+wwQ=H2=V03z7Han1K8t2aUsg@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T02:19:37Z","receivedAt":"2019-03-12T02:20:10Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 10:11 PM Jeffrey Walton wrote:\n>\n> On Mon, Mar 11, 2019 at 9:55 PM Jeffrey Walton wrote:\n> >\n> > On Mon, Mar 11, 2019 at 9:07 PM Jeffrey Walton wrote:\n> > >\n> > > On Mon, Mar 11, 2019 at 6:28 PM Eric Sunshine wrote:\n> > > >\n> > > > On Mon, Mar 11, 2019 at 05:43:55PM -0400, Jeffrey Walton wrote:\n> > > > > On Mon, Mar 11, 2019 at 5:15 PM Eric Sunshine wrote:\n> > > > > > On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n> > > > > > > I enabled self tests for Solaris. Solaris has some anemic utilities so\n> > > > > > > I put /usr/gnu/bin first on-path.\n> > > > > >\n> > > > > > The first question is if you are really running GNU 'sed'? My guess is\n> > > > > > \"no, it's still picking up Solaris's 'sed'\".\n> > > > >\n> > > > > I believe so. After modifying PATH, command -v returns:\n> > > > >\n> > > > > Solaris tools:\n> > > > > sed: /usr/gnu/bin/sed\n> > > > > awk: /usr/gnu/bin/awk\n> > > > > grep: /usr/gnu/bin/grep\n> > > > >\n> > > > > (This was added to my scripts to confirm).\n> > > > ...\n> > > >\n> > > > > Maybe Git would benefit from SED, AWK and GREP variables like PERL.\n> > > >\n> > > > Very possibly.\n> > >\n> > > Another potential workaround is to use Perl. Perl is already a\n> > > prerequisite, it get passed to the test gear through PERL_PATH, and it\n> > > avoids Solaris' anemic tools.\n> > >\n> > > Sadly my Perl sucks. You would be disappointed if I tried to whip up a patch.\n> >\n> > Oh man, you're using GNU make. I thought Git was using that anemic\n> > Posix Make. See attached.\n> >\n> > I think Solaris provides an older gawk. Is this an easier problem:\n> >\n> > awk: chainlint.sed:88: :squash\n> > awk: chainlint.sed:88: ^ syntax error\n> > awk: chainlint.sed:91: s/\\\\\\n//\n> > awk: chainlint.sed:91: ^ backslash not last character on line\n> > Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\n> > Usage: awk [POSIX or GNU style options] [--] 'program' file ...\n>\n> My bad , there was a typo... 'awk' got assigned to SED variable.\n>\n> This patch works as expected.\n\nhttps://github.com/git/git/pull/582\n"},{"id":"371214","messageId":"20190312024542.GA7411@flurp.local","threadId":"50711","inReplyTo":"CAH8yC8kD3XkYP2o6k3ioCcNB_+wwQ=H2=V03z7Han1K8t2aUsg@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2019-03-12T02:45:42Z","receivedAt":"2019-03-12T02:45:52Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Mar 11, 2019 at 10:11:41PM -0400, Jeffrey Walton wrote:\n> On Mon, Mar 11, 2019 at 9:55 PM Jeffrey Walton wrote:\n> > Oh man, you're using GNU make. I thought Git was using that anemic\n> > Posix Make. See attached.\n> >\n> > I think Solaris provides an older gawk. Is this an easier problem:\n> >\n> > awk: chainlint.sed:88: :squash\n> > awk: chainlint.sed:88: ^ syntax error\n> > awk: chainlint.sed:91: s/\\\\\\n//\n> > awk: chainlint.sed:91: ^ backslash not last character on line\n> > Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\n> > Usage: awk [POSIX or GNU style options] [--] 'program' file ...\n> \n> My bad , there was a typo... 'awk' got assigned to SED variable.\n> \n> This patch works as expected.\n>\n> diff --git a/t/Makefile b/t/Makefile\n> @@ -11,11 +11,25 @@ SHELL_PATH ?= $(SHELL)\n> TAR ?= $(TAR)\n> +AWK ?= $(AWK)\n> +SED ?= $(SED)\n> +GREP ?= $(GREP)\n> \n> +# Fix Solaris tools. These are Posix. GNU tools located at /usr/gnu/bin.\n> +ifneq ($(wildcard /usr/gnu/bin/grep),)\n> + GREP := /usr/gnu/bin/grep\n> +endif\n> +ifneq ($(wildcard /usr/gnu/bin/sed),)\n> + SED := /usr/gnu/bin/sed\n> +endif\n> +ifneq ($(wildcard /usr/gnu/bin/awk),)\n> + SED := /usr/gnu/bin/awk\n> +endif\n\nI think the last assignment ought to be \"AWK := ...\".\n\nAnyhow, this sort of platform-specific tool customization is typically done by config.mak.uname in the top-level Git directory. In fact, there's already a section for SunOS:\n\n ifeq ($(uname_S),SunOS)\n ...\n SANE_TOOL_PATH = /usr/xpg6/bin:/usr/xpg4/bin\n ...\n\nPrepending /usr/gnu/bin to SANE_TOOL_PATH might be a good idea as a first step toward fixing the problem you're seeing on Solaris, however, as Ævar mentioned in [1], SANE_TOOL_PATH isn't presently consulted when running tests. But, as he suggested, fixing the tests to respect SANE_TOOL_PATH might be a good solution overall.\n\nSo, rather than making platform-specific customizations to t/Makefile, an arguably better solution would be to update config.mak.uname to add /usr/gnu/bin to SANE_TOOL_PATH and then update the test system to respect that value (thus, these GREP, SED, AWK specializations can be avoided).\n\n[1]: http://public-inbox.org/git/877ek0rymz.fsf@evledraar.gmail.com/\n"},{"id":"371218","messageId":"CAH8yC8=gsuknMOAYyjVDyZENDiJoRia+MnwhsMhZdcnfuO=cbw@mail.gmail.com","threadId":"50711","inReplyTo":"20190312024542.GA7411@flurp.local","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Jeffrey Walton","fromEmail":"noloader@gmail.com","sentAt":"2019-03-12T03:57:53Z","receivedAt":"2019-03-12T03:58:27Z","isPatch":false,"sender":{"key":"noloader@gmail.com","avatar":null},"body":"On Mon, Mar 11, 2019 at 10:45 PM Eric Sunshine wrote:\n>\n> On Mon, Mar 11, 2019 at 10:11:41PM -0400, Jeffrey Walton wrote:\n> > On Mon, Mar 11, 2019 at 9:55 PM Jeffrey Walton wrote:\n> > > Oh man, you're using GNU make. I thought Git was using that anemic\n> > > Posix Make. See attached.\n> > >\n> > > I think Solaris provides an older gawk. Is this an easier problem:\n> > >\n> > > awk: chainlint.sed:88: :squash\n> > > awk: chainlint.sed:88: ^ syntax error\n> > > awk: chainlint.sed:91: s/\\\\\\n//\n> > > awk: chainlint.sed:91: ^ backslash not last character on line\n> > > Usage: awk [POSIX or GNU style options] -f progfile [--] file ...\n> > > Usage: awk [POSIX or GNU style options] [--] 'program' file ...\n> >\n> > My bad , there was a typo... 'awk' got assigned to SED variable.\n> >\n> > This patch works as expected.\n> >\n> > diff --git a/t/Makefile b/t/Makefile\n> > @@ -11,11 +11,25 @@ SHELL_PATH ?= $(SHELL)\n> > TAR ?= $(TAR)\n> > +AWK ?= $(AWK)\n> > +SED ?= $(SED)\n> > +GREP ?= $(GREP)\n> >\n> > +# Fix Solaris tools. These are Posix. GNU tools located at /usr/gnu/bin.\n> > +ifneq ($(wildcard /usr/gnu/bin/grep),)\n> > + GREP := /usr/gnu/bin/grep\n> > +endif\n> > +ifneq ($(wildcard /usr/gnu/bin/sed),)\n> > + SED := /usr/gnu/bin/sed\n> > +endif\n> > +ifneq ($(wildcard /usr/gnu/bin/awk),)\n> > + SED := /usr/gnu/bin/awk\n> > +endif\n>\n> I think the last assignment ought to be \"AWK := ...\".\n>\n> Anyhow, this sort of platform-specific tool customization is typically done by config.mak.uname in the top-level Git directory. In fact, there's already a section for SunOS:\n>\n> ifeq ($(uname_S),SunOS)\n> ...\n> SANE_TOOL_PATH = /usr/xpg6/bin:/usr/xpg4/bin\n> ...\n>\n> Prepending /usr/gnu/bin to SANE_TOOL_PATH might be a good idea as a first step toward fixing the problem you're seeing on Solaris, however, as Ćvar mentioned in [1], SANE_TOOL_PATH isn't presently consulted when running tests. But, as he suggested, fixing the tests to respect SANE_TOOL_PATH might be a good solution overall.\n>\n> So, rather than making platform-specific customizations to t/Makefile, an arguably better solution would be to update config.mak.uname to add /usr/gnu/bin to SANE_TOOL_PATH and then update the test system to respect that value (thus, these GREP, SED, AWK specializations can be avoided).\n\nI may have been blowing away SANE_BUILD_PATH.\n\nMy build script calls:\n\n SANE_BUILD_PATH=\" \\\n PKG_CONFIG_PATH=\"...\" \\\n ... \\\n./configure \\\n --prefix= \\\n --libdir=\"...\" \\\n ...\n\nI set SANE_BUILD_PATH in the environment because I want Git to use the\nnew tools being built for testing.\n\nOr, if Git is blowing away my SANE_BUILD_PATH , then that might\nexplain the encoding failures. The updated tools, like new iConv and\nUnistring, are not on PATH. The updated tools are located at\n/var/sanitize/bin, and are passed through SANE_BUILD_PATH .\n\nThis might also explain the intermittent crashes I see. Old tools from\n/usr/bin are used, but new libraries from /var/sanitize/lib are used\nbecause of LD_LIBRARY_PATH. Gotta love those fucking path problems\nthat have plagued Linux for the last 30 years. The idiot who figured\nit was a good idea to compile and link against one library, and then\nload the wrong library at runtime, should get a platinum Darwin award.\n\nIt may be better to set SANE_TOOL_PATH something like:\n\n SANE_TOOL_PATH = \"$SANE_TOOL_PATH:/usr/xpg6/bin:/usr/xpg4/bin\"\n\nJeff\n"},{"id":"371229","messageId":"87zhq0bgtu.fsf@evledraar.gmail.com","threadId":"50711","inReplyTo":"CAPig+cTCOfDCv=L3EWUqgVZx1VCdqt5ZjmSQMwHLX=R4C4hgTA@mail.gmail.com","subject":"Re: Solaris and sed: Too many commands, last: s/\\n//","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-03-12T09:08:29Z","receivedAt":"2019-03-12T09:08:34Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Mon, Mar 11 2019, Eric Sunshine wrote:\n\n> [cc:+Ævar]\n>\n> On Mon, Mar 11, 2019 at 4:32 PM Jeffrey Walton wrote:\n>> I enabled self tests for Solaris. Solaris has some anemic utilities so\n>> I put /usr/gnu/bin first on-path.\n>\n> The first question is if you are really running GNU 'sed'? My guess is\n> \"no, it's still picking up Solaris's 'sed'\".\n>\n>> gmake -C templates SHELL_PATH='/bin/bash' PERL_PATH='/usr/bin/perl'\n>> sed: Too many commands, last: s/\\n//\n>> --- chainlint/arithmetic-expansion.expect 2019-02-24\n>\n> Ævar had taken a stab at getting this to work on Solaris back in\n> September 2018. He did fix one or two issues, but the one you report\n> here was not addressed. He ultimately abandoned the attempt, I think,\n> after Junio's reminder that the project had long ago declared\n> Solaris's tools as a lost cause. See the relevant messages:\n>\n> http://public-inbox.org/git/87bm9cs5y6.fsf@evledraar.gmail.com/\n> http://public-inbox.org/git/CAPig+cRrwW662Hzp8V1sH51x8qYxPyqG=VHpOkfd-=RwUSSZbw@mail.gmail.com\n> http://public-inbox.org/git/877ek0rymz.fsf@evledraar.gmail.com\n> http://public-inbox.org/git/xmqqzhwv51wy.fsf@gitster-ct.c.googlers.com\n>\n>> Solaris in a VM sucks. I can provide SSH access to the hardware if\n>> anyone is interested. It is just an Solaris i86pc on an older Ivy\n>> Bridge.\n>\n> I wouldn't mind taking a look at it, though I don't promise anything,\n> and I suspect the only way forward is by ensuring that the GNU or XPG\n> tools are used instead of the Solaris ones.\n\nFWIW I gave up on Solaris sed after trying to fix it the last time\naround.\n\nIf you need another box to test it on you can apply for a GCC Farm\naccount: https://cfarm.tetaneutral.net/machines/list/\n"}]}