[PATCH v2 0/6] chainlint: improve robustness against "unusual" shell coding

This is a re-roll of [1] which improves chainlint's robustness in the face of unusual shell coding such as in contrib/subtree/t7900 which triggered a false-positive[2].

Changes since v1:

* recognize lowercase in here-doc tag names (in addition to uppercase)
* recognize 'quoted' here-doc tag names (in addition to \escaped)

Patch 2/6 is new. Range-diff below.

[1]: https://public-inbox.org/git/20180807082135.60913-1-sunshine@sunshineco.com/ [2]: https://public-inbox.org/git/20180730181356.GA156463@aiede.svl.corp.google.com/

Eric Sunshine (6):
  chainlint: match arbitrary here-docs tags rather than hard-coded names
  chainlint: match 'quoted' here-doc tags
  chainlint: recognize multi-line $(...) when command cuddled with "$("
  chainlint: let here-doc and multi-line string commence on same line
  chainlint: recognize multi-line quoted strings more robustly
  chainlint: add test of pathological case which triggered false
    positive

 t/chainlint.sed                               | 98 ++++++++++++-------
 t/chainlint/here-doc-close-subshell.expect    |  2 +
 t/chainlint/here-doc-close-subshell.test      |  5 +
 .../here-doc-multi-line-command-subst.expect  |  5 +
 .../here-doc-multi-line-command-subst.test    |  9 ++
 t/chainlint/here-doc-multi-line-string.expect |  4 +
 t/chainlint/here-doc-multi-line-string.test   |  8 ++
 t/chainlint/here-doc.expect                   |  4 +
 t/chainlint/here-doc.test                     | 14 +++
 ...ti-line-nested-command-substitution.expect | 11 ++-
 ...ulti-line-nested-command-substitution.test | 11 ++-
 t/chainlint/multi-line-string.expect          | 10 +-
 t/chainlint/multi-line-string.test            | 12 +++
 t/chainlint/nested-here-doc.expect            |  2 +
 t/chainlint/nested-here-doc.test              | 10 ++
 t/chainlint/subshell-here-doc.expect          |  5 +
 t/chainlint/subshell-here-doc.test            | 12 +++
 t/chainlint/t7900-subtree.expect              | 10 ++
 t/chainlint/t7900-subtree.test                | 22 +++++
 19 files changed, 213 insertions(+), 41 deletions(-)
 create mode 100644 t/chainlint/here-doc-close-subshell.expect
 create mode 100644 t/chainlint/here-doc-close-subshell.test
 create mode 100644 t/chainlint/here-doc-multi-line-command-subst.expect
 create mode 100644 t/chainlint/here-doc-multi-line-command-subst.test
 create mode 100644 t/chainlint/here-doc-multi-line-string.expect
 create mode 100644 t/chainlint/here-doc-multi-line-string.test
 create mode 100644 t/chainlint/t7900-subtree.expect
 create mode 100644 t/chainlint/t7900-subtree.test

Range-diff against v1:
1:  d9ed356b97 ! 1:  f4c606cb7a chainlint: match arbitrary here-docs tags rather than hard-coded names
    @@ -64,8 +64,8 @@
     -/<<[ 	]*[-\\]*EOF[ 	]*/ {
     -	s/[ 	]*<<[ 	]*[-\\]*EOF//
     -	h
    -+/<<[ 	]*[-\\]*[A-Z0-9_][A-Z0-9_]*/ {
    -+	s/^\(.*\)<<[ 	]*[-\\]*\([A-Z0-9_][A-Z0-9_]*\)/<\2>\1<</
    ++/<<[ 	]*[-\\]*[A-Za-z0-9_]/ {
    ++	s/^\(.*\)<<[ 	]*[-\\]*\([A-Za-z0-9_][A-Za-z0-9_]*\)/<\2>\1<</
     +	s/[ 	]*<<//
      	:hereslurp
      	N
    @@ -88,7 +88,7 @@
     -/<<[ 	]*[-\\]*EOF/bheredoc
     -/<<[ 	]*[-\\]*EOT/bheredoc
     -/<<[ 	]*[-\\]*INPUT_END/bheredoc
    -+/<<[ 	]*[-\\]*[A-Z0-9_][A-Z0-9_]*/bheredoc
    ++/<<[ 	]*[-\\]*[A-Za-z0-9_]/bheredoc
      # comment or empty line -- discard since final non-comment, non-empty line
      # before closing ")", "done", "elsif", "else", or "fi" will need to be
      # re-visited to drop "suspect" marking since final line of those constructs
    @@ -104,7 +104,7 @@
     -/EOF/{ s/[ 	]*<<[ 	]*[-\\]*EOF//; s/^/EOF/; }
     -/EOT/{ s/[ 	]*<<[ 	]*[-\\]*EOT//; s/^/EOT/; }
     -/INPUT_END/{ s/[ 	]*<<[ 	]*[-\\]*INPUT_END//; s/^/INPUT_END/; }
    -+s/^\(.*\)<<[ 	]*[-\\]*\([A-Z0-9_][A-Z0-9_]*\)/<\2>\1<</
    ++s/^\(.*\)<<[ 	]*[-\\]*\([A-Za-z0-9_][A-Za-z0-9_]*\)/<\2>\1<</
     +s/[ 	]*<<//
      :hereslurpsub
      N
    @@ -143,11 +143,11 @@
      EOF
      
     +# LINT: swallow here-doc with arbitrary tag
    -+cat <<-ARBITRARY >foo &&
    ++cat <<-Arbitrary_Tag_42 >foo &&
     +snoz
     +boz
     +woz
    -+ARBITRARY
    ++Arbitrary_Tag_42
     +
      # LINT: swallow here-doc (EOF is last line of test)
      horticulture <<\EOF
-:  ---------- > 2:  61c0d9c979 chainlint: match 'quoted' here-doc tags
2:  d63920cdd5 = 3:  b97a05aa9c chainlint: recognize multi-line $(...) when command cuddled with "$("
3:  a5078923ef = 4:  9d4d2b6c4d chainlint: let here-doc and multi-line string commence on same line
4:  4841ca6ac9 = 5:  0a7f533889 chainlint: recognize multi-line quoted strings more robustly
5:  e9eb45a2dc = 6:  d42bad0323 chainlint: add test of pathological case which triggered false positive