git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 12/15] chainlint.sed: don't mistake `<< word` in string as here-doc operator

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Dec 13, 2021, 06:30 UTC
Message-ID
<20211213063059.19424-13-sunshine@sunshineco.com>
In-Reply-To
<20211213063059.19424-1-sunshine@sunshineco.com>

Tighten here-doc recognition to prevent it from being fooled by text which looks like a here-doc operator but happens merely to be the content of a string, such as this real-world case from t7201:

    echo "<<<<<<< ours" &&
    echo ourside &&
    echo "=======" &&
    echo theirside &&
    echo ">>>>>>> theirs"

This problem went unnoticed because chainlint.sed is not a real parser, but rather applies heuristics to pretend to understand shell code. In this case, it saw what it thought was a here-doc operator (`<< ours`), and fell off the end of the test looking for the closing tag "ours" which it never found, thus swallowed the remainder of the test without checking it for &&-chain breakage.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.sed                |  8 ++++++--
 t/chainlint/not-heredoc.expect | 14 ++++++++++++++
 t/chainlint/not-heredoc.test   | 16 ++++++++++++++++
 3 files changed, 36 insertions(+), 2 deletions(-)
 create mode 100644 t/chainlint/not-heredoc.expect
 create mode 100644 t/chainlint/not-heredoc.test
diff --git a/t/chainlint.sed b/t/chainlint.sed
index b382746526..2f786f890d 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -89,6 +89,7 @@
 # here-doc -- swallow it to avoid false hits within its body (but keep the
 # command to which it was attached)
 /<<-*[ 	]*[\\'"]*[A-Za-z0-9_]/ {
+	/"[^"]*<<[^"]*"/bnotdoc
 	s/^\(.*\)<<-*[ 	]*[\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\2>\1<</
 	s/[ 	]*<<//
 	:hered
@@ -100,6 +101,7 @@
 	s/^<[^>]*>//
 	s/\n.*$//
 }
+:notdoc
 
 # one-liner "(...) &&"
 /^[ 	]*!*[ 	]*(..*)[ 	]*&&[ 	]*$/boneline
@@ -151,8 +153,10 @@ s/.*\n//
 	/"[^'"]*'[^'"]*"/!bsqstr
 }
 :folded
-# here-doc -- swallow it
-/<<-*[ 	]*[\\'"]*[A-Za-z0-9_]/bheredoc
+# here-doc -- swallow it (but not "<<" in a string)
+/<<-*[ 	]*[\\'"]*[A-Za-z0-9_]/{
+	/"[^"]*<<[^"]*"/!bheredoc
+}
 # comment or empty line -- discard since final non-comment, non-empty line
 # before closing ")", "done", "elsif", "else", or "fi" will need to be
 # re-visited to drop "suspect" marking since final line of those constructs
diff --git a/t/chainlint/not-heredoc.expect b/t/chainlint/not-heredoc.expect
new file mode 100644
index 0000000000..2e9bb135fe
--- /dev/null
+++ b/t/chainlint/not-heredoc.expect
@@ -0,0 +1,14 @@
+echo "<<<<<<< ours" &&
+echo ourside &&
+echo "=======" &&
+echo theirside &&
+echo ">>>>>>> theirs" &&
+
+(
+	echo "<<<<<<< ours" &&
+	echo ourside &&
+	echo "=======" &&
+	echo theirside &&
+	echo ">>>>>>> theirs" ?!AMP?!
+	poodle
+) >merged
diff --git a/t/chainlint/not-heredoc.test b/t/chainlint/not-heredoc.test
new file mode 100644
index 0000000000..9aa57346cd
--- /dev/null
+++ b/t/chainlint/not-heredoc.test
@@ -0,0 +1,16 @@
+# LINT: "<< ours" inside string is not here-doc
+echo "<<<<<<< ours" &&
+echo ourside &&
+echo "=======" &&
+echo theirside &&
+echo ">>>>>>> theirs" &&
+
+(
+# LINT: "<< ours" inside string is not here-doc
+	echo "<<<<<<< ours" &&
+	echo ourside &&
+	echo "=======" &&
+	echo theirside &&
+	echo ">>>>>>> theirs"
+	poodle
+) >merged
-- 
2.34.1.397.gfae76fe5da
Previous: Eric SunshineNext: Eric Sunshine
Message 29 of 34 in “generalize chainlint self-tests”
  1. 00/15 generalize chainlint self-testsEric Sunshine, Dec 13, 2021
  2. 01/15 t/chainlint/*.test: don't use invalid shell syntaxEric Sunshine, Dec 13, 2021
  3. 03/15 t/chainlint/*.test: generalize self-test commentaryEric Sunshine, Dec 13, 2021
  4. 02/15 t/chainlint/*.test: fix invalid test cases due to mixing quote typesEric Sunshine, Dec 13, 2021
  5. 04/15 t/chainlint/one-liner: avoid overly intimate chainlint.sed knowledgeEric Sunshine, Dec 13, 2021
  6. 05/15 t/Makefile: optimize chainlint self-testEric Sunshine, Dec 13, 2021
  7. Fabian StelzerDec 13, 2021
  8. Eric SunshineDec 13, 2021
  9. Fabian StelzerDec 13, 2021
  10. Eric SunshineDec 13, 2021
  11. Fabian StelzerDec 13, 2021
  12. Ævar Arnfjörð BjarmasonDec 13, 2021
  13. Eric SunshineDec 13, 2021
  14. Eric SunshineDec 13, 2021
  15. Ævar Arnfjörð BjarmasonDec 13, 2021
  16. Eric SunshineDec 13, 2021
  17. Ævar Arnfjörð BjarmasonDec 16, 2021
  18. Eric SunshineDec 16, 2021
  19. Ævar Arnfjörð BjarmasonDec 16, 2021
  20. t/Makefile: use dependency graph for "check-chainlint"Ævar Arnfjörð Bjarmason, Dec 13, 2021
  21. Eric SunshineDec 14, 2021
  22. Ævar Arnfjörð BjarmasonDec 14, 2021
  23. 06/15 chainlint.sed: improve ?!AMP?! placement accuracyEric Sunshine, Dec 13, 2021
  24. 07/15 chainlint.sed: improve ?!SEMI?! placement accuracyEric Sunshine, Dec 13, 2021
  25. 08/15 chainlint.sed: tolerate harmless ";" at end of last line in blockEric Sunshine, Dec 13, 2021
  26. 09/15 chainlint.sed: drop unnecessary distinction between ?!AMP?! and ?!SEMI?!Eric Sunshine, Dec 13, 2021
  27. 10/15 chainlint.sed: drop subshell-closing ">" annotationEric Sunshine, Dec 13, 2021
  28. 11/15 chainlint.sed: make here-doc "<<-" operator recognition more POSIX-likeEric Sunshine, Dec 13, 2021
  29. 12/15 chainlint.sed: don't mistake `<< word` in string as here-doc operatorEric Sunshine, Dec 13, 2021
  30. 13/15 chainlint.sed: stop throwing away here-doc tagsEric Sunshine, Dec 13, 2021
  31. 14/15 chainlint.sed: swallow comments consistentlyEric Sunshine, Dec 13, 2021
  32. 15/15 chainlint.sed: stop splitting "(..." into separate lines "(" and "..."Eric Sunshine, Dec 13, 2021
  33. Elijah NewrenDec 15, 2021
  34. Eric SunshineDec 15, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.