git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 13/15] chainlint.sed: stop throwing away here-doc tags

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Dec 13, 2021, 06:30 UTC
Message-ID
<20211213063059.19424-14-sunshine@sunshineco.com>
In-Reply-To
<20211213063059.19424-1-sunshine@sunshineco.com>

The purpose of chainlint is to highlight problems it finds in test code by inserting annotations at the location of each problem. Arbitrarily eliding bits of the code it is checking is not helpful, yet this is exactly what chainlint.sed does by cavalierly and unnecessarily dropping the here-doc operator and tag; i.e. `cat <<TAG` becomes simply `cat` in the output. This behavior can make it more difficult for the test writer to align the annotated output of chainlint.sed with the original test code. Address this by retaining here-doc tags.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.sed                               | 24 ++++++++++++-------
 t/chainlint/for-loop.expect                   |  2 +-
 t/chainlint/here-doc-close-subshell.expect    |  2 +-
 .../here-doc-multi-line-command-subst.expect  |  2 +-
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/here-doc.expect                   |  8 +++----
 t/chainlint/if-then-else.expect               |  2 +-
 t/chainlint/nested-here-doc.expect            |  6 ++---
 t/chainlint/subshell-here-doc.expect          | 10 ++++----
 t/chainlint/t7900-subtree.expect              |  4 ++--
 t/chainlint/while-loop.expect                 |  2 +-
 11 files changed, 35 insertions(+), 29 deletions(-)
diff --git a/t/chainlint.sed b/t/chainlint.sed
index 2f786f890d..0e575c0c62 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -62,20 +62,20 @@
 # receives similar treatment.
 #
 # Swallowing here-docs with arbitrary tags requires a bit of finesse. When a
-# line such as "cat <<EOF >out" is seen, the here-doc tag is moved to the front
-# of the line enclosed in angle brackets as a sentinel, giving "<EOF>cat >out".
+# line such as "cat <<EOF" is seen, the here-doc tag is copied to the front of
+# the line enclosed in angle brackets as a sentinel, giving "<EOF>cat <<EOF".
 # As each subsequent line is read, it is appended to the target line and a
 # (whitespace-loose) back-reference match /^<(.*)>\n\1$/ is attempted to see if
 # the content inside "<...>" matches the entirety of the newly-read line. For
 # instance, if the next line read is "some data", when concatenated with the
-# target line, it becomes "<EOF>cat >out\nsome data", and a match is attempted
+# target line, it becomes "<EOF>cat <<EOF\nsome data", and a match is attempted
 # to see if "EOF" matches "some data". Since it doesn't, the next line is
 # attempted. When a line consisting of only "EOF" (and possible whitespace) is
-# encountered, it is appended to the target line giving "<EOF>cat >out\nEOF",
+# encountered, it is appended to the target line giving "<EOF>cat <<EOF\nEOF",
 # in which case the "EOF" inside "<...>" does match the text following the
 # newline, thus the closing here-doc tag has been found. The closing tag line
 # and the "<...>" prefix on the target line are then discarded, leaving just
-# the target line "cat >out".
+# the target line "cat <<EOF".
 #------------------------------------------------------------------------------
 
 # incomplete line -- slurp up next line
@@ -90,8 +90,7 @@
 # command to which it was attached)
 /<<-*[ 	]*[\\'"]*[A-Za-z0-9_]/ {
 	/"[^"]*<<[^"]*"/bnotdoc
-	s/^\(.*\)<<-*[ 	]*[\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\2>\1<</
-	s/[ 	]*<<//
+	s/^\(.*<<-*[ 	]*\)[\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\2>\1\2/
 	:hered
 	N
 	/^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
@@ -238,6 +237,7 @@ s/.*\n//
 :cont
 # retrieve and print previous line
 x
+s/?!HERE?!/<</g
 n
 bslurp
 
@@ -278,8 +278,7 @@ bfolded
 # found here-doc -- swallow it to avoid false hits within its body (but keep
 # the command to which it was attached)
 :heredoc
-s/^\(.*\)<<-*[ 	]*[\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\2>\1<</
-s/[ 	]*<<//
+s/^\(.*\)<<\(-*[ 	]*\)[\\'"]*\([A-Za-z0-9_][A-Za-z0-9_]*\)['"]*/<\3>\1?!HERE?!\2\3/
 :hdocsub
 N
 /^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
@@ -293,6 +292,7 @@ bfolded
 # found "case ... in" -- pass through untouched
 :case
 x
+s/?!HERE?!/<</g
 n
 /^[ 	]*esac/bslurp
 bcase
@@ -320,6 +320,7 @@ bchkchn
 :nest
 x
 :nstslrp
+s/?!HERE?!/<</g
 n
 # closing ")" on own line -- stop nested slurp
 /^[ 	]*)/bnstcl
@@ -342,6 +343,7 @@ bchkchn
 # found multi-line "{...\n...}" block -- pass through untouched
 :block
 x
+s/?!HERE?!/<</g
 n
 # closing "}" -- stop block slurp
 /}/bchkchn
@@ -352,13 +354,17 @@ bblock
 :clssolo
 x
 s/\( ?!AMP?!\)* ?!AMP?!$//
+s/?!HERE?!/<</g
 p
 x
+s/?!HERE?!/<</g
 b
 
 # found closing "...)" -- exit subshell loop
 :close
 x
+s/?!HERE?!/<</g
 p
 x
+s/?!HERE?!/<</g
 b
diff --git a/t/chainlint/for-loop.expect b/t/chainlint/for-loop.expect
index b74df064c5..6671b8cd84 100644
--- a/t/chainlint/for-loop.expect
+++ b/t/chainlint/for-loop.expect
@@ -2,7 +2,7 @@
 	for i in a b c
 	do
 		echo $i ?!AMP?!
-		cat
+		cat <<-EOF
 	done ?!AMP?!
 	for i in a b c; do
 		echo $i &&
diff --git a/t/chainlint/here-doc-close-subshell.expect b/t/chainlint/here-doc-close-subshell.expect
index e748526570..2af9ced71c 100644
--- a/t/chainlint/here-doc-close-subshell.expect
+++ b/t/chainlint/here-doc-close-subshell.expect
@@ -1,2 +1,2 @@
 (
-	cat)
+	cat <<-INPUT)
diff --git a/t/chainlint/here-doc-multi-line-command-subst.expect b/t/chainlint/here-doc-multi-line-command-subst.expect
index f1248f8ade..f8b3aa73c4 100644
--- a/t/chainlint/here-doc-multi-line-command-subst.expect
+++ b/t/chainlint/here-doc-multi-line-command-subst.expect
@@ -1,5 +1,5 @@
 (
-	x=$(bobble &&
+	x=$(bobble <<-END &&
 		wiffle) ?!AMP?!
 	echo $x
 )
diff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect
index 7e883b252e..2578191ca8 100644
--- a/t/chainlint/here-doc-multi-line-string.expect
+++ b/t/chainlint/here-doc-multi-line-string.expect
@@ -1,4 +1,4 @@
 (
-	cat && echo "multi-line	string" ?!AMP?!
+	cat <<-TXT && echo "multi-line	string" ?!AMP?!
 	bap
 )
diff --git a/t/chainlint/here-doc.expect b/t/chainlint/here-doc.expect
index 8449eb2e02..110059ba58 100644
--- a/t/chainlint/here-doc.expect
+++ b/t/chainlint/here-doc.expect
@@ -1,7 +1,7 @@
-boodle wobba        gorgo snoot        wafta snurb &&
+boodle wobba        gorgo snoot        wafta snurb <<EOF &&
 
-cat >foo &&
+cat <<-Arbitrary_Tag_42 >foo &&
 
-cat >boo &&
+cat <<zump >boo &&
 
-horticulture
+horticulture <<EOF
diff --git a/t/chainlint/if-then-else.expect b/t/chainlint/if-then-else.expect
index debcf7b756..44d86c3597 100644
--- a/t/chainlint/if-then-else.expect
+++ b/t/chainlint/if-then-else.expect
@@ -8,7 +8,7 @@
 		echo foo
 	else
 		echo foo &&
-		cat
+		cat <<-EOF
 	fi ?!AMP?!
 	echo poodle
 ) &&
diff --git a/t/chainlint/nested-here-doc.expect b/t/chainlint/nested-here-doc.expect
index 2a51205d32..e3bef63f75 100644
--- a/t/chainlint/nested-here-doc.expect
+++ b/t/chainlint/nested-here-doc.expect
@@ -1,7 +1,7 @@
-cat >foop &&
+cat <<ARBITRARY >foop &&
 
 (
-	cat &&
-	cat ?!AMP?!
+	cat <<-INPUT_END &&
+	cat <<-EOT ?!AMP?!
 	foobar
 )
diff --git a/t/chainlint/subshell-here-doc.expect b/t/chainlint/subshell-here-doc.expect
index b7250ca753..029d129299 100644
--- a/t/chainlint/subshell-here-doc.expect
+++ b/t/chainlint/subshell-here-doc.expect
@@ -1,10 +1,10 @@
 (
-	echo wobba 	       gorgo snoot 	       wafta snurb &&
-	cat >bip ?!AMP?!
-	echo >bop
+	echo wobba 	       gorgo snoot 	       wafta snurb <<-EOF &&
+	cat <<EOF >bip ?!AMP?!
+	echo <<-EOF >bop
 ) &&
 (
-	cat >bup &&
-	cat >bup3 &&
+	cat <<-ARBITRARY >bup &&
+	cat <<-ARBITRARY3 >bup3 &&
 	meep
 )
diff --git a/t/chainlint/t7900-subtree.expect b/t/chainlint/t7900-subtree.expect
index 215aca01c2..1cccc7bf7e 100644
--- a/t/chainlint/t7900-subtree.expect
+++ b/t/chainlint/t7900-subtree.expect
@@ -1,9 +1,9 @@
 (
 	chks="sub1sub2sub3sub4" &&
-	chks_sub=$(cat | sed "s,^,sub dir/,"
+	chks_sub=$(cat <<TXT | sed "s,^,sub dir/,"
 ) &&
 	chkms="main-sub1main-sub2main-sub3main-sub4" &&
-	chkms_sub=$(cat | sed "s,^,sub dir/,"
+	chkms_sub=$(cat <<TXT | sed "s,^,sub dir/,"
 ) &&
 	subfiles=$(git ls-files) &&
 	check_equal "$subfiles" "$chkms$chks"
diff --git a/t/chainlint/while-loop.expect b/t/chainlint/while-loop.expect
index e2813b378e..0d3a9b3d12 100644
--- a/t/chainlint/while-loop.expect
+++ b/t/chainlint/while-loop.expect
@@ -2,7 +2,7 @@
 	while true
 	do
 		echo foo ?!AMP?!
-		cat
+		cat <<-EOF
 	done ?!AMP?!
 	while true; do
 		echo foo &&

-- 
2.34.1.397.gfae76fe5da
Previous: Eric SunshineNext: Eric Sunshine
Message 30 of 34 in “generalize chainlint self-tests”
  1. 00/15 generalize chainlint self-testsEric Sunshine, Dec 13, 2021
  2. 01/15 t/chainlint/*.test: don't use invalid shell syntaxEric Sunshine, Dec 13, 2021
  3. 03/15 t/chainlint/*.test: generalize self-test commentaryEric Sunshine, Dec 13, 2021
  4. 02/15 t/chainlint/*.test: fix invalid test cases due to mixing quote typesEric Sunshine, Dec 13, 2021
  5. 04/15 t/chainlint/one-liner: avoid overly intimate chainlint.sed knowledgeEric Sunshine, Dec 13, 2021
  6. 05/15 t/Makefile: optimize chainlint self-testEric Sunshine, Dec 13, 2021
  7. Fabian StelzerDec 13, 2021
  8. Eric SunshineDec 13, 2021
  9. Fabian StelzerDec 13, 2021
  10. Eric SunshineDec 13, 2021
  11. Fabian StelzerDec 13, 2021
  12. Ævar Arnfjörð BjarmasonDec 13, 2021
  13. Eric SunshineDec 13, 2021
  14. Eric SunshineDec 13, 2021
  15. Ævar Arnfjörð BjarmasonDec 13, 2021
  16. Eric SunshineDec 13, 2021
  17. Ævar Arnfjörð BjarmasonDec 16, 2021
  18. Eric SunshineDec 16, 2021
  19. Ævar Arnfjörð BjarmasonDec 16, 2021
  20. t/Makefile: use dependency graph for "check-chainlint"Ævar Arnfjörð Bjarmason, Dec 13, 2021
  21. Eric SunshineDec 14, 2021
  22. Ævar Arnfjörð BjarmasonDec 14, 2021
  23. 06/15 chainlint.sed: improve ?!AMP?! placement accuracyEric Sunshine, Dec 13, 2021
  24. 07/15 chainlint.sed: improve ?!SEMI?! placement accuracyEric Sunshine, Dec 13, 2021
  25. 08/15 chainlint.sed: tolerate harmless ";" at end of last line in blockEric Sunshine, Dec 13, 2021
  26. 09/15 chainlint.sed: drop unnecessary distinction between ?!AMP?! and ?!SEMI?!Eric Sunshine, Dec 13, 2021
  27. 10/15 chainlint.sed: drop subshell-closing ">" annotationEric Sunshine, Dec 13, 2021
  28. 11/15 chainlint.sed: make here-doc "<<-" operator recognition more POSIX-likeEric Sunshine, Dec 13, 2021
  29. 12/15 chainlint.sed: don't mistake `<< word` in string as here-doc operatorEric Sunshine, Dec 13, 2021
  30. 13/15 chainlint.sed: stop throwing away here-doc tagsEric Sunshine, Dec 13, 2021
  31. 14/15 chainlint.sed: swallow comments consistentlyEric Sunshine, Dec 13, 2021
  32. 15/15 chainlint.sed: stop splitting "(..." into separate lines "(" and "..."Eric Sunshine, Dec 13, 2021
  33. Elijah NewrenDec 15, 2021
  34. Eric SunshineDec 15, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.