git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/5] tests: run internal chain-linter under "make test"

From
Jeff King <peff@peff.net>
Date
Mar 30, 2023, 19:27 UTC
Message-ID
<20230330192738.GA27989@coredump.intra.peff.net>
In-Reply-To
<20230330192712.GA27719@coredump.intra.peff.net>

Since 69b9924b875 (t/Makefile: teach `make test` and `make prove` to run chainlint.pl, 2022-09-01), we run a single chainlint.pl process for all scripts, and then instruct each individual script to run with the equivalent of --no-chain-lint, which tells them not to redundantly run the chainlint script themselves.

However, this also disables the internal linter run within the shell by eval-ing "(exit 117) && $1" and confirming we get code 117. In theory the external linter produces a superset of complaints, and we don't need the internal one anymore. However, we know there is at least one case where they differ. A test like:

	test_expect_success 'should fail linter' '
		false &&
		sleep 2 &
		pid=$! &&
		kill $pid
	'

is buggy (it ignores the failure from "false", because it is backgrounded along with the sleep). The internal linter catches this, but the external one doesn't (and teaching it to do so is complicated[1]). So not only does "make test" miss this problem, but it's doubly confusing because running the script standalone does complain.

Let's teach the suppression in the Makefile to only turn off the external linter (which we know is redundant, as it was already run) and leave the internal one intact.

I've used a new environment variable to do this here, and intentionally did not add a "--no-ext-chain-lint" option. This is an internal optimization used by the Makefile, and not something that ordinary users would need to tweak.

[1] For discussion of chainlint.pl and this case, see:
    https://lore.kernel.org/git/CAPig+cQtLFX4PgXyyK_AAkCvg4Aw2RAC5MmLbib-aHHgTBcDuw@mail.gmail.com/
Signed-off-by: Jeff King <peff@peff.net>
---
Same as before, but tweaking the Makefile comment.
 t/Makefile    | 4 ++--
 t/test-lib.sh | 3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/t/Makefile b/t/Makefile
index 88fa5049573..3e00cdd801d 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -44,8 +44,8 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl
 
 # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`)
 # checks all tests in all scripts via a single invocation, so tell individual
-# scripts not to "chainlint" themselves
-CHAINLINTSUPPRESS = GIT_TEST_CHAIN_LINT=0 && export GIT_TEST_CHAIN_LINT &&
+# scripts not to run the external "chainlint.pl" script themselves
+CHAINLINTSUPPRESS = GIT_TEST_EXT_CHAIN_LINT=0 && export GIT_TEST_EXT_CHAIN_LINT &&
 
 all: $(DEFAULT_TEST_TARGET)
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 62136caee5a..09789566374 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1593,7 +1593,8 @@ then
 	BAIL_OUT_ENV_NEEDS_SANITIZE_LEAK "GIT_TEST_SANITIZE_LEAK_LOG=true"
 fi
 
-if test "${GIT_TEST_CHAIN_LINT:-1}" != 0
+if test "${GIT_TEST_CHAIN_LINT:-1}" != 0 &&
+   test "${GIT_TEST_EXT_CHAIN_LINT:-1}" != 0
 then
 	"$PERL_PATH" "$TEST_DIRECTORY/chainlint.pl" "$0" ||
 		BUG "lint error (see '?!...!? annotations above)"
-- 
2.40.0.692.g7c4c956fc5c
Previous: Jeff KingNext: Jeff King
Message 25 of 32 in “some chainlint fixes and performance improvements”
  1. 0/4 some chainlint fixes and performance improvementsJeff King, Mar 28, 2023
  2. 1/4 tests: run internal chain-linter under "make test"Jeff King, Mar 28, 2023
  3. Ævar Arnfjörð BjarmasonMar 29, 2023
  4. Junio C HamanoMar 29, 2023
  5. Jeff KingMar 29, 2023
  6. Junio C HamanoMar 30, 2023
  7. 2/4 tests: replace chainlint subshell with a functionJeff King, Mar 28, 2023
  8. Junio C HamanoMar 28, 2023
  9. 3/4 tests: drop here-doc check from internal chain-linterJeff King, Mar 28, 2023
  10. Junio C HamanoMar 28, 2023
  11. Jeff KingMar 29, 2023
  12. Jeff KingMar 29, 2023
  13. Eric SunshineMar 29, 2023
  14. Eric SunshineMar 29, 2023
  15. Eric SunshineMar 29, 2023
  16. Jeff KingMar 29, 2023
  17. Eric SunshineMar 29, 2023
  18. Eric SunshineMar 29, 2023
  19. Jeff KingMar 29, 2023
  20. 4/4 tests: skip test_eval_ in internal chain-lintJeff King, Mar 28, 2023
  21. Jeff KingMar 28, 2023
  22. Jeff KingMar 30, 2023
  23. Junio C HamanoMar 30, 2023
  24. 0/5 some chainlint fixes and performance improvementsJeff King, Mar 30, 2023
  25. 1/5 tests: run internal chain-linter under "make test"Jeff King, Mar 30, 2023
  26. 2/5 tests: replace chainlint subshell with a functionJeff King, Mar 30, 2023
  27. 3/5 tests: diagnose unclosed here-doc in chainlint.plJeff King, Mar 30, 2023
  28. Eric SunshineMar 30, 2023
  29. 4/5 tests: drop here-doc check from internal chain-linterJeff King, Mar 30, 2023
  30. 5/5 tests: skip test_eval_ in internal chain-lintJeff King, Mar 30, 2023
  31. Junio C HamanoMar 30, 2023
  32. Jeff KingMar 30, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.