git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 4/9] chainlint.pl: force CRLF conversion when opening input files

From
Jeff King <peff@peff.net>
Date
Jul 10, 2024, 08:37 UTC
Message-ID
<20240710083730.GD2060601@coredump.intra.peff.net>
In-Reply-To
<20240710083416.GA2060328@coredump.intra.peff.net>

The lexer in chainlint.pl can't handle CRLF line endings; it complains about an internal error in scan_token() if we see one. For example, in our Windows CI environment:

  $ perl chainlint.pl chainlint/for-loop.test | cat -v
  Thread 2 terminated abnormally: internal error scanning character '^M'

This doesn't break "make check-chainlint" (yet), because we assemble a concatenated input by passing the contents of each file through "sed". And the "sed" we use will strip out the CRLFs. But the next patch is going to rework this a bit, which does break check-chainlint on Windows. Plus it's probably nicer to folks on Windows who might work on chainlint itself and write new tests.

In theory we could fix the parser to handle this, but it's not really worth the trouble. We should be able to ask the input layer to translate the line endings for us. In fact, I'd expect this to happen by default, as perl's documentation claims Win32 uses the ":unix:crlf" PERLIO layer by default ("unix" here just refers to using read/write syscalls, and then "crlf" layers the translation on top). However, this doesn't seem to be the case in our Windows CI environment. I didn't dig into the exact reason, but it is perhaps because we are using an msys build of perl rather than a "true" Win32 build.

At any rate, it is easy-ish to just ask explicitly for the conversion. In the above example, setting PERLIO=crlf in the environment is enough to make it work. Curiously, though, this doesn't work when invoking chainlint via "make". Again, I didn't dig into it, but it may have to do with msys programs calling Windows programs or vice versa.

We can make it work consistently by just explicitly asking for CRLF translation when we open the files. This will even work on non-Windows platforms, though we wouldn't really expect to find CRLF files there.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/chainlint.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/chainlint.pl b/t/chainlint.pl
index 118a229a96..fb749d3d5c 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -762,7 +762,7 @@ sub check_script {
 	while (my $path = $next_script->()) {
 		$nscripts++;
 		my $fh;
-		unless (open($fh, "<", $path)) {
+		unless (open($fh, "<:unix:crlf", $path)) {
 			$emit->("?!ERR?! $path: $!\n");
 			next;
 		}
-- 
2.45.2.1249.gb036353db5
Previous: Jeff KingNext: Jeff King
Message 54 of 65 in “here-doc test bodies”
  1. 0/2 here-doc test bodiesJeff King, Jul 1, 2024
  2. 1/2 test-lib: allow test snippets as here-docsJeff King, Jul 1, 2024
  3. Eric SunshineJul 1, 2024
  4. Junio C HamanoJul 1, 2024
  5. Jeff KingJul 2, 2024
  6. Jeff KingJul 2, 2024
  7. Eric SunshineJul 2, 2024
  8. Jeff KingJul 6, 2024
  9. Jeff KingJul 2, 2024
  10. Eric SunshineJul 2, 2024
  11. Jeff KingJul 6, 2024
  12. Eric SunshineJul 2, 2024
  13. Eric SunshineJul 2, 2024
  14. Eric SunshineJul 2, 2024
  15. Jeff KingJul 6, 2024
  16. Jeff KingJul 6, 2024
  17. Eric SunshineJul 6, 2024
  18. Eric SunshineJul 6, 2024
  19. Jeff KingJul 6, 2024
  20. Eric SunshineJul 6, 2024
  21. Jeff KingJul 6, 2024
  22. 2/2 t: convert some here-doc test bodiesJeff King, Jul 1, 2024
  23. chainlint.pl: recognize test bodies defined via heredocEric Sunshine, Jul 2, 2024
  24. Jeff KingJul 6, 2024
  25. 1/3 chainlint.pl: fix line number reportingJeff King, Jul 6, 2024
  26. Eric SunshineJul 8, 2024
  27. Jeff KingJul 8, 2024
  28. 2/3 t/chainlint: add test_expect_success call to test snippetsJeff King, Jul 6, 2024
  29. Jeff KingJul 6, 2024
  30. Eric SunshineJul 8, 2024
  31. 3/3 t/chainlint: add tests for test body in heredocJeff King, Jul 6, 2024
  32. Eric SunshineJul 8, 2024
  33. Jeff KingJul 8, 2024
  34. Junio C HamanoJul 6, 2024
  35. Jeff KingJul 6, 2024
  36. Eric SunshineJul 8, 2024
  37. Jeff KingJul 8, 2024
  38. Eric SunshineJul 8, 2024
  39. Eric SunshineJul 8, 2024
  40. Jeff KingJul 10, 2024
  41. Jeff KingJul 10, 2024
  42. Eric SunshineJul 10, 2024
  43. Jeff KingJul 10, 2024
  44. Eric SunshineJul 10, 2024
  45. Eric SunshineJul 8, 2024
  46. Jeff KingJul 8, 2024
  47. Eric SunshineJul 8, 2024
  48. Jeff KingJul 10, 2024
  49. Eric SunshineJul 10, 2024
  50. 0/9 here-doc test bodies (now with 100% more chainlinting)Jeff King, Jul 10, 2024
  51. 1/9 chainlint.pl: add test_expect_success call to test snippetsJeff King, Jul 10, 2024
  52. 2/9 chainlint.pl: only start threads if jobs > 1Jeff King, Jul 10, 2024
  53. 3/9 chainlint.pl: do not spawn more threads than we have scriptsJeff King, Jul 10, 2024
  54. 4/9 chainlint.pl: force CRLF conversion when opening input filesJeff King, Jul 10, 2024
  55. 5/9 chainlint.pl: check line numbers in expected outputJeff King, Jul 10, 2024
  56. Eric SunshineAug 21, 2024
  57. Jeff KingAug 21, 2024
  58. Eric SunshineAug 21, 2024
  59. 6/9 chainlint.pl: recognize test bodies defined via heredocJeff King, Jul 10, 2024
  60. 7/9 chainlint.pl: add tests for test body in heredocJeff King, Jul 10, 2024
  61. 8/9 test-lib: allow test snippets as here-docsJeff King, Jul 10, 2024
  62. 9/9 t: convert some here-doc test bodiesJeff King, Jul 10, 2024
  63. 10/9 t/.gitattributes: ignore whitespace in chainlint expect filesJeff King, Jul 10, 2024
  64. Junio C HamanoJul 10, 2024
  65. Eric SunshineAug 21, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.