git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/2] t: work around multibyte bug in quoted heredocs with Dash v0.5.13

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 1, 2026, 10:42 UTC
Message-ID
<20260401-pks-tests-with-dash-v1-1-d70b5040aa5d@pks.im>
In-Reply-To
<20260401-pks-tests-with-dash-v1-0-d70b5040aa5d@pks.im>

When executing our test suite with Dash v0.5.13.2 one can observe several test failures that all have the same symptoms: we have a quoted heredoc that contains multibyte characters, but the final data does not match what we actually wanted to write. One such example is in t0300, where we see the diffs like the following:

  --- expect-stdout	2026-04-01 07:25:45.249919440 +0000
  +++ stdout	2026-04-01 07:25:45.254919509 +0000
  @@ -1,5 +1,5 @@
   protocol=https
   host=example.com
  -path=perú.git
  +path=perú.git
   username=foo
   password=bar

While seemingly the same, the data that we've written via the heredoc contains some invisible bytes. The expected hex representation of the string is:

  7065 72c3 ba2e 6769 74                 per...git
But what we actually get instead is this string:
  7065 7285 02c3 ba02 852e 6769 74       per.......git

What's important to note here is that the multibyte character exists in both versions. But in the broken version we see that the bytes are wrapped in a sequence of "85 02" and "02 85". This is the CTLMBCHAR byte sequence of Dash, which it uses internally to quote multibyte sequences.

As it turns out, this bug was introduced in c5bf970 (expand: Add multi-byte support to pmatch, 2024-06-02), which adds multibyte support to more contexts of Dash. One of these contexts seems to be in heredocs, and Dash _does_ correctly unquote these multibyte sequences when using an unquoted heredoc. But the bug seems to be that this unquoting does not happen in quoted heredocs, and the bug still exists on the latest "master" branch.

For now, work around the bug by using unquoted heredocs instead.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t0300-credentials.sh   |  2 +-
 t/t3430-rebase-merges.sh |  4 ++--
 t/t3902-quoted.sh        | 14 +++++++-------
 t/t4014-format-patch.sh  |  8 ++++----
 t/t4201-shortlog.sh      |  2 +-
 t/t9001-send-email.sh    |  6 +++---
 6 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh
index 07aa834d33..fda6760955 100755
--- a/t/t0300-credentials.sh
+++ b/t/t0300-credentials.sh
@@ -675,7 +675,7 @@ test_expect_success 'match percent-encoded values' '
 test_expect_success 'match percent-encoded UTF-8 values in path' '
 	test_config credential.https://example.com.useHttpPath true &&
 	test_config credential.https://example.com/perú.git.helper "$HELPER" &&
-	check fill <<-\EOF
+	check fill <<-EOF
 	url=https://example.com/per%C3%BA.git
 	--
 	protocol=https
diff --git a/t/t3430-rebase-merges.sh b/t/t3430-rebase-merges.sh
index cc627e34a7..d3dffbb830 100755
--- a/t/t3430-rebase-merges.sh
+++ b/t/t3430-rebase-merges.sh
@@ -507,9 +507,9 @@ test_expect_success 'octopus merges' '
 	git rebase -i --force-rebase -r HEAD^^ &&
 	test "Hank" = "$(git show -s --format=%an HEAD)" &&
 	test "$before" != $(git rev-parse HEAD) &&
-	test_cmp_graph HEAD^^.. <<-\EOF
+	test_cmp_graph HEAD^^.. <<-EOF
 	*-.   Tüntenfüsch
-	|\ \
+	|\\ \\
 	| | * three
 	| * | two
 	| |/
diff --git a/t/t3902-quoted.sh b/t/t3902-quoted.sh
index f528008c36..c3b45d991e 100755
--- a/t/t3902-quoted.sh
+++ b/t/t3902-quoted.sh
@@ -60,16 +60,16 @@ With SP in it
 "\346\277\261\351\207\216\347\264\224"
 EOF
 
-cat >expect.raw <<\EOF
+cat >expect.raw <<EOF
 Name
-"Name and a\nLF"
-"Name and an\tHT"
-"Name\""
+"Name and a\\nLF"
+"Name and an\\tHT"
+"Name\\""
 With SP in it
-"濱野\t純"
-"濱野\n純"
+"濱野\\t純"
+"濱野\\n純"
 濱野 純
-"濱野\"純"
+"濱野\\"純"
 濱野/file
 濱野純
 EOF
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index bcdb944017..d22b7f348e 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -1285,7 +1285,7 @@ test_expect_success 'format-patch wraps extremely long from-header (rfc2047)' '
 	check_author "Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar"
 '
 
-cat >expect <<'EOF'
+cat >expect <<EOF
 From: Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar
  Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo
  Bar Foo Bar Foo Bar Foo Bar <author@example.com>
@@ -1300,7 +1300,7 @@ test_expect_success 'format-patch wraps extremely long from-header (non-ASCII wi
 	test_cmp expect actual
 '
 
-cat >expect <<'EOF'
+cat >expect <<EOF
 Subject: [PATCH] Foö
 EOF
 test_expect_success 'subject lines are unencoded with --no-encode-email-headers' '
@@ -1312,7 +1312,7 @@ test_expect_success 'subject lines are unencoded with --no-encode-email-headers'
 	test_cmp expect actual
 '
 
-cat >expect <<'EOF'
+cat >expect <<EOF
 Subject: [PATCH] Foö
 EOF
 test_expect_success 'subject lines are unencoded with format.encodeEmailHeaders=false' '
@@ -1531,7 +1531,7 @@ test_expect_success 'in-body headers trigger content encoding' '
 	test_env GIT_AUTHOR_NAME="éxötìc" test_commit exotic &&
 	test_when_finished "git reset --hard HEAD^" &&
 	git format-patch -1 --stdout --from >patch &&
-	cat >expect <<-\EOF &&
+	cat >expect <<-EOF &&
 	From: C O Mitter <committer@example.com>
 	Content-Type: text/plain; charset=UTF-8
 
diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index 5f23fc147b..d73c9f5204 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -105,7 +105,7 @@ test_expect_success 'output from user-defined format is re-wrapped' '
 '
 
 test_expect_success !MINGW,ICONV 'shortlog wrapping' '
-	cat >expect <<\EOF &&
+	cat >expect <<EOF &&
 A U Thor (5):
       Test
       This is a very, very long first line for the commit message to see if
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 24f6c76aee..3612d32d39 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -1649,7 +1649,7 @@ test_expect_success $PREREQ 'To headers from files reset each patch' '
 '
 
 test_expect_success $PREREQ 'setup expect' '
-cat >email-using-8bit <<\EOF
+cat >email-using-8bit <<EOF
 From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
 Message-ID: <bogus-message-id@example.com>
 From: author@example.com
@@ -1735,7 +1735,7 @@ test_expect_success $PREREQ '--8bit-encoding overrides sendemail.8bitEncoding' '
 '
 
 test_expect_success $PREREQ 'setup expect' '
-	cat >email-using-8bit <<-\EOF
+	cat >email-using-8bit <<-EOF
 	From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
 	Message-ID: <bogus-message-id@example.com>
 	From: author@example.com
@@ -1764,7 +1764,7 @@ test_expect_success $PREREQ '--8bit-encoding also treats subject' '
 '
 
 test_expect_success $PREREQ 'setup expect' '
-	cat >email-using-8bit <<-\EOF
+	cat >email-using-8bit <<-EOF
 	From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
 	Message-ID: <bogus-message-id@example.com>
 	From: A U Thor <author@example.com>

-- 
2.53.0.1185.g05d4b7b318.dirty
Previous: Patrick SteinhardtNext: Eric Sunshine
Message 2 of 10 in “t: work around bugs in Dash v0.5.13”
  1. 0/2 t: work around bugs in Dash v0.5.13Patrick Steinhardt, Apr 1, 2026
  2. 1/2 t: work around multibyte bug in quoted heredocs with Dash v0.5.13Patrick Steinhardt, Apr 1, 2026
  3. Eric SunshineApr 1, 2026
  4. Patrick SteinhardtApr 2, 2026
  5. 2/2 t9300: work around partial read bug in Dash v0.5.13Patrick Steinhardt, Apr 1, 2026
  6. 0/2 t: work around bugs in Dash v0.5.13Patrick Steinhardt, Apr 2, 2026
  7. 1/2 t: work around multibyte bug in quoted heredocs with Dash v0.5.13Patrick Steinhardt, Apr 2, 2026
  8. parser: Fix multi-byte output in here-doc with quoted delimiterHerbert Xu, May 7, 2026
  9. Herbert XuMay 7, 2026
  10. 2/2 t9300: work around partial read bug in Dash v0.5.13Patrick Steinhardt, Apr 2, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.