[PATCH v2 0/2] t: work around bugs in Dash v0.5.13

Hi,

while testing my `set -e` change for our test suite I was also playing around with Dash v0.5.13.1 and noticed that multiple of our tests fail with that version. As it turns out, some of these test failures are actual bugs in Dash itself that have been introduced in v0.5.13.

There's ultimately two bugs:

  - Dash inserts CTLMBCHAR byte sequences into heredocs around multibyte
    characters. This bug still exists with the current "master" branch
    of Dash.

  - Dash may swallow some bytes when reading data, which has already
    been fixed.

I've Cc'd Herbert, maintainer of Dash.

Changes in v2:
  - Add comment to heredocs explaining why they should never be quoted.
  - Link to v1: https://patch.msgid.link/20260401-pks-tests-with-dash-v1-0-d70b5040aa5d@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (2):
      t: work around multibyte bug in quoted heredocs with Dash v0.5.13
      t9300: work around partial read bug in Dash v0.5.13

 t/t0300-credentials.sh   |  4 +++-
 t/t3430-rebase-merges.sh |  6 ++++--
 t/t3902-quoted.sh        | 16 +++++++++-------
 t/t4014-format-patch.sh  | 16 ++++++++++++----
 t/t4201-shortlog.sh      |  4 +++-
 t/t9001-send-email.sh    | 12 +++++++++---
 t/t9300-fast-import.sh   | 32 ++++++++++++++------------------
 7 files changed, 54 insertions(+), 36 deletions(-)

Range-diff versus v1:

1:  192b5b4330 ! 1:  5367e05e0f t: work around multibyte bug in quoted heredocs with Dash v0.5.13
    @@ t/t0300-credentials.sh: test_expect_success 'match percent-encoded values' '
      	test_config credential.https://example.com.useHttpPath true &&
      	test_config credential.https://example.com/perú.git.helper "$HELPER" &&
     -	check fill <<-\EOF
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	check fill <<-EOF
      	url=https://example.com/per%C3%BA.git
      	--
    @@ t/t3430-rebase-merges.sh: test_expect_success 'octopus merges' '
      	test "Hank" = "$(git show -s --format=%an HEAD)" &&
      	test "$before" != $(git rev-parse HEAD) &&
     -	test_cmp_graph HEAD^^.. <<-\EOF
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	test_cmp_graph HEAD^^.. <<-EOF
      	*-.   Tüntenfüsch
     -	|\ \
    @@ t/t3902-quoted.sh: With SP in it
      EOF
      
     -cat >expect.raw <<\EOF
    ++# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++# that contain multibyte chars.
     +cat >expect.raw <<EOF
      Name
     -"Name and a\nLF"
    @@ t/t4014-format-patch.sh: test_expect_success 'format-patch wraps extremely long
      '
      
     -cat >expect <<'EOF'
    ++# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++# that contain multibyte chars.
     +cat >expect <<EOF
      From: Foö Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar
       Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo Bar Foo
    @@ t/t4014-format-patch.sh: test_expect_success 'format-patch wraps extremely long
      '
      
     -cat >expect <<'EOF'
    ++# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++# that contain multibyte chars.
     +cat >expect <<EOF
      Subject: [PATCH] Foö
      EOF
    @@ t/t4014-format-patch.sh: test_expect_success 'subject lines are unencoded with -
      '
      
     -cat >expect <<'EOF'
    ++# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++# that contain multibyte chars.
     +cat >expect <<EOF
      Subject: [PATCH] Foö
      EOF
    @@ t/t4014-format-patch.sh: test_expect_success 'in-body headers trigger content en
      	test_when_finished "git reset --hard HEAD^" &&
      	git format-patch -1 --stdout --from >patch &&
     -	cat >expect <<-\EOF &&
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	cat >expect <<-EOF &&
      	From: C O Mitter <committer@example.com>
      	Content-Type: text/plain; charset=UTF-8
    @@ t/t4201-shortlog.sh: test_expect_success 'output from user-defined format is re-
      
      test_expect_success !MINGW,ICONV 'shortlog wrapping' '
     -	cat >expect <<\EOF &&
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	cat >expect <<EOF &&
      A U Thor (5):
            Test
    @@ t/t9001-send-email.sh: test_expect_success $PREREQ 'To headers from files reset
      
      test_expect_success $PREREQ 'setup expect' '
     -cat >email-using-8bit <<\EOF
    ++# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++# that contain multibyte chars.
     +cat >email-using-8bit <<EOF
      From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
      Message-ID: <bogus-message-id@example.com>
    @@ t/t9001-send-email.sh: test_expect_success $PREREQ '--8bit-encoding overrides se
      
      test_expect_success $PREREQ 'setup expect' '
     -	cat >email-using-8bit <<-\EOF
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	cat >email-using-8bit <<-EOF
      	From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
      	Message-ID: <bogus-message-id@example.com>
    @@ t/t9001-send-email.sh: test_expect_success $PREREQ '--8bit-encoding also treats
      
      test_expect_success $PREREQ 'setup expect' '
     -	cat >email-using-8bit <<-\EOF
    ++	# NOTE: do not quote this heredoc, Dash 0.5.13 has a bug with heredocs
    ++	# that contain multibyte chars.
     +	cat >email-using-8bit <<-EOF
      	From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
      	Message-ID: <bogus-message-id@example.com>
2:  db99459b3f = 2:  364d37fcc7 t9300: work around partial read bug in Dash v0.5.13

--- base-commit: 270e10ad6dda3379ea0da7efd11e4fbf2cd7a325 change-id: 20260401-pks-tests-with-dash-ea71cbb36958