git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/3] mailinfo.c::convert_to_utf8: reuse strlen info

From
Đoàn Trần Công Danh <congdanhqx@gmail.com>
Date
Apr 19, 2020, 11:00 UTC
Message-ID
<e3e542d38818b762f8d6d6b50bc42e01b070772b.1587289680.git.congdanhqx@gmail.com>
In-Reply-To
<cover.1587289680.git.congdanhqx@gmail.com>

We're passing buffer from strbuf to reencode_string, which will call strlen(3) on that buffer, and discard the length of newly created buffer. Then, we compute the length of the return buffer to attach to strbuf.

During this process, we introduce a discrimination between mail originally written in utf-8 and other encoding.

* if the email was written in utf-8, we leave it as is. If there is
  a NUL character in that line, we complains loudly:
  	error: a NUL byte in commit log message not allowed.
* if the email was written in other encoding, we truncate the data as
  the NUL character in that line, then we used the truncated line for
  the metadata.

We can do better by reusing all the available information, and call the underlying lower level function that will be called indirectly by reencode_string. By doing this, we will also postpone the NUL character processing to the commit step, which will complains about the faulty metadata.

Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
---
 mailinfo.c            |  6 ++++--
 t/t4254-am-corrupt.sh | 44 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/mailinfo.c b/mailinfo.c
index 742fa376ab..0e9911df6d 100644
--- a/mailinfo.c
+++ b/mailinfo.c
@@ -447,19 +447,21 @@ static int convert_to_utf8(struct mailinfo *mi,
 			   struct strbuf *line, const char *charset)
 {
 	char *out;
+	size_t out_len;
 
 	if (!mi->metainfo_charset || !charset || !*charset)
 		return 0;
 
 	if (same_encoding(mi->metainfo_charset, charset))
 		return 0;
-	out = reencode_string(line->buf, mi->metainfo_charset, charset);
+	out = reencode_string_len(line->buf, line->len,
+				  mi->metainfo_charset, charset, &out_len);
 	if (!out) {
 		mi->input_error = -1;
 		return error("cannot convert from %s to %s",
 			     charset, mi->metainfo_charset);
 	}
-	strbuf_attach(line, out, strlen(out), strlen(out));
+	strbuf_attach(line, out, out_len, out_len);
 	return 0;
 }
 
diff --git a/t/t4254-am-corrupt.sh b/t/t4254-am-corrupt.sh
index ddd35498db..98cda32d0a 100755
--- a/t/t4254-am-corrupt.sh
+++ b/t/t4254-am-corrupt.sh
@@ -3,6 +3,36 @@
 test_description='git am with corrupt input'
 . ./test-lib.sh
 
+write_nul_patch() {
+	space=' '
+	qNUL=
+	case "$1" in
+		subject) qNUL='=00' ;;
+	esac
+	cat <<-EOF
+	From ec7364544f690c560304f5a5de9428ea3b978b26 Mon Sep 17 00:00:00 2001
+	From: A U Thor <author@example.com>
+	Date: Sun, 19 Apr 2020 13:42:07 +0700
+	Subject: [PATCH] =?ISO-8859-1?q?=C4=CB${qNUL}=D1=CF=D6?=
+	MIME-Version: 1.0
+	Content-Type: text/plain; charset=ISO-8859-1
+	Content-Transfer-Encoding: 8bit
+
+	EOF
+	if test "$1" = body
+	then
+		printf "%s\0%s\n" abc def
+	fi
+	cat <<-\EOF
+	---
+	diff --git a/afile b/afile
+	new file mode 100644
+	index 0000000000..e69de29bb2
+	--$space
+	2.26.1
+	EOF
+}
+
 test_expect_success setup '
 	# Note the missing "+++" line:
 	cat >bad-patch.diff <<-\EOF &&
@@ -32,4 +62,18 @@ test_expect_success 'try to apply corrupted patch' '
 	test_i18ncmp expected actual
 '
 
+test_expect_success "NUL in commit message's body" '
+	test_when_finished "git am --abort" &&
+	write_nul_patch body >body.patch &&
+	test_must_fail git am body.patch 2>err &&
+	grep "a NUL byte in commit log message not allowed" err
+'
+
+test_expect_failure "NUL in commit message's header" '
+	test_when_finished "git am --abort" &&
+	write_nul_patch subject >subject.patch &&
+	test_must_fail git am subject.patch 2>err &&
+	grep "a NUL byte in Subject is not allowed" err
+'
+
 test_done
-- 
2.26.1.301.g55bc3eb7cb
Previous: Danh DoanNext: Martin Ågren
Message 31 of 39 in “mailinfo.c::convert_to_utf8: reuse strlen info”
  1. mailinfo.c::convert_to_utf8: reuse strlen infoĐoàn Trần Công Danh, Apr 18, 2020
  2. Martin ÅgrenApr 18, 2020
  3. 0/6 strbuf: simplify `strbuf_attach()` usageMartin Ågren, Apr 18, 2020
  4. 1/6 am: use `strbuf_attach()` correctlyMartin Ågren, Apr 18, 2020
  5. 2/6 strbuf_attach: correctly pass in `strlen() + 1` for `alloc`Martin Ågren, Apr 18, 2020
  6. 3/6 strbuf: use `strbuf_attach()` correctlyMartin Ågren, Apr 18, 2020
  7. 4/6 fast-import: avoid awkward use of `strbuf_attach()`Martin Ågren, Apr 18, 2020
  8. 5/6 rerere: avoid awkward use of `strbuf_attach()`Martin Ågren, Apr 18, 2020
  9. 6/6 strbuf: simplify `strbuf_attach()` usageMartin Ågren, Apr 18, 2020
  10. Martin ÅgrenApr 19, 2020
  11. 0/4 strbuf: fix doc for `strbuf_attach()` and avoid itMartin Ågren, Apr 19, 2020
  12. 1/4 strbuf: fix doc for `strbuf_attach()`Martin Ågren, Apr 19, 2020
  13. Junio C HamanoApr 20, 2020
  14. Martin ÅgrenApr 21, 2020
  15. 2/4 strbuf: introduce `strbuf_attachstr_len()`Martin Ågren, Apr 19, 2020
  16. 3/4 strbuf: introduce `strbuf_attachstr()`Martin Ågren, Apr 19, 2020
  17. Junio C HamanoApr 20, 2020
  18. Martin ÅgrenApr 21, 2020
  19. 4/4 strbuf_attach: prefer `strbuf_attachstr_len()`Martin Ågren, Apr 19, 2020
  20. Junio C HamanoApr 18, 2020
  21. Danh DoanApr 19, 2020
  22. Martin ÅgrenApr 19, 2020
  23. Junio C HamanoApr 19, 2020
  24. 0/3 mailinfo: disallow and complains about NUL characterĐoàn Trần Công Danh, Apr 19, 2020
  25. 1/3 t4254: merge 2 steps of a single testĐoàn Trần Công Danh, Apr 19, 2020
  26. Martin ÅgrenApr 19, 2020
  27. Danh DoanApr 19, 2020
  28. 3/3 mailinfo: disallow NUL character in mail's headerĐoàn Trần Công Danh, Apr 19, 2020
  29. Martin ÅgrenApr 19, 2020
  30. Danh DoanApr 19, 2020
  31. 2/3 mailinfo.c::convert_to_utf8: reuse strlen infoĐoàn Trần Công Danh, Apr 19, 2020
  32. Martin ÅgrenApr 19, 2020
  33. Danh DoanApr 19, 2020
  34. Junio C HamanoApr 20, 2020
  35. Danh DoanApr 20, 2020
  36. 0/3 Disallow NUL character in mailinfoĐoàn Trần Công Danh, Apr 20, 2020
  37. 1/3 t4254: merge 2 steps of a single testĐoàn Trần Công Danh, Apr 20, 2020
  38. 2/3 mailinfo.c: avoid strlen on strings that can contains NULĐoàn Trần Công Danh, Apr 20, 2020
  39. 3/3 mailinfo: disallow NUL character in mail's headerĐoàn Trần Công Danh, Apr 20, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.