git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v13 04/10] utf8: teach same_encoding() alternative UTF encoding names

From
lars.schneider@autodesk.com <lars.schneider@autodesk.com>
Date
Apr 15, 2018, 18:16 UTC
Message-ID
<20180415181610.1612-5-lars.schneider@autodesk.com>
In-Reply-To
<20180415181610.1612-1-lars.schneider@autodesk.com>
From: Lars Schneider <larsxschneider@gmail.com>

The function same_encoding() could only recognize alternative names for UTF-8 encodings. Teach it to recognize all kinds of alternative UTF encoding names (e.g. utf16).

While we are at it, fix a crash that would occur if same_encoding() was called with a NULL argument and a non-NULL argument.

This function is used in a subsequent commit.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 utf8.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
diff --git a/utf8.c b/utf8.c
index 2c27ce0137..40f4b142d6 100644
--- a/utf8.c
+++ b/utf8.c
@@ -401,18 +401,40 @@ void strbuf_utf8_replace(struct strbuf *sb_src, int pos, int width,
 	strbuf_release(&sb_dst);
 }
 
+/*
+ * Returns true (1) if the src encoding name matches the dst encoding
+ * name directly or one of its alternative names. E.g. UTF-16BE is the
+ * same as UTF16BE.
+ */
+static int same_utf_encoding(const char *src, const char *dst)
+{
+	if (istarts_with(src, "utf") && istarts_with(dst, "utf")) {
+		/* src[3] or dst[3] might be '\0' */
+		int i = (src[3] == '-' ? 4 : 3);
+		int j = (dst[3] == '-' ? 4 : 3);
+		return !strcasecmp(src+i, dst+j);
+	}
+	return 0;
+}
+
 int is_encoding_utf8(const char *name)
 {
 	if (!name)
 		return 1;
-	if (!strcasecmp(name, "utf-8") || !strcasecmp(name, "utf8"))
+	if (same_utf_encoding("utf-8", name))
 		return 1;
 	return 0;
 }
 
 int same_encoding(const char *src, const char *dst)
 {
-	if (is_encoding_utf8(src) && is_encoding_utf8(dst))
+	static const char utf8[] = "UTF-8";
+
+	if (!src)
+		src = utf8;
+	if (!dst)
+		dst = utf8;
+	if (same_utf_encoding(src, dst))
 		return 1;
 	return !strcasecmp(src, dst);
 }
-- 
2.16.2
Previous: lars.schneider@autodesk.comNext: lars.schneider@autodesk.com
Message 4 of 11 in “convert: add support for different encodings”
  1. 00/10 convert: add support for different encodingslars.schneider@autodesk.com, Apr 15, 2018
  2. 01/10 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Apr 15, 2018
  3. 02/10 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Apr 15, 2018
  4. 04/10 utf8: teach same_encoding() alternative UTF encoding nameslars.schneider@autodesk.com, Apr 15, 2018
  5. 05/10 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Apr 15, 2018
  6. 03/10 strbuf: add a case insensitive starts_with()lars.schneider@autodesk.com, Apr 15, 2018
  7. 06/10 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Apr 15, 2018
  8. 08/10 convert: check for detectable errors in UTF encodingslars.schneider@autodesk.com, Apr 15, 2018
  9. 07/10 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Apr 15, 2018
  10. 09/10 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Apr 15, 2018
  11. 10/10 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Apr 15, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.