git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v11 08/10] convert: advise canonical UTF encoding names

From
lars.schneider@autodesk.com <lars.schneider@autodesk.com>
Date
Mar 9, 2018, 17:35 UTC
Message-ID
<20180309173536.62012-9-lars.schneider@autodesk.com>
In-Reply-To
<20180309173536.62012-1-lars.schneider@autodesk.com>
From: Lars Schneider <larsxschneider@gmail.com>

The canonical name of an UTF encoding has the format UTF, dash, number, and an optionally byte order in upper case (e.g. UTF-8 or UTF-16BE). Some iconv versions support alternative names without a dash or with lower case characters.

To avoid problems between different iconv version always suggest the canonical UTF names in advise messages.

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 convert.c                        | 21 +++++++++++++++++----
 t/t0028-working-tree-encoding.sh | 10 ++++++++--
 2 files changed, 25 insertions(+), 6 deletions(-)
diff --git a/convert.c b/convert.c
index b80d666a6b..9a3ae7cce1 100644
--- a/convert.c
+++ b/convert.c
@@ -279,12 +279,20 @@ static int validate_encoding(const char *path, const char *enc,
 				"BOM is prohibited in '%s' if encoded as %s");
 			/*
 			 * This advice is shown for UTF-??BE and UTF-??LE encodings.
+			 * We cut off the last two characters of the encoding name
+			 # to generate the encoding name suitable for BOMs.
 			 */
 			const char *advise_msg = _(
 				"The file '%s' contains a byte order "
-				"mark (BOM). Please use %.6s as "
+				"mark (BOM). Please use UTF-%s as "
 				"working-tree-encoding.");
-			advise(advise_msg, path, enc);
+			const char *stripped = "";
+			char *upper = xstrdup_toupper(enc);
+			upper[strlen(upper)-2] = '\0';
+			if (!skip_prefix(upper, "UTF-", &stripped))
+				skip_prefix(stripped, "UTF", &stripped);
+			advise(advise_msg, path, stripped);
+			free(upper);
 			if (die_on_error)
 				die(error_msg, path, enc);
 			else {
@@ -296,10 +304,15 @@ static int validate_encoding(const char *path, const char *enc,
 				"BOM is required in '%s' if encoded as %s");
 			const char *advise_msg = _(
 				"The file '%s' is missing a byte order "
-				"mark (BOM). Please use %sBE or %sLE "
+				"mark (BOM). Please use UTF-%sBE or UTF-%sLE "
 				"(depending on the byte order) as "
 				"working-tree-encoding.");
-			advise(advise_msg, path, enc, enc);
+			const char *stripped = "";
+			char *upper = xstrdup_toupper(enc);
+			if (!skip_prefix(upper, "UTF-", &stripped))
+				skip_prefix(stripped, "UTF", &stripped);
+			advise(advise_msg, path, stripped, stripped);
+			free(upper);
 			if (die_on_error)
 				die(error_msg, path, enc);
 			else {
diff --git a/t/t0028-working-tree-encoding.sh b/t/t0028-working-tree-encoding.sh
index e8408dfe5c..1bee7b9f71 100755
--- a/t/t0028-working-tree-encoding.sh
+++ b/t/t0028-working-tree-encoding.sh
@@ -74,18 +74,22 @@ do
 		cp bebom.utf${i}be.raw bebom.utf${i}be &&
 		test_must_fail git add bebom.utf${i}be 2>err.out &&
 		test_i18ngrep "fatal: BOM is prohibited .* utf-${i}be" err.out &&
+		test_i18ngrep "use UTF-${i} as working-tree-encoding" err.out &&
 
 		cp lebom.utf${i}le.raw lebom.utf${i}be &&
 		test_must_fail git add lebom.utf${i}be 2>err.out &&
 		test_i18ngrep "fatal: BOM is prohibited .* utf-${i}be" err.out &&
+		test_i18ngrep "use UTF-${i} as working-tree-encoding" err.out &&
 
 		cp bebom.utf${i}be.raw bebom.utf${i}le &&
 		test_must_fail git add bebom.utf${i}le 2>err.out &&
 		test_i18ngrep "fatal: BOM is prohibited .* utf-${i}LE" err.out &&
+		test_i18ngrep "use UTF-${i} as working-tree-encoding" err.out &&
 
 		cp lebom.utf${i}le.raw lebom.utf${i}le &&
 		test_must_fail git add lebom.utf${i}le 2>err.out &&
-		test_i18ngrep "fatal: BOM is prohibited .* utf-${i}LE" err.out
+		test_i18ngrep "fatal: BOM is prohibited .* utf-${i}LE" err.out &&
+		test_i18ngrep "use UTF-${i} as working-tree-encoding" err.out
 	'
 
 	test_expect_success "check required UTF-${i} BOM" '
@@ -96,10 +100,12 @@ do
 		cp nobom.utf${i}be.raw nobom.utf${i} &&
 		test_must_fail git add nobom.utf${i} 2>err.out &&
 		test_i18ngrep "fatal: BOM is required .* utf-${i}" err.out &&
+		test_i18ngrep "use UTF-${i}BE or UTF-${i}LE" err.out &&
 
 		cp nobom.utf${i}le.raw nobom.utf${i} &&
 		test_must_fail git add nobom.utf${i} 2>err.out &&
-		test_i18ngrep "fatal: BOM is required .* utf-${i}" err.out
+		test_i18ngrep "fatal: BOM is required .* utf-${i}" err.out &&
+		test_i18ngrep "use UTF-${i}BE or UTF-${i}LE" err.out
 	'
 
 	test_expect_success "eol conversion for UTF-${i} encoded files on checkout" '

-- 
2.16.2
Previous: lars.schneider@autodesk.comNext: Junio C Hamano
Message 3 of 25 in “convert: add support for different encodings”
  1. 00/10 convert: add support for different encodingslars.schneider@autodesk.com, Mar 9, 2018
  2. 02/10 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Mar 9, 2018
  3. 08/10 convert: advise canonical UTF encoding nameslars.schneider@autodesk.com, Mar 9, 2018
  4. Junio C HamanoMar 9, 2018
  5. Lars SchneiderMar 15, 2018
  6. 03/10 strbuf: add a case insensitive starts_with()lars.schneider@autodesk.com, Mar 9, 2018
  7. 09/10 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 9, 2018
  8. 07/10 convert: check for detectable errors in UTF encodingslars.schneider@autodesk.com, Mar 9, 2018
  9. Junio C HamanoMar 9, 2018
  10. Lars SchneiderMar 9, 2018
  11. Junio C HamanoMar 9, 2018
  12. 06/10 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Mar 9, 2018
  13. Junio C HamanoMar 9, 2018
  14. Lars SchneiderMar 15, 2018
  15. Torsten BögershausenMar 18, 2018
  16. Lars SchneiderApr 1, 2018
  17. Torsten BögershausenApr 5, 2018
  18. Lars SchneiderApr 15, 2018
  19. 10/10 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Mar 9, 2018
  20. Eric SunshineMar 9, 2018
  21. Junio C HamanoMar 9, 2018
  22. Eric SunshineMar 9, 2018
  23. 01/10 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Mar 9, 2018
  24. 05/10 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Mar 9, 2018
  25. 04/10 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Mar 9, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.