git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v7 08/12] gettext: add is_utf8_locale()

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Feb 14, 2016, 11:49 UTC
Message-ID
<1455450596-11904-9-git-send-email-pclouds@gmail.com>
In-Reply-To
<1455450596-11904-1-git-send-email-pclouds@gmail.com>

This function returns true if git is running under an UTF-8 locale. pcre in the next patch will need this.

is_encoding_utf8() is used instead of strcmp() to catch both "utf-8" and "utf8" suffixes.

When built with no gettext support, we peek in several env variables to detect UTF-8. pcre library might support utf-8 even if libc is built without locale support.. The peeking code is a copy from compat/regex/regcomp.c

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 gettext.c | 24 ++++++++++++++++++++++--
 gettext.h |  1 +
 2 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/gettext.c b/gettext.c
index a268a2c..db727ea 100644
--- a/gettext.c
+++ b/gettext.c
@@ -18,6 +18,8 @@
 #	endif
 #endif
 
+static const char *charset;
+
 /*
  * Guess the user's preferred languages from the value in LANGUAGE environment
  * variable and LC_MESSAGES locale category if NO_GETTEXT is not defined.
@@ -65,7 +67,6 @@ static int test_vsnprintf(const char *fmt, ...)
 	return ret;
 }
 
-static const char *charset;
 static void init_gettext_charset(const char *domain)
 {
 	/*
@@ -172,8 +173,27 @@ int gettext_width(const char *s)
 {
 	static int is_utf8 = -1;
 	if (is_utf8 == -1)
-		is_utf8 = !strcmp(charset, "UTF-8");
+		is_utf8 = is_utf8_locale();
 
 	return is_utf8 ? utf8_strwidth(s) : strlen(s);
 }
 #endif
+
+int is_utf8_locale(void)
+{
+#ifdef NO_GETTEXT
+	if (!charset) {
+		const char *env = getenv("LC_ALL");
+		if (!env || !*env)
+			env = getenv("LC_CTYPE");
+		if (!env || !*env)
+			env = getenv("LANG");
+		if (!env)
+			env = "";
+		if (strchr(env, '.'))
+			env = strchr(env, '.') + 1;
+		charset = xstrdup(env);
+	}
+#endif
+	return is_encoding_utf8(charset);
+}
diff --git a/gettext.h b/gettext.h
index 33696a4..7eee64a 100644
--- a/gettext.h
+++ b/gettext.h
@@ -90,5 +90,6 @@ const char *Q_(const char *msgid, const char *plu, unsigned long n)
 #endif
 
 const char *get_preferred_languages(void);
+extern int is_utf8_locale(void);
 
 #endif
-- 
2.7.0.377.g4cd97dd
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 45 of 56 in “Fix icase grep on non-ascii”
  1. 00/10 Fix icase grep on non-asciiNguyễn Thái Ngọc Duy, Jan 28, 2016
  2. 01/10 grep: allow -F -i combinationNguyễn Thái Ngọc Duy, Jan 28, 2016
  3. 02/10 grep: break down an "if" stmt in preparation for next changesNguyễn Thái Ngọc Duy, Jan 28, 2016
  4. 03/10 test-regex: expose full regcomp() to the command lineNguyễn Thái Ngọc Duy, Jan 28, 2016
  5. Eric SunshineJan 29, 2016
  6. Ramsay JonesJan 29, 2016
  7. 04/10 grep/icase: avoid kwsset on literal non-ascii stringsNguyễn Thái Ngọc Duy, Jan 28, 2016
  8. Eric SunshineJan 29, 2016
  9. Eric SunshineJan 29, 2016
  10. 05/10 grep/icase: avoid kwsset when -F is specifiedNguyễn Thái Ngọc Duy, Jan 28, 2016
  11. Eric SunshineJan 29, 2016
  12. 06/10 grep/pcre: prepare locale-dependent tables for icase matchingNguyễn Thái Ngọc Duy, Jan 28, 2016
  13. 07/10 gettext: add is_utf8_locale()Nguyễn Thái Ngọc Duy, Jan 28, 2016
  14. 08/10 grep/pcre: support utf-8Nguyễn Thái Ngọc Duy, Jan 28, 2016
  15. 09/10 diffcore-pickaxe: "share" regex error handling codeNguyễn Thái Ngọc Duy, Jan 28, 2016
  16. 10/10 diffcore-pickaxe: support case insensitive match on non-asciiNguyễn Thái Ngọc Duy, Jan 28, 2016
  17. Eric SunshineJan 29, 2016
  18. Junio C HamanoJan 28, 2016
  19. 00/11 Fix icase grep on non-asciiNguyễn Thái Ngọc Duy, Feb 6, 2016
  20. 01/11 grep: allow -F -i combinationNguyễn Thái Ngọc Duy, Feb 6, 2016
  21. Junio C HamanoJun 17, 2016
  22. Duy NguyenJun 18, 2016
  23. 02/11 grep: break down an "if" stmt in preparation for next changesNguyễn Thái Ngọc Duy, Feb 6, 2016
  24. Junio C HamanoFeb 9, 2016
  25. 03/11 test-regex: isolate the bug test codeNguyễn Thái Ngọc Duy, Feb 6, 2016
  26. 04/11 test-regex: expose full regcomp() to the command lineNguyễn Thái Ngọc Duy, Feb 6, 2016
  27. Eric SunshineFeb 7, 2016
  28. Junio C HamanoFeb 9, 2016
  29. 05/11 grep/icase: avoid kwsset on literal non-ascii stringsNguyễn Thái Ngọc Duy, Feb 6, 2016
  30. 06/11 grep/icase: avoid kwsset when -F is specifiedNguyễn Thái Ngọc Duy, Feb 6, 2016
  31. 07/11 grep/pcre: prepare locale-dependent tables for icase matchingNguyễn Thái Ngọc Duy, Feb 6, 2016
  32. 08/11 gettext: add is_utf8_locale()Nguyễn Thái Ngọc Duy, Feb 6, 2016
  33. 09/11 grep/pcre: support utf-8Nguyễn Thái Ngọc Duy, Feb 6, 2016
  34. 10/11 diffcore-pickaxe: "share" regex error handling codeNguyễn Thái Ngọc Duy, Feb 6, 2016
  35. 11/11 diffcore-pickaxe: support case insensitive match on non-asciiNguyễn Thái Ngọc Duy, Feb 6, 2016
  36. Eric SunshineFeb 7, 2016
  37. 00/12 nd/icase updatesNguyễn Thái Ngọc Duy, Feb 14, 2016
  38. 01/12 grep: allow -F -i combinationNguyễn Thái Ngọc Duy, Feb 14, 2016
  39. 02/12 grep: break down an "if" stmt in preparation for next changesNguyễn Thái Ngọc Duy, Feb 14, 2016
  40. 03/12 test-regex: isolate the bug test codeNguyễn Thái Ngọc Duy, Feb 14, 2016
  41. 04/12 test-regex: expose full regcomp() to the command lineNguyễn Thái Ngọc Duy, Feb 14, 2016
  42. 05/12 grep/icase: avoid kwsset on literal non-ascii stringsNguyễn Thái Ngọc Duy, Feb 14, 2016
  43. 06/12 grep/icase: avoid kwsset when -F is specifiedNguyễn Thái Ngọc Duy, Feb 14, 2016
  44. 07/12 grep/pcre: prepare locale-dependent tables for icase matchingNguyễn Thái Ngọc Duy, Feb 14, 2016
  45. 08/12 gettext: add is_utf8_locale()Nguyễn Thái Ngọc Duy, Feb 14, 2016
  46. 09/12 grep/pcre: support utf-8Nguyễn Thái Ngọc Duy, Feb 14, 2016
  47. 10/12 diffcore-pickaxe: "share" regex error handling codeNguyễn Thái Ngọc Duy, Feb 14, 2016
  48. 11/12 diffcore-pickaxe: support case insensitive match on non-asciiNguyễn Thái Ngọc Duy, Feb 14, 2016
  49. 12/12 grep.c: reuse "icase" variableNguyễn Thái Ngọc Duy, Feb 14, 2016
  50. Junio C HamanoJun 17, 2016
  51. Duy NguyenJun 18, 2016
  52. Duy NguyenJun 22, 2016
  53. Junio C HamanoJun 22, 2016
  54. Duy NguyenJun 22, 2016
  55. Junio C HamanoJun 22, 2016
  56. Junio C HamanoJun 22, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.