threads / discuss / 35393

gettext CTYPE for libc

Subject: gettext CTYPE for libc

## tl;dr

5 messages between Nov 23, 2013 and Nov 25, 2013.

replies: 4people: 4as markdown or json

Trần Ngọc Quân· Nov 23, 2013, 02:15 UTC · lore

Hello,

$ mkdir xyz $ cd xyz $ rmdir ../xyz $ git status fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c th? m?c nh? v?y

So, somthing wrong with our charset.

$ strace git status 2>&1 | grep open open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 open("/lib/i386-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3 open("/lib/i386-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3 open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3 open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 open("/dev/null", O_RDWR|O_LARGEFILE) = 3 open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3 open("/usr/share/locale/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/vi/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale-langpack/vi/LC_MESSAGES/libc.mo", O_RDONLY) = 3 open("/usr/lib/i386-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3

We will see, this string come from libc.mo $ gettext --domain=libc "No such file or directory" Không có tập tin hoặc thư mục như vậy

in git's gettext.c, it not allow CTYPE="" for all domain, so we will set this one individually. In this ex. I set it for libc:

$ git diff

diff --git a/gettext.c b/gettext.c
index 71e9545..abd3978 100644
--- a/gettext.c
+++ b/gettext.c
@@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
        setlocale(LC_CTYPE, "");
        charset = locale_charset();
        bind_textdomain_codeset(domain, charset);
+       bind_textdomain_codeset("libc", charset);
        setlocale(LC_CTYPE, "C");
 }

And it work as I expect!

-- 
Trần Ngọc Quân.

Thomas Rast· Nov 24, 2013, 09:05 UTC · re: Trần Ngọc Quân · lore

Re: gettext CTYPE for libc

Trần Ngọc Quân <vnwildman@gmail.com> writes:

Show 5 quoted lines

> $ git status
> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
> th? m?c nh? v?y
>
> So, somthing wrong with our charset.

[...]

Show 18 quoted lines

> $ gettext --domain=libc "No such file or directory"
> Không có tập tin hoặc thư mục như vậy
>
> in git's gettext.c, it not allow CTYPE="" for all domain, so we will set
> this one individually. In this ex. I set it for libc:
>
> $ git diff
> diff --git a/gettext.c b/gettext.c
> index 71e9545..abd3978 100644
> --- a/gettext.c
> +++ b/gettext.c
> @@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
>         setlocale(LC_CTYPE, "");
>         charset = locale_charset();
>         bind_textdomain_codeset(domain, charset);
> +       bind_textdomain_codeset("libc", charset);
>         setlocale(LC_CTYPE, "C");
>  }

Do you know why this "suddenly" broke? The long comment in init_gettext_charset() suggests that the *existing* code is there to handle exactly this problem, and apparently it doesn't. Why? Has libc moved the perror() strings into a separate domain in some version?

-- 
Thomas Rast
tr@thomasrast.ch

Trần Ngọc Quân· Nov 25, 2013, 01:32 UTC · re: Thomas Rast · lore

Re: gettext CTYPE for libc

On 24/11/2013 16:05, Thomas Rast wrote:

Show 9 quoted lines

> Trần Ngọc Quân <vnwildman@gmail.com> writes:
>
>> $ git status
>> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
>> th? m?c nh? v?y
>>
>> So, somthing wrong with our charset.
> [...]
> Do you know why this "suddenly" broke?

I think git set CTYPE="C" for libc, so charset become 7-bit ASCII, but it don't set LC_MESSAGES="C" for libc and libc will get this one from system variable.

> The long comment in
> init_gettext_charset() suggests that the *existing* code is there to
> handle exactly this problem, and apparently it doesn't.  Why?  Has libc
> moved the perror() strings into a separate domain in some version?

See setlocale(3) [1] I'm a newbie in GIT. I'm not sure about git work correctly [2] if set git's charset to same with system. I don't think libc moved perror() string in separate domain. It use its own domain.

[1] http://man7.org/linux/man-pages/man3/setlocale.3.html [2] incorrect if some function need work in ASCII mode

-- 
Trần Ngọc Quân.

Duy Nguyen· Nov 25, 2013, 07:20 UTC · re: Trần Ngọc Quân · lore

Re: gettext CTYPE for libc

Jonathan I see you participated in bug 6530, mentioned in the big comment block in init_gettext_charset(). The bug seems fixed since glibc 2.17. Do you think it's ok to revert the workaround if we detect the running glibc is fixed (or if it does not run glibc at all)? I think we could use gnu_get_libc_version() to detect it.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=6530

On Sat, Nov 23, 2013 at 9:15 AM, Trần Ngọc Quân <vnwildman@gmail.com> wrote:

Show 59 quoted lines

> Hello,
>
> $ mkdir xyz
> $ cd xyz
> $ rmdir ../xyz
> $ git status
> fatal: Unable to read current working directory: Kh?ng c? t?p tin ho?c
> th? m?c nh? v?y
>
> So, somthing wrong with our charset.
>
> $ strace git status 2>&1 | grep open
> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libcrypto.so.1.0.0", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
> open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
> open("/dev/null", O_RDWR|O_LARGEFILE)   = 3
> open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
> open("/usr/share/locale/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) = -1
> ENOENT (No such file or directory)
> open("/usr/share/locale/vi/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT
> (No such file or directory)
> open("/usr/share/locale-langpack/vi_VN/LC_MESSAGES/libc.mo", O_RDONLY) =
> -1 ENOENT (No such file or directory)
> open("/usr/share/locale-langpack/vi/LC_MESSAGES/libc.mo", O_RDONLY) = 3
> open("/usr/lib/i386-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 3
>
> We will see, this string come from libc.mo
> $ gettext --domain=libc "No such file or directory"
> Không có tập tin hoặc thư mục như vậy
>
> in git's gettext.c, it not allow CTYPE="" for all domain, so we will set
> this one individually. In this ex. I set it for libc:
>
> $ git diff
> diff --git a/gettext.c b/gettext.c
> index 71e9545..abd3978 100644
> --- a/gettext.c
> +++ b/gettext.c
> @@ -115,6 +115,7 @@ static void init_gettext_charset(const char *domain)
>         setlocale(LC_CTYPE, "");
>         charset = locale_charset();
>         bind_textdomain_codeset(domain, charset);
> +       bind_textdomain_codeset("libc", charset);
>         setlocale(LC_CTYPE, "C");
>  }
>
> And it work as I expect!
>
> --
> Trần Ngọc Quân.
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Duy

Jonathan Nieder· Nov 25, 2013, 19:31 UTC · re: Duy Nguyen · lore

Re: gettext CTYPE for libc

Duy Nguyen wrote:

>             Do you think it's ok to revert the workaround if we detect
> the running glibc is fixed (or if it does not run glibc at all)? I
> think we could use gnu_get_libc_version() to detect it.

That would be wonderful.

Thanks, Jonathan