Re: non-US-ASCII file names (e.g. Hiragana) on Windows

From: Reece Dunn <msclrhd@googlemail.com>
Date: Nov 28, 2009, 23:37 UTC
Message-ID: <3f4fd2640911281537m3a3d7f5dl7836a613b48128f2@mail.gmail.com>
In-Reply-To: <4B1168D4.5010902@syntevo.com>

2009/11/28 Thomas Singer <thomas.singer@syntevo.com>:

Show 11 quoted lines

>
> When launching 'git status' from the git shell (msys 1.6.5.1.1367.gcd48 from
> 7zip-bundle) it only shows me 4 question marks. I would have expected to see
> the non-displayable characters escaped like it did with the umlauts on OS X.
>
> Even adding fails:
>
> $ git add .
> fatal: unable to stat '????': No such file or directory
>
> What should I do to make Git recognize these characters?

This is a bug in git's character encoding/conversion logic. It looks like git is taking the source string and converting it to ascii to be displayed on the console output (e.g. by using the WideCharToMultiByte conversion API) -- these APIs will use a '?' character for characters that it cannot map to the target character encoding (like the Hiragana characters that you are using).

SetConsoleOutputCP can be used to change the console output codepage [http://msdn.microsoft.com/en-us/library/ms686036%28VS.85%29.aspx] and SetConsoleCP is the equivalent for input [http://msdn.microsoft.com/en-us/library/ms686013%28VS.85%29.aspx]. e.g.

    SetConsoleCP(CP_UTF8);
    SetConsoleOutputCP(CP_UTF8);

should make the console process UTF-8 characters, so git shouldn't need to do any character conversions on Windows when reading/writing it's data.

NOTE: I have not tested this, just noting what I have found via Google.

- Reece