Re: [RFC PATCH] Windows: Assume all file names to be UTF-8 encoded.

From: Peter Krefting <peter@softwolves.pp.se>
Date: Mar 2, 2009, 20:52 UTC
Message-ID: <alpine.DEB.2.00.0903022146510.20047@perkele.intern.softwolves.pp.se>
In-Reply-To: <200903022058.33392.robin.rosenberg.lists@dewire.com>

Robin Rosenberg:

> Pipes are just bytes so you have to know what you're piping by convention 
> or protocol. You can ask for the console output page, which may be set to 
> a multibyte locale or unicode and maybe trust that.... (just guessing, 
> really).

You can get cmd.exe to write data to pipes and redirections as UTF-16 Unicode (cmd.exe /u), perhaps there is a way to capitalise on that? "Unfortunately", the Git stuff is mostly called from a bash shell inside msys, so it requires a "bit" more work...

> architecture? Like the "architecture" of species? No, it's evolution.

There's still an architecture there, somewhere. Perhaps not intended or specified, but there definitely is one :-)

> http://www.jgit.org/cgi-bin/gitweb/gitweb.cgi?p=GIT.git;a=shortlog;h=i18n
>
> The goal is locale neutrality yielding the "expected", in the users eyes, 
> result regardless of locale as much as possible.

Ah, yes, that looks like an interesting starting point. I already assumed that Git on Linux would use UTF-8 for everything already, since it already does that for the commit messages despite me using an iso8859-1 locale. Apparently I haven't done my homework.

> We let the runtime decide on how to encode file names in the file system 
> using the user's locale.

That's good. That's what I'm trying to achieve. Or, rather, avoid the user locale altogether (which is easy on Windows since the file names are always stored in Unicode, and the user locale can be bypassed).

> I'd be almost happy with a solution that works when people are interacting 
> using the subset that is convertible between the character sets in use.

You mean like the "invariant" character set? :-) Using Unicode internally (in whatever encoding) is nice, the problem is when you have to interact with the world around you.

-- 
\\// Peter - http://www.softwolves.pp.se/