Re: git-rebase is ignoring working-tree-encoding

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Dec 26, 2018, 19:25 UTC
Message-ID: <20181226192525.GB423984@genre.crustytoothpaste.net>
In-Reply-To: <002201d49cb5$cc554160$64ffc420$@gmail.com>

On Tue, Dec 25, 2018 at 04:56:11PM -0800, Alexandre Grigoriev wrote:

Show 14 quoted lines

> Many tools in Windows still do not understand UTF-8, although it's getting
> better. I think Windows is about the only OS where tools still require
> UTF-16 for full internationalization.
> Many tools written in C use MSVC RTL, where fopen(), unfortunately, doesn't
> understand UTF-16BE (though such a rudimentary program as Notepad does).
> 
> For this reason, it's very reasonable to ask that the programming tools
> produce UTF-16 files with particular endianness, natural for the platform
> they're running on.
> 
> The iconv programmers' boneheaded decision to always produce UTF-16BE with
> BOM for UTF-16 output doesn't make sense.
> Again, git and iconv/libiconv in Centos on x86 do the right thing and
> produce UTF-16LE with BOM in this case.

A program which claims to support "UTF-16" must support both endiannesses, according to RFC 2781. A program writing UTF-16-LE must not write a BOM at the beginning. I realize this is inconvenient, but the bad behavior of some Windows programs doesn't mean that Git should ignore interoperability with non-Windows systems using UTF-16 correctly in favor of Windows.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204