Re: [PATCH v9 4/8] utf8: add function to detect a missing UTF-16/32 BOM

From: Torsten Bögershausen <tboegi@web.de>
Date: Mar 7, 2018, 17:39 UTC
Message-ID: <20180307173911.GA19262@tor.lan>
In-Reply-To: <xmqqpo4g22tv.fsf@gitster-ct.c.googlers.com>

On Tue, Mar 06, 2018 at 03:37:16PM -0800, Junio C Hamano wrote:

Show 14 quoted lines

> Lars Schneider <larsxschneider@gmail.com> writes:
> 
> > After thinking about it I wonder if we should barf on "utf16" without
> > dash. Your Linux iconv would handle this correctly. My macOS iconv would not.
> > That means the repo would checkout correctly on your machine but not on mine.
> >
> > What do you think?
> 
> To be bluntly honest, I prefer not to have excess "sanity checks";
> there is no need to barf on utf16 in a project run by those who are
> without macOS friends, for example.  For that matter, while I do not
> hate it so much to reject it, I am not all that supportive of this
> "The consortium says without -LE or -BE suffix there must be BOM,
> and this lacks one, so barf loudly" step in this topic myself.

Loosing or adding a BOM couild render a file useless, depending on the SW that reads and processes it.

The main reason for being critacal is to make sure that the material in the working-tree does a safe roundtrip when commited, pushed, pulled, checked out...

The best thing we can do is probably to follow the specification as good as possible.

Having a clear specification (UTF-16LE/BE has no BOM, UTF-16 has none, UTF-16 has a BOM) would even open the chance to work around a buggy iconv library, because Git can check the BOM. If, and only if, a platform messes it up, and we want to make a (compile time) workaround in Git for this very platform.

A consistant behavior across all platforms is a good thing, sometimes I have a network share mounted to Linux, MacOS and Windows and it doesn't matter under which Git on which machine I checkout or commit.

Oh, I see, there is a new patch coming...