Re: [PATCH/RFC v5 7/7] Careful with CRLF when using e.g. UTF-16 for working-tree-encoding

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 7, 2018, 18:12 UTC
Message-ID: <xmqq372c8y9n.fsf@gitster-ct.c.googlers.com>
In-Reply-To: <20180207063147.GA22714@tor.lan>

Torsten Bögershausen <tboegi@web.de> writes:

Show 9 quoted lines

>> the user explicitly tells us it is in UTF-16, right?  Is there such a
>> thing as UTF-16 binary?
>
> I don't think so, by definiton UTF-16 is ment to be text.
> (this means that git ls-files --eol needs some update, I can have a look)
>
> Do we agree that UTF-16 is text ?
> If yes, could Git assume that the "text" attribute is set when
> working-tree-encoding is set ?

These are two different questions. It seems that between the two of us, we established that "UTF-16 binary" is a nonsense, and a path that is given working-tree-encoding=UTF-16 must be treated as holding a text file. But that does not have direct relevance to the other question you are asking: "is a question 'does this path have text attribute set?' be answered with 'yes' if the path has wte attribute set to UTF-16?" I think the answer to that latter question ought to be "no".

By the way, a related tangent is if it makes sense to give working-tree-encoding to anything that is binary, regardless of the value---I am inclined to say it is not, so the issue here is not "by definition UTF-16 is text", but "any path that has wte set to some enconding should be treated the same way as if the path also has text attribute set as far as convert machinery is concerned.".