git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 5/7] convert: add 'working-tree-encoding' attribute

From
Lars Schneider <larsxschneider@gmail.com>
Date
Feb 14, 2018, 13:22 UTC
Message-ID
<E13489D5-5BED-4C30-A153-4146822C3DC6@gmail.com>
In-Reply-To
<20180210094852.GB11525@tor.lan>
Show 14 quoted lines
> On 10 Feb 2018, at 10:48, Torsten Bögershausen <tboegi@web.de> wrote:
> 
> On Fri, Feb 09, 2018 at 02:28:28PM +0100, lars.schneider@autodesk.com wrote:
>> From: Lars Schneider <larsxschneider@gmail.com>
>> 
>> ...
>> 
>> +Please note that using the `working-tree-encoding` attribute may have a
>> +number of pitfalls:
>> +
>> +- Git clients that do not support the `working-tree-encoding` attribute
> 
> A client to Git ?
> Or may be "third party Git implementations"
OK, I'll go with "Third party Git implementations".
Show 31 quoted lines
>> 
>> +As an example, use the following attributes if your '*.proj' files are
>> +UTF-16 encoded with byte order mark (BOM) and you want Git to perform
>> +automatic line ending conversion based on your platform.
>> +
>> +------------------------
>> +*.proj		text working-tree-encoding=UTF-16
>> +------------------------
>> +
>> +Use the following attributes if your '*.proj' files are UTF-16 little
>> +endian encoded without BOM and you want Git to use Windows line endings
>> +in the working directory. Please note, it is highly recommended to
>> +explicitly define the line endings with `eol` if the `working-tree-encoding`
>> +attribute is used to avoid ambiguity.
>> +
>> +------------------------
>> +*.proj 		working-tree-encoding=UTF-16LE text eol=CRLF
>> +------------------------
>> +
>> +You can get a list of all available encodings on your platform with the
>> +following command:
> 
> One question:
> +*.proj		text working-tree-encoding=UTF-16
> vs
> *.proj 		working-tree-encoding=UTF-16LE text eol=CRLF
> 
> Technically the order of attributes doesn't matter, but that is not what we
> want to demonstrate here and now.
> I would probably move the "text" attribute to the end of the line.
> So that readers don't start to wonder if the order is important.

I agree in general. However, I would move "text" to the beginning to be consistent with the gitattribute pattern above. OK?

Show 26 quoted lines
>> 
>> +	if (has_prohibited_utf_bom(enc->name, src, src_len)) {
>> +		const char *error_msg = _(
>> +			"BOM is prohibited for '%s' if encoded as %s");
>> +		const char *advise_msg = _(
>> +			"You told Git to treat '%s' as %s. A byte order mark "
>> +			"(BOM) is prohibited with this encoding. Either use "
>> +			"%.6s as working tree encoding or remove the BOM from the "
>> +			"file.");
> 
> "You told Git" is probly right from Gits point of view, and advises are really helpfull.
> But what should the user do about it ?
> Could we give a better advise ?
> 
> 
> "A byte order mark (BOM) is prohibited with %s.
> Please remove the BOM from the file %s 
> or use "%s as working-tree-encoding"
> 
> I would probably suspect that a tool wrote the BOM, and that is
> good and can or should not be changed by a user.
> 
> So a simply message like this could be the preferred (and only)
> solution for a user:
> "A byte order mark (BOM) is prohibited with %s.
> Please use "%s as working-tree-encoding"
OK. I like the last one!
> (And why %.6s and not simply %s ?)

The encodings is UTF-16LE, UTF-16BE, UTF-32LE, or UTF-32BE. I just use the first 6 characters to print the encoding that allows BOMs (UTF-16 or UTF-32). I'll add a comment to explain the trickery in the code!

Thanks, Lars

Previous: Torsten BögershausenNext: lars.schneider@autodesk.com
Message 10 of 14 in “convert: add support for different encodings”
  1. 0/7 convert: add support for different encodingslars.schneider@autodesk.com, Feb 9, 2018
  2. 1/7 strbuf: remove unnecessary NUL assignment in xstrdup_tolower()lars.schneider@autodesk.com, Feb 9, 2018
  3. 2/7 strbuf: add xstrdup_toupper()lars.schneider@autodesk.com, Feb 9, 2018
  4. 3/7 utf8: add function to detect prohibited UTF-16/32 BOMlars.schneider@autodesk.com, Feb 9, 2018
  5. 4/7 utf8: add function to detect a missing UTF-16/32 BOMlars.schneider@autodesk.com, Feb 9, 2018
  6. Junio C HamanoFeb 9, 2018
  7. Lars SchneiderFeb 9, 2018
  8. 5/7 convert: add 'working-tree-encoding' attributelars.schneider@autodesk.com, Feb 9, 2018
  9. Torsten BögershausenFeb 10, 2018
  10. Lars SchneiderFeb 14, 2018
  11. 7/7 convert: add round trip check based on 'core.checkRoundtripEncoding'lars.schneider@autodesk.com, Feb 9, 2018
  12. 6/7 convert: add tracing for 'working-tree-encoding' attributelars.schneider@autodesk.com, Feb 9, 2018
  13. Junio C HamanoFeb 9, 2018
  14. Lars SchneiderFeb 10, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.