Re: Feature request: number of added and deleted lines for UTF-16 files

Александр Третьяк· Mar 8, 2020, 21:37 UTC · lore

Some source files in MSVC (e.g. resource.h) are encoded in UTF-16 (LE)
with Byte Order Mark (i.e. signature: FF FE for little-endian and FE
FF for big-endian), and `git diff --numstat` shows `-` in place of
number of added/removed lines, e.g.:
-       -       clientapp/resource.h

Can you auto detect UTF-16 by reading first 2 bytes of the file, and correctly show number of lines added/removed?

Torsten Bögershausen· Mar 9, 2020, 18:35 UTC · re: Александр Третьяк · lore

On Mon, Mar 09, 2020 at 07:37:56AM +1000, Александр Третьяк wrote:

Show 8 quoted lines

> Some source files in MSVC (e.g. resource.h) are encoded in UTF-16 (LE)
> with Byte Order Mark (i.e. signature: FF FE for little-endian and FE
> FF for big-endian), and `git diff --numstat` shows `-` in place of
> number of added/removed lines, e.g.:
> -       -       clientapp/resource.h
>
> Can you auto detect UTF-16 by reading first 2 bytes of the file, and
> correctly show number of lines added/removed?

There is an attribute "working-tree-encoding" that can be used to have the files in "UTF-16LE-BOM" (in your case) in your working tree and in UTF-8 inside the repo. This will allow commands like "git diff" to display added/deleted lines in UTF-8.

More details may be found here: https://git-scm.com/docs/gitattributes

I hope that helps