Re: [BUG?] iconv used as textconv, and spurious ^M on added lines on Windows

From: Jakub Narębski <jnareb@gmail.com>
Date: Apr 1, 2017, 18:31 UTC
Message-ID: <63eb5546-0dce-2f69-c2f8-1e777b97c532@gmail.com>
In-Reply-To: <20170401060800.hrpqqgdx6t262c7f@sigill.intra.peff.net>

W dniu 01.04.2017 o 08:08, Jeff King pisze:

Show 11 quoted lines

> On Fri, Mar 31, 2017 at 03:24:48PM +0200, Jakub Narębski wrote:
> 
>>> I suspect in the normal case that git is doing line-ending conversion,
>>> but it's suppressed when textconv is in use.
>>
>> I would not consider this a bug if not for the fact that there is no ^M
>> without using iconv as textconv.
> 
> I don't think it's a bug, though. You have told Git that you will
> convert the contents (whatever their format) into the canonical format,
> but your program to do so includes a CR.

Well, I have not declared file binary with "binary = true" in diff driver definition, isn't it?

Show 5 quoted lines

> 
> We _could_ further process with other canonicalizations, but I'm not
> sure that is a good idea (line-endings sound reasonably harmless, but
> almost certainly we should not be doing clean/smudge filtering). And I'm
> not sure if there would be any compatibility fallouts.

Yes, gitattributes(5) defines interaction between 'text'/'eol', 'ident' and 'filter' attributes, but nothing about 'diff' and 'text'/'eol'.

Show 7 quoted lines

> 
> So I think the behavior is perhaps not what you want, but it's not an
> unreasonable one. And the solution is to define your textconv such that
> it produces clean LF-only output. Perhaps:
> 
>   [diff.whatever]
>   textconv = "iconv ... | tr -d '\r'"

Well, either this (or equivalent using dos2unix), or using 'whitespace' attribute (or 'core.whitespace') with cr-at-eol.

P.S. What do you think about Git supporting 'encoding' attribute (or 'core.encoding' config) plus 'core.outputEncoding' in-core?

Best,

-- 
Jakub Narębski