Re: [RFC] Print diffs of UTF-16 to console / patches to email as UTF-8...?

From: Jakub Narebski <jnareb@gmail.com>
Date: Oct 22, 2010, 17:48 UTC
Message-ID: <m31v7iktkn.fsf@localhost.localdomain>
In-Reply-To: <1287768426.31218.75.camel@drew-northup.unet.maine.edu>

Drew Northup <drew.northup@maine.edu> writes:

Show 22 quoted lines

> On Fri, 2010-10-22 at 12:12 -0500, Jonathan Nieder wrote:
> > 
> > Drew Northup wrote:
> > 
> > > That still doesn't fix the crlf issue, for starters. Also, I would like
> > > to be able to email patches for files that are in UTF-16 and properly
> > > re-import them. Unless I'm missing something really big there's not much
> > > that a display filter is going to do for me there.
> > 
> > Right, I think you're missing something big.  textconv is a display
> > filter.  clean/smudge convert between internal and external
> > representation (and your clean/smudge scripts could take care of CRLF
> > themselves if desired).
> > 
> > That said, I wouldn't be surprised if clean/smudge filters don't do
> > everything you want.  If you do go that way, please keep the list
> > posted so the mechanism can be improved.
> 
> Well I shall plumb the documentation again.... just in case. I'm not
> holding my breath that it will do what I (and frankly a fair number of
> other people) want. We just want version control that treats text like
> text. FULL STOP. Why isn't UTF-16 text???????

If you are asking why Git detects files with text in UTF-16 / USC-2 as binary, it is because Git (re)uses the same heuristic that e.g. GNU diff (and probably also -T file test in Perl), and one of heuristics is that if file contains NUL ("\0") character, then it is most porbably binary (because legacy C programs for text would have troubles with NUL characters).

That probably doesn't help you any...

-- 
Jakub Narebski
Poland
ShadeHawk on #git