threads / discuss / 6270

Re: OT: character encodings (was: Linux 2.6.20-rc4)

Subject: Re: OT: character encodings (was: Linux 2.6.20-rc4)

## tl;dr

4 messages between Jan 7, 2007 and Jan 8, 2007.

replies: 3people: 4as markdown or json

Sean· Jan 7, 2007, 20:15 UTC · lore

On Sun, 7 Jan 2007 15:05:53 -0500 Dave Jones <davej@redhat.com> wrote:

Including the Git list...

Show 31 quoted lines

> On Sun, Jan 07, 2007 at 07:17:30PM +0000, Russell King wrote:
> 
>  > commit 24ebead82bbf9785909d4cf205e2df5e9ff7da32
>  > tree 921f686860e918a01c3d3fb6cd106ba82bf4ace6
>  > parent 264166e604a7e14c278e31cadd1afb06a7d51a11
>  > author Rafa³ Bilski <rafalbilski@interia.pl> 1167691774 +0100
>  > committer Dave Jones <davej@redhat.com> 1167799119 -0500
>  > 
>  > and looking at that "author" closer with od:
>  > 
>  > 0000140 74 68 6f 72 20 52 61 66 61 b3 20 42 69 6c 73 6b
>  >           t   h   o   r       R   a   f   a   ³       B   i   l   s   k
>  > 
>  > clearly not UTF-8.  I doubt whether any of the commits I do on my
>  > en_GB ISO-8859-1 systems end up being UTF-8 encoded.
> 
> This has been bugging me for a while.
> Viewing the mail I applied in mutt shows his name correctly as Rafał
> Applying it with git-applymbox and viewing the log on master.kernel.org
> with git log shows Rafa<B3>   And then later when put into email
> it turns into Rafa³
> 
>  > But the point is there is charset damage which has happened _long_ before
>  > Linus' action.  There is no character set defined for the contents of git
>  > repositories, and as such the output of the git tools can not be
>  > interpreted as any one single character set.
> 
> If there's something I should be doing when I commit that I'm not,
> I'll be happy to change my scripts.  My $LANG is set to en_US.UTF-8
> which should DTRT to the best of my knowledge, but clearly, that isn't
> the case.

Jan Engelhardt· Jan 7, 2007, 20:40 UTC · re: Sean · lore

Show 7 quoted lines

>On Sun, 7 Jan 2007 15:05:53 -0500
>Dave Jones <davej@redhat.com> wrote:
>
>> If there's something I should be doing when I commit that I'm not,
>> I'll be happy to change my scripts.  My $LANG is set to en_US.UTF-8
>> which should DTRT to the best of my knowledge, but clearly, that isn't
>> the case.

No, LC_CTYPE defines what charset you use. (I may be wrong, though.)

	-`J'

Xavier Bestel· Jan 7, 2007, 21:07 UTC · re: Jan Engelhardt · lore

Le dimanche 07 janvier 2007 à 21:40 +0100, Jan Engelhardt a écrit :

Show 9 quoted lines

> >On Sun, 7 Jan 2007 15:05:53 -0500
> >Dave Jones <davej@redhat.com> wrote:
> >
> >> If there's something I should be doing when I commit that I'm not,
> >> I'll be happy to change my scripts.  My $LANG is set to en_US.UTF-8
> >> which should DTRT to the best of my knowledge, but clearly, that isn't
> >> the case.
> 
> No, LC_CTYPE defines what charset you use. (I may be wrong, though.)

IIRC LANG is a superset for all LC_* - i.e. if only LANG is defined, it sets all your locales, but you can individually set the charset, numeric format, date format, etc.

Xav

David Woodhouse· Jan 8, 2007, 01:53 UTC · lore

Re: OT: character encodings

On Sun, 2007-01-07 at 23:37 +0100, Tilman Schmidt wrote:

Show 5 quoted lines

> > That depends on your point of view.  David's assertion was that git
> > internally uses UTF-8.
> 
> Well, I'm inclined to say he was wrong. Git internally is completely
> oblivious to character encoding.

See the 'i18n.commitencoding' property, stored in the repository config and defaulting to UTF-8. Git is certainly capable of converting to its internal storage encoding when you commit; if it doesn't do so by default then that would be a bug.

-- 
dwmw2