threads / discuss / 14831

StGit and charsets

Subject: StGit and charsets

## tl;dr

3 messages between Aug 4, 2008 and Aug 4, 2008.

replies: 2people: 2as markdown or json

Jon Smirl· Aug 4, 2008, 14:21 UTC · lore

Do you have tests in place to handle the names and comments in patches being in different charsets? When I was working with the mailmap file a large source of errors was from mangling names in alternate charsets. I recall errors in Finnish, Japanese and Chinese names for sure. I don't know which tools did the charset mangling.

I don't work much with international charsets. If someone is using something like Russian or Finish locally, is the metadata in the patch converted to UTF8 before exporting or sending it as mail? Comments should be in English, but people's names may need UTF8. And what about email addresses, does DNS allow Unicode names now?

-- 
Jon Smirl
jonsmirl@gmail.com

Karl Hasselström· Aug 4, 2008, 15:31 UTC · re: Jon Smirl · lore

Re: StGit and charsets

On 2008-08-04 10:21:20 -0400, Jon Smirl wrote:

> Do you have tests in place to handle the names and comments in
> patches being in different charsets?

There are some tests in t1800 and t1900 that use non-ascii names. Might be others, but a quick grep didn't find them.

> I don't work much with international charsets. If someone is using
> something like Russian or Finish locally, is the metadata in the
> patch converted to UTF8 before exporting or sending it as mail?

I think what happens is that it's assumed to be utf8. No one has complained that their non-utf8 locale doesn't work, but my guess is that's because those people just haven't tried StGit yet.

> Comments should be in English, but people's names may need UTF8.

Mine does, for example. I started my StGit career by making sure it could send out patches that didn't mangle my name. (That's a while ago, though, so my memory's getting a bit rusty.)

> And what about email addresses, does DNS allow Unicode names now?

I think it's up to each TLD whether they want to allow it. See e.g.

  http://en.wikipedia.org/wiki/Internationalized_domain_name

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Jon Smirl· Aug 4, 2008, 15:21 UTC · re: Karl Hasselström · lore

Re: StGit and charsets

On 8/4/08, Karl Hasselström <kha@treskal.com> wrote:

Show 9 quoted lines

> On 2008-08-04 10:21:20 -0400, Jon Smirl wrote:
>  > I don't work much with international charsets. If someone is using
>  > something like Russian or Finish locally, is the metadata in the
>  > patch converted to UTF8 before exporting or sending it as mail?
>
>
> I think what happens is that it's assumed to be utf8. No one has
>  complained that their non-utf8 locale doesn't work, but my guess is
>  that's because those people just haven't tried StGit yet.

This might be worth testing for Asian locales. If Asian locales make it through ok everything else probably will too.

I don't know enough about Unicode to decode the mangled names, I suspect they are still in the local charset and haven't been UTF8 converted. Of course the name mangling may be coming from other tools. There is no way to tell.

This problem happens in other ways in the kernel. There is a file in the DRM code where two people's names have been inserted in different charsets. If I set my editor to display one right the other is always wrong. checkpatch probably needs to make sure that patches are completely valid UTF8.

-- 
Jon Smirl
jonsmirl@gmail.com