Re: Git-mediawiki : Encoding problems in perl
- From
Jakub Narebski <jnareb@gmail.com>
- Date
- Jun 8, 2011, 17:04 UTC
- Message-ID
- <m3y61cdzmi.fsf@localhost.localdomain>
- In-Reply-To
- <BANLkTimy85b3nu05FBjXzdnTJP0RBWdxiQ@mail.gmail.com>
Jérémie NIKAES <jeremie.nikaes@gmail.com> writes:
Show 15 quoted lines
> While working on the git-mediawiki project[1], we ran into some
> problems regarding utf8 encoding of files. Most of them have been
> solved, however, one is still pretty annoying.
> Let me illustrate it :
>
> I want to edit a page on mediawiki using the API, with a very simple example :
>
> my $mw = MediaWiki::API->new();
> $mw->edit( {
> action => 'edit',
> title => 'Main_page',
> text => 'été',
> } ) ;
>
> But, when I look at the page on mediawiki, I see weird characters : été.Take a look at
http://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default
especially accepted answer.
In short (I don't agree with everything there, and not everything is needed for all but extremal Unicode usage): if your script is written using UTF-8 like in above examples, use
use utf8;
If this is simplification, and this text comes from other file or is result of output of some command, use
use utf8::all;
or take a look what it does and put relevant parts in your script.
Show 11 quoted lines
> I tried text => encode_utf8('été') with no success.
>
> This makes pushing changes from git to mediawiki buggy since pulling a
> file with accentuated characters and pushing it right after changes
> things on the wiki.
>
> While googling (a lot), I found that utf8 was pretty tricky in perl...
> The only thing that seems to solve things is a simple addition of 'use
> encoding utf8' at the top of our script.
> However
> A) Adding this line requires that I remove 'use strict;'use encoding ':utf8';
or
use encoding 'utf8';
> B) I found some information about this pragma encoding and it seems to > be unadvised to use it
-- Jakub Narebski Poland ShadeHawk on #git