Re: [PATCH] gitweb: Fix chop_str not to cut in middle of utf8 multibyte chars.

From: Jakub Narebski <jnareb@gmail.com>
Date: May 20, 2008, 22:19 UTC
Message-ID: <m3lk244o16.fsf@localhost.localdomain>
In-Reply-To: <g0vdse$rj6$1@ger.gmane.org>

Anders Waldenborg <anders@0x63.nu> writes:

> Without this fix at least author name in short log may cut in middle of a
> multibyte char. When the result comes to esc_html to_utf8 is called again,
> which doesn't find valid utf8 and decodes using $fallback_encoding making
> it even worse.

Thanks a lot. This is certainly a good thing, although I think that the proper solution (but which would need much more work) would be to ensure that all information is stored in Perl internal form, and not only ensured on output.

I would change title of this commit to be more descriptive what this commit does; or rather make current subject the first sentence of commit summary, have something like the following for commit message:

gitweb: Convert string to Perl internal form before chopping in chop_str

Fix chop_str not to cut in middle of utf8 multibyte chars.  Without
this fix at least author name in short log may cut in middle of a
multibyte char. When the result comes to esc_html to_utf8 is called
again, which doesn't find valid utf8 and decodes using
$fallback_encoding making it even worse.

Show 17 quoted lines

> Signed-off-by: Anders Waldenborg <anders@0x63.nu>
> ---
>   gitweb/gitweb.perl |    4 ++++
>   1 files changed, 4 insertions(+), 0 deletions(-)
> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 2facf2d..8308e22 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -866,6 +866,10 @@ sub chop_str {
>          my $add_len = shift || 10;
>          my $where = shift || 'right'; # 'left' | 'center' | 'right'
> 
> +       # Make sure perl knows it is utf8 encoded so we don't
> +       # cut in the middle of a utf8 multibyte char.
> +       $str = to_utf8($str);
> +

I like the comment here. It explains the whys of code.

>          # allow only $len chars, but don't cut a word if it would fit in $add_len
>          # if it doesn't fit, cut it if it's still longer than the dots we would add
>          # remove chopped character entities entirely
>

This patch is whitespace damaged, by the way.

-- 
Jakub Narebski
Poland
ShadeHawk on #git