RFC patchgitweb: parse_commit_text encoding fix

>
> Thanks.
>
>
> Still, I do wonder if it would be possible to simply do the following:
>
>  -binmode STDOUT, ':utf8';
>  +use open qw(:std :utf8);
>
> ...but it unfortunately doesn't work.  It was tried in
>  http://thread.gmane.org/gmane.comp.version-control.git/87129/focus=87135
>
> to_utf8() has at least (possible) fallback if it encounters characters
> outside of UTF-8 coding.
>

> Call to_utf8 when parsing author and committer names, otherwise they will appear
> with bad encoding if they written by using chop_and_escape_str.
>
> Signed-off-by: Zoltán Füzesi <zfuzesi@eaglet.hu>
> ---

>  gitweb/gitweb.perl |    5 ++---
>  1 files changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 7fbd5ff..4f05194 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -2570,7 +2570,7 @@ sub parse_commit_text {
>  		} elsif ((!defined $withparents) && ($line =~ m/^parent ([0-9a-fA-F]{40})$/)) {
>  			push @parents, $1;
>  		} elsif ($line =~ m/^author (.*) ([0-9]+) (.*)$/) {
> -			$co{'author'} = $1;
> +			$co{'author'} = to_utf8($1);
>  			$co{'author_epoch'} = $2;
>  			$co{'author_tz'} = $3;
>  			if ($co{'author'} =~ m/^([^<]+) <([^>]*)>/) {
> @@ -2580,10 +2580,9 @@ sub parse_commit_text {
>  				$co{'author_name'} = $co{'author'};
>  			}
>  		} elsif ($line =~ m/^committer (.*) ([0-9]+) (.*)$/) {
> -			$co{'committer'} = $1;
> +			$co{'committer'} = to_utf8($1);
>  			$co{'committer_epoch'} = $2;
>  			$co{'committer_tz'} = $3;
> -			$co{'committer_name'} = $co{'committer'};
>  			if ($co{'committer'} =~ m/^([^<]+) <([^>]*)>/) {
>  				$co{'committer_name'}  = $1;
>  				$co{'committer_email'} = $2;
> -- 
> 1.6.4.13.ge6580

>
> Thanks, Zoltán.
>
> We should be able to set up a script that scrapes the output to test this
> kind of thing.  We may not want to have a test pattern that matches too
> strictly for the current structure and appearance of the output
> (e.g. counting nested <div>s, presentation styles and such), but if we can
> robustly scrape off HTML tags (e.g. "elinks -dump") and check the
> remaining payload, it might be enough.
>
> Jakub what do you think?  I suspect that scraping approach may turn out to
> be too fragile for tests to be worth doing, but I am just throwing out a
> thought.
>

> 2009/8/4 Junio C Hamano <gitster@pobox.com>:
> >
> > Thanks, Zoltán.
> >
> > We should be able to set up a script that scrapes the output to test this
> > kind of thing.  We may not want to have a test pattern that matches too
> > strictly for the current structure and appearance of the output
> > (e.g. counting nested <div>s, presentation styles and such), but if we can
> > robustly scrape off HTML tags (e.g. "elinks -dump") and check the
> > remaining payload, it might be enough.
> >
> > Jakub what do you think?  I suspect that scraping approach may turn out to
> > be too fragile for tests to be worth doing, but I am just throwing out a
> > thought.
> >
> 
> This issue comes out when chop_and_escape_str function is called with
> a non-ascii string (like my name :)) without before calling to_utf8 on
> it. "author_name" and "committer_name" are two examples, and
> "author_name" shows up with bad encoding in HTML.
> 
> Example from one of my repos (little piece from shortlog output):
> <td class="author"><span title="Füzesi Zoltán">Füzesi Zoltán</span></td>
> After applying the patch:
> <td class="author">Füzesi Zoltán</td>
> 
> This is an "old" (seen in 1.5.6 version too) and (I think) minor issue.
> I haven't spent time on thinking how a test script could show this yet.
> Waiting for Jakub's reaction.

> Zoltán Füzesi <zfuzesi@eaglet.hu> writes:
> 
> > Call to_utf8 when parsing author and committer names, otherwise they will appear
> > with bad encoding if they written by using chop_and_escape_str.
> >
> > Signed-off-by: Zoltán Füzesi <zfuzesi@eaglet.hu>
> > ---
> 
> Thanks, Zoltán.
> 
> We should be able to set up a script that scrapes the output to test this
> kind of thing.  We may not want to have a test pattern that matches too
> strictly for the current structure and appearance of the output
> (e.g. counting nested <div>s, presentation styles and such), but if we can
> robustly scrape off HTML tags (e.g. "elinks -dump") and check the
> remaining payload, it might be enough.
> 
> Jakub what do you think?  I suspect that scraping approach may turn out to
> be too fragile for tests to be worth doing, but I am just throwing out a
> thought.