"; print $cgi->a({-href => href(action=>"tree", hash_base=>$hb), - -title => 'tree root'}, to_utf8("[$project]")); + -title => 'tree root'}, decode_utf8("[$project]")); print " / "; if (defined $name) { my @dirname = split '/', $name; @@ -2584,7 +2578,7 @@ ($pr->{'age'}, $pr->{'age_string'}) = @aa; if (!defined $pr->{'descr'}) { my $descr = git_get_project_description($pr->{'path'}) || ""; - $pr->{'descr_long'} = to_utf8($descr); + $pr->{'descr_long'} = decode_utf8($descr); $pr->{'descr'} = chop_str($descr, 25, 5); } if (!defined $pr->{'owner'}) { @@ -3616,7 +3610,7 @@ $hash = git_get_head_hash($project); } - my $filename = to_utf8(basename($project)) . "-$hash.tar.$suffix"; + my $filename = decode_utf8(basename($project)) . "-$hash.tar.$suffix"; print $cgi->header( -type => "application/$ctype", ``` ## Junio C Hamano, 2007-05-01 21:39 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <7v8xc85ill.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7v8xc85ill.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <200705020012.13302.ismail@pardus.org.tr> ``` Ismail Dönmez writes: > Ok found out the reason. decode() tries to decode data that is already UTF-8 > and borks. > > This is from Encode.pm : > > sub decode_utf8($;$) { > my ( $str, $check ) = @_; > return $str if is_utf8($str); <--- Checks if the $str is already UTF-8 > if ($check) { > return decode( "utf8", $str, $check ); <--- Else do what gitweb does > [...] > > So my patch is indeed correct. Ok, I think that makes it an improvement from the current code, so I'd apply. But at the same time I wonder why should the callers be feeding an already decoded string to to_utf8(). It might be that some callers needs fixing. ``` ## Ismail Dönmez, 2007-05-01 21:44 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200705020044.47171.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200705020044.47171.ismail%40pardus.org.tr In-Reply-To: <7v8xc85ill.fsf@assigned-by-dhcp.cox.net> ``` On Wednesday 02 May 2007 00:39:34 you wrote: > Ismail Dönmez writes: > > Ok found out the reason. decode() tries to decode data that is already > > UTF-8 and borks. > > > > This is from Encode.pm : > > > > sub decode_utf8($;$) { > > my ( $str, $check ) = @_; > > return $str if is_utf8($str); <--- Checks if the $str is already > > UTF-8 if ($check) { > > return decode( "utf8", $str, $check ); <--- Else do what gitweb > > does [...] > > > > So my patch is indeed correct. > > Ok, I think that makes it an improvement from the current code, > so I'd apply. > > But at the same time I wonder why should the callers be feeding > an already decoded string to to_utf8(). It might be that some > callers needs fixing. FWIW it was passing my name "İsmail Dönmez" based on user info I guess. Regards, ismail ``` ## Ismail Dönmez, 2007-05-01 21:48 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200705020048.53853.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200705020048.53853.ismail%40pardus.org.tr In-Reply-To: <200705020044.47171.ismail@pardus.org.tr> ``` On Wednesday 02 May 2007 00:44:46 you wrote: > On Wednesday 02 May 2007 00:39:34 you wrote: > > Ismail Dönmez writes: > > > Ok found out the reason. decode() tries to decode data that is already > > > UTF-8 and borks. > > > > > > This is from Encode.pm : > > > > > > sub decode_utf8($;$) { > > > my ( $str, $check ) = @_; > > > return $str if is_utf8($str); <--- Checks if the $str is already > > > UTF-8 if ($check) { > > > return decode( "utf8", $str, $check ); <--- Else do what gitweb > > > does [...] > > > > > > So my patch is indeed correct. > > > > Ok, I think that makes it an improvement from the current code, > > so I'd apply. > > > > But at the same time I wonder why should the callers be feeding > > an already decoded string to to_utf8(). It might be that some > > callers needs fixing. > > FWIW it was passing my name "İsmail Dönmez" based on user info I guess. I guess its line 1116: if (check_export_ok("$projectroot/$path")) { my $pr = { path => $path, owner => to_utf8($owner), <---- Here }; My system is configured for UTF-8 so $owner will be UTF-8 but in some systems it might not be so I don't think there is anything to fix here. Regards, ismail ``` ## Ismail Dönmez, 2007-05-03 19:22 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200705032222.37387.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200705032222.37387.ismail%40pardus.org.tr In-Reply-To: <7v8xc85ill.fsf@assigned-by-dhcp.cox.net> ``` Hi, On Wednesday 02 May 2007 00:39:34 Junio C Hamano wrote: > Ismail Dönmez writes: > > Ok found out the reason. decode() tries to decode data that is already > > UTF-8 and borks. > > > > This is from Encode.pm : > > > > sub decode_utf8($;$) { > > my ( $str, $check ) = @_; > > return $str if is_utf8($str); <--- Checks if the $str is already > > UTF-8 if ($check) { > > return decode( "utf8", $str, $check ); <--- Else do what gitweb > > does [...] > > > > So my patch is indeed correct. > > Ok, I think that makes it an improvement from the current code, > so I'd apply. > > But at the same time I wonder why should the callers be feeding > an already decoded string to to_utf8(). It might be that some > callers needs fixing. Is the patch OK do you want more investigation? Asking because its still not in git.git. Regards, ismail ``` ## Junio C Hamano, 2007-05-03 19:26 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <7vsladzp29.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vsladzp29.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <200705032222.37387.ismail@pardus.org.tr> ``` Ismail Dönmez writes: >> But at the same time I wonder why should the callers be feeding >> an already decoded string to to_utf8(). It might be that some >> callers needs fixing. > > Is the patch OK do you want more investigation? Asking because its still not > in git.git. I would say that the patch is an improvement from the current code so it should hit 'master'; I was a bit busy lately and then am sick, and also we are post -rc1 freeze now and I was being cautious, just in case some nacks from more informed parties arrive late. ``` ## Alexandre Julliard, 2007-06-01 13:45 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <87zm3ju6tg.fsf@wine.dyndns.org> URL: https://gitlist.dev/e/87zm3ju6tg.fsf%40wine.dyndns.org In-Reply-To: <7vsladzp29.fsf@assigned-by-dhcp.cox.net> ``` Junio C Hamano writes: > I would say that the patch is an improvement from the current > code so it should hit 'master'; I was a bit busy lately and then > am sick, and also we are post -rc1 freeze now and I was being > cautious, just in case some nacks from more informed parties > arrive late. Sorry for the late nack, but it turns out that this patch breaks diff output on the Wine server for files that are not utf-8. The cause is apparently that decode_utf8() returns undef for invalid sequences instead of substituting a replacement char like decode("utf8") does. That may be considered an Encode bug since we are running a fairly old version (1.99, coming with Debian 3.1), but I'd rather not upgrade perl on the server. Could the patch be reverted, or done differently? -- Alexandre Julliard julliard@winehq.org ``` ## Ismail Dönmez, 2007-06-01 13:50 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200706011650.10650.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200706011650.10650.ismail%40pardus.org.tr In-Reply-To: <87zm3ju6tg.fsf@wine.dyndns.org> ``` On Friday 01 June 2007 16:45:31 Alexandre Julliard wrote: > Junio C Hamano writes: > > I would say that the patch is an improvement from the current > > code so it should hit 'master'; I was a bit busy lately and then > > am sick, and also we are post -rc1 freeze now and I was being > > cautious, just in case some nacks from more informed parties > > arrive late. > > Sorry for the late nack, but it turns out that this patch breaks diff > output on the Wine server for files that are not utf-8. Isn't UTF-8 default even for Linux kernel now? > The cause is apparently that decode_utf8() returns undef for invalid > sequences instead of substituting a replacement char like > decode("utf8") does. > > That may be considered an Encode bug since we are running a fairly old > version (1.99, coming with Debian 3.1), but I'd rather not upgrade > perl on the server. Could the patch be reverted, or done differently? Sorry but thats too old. Of course I am not the maintainer of GIT so its not for me to decide but well as David Woodhouse puts it, please join us in 21st century and start using UTF-8. /ismail -- Perfect is the enemy of good ``` ## Alexandre Julliard, 2007-06-01 16:51 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <87ejkvty7k.fsf@wine.dyndns.org> URL: https://gitlist.dev/e/87ejkvty7k.fsf%40wine.dyndns.org In-Reply-To: <200706011650.10650.ismail@pardus.org.tr> ``` Ismail Dönmez writes: > Sorry but thats too old. Of course I am not the maintainer of GIT so its not > for me to decide but well as David Woodhouse puts it, please join us in 21st > century and start using UTF-8. That's not very helpful. There can be many valid reasons for not using utf-8, in our case compatibility with Windows tools is the main reason. And even if we were to convert all our files today, it wouldn't help when browsing older versions. I'm not asking gitweb to magically guess the encoding of the files, I'm happy with it replacing invalid sequences with some substitution char, like it did before 1.5.2. But now it is deleting whole lines from the diff, without any indication that something went wrong. That's not an improvement IMNSHO. -- Alexandre Julliard julliard@winehq.org ``` ## Junio C Hamano, 2007-06-01 19:44 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <7vr6ovzcgr.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vr6ovzcgr.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <87zm3ju6tg.fsf@wine.dyndns.org> ``` Alexandre Julliard writes: > Sorry for the late nack, but it turns out that this patch breaks diff > output on the Wine server for files that are not utf-8. > > The cause is apparently that decode_utf8() returns undef for invalid > sequences instead of substituting a replacement char like > decode("utf8") does. Thanks for noticing. Will revert. ``` ## Ismail Dönmez, 2007-06-01 19:47 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200706012247.57273.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200706012247.57273.ismail%40pardus.org.tr In-Reply-To: <7vr6ovzcgr.fsf@assigned-by-dhcp.cox.net> ``` On Friday 01 June 2007 22:44:36 Junio C Hamano wrote: > Alexandre Julliard writes: > > Sorry for the late nack, but it turns out that this patch breaks diff > > output on the Wine server for files that are not utf-8. > > > > The cause is apparently that decode_utf8() returns undef for invalid > > sequences instead of substituting a replacement char like > > decode("utf8") does. > > Thanks for noticing. Will revert. Why are reverting a correct bugfix? :( He's at most using outdated software. *sigh* /ismail -- Perfect is the enemy of good ``` ## Junio C Hamano, 2007-06-01 20:00 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <7vbqfzzbq8.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vbqfzzbq8.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <200706012247.57273.ismail@pardus.org.tr> ``` Ismail Dönmez writes: > On Friday 01 June 2007 22:44:36 Junio C Hamano wrote: >> Alexandre Julliard writes: >> > Sorry for the late nack, but it turns out that this patch breaks diff >> > output on the Wine server for files that are not utf-8. >> > >> > The cause is apparently that decode_utf8() returns undef for invalid >> > sequences instead of substituting a replacement char like >> > decode("utf8") does. >> >> Thanks for noticing. Will revert. > > Why are reverting a correct bugfix? :( He's at most using outdated software. > *sigh* I would assume that on top of a revert, with an additional return $str if is_utf8($str); to to_utf8() you should be able to fix both installations that has old or new Encode.pm? ``` ## Ismail Dönmez, 2007-06-01 20:08 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200706012308.41335.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200706012308.41335.ismail%40pardus.org.tr In-Reply-To: <7vbqfzzbq8.fsf@assigned-by-dhcp.cox.net> ``` On Friday 01 June 2007 23:00:31 you wrote: > Ismail Dönmez writes: > > On Friday 01 June 2007 22:44:36 Junio C Hamano wrote: > >> Alexandre Julliard writes: > >> > Sorry for the late nack, but it turns out that this patch breaks diff > >> > output on the Wine server for files that are not utf-8. > >> > > >> > The cause is apparently that decode_utf8() returns undef for invalid > >> > sequences instead of substituting a replacement char like > >> > decode("utf8") does. > >> > >> Thanks for noticing. Will revert. > > > > Why are reverting a correct bugfix? :( He's at most using outdated > > software. *sigh* > > I would assume that on top of a revert, with an additional > > return $str if is_utf8($str); > > to to_utf8() you should be able to fix both installations that > has old or new Encode.pm? I can try the patch if you can send me what you propose. /ismail -- Perfect is the enemy of good ``` ## Jakub Narebski, 2007-06-02 08:22 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200706021022.31812.jnareb@gmail.com> URL: https://gitlist.dev/e/200706021022.31812.jnareb%40gmail.com In-Reply-To: <87zm3ju6tg.fsf@wine.dyndns.org> ``` On Fri, 1 Jun 2007, Alexandre Julliard wrote: > The cause is apparently that decode_utf8() returns undef for invalid > sequences instead of substituting a replacement char like > decode("utf8") does. > > That may be considered an Encode bug since we are running a fairly old > version (1.99, coming with Debian 3.1), but I'd rather not upgrade > perl on the server. Could the patch be reverted, or done differently? Could you put modern (without this decode_utf8 bug) version of Encode.pm in the directory with gitweb.cgi, so gitweb uses new local version and not the one that is installed system-wide? -- Jakub Narebski Poland ``` ## Junio C Hamano, 2007-06-03 22:06 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <7vmyzgn14w.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vmyzgn14w.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <200706012308.41335.ismail@pardus.org.tr> ``` Ismail Dönmez writes: > I can try the patch if you can send me what you propose. Does the recent one from Jakub work for you? ``` ## Ismail Dönmez, 2007-06-03 22:13 Subject: Re: [PATCH] gitweb: use decode_utf8 directly Message-ID: <200706040113.56055.ismail@pardus.org.tr> URL: https://gitlist.dev/e/200706040113.56055.ismail%40pardus.org.tr In-Reply-To: <7vmyzgn14w.fsf@assigned-by-dhcp.cox.net> ``` On Monday 04 June 2007 01:06:55 Junio C Hamano wrote: > Ismail Dönmez writes: > > I can try the patch if you can send me what you propose. > > Does the recent one from Jakub work for you? It works fine here. Regards, ismail -- Perfect is the enemy of good ```