{"thread":{"id":"33441","subject":"[PATCH 2/4] gitweb: Make feed title valid utf8","startedAt":"2013-04-08T20:09:14Z","lastAt":"2013-04-09T19:58:41Z","messageCount":6,"participants":["Jürgen Kreileder","Jakub Narębski"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"213599","messageId":"m2y5csbx91.fsf@blackdown.de","threadId":"33441","inReplyTo":null,"subject":"[PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jürgen Kreileder","fromEmail":"jk@blackdown.de","sentAt":"2013-04-08T20:09:14Z","receivedAt":"2013-04-08T20:09:14Z","isPatch":true,"sender":{"key":"jk@blackdown.de","avatar":"https://avatars.githubusercontent.com/u/4390544?v=4"},"body":"Properly encode site and project names for RSS and Atom feeds.\n\nSigned-off-by: Jürgen Kreileder \n---\n gitweb/gitweb.perl | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\nindex 9cfe5b5..09294eb 100755\n--- a/gitweb/gitweb.perl\n+++ b/gitweb/gitweb.perl\n@@ -8056,7 +8056,7 @@ sub git_feed {\n \treturn if ($cgi->request_method() eq 'HEAD');\n \n \t# header variables\n-\tmy $title = \"$site_name - $project/$action\";\n+\tmy $title = to_utf8($site_name) . \" - \" . to_utf8($project) . \"/$action\";\n \tmy $feed_type = 'log';\n \tif (defined $hash) {\n \t\t$title .= \" - '$hash'\";\n-- \n1.7.10.4\n"},{"id":"213662","messageId":"51642F67.5010501@gmail.com","threadId":"33441","inReplyTo":"m2y5csbx91.fsf@blackdown.de","subject":"Re: [PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2013-04-09T15:10:31Z","receivedAt":"2013-04-09T15:10:31Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Jürgen Kreileder wrote:\n\n> Properly encode site and project names for RSS and Atom feeds.\n> \n> Signed-off-by: Jürgen Kreileder \n> ---\n> gitweb/gitweb.perl | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n> \n> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\n> index 9cfe5b5..09294eb 100755\n> --- a/gitweb/gitweb.perl\n> +++ b/gitweb/gitweb.perl\n> @@ -8056,7 +8056,7 @@ sub git_feed {\n> \treturn if ($cgi->request_method() eq 'HEAD');\n> \n> \t# header variables\n> -\tmy $title = \"$site_name - $project/$action\";\n> +\tmy $title = to_utf8($site_name) . \" - \" . to_utf8($project) . \"/$action\";\n> \tmy $feed_type = 'log';\n> \tif (defined $hash) {\n> \t\t$title .= \" - '$hash'\";\n> \n\nWas this patch triggered by some bug?\n\nBecause the above is not necessary, as git_feed() has\n\n\t$title = esc_html($title);\n\na bit later, which does to_utf8() internally.\n-- \nJakub Narębski\n"},{"id":"213682","messageId":"m2r4ija9gh.fsf@zahir.fritz.box","threadId":"33441","inReplyTo":"51642F67.5010501@gmail.com","subject":"Re: [PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jürgen Kreileder","fromEmail":"jk@blackdown.de","sentAt":"2013-04-09T17:40:46Z","receivedAt":"2013-04-09T17:40:46Z","isPatch":true,"sender":{"key":"jk@blackdown.de","avatar":"https://avatars.githubusercontent.com/u/4390544?v=4"},"body":"Jakub Narębski writes:\n\n> Jürgen Kreileder wrote:\n>\n>> Properly encode site and project names for RSS and Atom feeds.\n>> \n>> Signed-off-by: Jürgen Kreileder \n>> ---\n>> gitweb/gitweb.perl | 2 +-\n>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>> \n>> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl\n>> index 9cfe5b5..09294eb 100755\n>> --- a/gitweb/gitweb.perl\n>> +++ b/gitweb/gitweb.perl\n>> @@ -8056,7 +8056,7 @@ sub git_feed {\n>> \treturn if ($cgi->request_method() eq 'HEAD');\n>> \n>> \t# header variables\n>> -\tmy $title = \"$site_name - $project/$action\";\n>> +\tmy $title = to_utf8($site_name) . \" - \" . to_utf8($project) . \"/$action\";\n>> \tmy $feed_type = 'log';\n>> \tif (defined $hash) {\n>> \t\t$title .= \" - '$hash'\";\n>> \n>\n> Was this patch triggered by some bug?\n\nYes, I actually see broken encoding with the old code, e.g on \nhttps://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss\nmy first name is messed up in the title tag.\n\nNew version: https://git.blackdown.de/?p=contactalbum.git;a=rss\n\n> Because the above is not necessary, as git_feed() has\n>\n> \t$title = esc_html($title);\n>\n> a bit later, which does to_utf8() internally.\n\nGood point. But it doesn't fix the string in question:\nIt looks like to_utf8(\"$a $b\") != (to_utf8($a) . \" \" . to_utf($b)).\n\n\nJuergen\n"},{"id":"213695","messageId":"51645D99.6000106@gmail.com","threadId":"33441","inReplyTo":"m2r4ija9gh.fsf@zahir.fritz.box","subject":"Re: [PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2013-04-09T18:27:37Z","receivedAt":"2013-04-09T18:27:37Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 09.04.2013 19:40, Jürgen Kreileder napisał:\n> Jakub Narębski writes:\n>> Jürgen Kreileder wrote:\n>>\n>>> Properly encode site and project names for RSS and Atom feeds.\n\n>>> -\tmy $title = \"$site_name - $project/$action\";\n>>> +\tmy $title = to_utf8($site_name) . \" - \" . to_utf8($project) . \"/$action\";\n\n>> Was this patch triggered by some bug?\n> \n> Yes, I actually see broken encoding with the old code, e.g on \n> https://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss\n> my first name is messed up in the title tag.\n> \n> New version: https://git.blackdown.de/?p=contactalbum.git;a=rss\n> \n>> Because the above is not necessary, as git_feed() has\n>>\n>> \t$title = esc_html($title);\n>>\n>> a bit later, which does to_utf8() internally.\n> \n> Good point. But it doesn't fix the string in question:\n> It looks like to_utf8(\"$a $b\") != (to_utf8($a) . \" \" . to_utf8($b)).\n\nStrange. I wonder if the bug is in our to_utf8() implementation,\nor in Encode, or in Perl... and whether this bug can be triggered\nanywhere else in gitweb.\n\nWhat Perl version and Encode module version do you use?\n-- \nJakub Narębski\n"},{"id":"213706","messageId":"m2ip3va4ro.fsf@zahir.fritz.box","threadId":"33441","inReplyTo":"51645D99.6000106@gmail.com","subject":"Re: [PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jürgen Kreileder","fromEmail":"jk@blackdown.de","sentAt":"2013-04-09T19:22:03Z","receivedAt":"2013-04-09T19:22:03Z","isPatch":true,"sender":{"key":"jk@blackdown.de","avatar":"https://avatars.githubusercontent.com/u/4390544?v=4"},"body":"Jakub Narębski writes:\n\n> W dniu 09.04.2013 19:40, Jürgen Kreileder napisał:\n>> Jakub Narębski writes:\n>>> Jürgen Kreileder wrote:\n>>>\n>>>> Properly encode site and project names for RSS and Atom feeds.\n>\n>>>> -\tmy $title = \"$site_name - $project/$action\";\n>>>> +\tmy $title = to_utf8($site_name) . \" - \" . to_utf8($project) . \"/$action\";\n>\n>>> Was this patch triggered by some bug?\n>> \n>> Yes, I actually see broken encoding with the old code, e.g on \n>> https://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss\n>> my first name is messed up in the title tag.\n>> \n>> New version: https://git.blackdown.de/?p=contactalbum.git;a=rss\n>> \n>>> Because the above is not necessary, as git_feed() has\n>>>\n>>> \t$title = esc_html($title);\n>>>\n>>> a bit later, which does to_utf8() internally.\n>> \n>> Good point. But it doesn't fix the string in question:\n>> It looks like to_utf8(\"$a $b\") != (to_utf8($a) . \" \" . to_utf8($b)).\n>\n> Strange. I wonder if the bug is in our to_utf8() implementation,\n> or in Encode, or in Perl... and whether this bug can be triggered\n> anywhere else in gitweb.\n\nI don't think it's a bug, more like a consequence of concatenating utf8\nand non-utf8 strings:\n\n my $a = \"ü\";\n my $b = \"ü\";\n my $c = \"$a - $b\";\n print \"$c -> \". to_utf8($c) . \": \" . (utf8::is_utf8($c) ? \"utf8\" : \"not utf8\") . \"\\n\"; # GOOD\n $b = to_utf8($b);\n $c = \"$a - $b\";\n print \"$c -> \". to_utf8($c) . \": \" . (utf8::is_utf8($c) ? \"utf8\" : \"not utf8\") . \"\\n\"; # GOOD\n\nyields (hopefully the broken encoding shows up correctly here):\n\n ü - ü -> ü - ü: not utf8\n ü - ü -> ü - ü: utf8\n\n\nIn gitweb we have the bad case: \n\n my $title = \"$site_name - $project/$action\";\n\n$project and $action are apparently utf8 already but $site_name isn't.\nThe resulting string is marked as utf8 - although the encoding of\n$site_name was never fixed. The to_utf8() in esc_html() returns the string\nwithout fixing anything because of that.\n\n> What Perl version and Encode module version do you use?\n\n5.14.2 and 2.42_01 on Ubuntu. Same results with 5.12.4 and 2.39 on OS X.\n\n\n Juergen\n"},{"id":"213710","messageId":"516472F1.4060903@gmail.com","threadId":"33441","inReplyTo":"m2ip3va4ro.fsf@zahir.fritz.box","subject":"Re: [PATCH 2/4] gitweb: Make feed title valid utf8","fromName":"Jakub Narębski","fromEmail":"jnareb@gmail.com","sentAt":"2013-04-09T19:58:41Z","receivedAt":"2013-04-09T19:58:41Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"W dniu 09.04.2013 21:22, Jürgen Kreileder napisał:\n> Jakub Narębski writes: \n>> W dniu 09.04.2013 19:40, Jürgen Kreileder napisał:\n>>> Jakub Narębski writes:\n>>>> Jürgen Kreileder wrote:\n>>>>\n>>>>> Properly encode site and project names for RSS and Atom feeds.\n\n>>> Good point. But it doesn't fix the string in question:\n>>> It looks like to_utf8(\"$a $b\") != (to_utf8($a) . \" \" . to_utf8($b)).\n>>\n>> Strange. I wonder if the bug is in our to_utf8() implementation,\n>> or in Encode, or in Perl... and whether this bug can be triggered\n>> anywhere else in gitweb.\n> \n> I don't think it's a bug, more like a consequence of concatenating utf8\n> and non-utf8 strings:\n> \n> my $a = \"ü\";\n> my $b = \"ü\";\n> my $c = \"$a - $b\";\n> print \"$c -> \". to_utf8($c) . \": \" . (utf8::is_utf8($c) ? \"utf8\" : \"not utf8\") . \"\\n\"; # GOOD\n> $b = to_utf8($b);\n> $c = \"$a - $b\";\n> print \"$c -> \". to_utf8($c) . \": \" . (utf8::is_utf8($c) ? \"utf8\" : \"not utf8\") . \"\\n\"; # GOOD\n> \n> yields (hopefully the broken encoding shows up correctly here):\n> \n> ü - ü -> ü - ü: not utf8\n> ü - ü -> ü - ü: utf8\n\nAh, so it looks like it is misfeature of the way Perl handles Unicode;\nconcatenating adds 'UTF8' flag if either of concatenates strings has\nit to the result.\n\n[Which I have checked using Devel::Peek with\n perl -MDevel::Peek -E '\n my $a = \"ż\"; my $b = \"\\x{17c}\";\n Dump $a; Dump $b; Dump \"$b - $a\"'\n]\n\n> In gitweb we have the bad case: \n> \n> my $title = \"$site_name - $project/$action\";\n> \n> $project and $action are apparently utf8 already but $site_name isn't.\n\n$project and $action are taken from URL, and we have to run decode_utf8\n(at least for query params) for gitweb to work correctly.\n\n$site_name is usually taken from config file, and gitweb doesn't have\n\"use utf8\" pragma.\n\n> The resulting string is marked as utf8 - although the encoding of\n> $site_name was never fixed. The to_utf8() in esc_html() returns the string\n> without fixing anything because of that.\n\nO.K.\n\n_Maybe_ it would be worth adding explanation of this to commit message\n(and I see I should audit gitweb for similar problems elsewhere), but anyway\n\nAcked-by: Jakub Narebski \n\n-- \nJakub Narębski\n"}]}