{"thread":{"id":"8270","subject":"gitweb - encoding problems","startedAt":"2007-05-21T20:57:21Z","lastAt":"2007-05-22T07:50:34Z","messageCount":4,"participants":["Martin Koegler","Ismail Dönmez","David Woodhouse","Jakub Narebski"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"42924","messageId":"20070521205721.GA21771@auto.tuwien.ac.at","threadId":"8270","inReplyTo":null,"subject":"gitweb - encoding problems","fromName":"Martin Koegler","fromEmail":"mkoegler@auto.tuwien.ac.at","sentAt":"2007-05-21T20:57:21Z","receivedAt":"2007-05-21T20:57:21Z","isPatch":false,"sender":{"key":"mkoegler@auto.tuwien.ac.at","avatar":null},"body":"I use ISO-8859-1 as my locale, so my blobs, commits and tags are in\nthis encoding.\n\nOn perl v5.8.6, decode_utf8 of any non utf-8 value returns undefined:\n\n$cat xy\n#!/usr/bin/perl\nuse strict;\nuse warnings;\nuse CGI qw(:standard :escapeHTML -nosticky);\nuse CGI::Util qw(unescape);\nuse CGI::Carp qw(fatalsToBrowser);\nuse Encode;\nuse Fcntl ':mode';\nuse File::Find qw();\nuse File::Basename qw(basename);\n\nbinmode STDOUT, ':utf8';\n\nprint decode_utf8('äöü');\n$perl xy\n[Mon May 21 22:00:00 2007] xy: Use of uninitialized value in print at xy line 14.\n\nIf gitweb encounters, eg. an \"Umlaut\" (äöü) in a commit/tag, use of\nuninitialized value message are generated. In one case,\ndecode_utf8($long) in format_subject_html is undefined, which results\nin a invalid link (a tag contains only title without any value\nassignment) and a browser message, that the html is not valid.\n\nThe previous installed version of git/gitweb (1.5.0rc3) showed only\nsmall black rhombuses, but didn't generate \"uninitialized value\"\nmessages or invalid html.\n\nSo there is regression between git-1.5.0 and git-1.5.2.\n\nAdding $var = encode_utf8($var) if (!defined decode_utf8($var)) for\neach \"uninitialized value\" message results in a correct result for me.\n\nI wanted to post a patch with these changes, as it solved my locale problem.\nBut then I tried the same a different computer with a newer perl (v5.8.8).\n$ cat x\n#!/usr/bin/perl\nuse strict;\nuse warnings;\nuse CGI qw(:standard :escapeHTML -nosticky);\nuse CGI::Util qw(unescape);\nuse CGI::Carp qw(fatalsToBrowser);\nuse Encode;\nuse Fcntl ':mode';\nuse File::Find qw();\nuse File::Basename qw(basename);\n\nbinmode STDOUT, ':utf8';\n\nprint decode_utf8('äöü');\n$ perl x\nￜￜï¿\n\nHere perl decodes the ISO-8859-1 text to something differnent:\n00000000  ef bf bd ef bf bd ef bf  bd                       |ￜￜï¿\n\nThe result is, that all \"Umlaute\" are shown as a small black rhombus\nin gitweb (and no invalid html).\n\nmfg Martin Kögler\n\ncat x |hexdump -C\n00000000  23 21 2f 75 73 72 2f 62  69 6e 2f 70 65 72 6c 0a  |#!/usr/bin/perl.|\n00000010  75 73 65 20 73 74 72 69  63 74 3b 0a 75 73 65 20  |use strict;.use |\n00000020  77 61 72 6e 69 6e 67 73  3b 0a 75 73 65 20 43 47  |warnings;.use CG|\n00000030  49 20 71 77 28 3a 73 74  61 6e 64 61 72 64 20 3a  |I qw(:standard :|\n00000040  65 73 63 61 70 65 48 54  4d 4c 20 2d 6e 6f 73 74  |escapeHTML -nost|\n00000050  69 63 6b 79 29 3b 0a 75  73 65 20 43 47 49 3a 3a  |icky);.use CGI::|\n00000060  55 74 69 6c 20 71 77 28  75 6e 65 73 63 61 70 65  |Util qw(unescape|\n00000070  29 3b 0a 75 73 65 20 43  47 49 3a 3a 43 61 72 70  |);.use CGI::Carp|\n00000080  20 71 77 28 66 61 74 61  6c 73 54 6f 42 72 6f 77  | qw(fatalsToBrow|\n00000090  73 65 72 29 3b 0a 75 73  65 20 45 6e 63 6f 64 65  |ser);.use Encode|\n000000a0  3b 0a 75 73 65 20 46 63  6e 74 6c 20 27 3a 6d 6f  |;.use Fcntl ':mo|\n000000b0  64 65 27 3b 0a 75 73 65  20 46 69 6c 65 3a 3a 46  |de';.use File::F|\n000000c0  69 6e 64 20 71 77 28 29  3b 0a 75 73 65 20 46 69  |ind qw();.use Fi|\n000000d0  6c 65 3a 3a 42 61 73 65  6e 61 6d 65 20 71 77 28  |le::Basename qw(|\n000000e0  62 61 73 65 6e 61 6d 65  29 3b 0a 0a 62 69 6e 6d  |basename);..binm|\n000000f0  6f 64 65 20 53 54 44 4f  55 54 2c 20 27 3a 75 74  |ode STDOUT, ':ut|\n00000100  66 38 27 3b 0a 0a 70 72  69 6e 74 20 64 65 63 6f  |f8';..print deco|\n00000110  64 65 5f 75 74 66 38 28  27 e4 f6 fc 27 29 3b 0a  |de_utf8('äöü');.|\n00000120\n"},{"id":"42926","messageId":"200705220009.09911.ismail@pardus.org.tr","threadId":"8270","inReplyTo":"20070521205721.GA21771@auto.tuwien.ac.at","subject":"Re: gitweb - encoding problems","fromName":"Ismail Dönmez","fromEmail":"ismail@pardus.org.tr","sentAt":"2007-05-21T21:09:09Z","receivedAt":"2007-05-21T21:09:09Z","isPatch":false,"sender":{"key":"ismail@pardus.org.tr","avatar":null},"body":"On Monday 21 May 2007 23:57:21 you wrote:\n> binmode STDOUT, ':utf8';\n>\n> print decode_utf8('äöü');\n\n[~]> perl test.pl\näöü\n\n[~]> cat test.pl\nuse Encode;\nbinmode STDOUT, ':utf8';\n\nprint decode_utf8('äöü'),\"\\n\";\n[cartman@southpark][00:08:15]\n[~]> perl --version\n\nThis is perl, v5.8.8 built for i686-linux\n\nCopyright 1987-2006, Larry Wall\n\nPerl may be copied only under the terms of either the Artistic License or the\nGNU General Public License, which may be found in the Perl 5 source kit.\n\nComplete documentation for Perl, including FAQ lists, should be found on\nthis system using \"man perl\" or \"perldoc perl\".  If you have access to the\nInternet, point your browser at http://www.perl.org/, the Perl Home Page.\n\nYou got an old Encode.\n\n-- \nPerfect is the enemy of good\n"},{"id":"42940","messageId":"1179794012.2771.112.camel@shinybook.infradead.org","threadId":"8270","inReplyTo":"20070521205721.GA21771@auto.tuwien.ac.at","subject":"Re: gitweb - encoding problems","fromName":"David Woodhouse","fromEmail":"dwmw2@infradead.org","sentAt":"2007-05-22T00:33:32Z","receivedAt":"2007-05-22T00:33:32Z","isPatch":false,"sender":{"key":"dwmw2@infradead.org","avatar":"https://gravatar.com/avatar/7afd4f07e0cf7d7e046ae2d23678296b37777c96488e6f3451e78a5514154ebd?d=mp&s=160"},"body":"On Mon, 2007-05-21 at 22:57 +0200, Martin Koegler wrote:\n> I use ISO-8859-1 as my locale, so my blobs, commits and tags are in\n> this encoding. \n\nThat's a very strange thing for anyone to do in the 21st century.\nDid you configure this archaic thing correctly in .git/config?\n\nOtherwise, gitweb will assume that you're using utf-8 like any normal\nperson would, and of course you'll have problems when it tries to deal\nwith your legacy character set as if it were something sensible.\n\n-- \ndwmw2\n"},{"id":"42965","messageId":"200705220950.35493.jnareb@gmail.com","threadId":"8270","inReplyTo":"1179794012.2771.112.camel@shinybook.infradead.org","subject":"Re: gitweb - encoding problems","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2007-05-22T07:50:34Z","receivedAt":"2007-05-22T07:50:34Z","isPatch":false,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"On Thu, 22 May 2007, David Woodhouse wrote:\n> On Mon, 2007-05-21 at 22:57 +0200, Martin Koegler wrote:\n\n>> I use ISO-8859-1 as my locale, so my blobs, commits and tags are in\n>> this encoding. \n> \n> That's a very strange thing for anyone to do in the 21st century.\n> Did you configure this archaic thing correctly in .git/config?\n> \n> Otherwise, gitweb will assume that you're using utf-8 like any normal\n> person would, and of course you'll have problems when it tries to deal\n> with your legacy character set as if it were something sensible.\n\nActually gitweb does not respect i18n.* configuration variables and\nhappily assumes that everything is in utf-8, with the exception of \n*_plain views, which are send :raw.\n\nIf you decide to implement supporting encodings other that utf-8 in \ngitweb, please remember that some (like git-show, git-log) but not all \nparts (like git-rev-list or --pretty=raw) do the decoding/encoding. And \nthat git can be compiled without iconv support. And that comits might \nbe in different encodings, which should be given by 'encoding' header, \nbut there is no way to guess encoding for a blob, or for a file names.\n\n\ngit-commit(1):\n i18n.commitEncoding::\n     Character encoding the commit messages are stored in; git itself\n     does not care per se, but this information is necessary e.g. when\n     importing commits from emails or in the gitk graphical history\n     browser (and possibly at other places in the future or in other\n     porcelains). See e.g. gitlink:git-mailinfo[1]. Defaults to 'utf-8'.\n\n i18n.logOutputEncoding::\n     Character encoding the commit messages are converted to when\n     running `git-log` and friends.\n\n-- \nJakub Narebski\nPoland\n"}]}