{"thread":{"id":"2208","subject":"Re: gitweb: charset problem","startedAt":"2005-10-24T13:56:10Z","lastAt":"2005-10-25T17:44:47Z","messageCount":6,"participants":["Horst von Brand","Daniel Barkalow","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"10533","messageId":"200510241356.j9ODuAHn005060@laptop11.inf.utfsm.cl","threadId":"2208","inReplyTo":"nico-linux-git@schottelius.org","subject":"Re: gitweb: charset problem","fromName":"Horst von Brand","fromEmail":"vonbrand@inf.utfsm.cl","sentAt":"2005-10-24T13:56:10Z","receivedAt":"2005-10-24T13:56:10Z","isPatch":false,"sender":{"key":"vonbrand@inf.utfsm.cl","avatar":"https://avatars.githubusercontent.com/u/211384?v=4"},"body":"Nico -telmich- Schottelius wrote:\n> gitweb (my $version = \"247\";) seems to send utf-8 as meta tag encoding\n> ().\n\n> The problem is that the name of the user \"HansjOErg\" (OE is the german\n> umlaut) is in iso8859-1 in /etc/passwd. This is guessed, but it does not\n> look like utf-8, as it's a one byte encoding:\n\n> 00007b0: 3031 323a 3130 303a 4861 6e73 6af6 7267 012:100:Hansj.rg\n\n> What would be the correct way to fix that? Change the username to utf-8?\n> (Is this possible without causing problems in other programs?)\n> Or tell gitweb that it should convert non-UTF-8 to UTF-8?\n\nI'd be /very/ wary of usernames that aren't plain ASCII, just lovercase\nletters and digits, not starting with a digit, at most 8 characters long.\nI've seen more than enough hard-to-debug funnies in the most surprising\nplaces otherwise.\n\n> But we also have another problem: Sometimes we have umlauts in the commit\n> messages. Those are also displayed incorrectly. When I switch to\n> iso-8859-1 encoding in mozilla, the characters in the username and in the\n> commit message are ok.\n\nI believe the Emperor Penguin decreed messages have to be ASCII, or else\nUTF-8. Please don't add to the mess by using non-portable encodings!\n-- \nDr. Horst H. von Brand User #22616 counter.li.org\nDepartamento de Informatica Fono: +56 32 654431\nUniversidad Tecnica Federico Santa Maria +56 32 654239\nCasilla 110-V, Valparaiso, Chile Fax: +56 32 797513\n"},{"id":"10555","messageId":"Pine.LNX.4.64.0510241743280.25300@iabervon.org","threadId":"2208","inReplyTo":"200510241356.j9ODuAHn005060@laptop11.inf.utfsm.cl","subject":"Re: gitweb: charset problem","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-10-24T21:55:30Z","receivedAt":"2005-10-24T21:55:30Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Mon, 24 Oct 2005, Horst von Brand wrote:\n\n> > But we also have another problem: Sometimes we have umlauts in the commit\n> > messages. Those are also displayed incorrectly. When I switch to\n> > iso-8859-1 encoding in mozilla, the characters in the username and in the\n> > commit message are ok.\n> \n> I believe the Emperor Penguin decreed messages have to be ASCII, or else\n> UTF-8. Please don't add to the mess by using non-portable encodings!\n\nShould we possibly reject non-UTF-8 input to commits?\n\nIIRC, we actually define that to be UTF-8, unlike most of the other stuff, \nfor which we don't actually insist on a policy.\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"10558","messageId":"7vwtk2k08z.fsf@assigned-by-dhcp.cox.net","threadId":"2208","inReplyTo":"Pine.LNX.4.64.0510241743280.25300@iabervon.org","subject":"Re: gitweb: charset problem","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-24T22:39:56Z","receivedAt":"2005-10-24T22:39:56Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow writes:\n\n> On Mon, 24 Oct 2005, Horst von Brand wrote:\n>\n>> I believe the Emperor Penguin decreed messages have to be\n>> ASCII, or else UTF-8. Please don't add to the mess by using\n>> non-portable encodings!\n>\n> Should we possibly reject non-UTF-8 input to commits?\n\nPlease, don't.\n\n> IIRC, we actually define that to be UTF-8, unlike most of the\n> other stuff, for which we don't actually insist on a policy.\n\nNo, we do not define nor insist on a particluar policy as far as\nI know. We suggest the use of UTF-8 merely from common sense to\nhelp interoperability, and make UTF-8 slightly easier to use\nthan other encodings by giving specific support for it in some\ntools, namely -u flag in git-mailinfo.\n\nIt is perfectly reasonable if a company internal project that\nworks in Russia to standardize on KOI, or in Japan on EUC-JP.\nWe simply allow it without encouraging nor discouraging it. If\ngitweb can take a configuration mechanism to override the\nbuilt-in UTF-8 header, that is perfectly a valid thing to do to\nhelp such an environment.\n\nHowever, we suggest UTF-8 if the project does not have a\ncompelling reason to do otherwise [*1*]. If you want to be\nprepared for the day your project might have wider participants\nthan you originally envisioned, that is the most sensible thing\nto do. This is especially true because the commit logs cannot\nbe re-encoded after the fact.\n\n[Footnote]\n\n*1* For example, I've never made GNU emacs to work well with\nJapanese in UTF-8 , so if people in my company internal project\nwanted to use Japanese in commit logs, I would probably\nstandardize on EUC-JP for such a project. Luckily so far I have\nnot been forced to make that decision.\n"},{"id":"10586","messageId":"Pine.LNX.4.64.0510251138290.25300@iabervon.org","threadId":"2208","inReplyTo":"7vwtk2k08z.fsf@assigned-by-dhcp.cox.net","subject":"Re: gitweb: charset problem","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2005-10-25T16:01:15Z","receivedAt":"2005-10-25T16:01:15Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Mon, 24 Oct 2005, Junio C Hamano wrote:\n\n> > IIRC, we actually define that to be UTF-8, unlike most of the\n> > other stuff, for which we don't actually insist on a policy.\n> \n> No, we do not define nor insist on a particluar policy as far as\n> I know. We suggest the use of UTF-8 merely from common sense to\n> help interoperability, and make UTF-8 slightly easier to use\n> than other encodings by giving specific support for it in some\n> tools, namely -u flag in git-mailinfo.\n\nI thought we'd decided on uninterpreted byte values for blobs, filenames, \nand trees (and everything in the working tree), but using UTF-8 for tag \nand commit objects.\n\nConsider if you started a project in EUC-JP, and then decided to switch to \nUTF-8 later (when your environment handled it cleanly, perhaps). You could \nconvert all the file contents and move files to re-encoded names, but \nyou'd then want to commit these changes and have the log before and after \nsimultaneously intelligable.\n\n> [Footnote]\n> \n> *1* For example, I've never made GNU emacs to work well with\n> Japanese in UTF-8 , so if people in my company internal project\n> wanted to use Japanese in commit logs, I would probably\n> standardize on EUC-JP for such a project. Luckily so far I have\n> not been forced to make that decision.\n\nIt wouldn't be hard to convert at some point between the editor and the \ncommit object, and you don't re-edit the commit objects like you do \ntracked files. It probably wouldn't even be hard for commit-tree to \nconvert its input based on locale. (And stuff which prints commit contents \nfor user consumption probably ought to re-encode it if necessary, too)\n\n\t-Daniel\n*This .sig left intentionally blank*\n"},{"id":"10593","messageId":"7vu0f5fqps.fsf@assigned-by-dhcp.cox.net","threadId":"2208","inReplyTo":"Pine.LNX.4.64.0510251138290.25300@iabervon.org","subject":"Re: gitweb: charset problem","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-25T17:31:43Z","receivedAt":"2005-10-25T17:31:43Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow writes:\n\n> Consider if you started a project in EUC-JP, and then decided to switch to \n> UTF-8 later (when your environment handled it cleanly, perhaps). You could \n> convert...\n\nI am not saying that using UTF-8 is impossible in some\nsituations. I am saying that if all involved parties agree to\nuse something else in a private project, that is their choice\nand there is no reason to forbid it. They may be shooting in\nthe foot in the long run, they may be not.\n\nFor the internal project I was using as an example, I do not\nforsee anybody who does not do Japanese ever touching it, nor it\nneeds to record any other language in the future (this comes\nfrom the nature of the project -- keeping track of some\ndocuments that are written in Japanese and we are not in\ntranslation business). Log and contents being encoded in EUC-JP\nis perfectly valid right now and in the future in that project.\nIn such an application there is nothing gained by using UTF-8,\nand it will only inconvenience the users if we insisted on\nUTF-8.\n"},{"id":"10596","messageId":"7v64rlfq40.fsf@assigned-by-dhcp.cox.net","threadId":"2208","inReplyTo":"Pine.LNX.4.64.0510251138290.25300@iabervon.org","subject":"Re: gitweb: charset problem","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-10-25T17:44:47Z","receivedAt":"2005-10-25T17:44:47Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Daniel Barkalow writes:\n\n> It wouldn't be hard to convert at some point between the editor and the \n> commit object, and you don't re-edit the commit objects like you do \n> tracked files. It probably wouldn't even be hard for commit-tree to \n> convert its input based on locale. (And stuff which prints commit contents \n> for user consumption probably ought to re-encode it if necessary, too)\n\nDon't get me wrong. I am not opposed to giving preferential\ntreatment to UTF-8 by supporting it better. I think it may be a\ngood idea to have an *option* in commit-tree and mktag to\nconvert from LC_CTYPE to utf-8, just like mailinfo does.\n\nI am just opposing to make UTF-8 mandatory.\n"}]}