{"thread":{"id":"7284","subject":"UTF-8 trouble with GIT 1.5.0.3","startedAt":"2007-03-17T10:27:30Z","lastAt":"2007-03-17T10:27:30Z","messageCount":1,"participants":["Willy Tarreau"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"37308","messageId":"20070317102730.GA15256@1wt.eu","threadId":"7284","inReplyTo":null,"subject":"UTF-8 trouble with GIT 1.5.0.3","fromName":"Willy Tarreau","fromEmail":"w@1wt.eu","sentAt":"2007-03-17T10:27:30Z","receivedAt":"2007-03-17T10:27:30Z","isPatch":false,"sender":{"key":"w@1wt.eu","avatar":"https://avatars.githubusercontent.com/u/8141789?v=4"},"body":"Hi Junio,\n\nafter upgrading from GIT 1.4 to 1.5, I noticed that I had got\nerrors when applying patches from mailboxes and that I had to \nmanually add \"--no-utf8\" to git-am. The error was always :\n\n  \"fatal: cannot convert from latin1 to utf-8\".\n\nThis morning, I decided to take a closer look at it, first\ntrying to fix it in the configuration. Even this fails :\n\n[i18n]\n        commitencoding = latin1\n\n$ git-am -3 username.patch \nfatal: cannot convert from latin1 to latin1\n\n\nSo I took a look at the code and noticed that in builtin-mailinfo,\nconvert_to_utf8() is always called because metainfo_charset is\nalways set. Also, this function always calls reencode_string(),\nno matter what input and outputs are passed. And since it fails\nfor an unknown reason, I get the error message.\n\nNow I have several options :\n \n - always pass --no-utf8 with git-am to leave metainfo_charset\n   NULL and avoid the call to convert_to_utf8()\n   => very annoying and I have to rm -rf .dotest every time I forget\n      about it\n\n - add a check to convert_to_utf8() so that when input and output\n   charsets are equal, it does not malloc/convert/copy/free the\n   result. This would seem necessary anyway.\n\n - check for empty charset name instead of just a NULL pointer,\n   so that it is still possible by configuration to stay compatible\n   with old default behaviour which is : not to convert by default.\n\n - switch back to 1.4 : this will be my immediate option while\n   waiting for any advice on the right long-term solution.\n\nI would be glad to get a recommendation for one of the solutions\nabove. I can workout a patch, but I don't want to waste my time\ndoing crappy patches that will not be retained.\n\nPlease note that I don't know a damn thing about the way iconv\nworks, which may be one of the reason I really hate it. So it\nwill take far too much time to me to dig below reencode_string().\n\nAlso, I am really worried for buffer overflows here. Looking at\nthe code, it seems like headers from input mails are passed to\ndecode_header(), which itself calls decode_header_bq() which in\nturns calls convert_to_utf8() with input and output buffers of\n256 bytes. Same for mailinfo() which calls check_header_line()\nwith <line> which has been filled up to sizeof(line) (1000 bytes).\n\nUnless I'm mistaken, encoding from latin1 to utf-8 can grow the\nstring by a 5-to-1 ratio. So this means that the blind strcpy()\nin convert_to_utf8() could overflow on user-crafted mail headers.\nMaybe I overlooked something :-/\n\nRegards,\nWilly\n"}]}