{"thread":{"id":"31866","subject":"fatal: cannot convert from utf8 to UTF-8","startedAt":"2012-10-19T00:03:26Z","lastAt":"2012-10-19T05:50:45Z","messageCount":2,"participants":["Cristian Tibirna","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"201543","messageId":"1532361.Y42DjGJIX1@leto","threadId":"31866","inReplyTo":null,"subject":"fatal: cannot convert from utf8 to UTF-8","fromName":"Cristian Tibirna","fromEmail":"ctibirna@giref.ulaval.ca","sentAt":"2012-10-19T00:03:26Z","receivedAt":"2012-10-19T00:03:26Z","isPatch":false,"sender":{"key":"ctibirna@giref.ulaval.ca","avatar":null},"body":"\nThis error:\n\nfatal: cannot convert from utf8 to UTF-8\n\noccured in two distinct situations in our work group with git binaries older \nor equal to 1.7.7. Once during a commit, the other time during a rebase. Both \noccurences are 100% reproductible. But the commit that gives the error during \na rebase doesn't do so in a cherry-pick.\n\nThis is in part our fault: during the standardisation of our git environment, \nwe (re)enforced UTF-8 encodings by setting \"i18n.commitenconding\" and \n\"i18n.logOutputEncoding\" to \"utf8\".\n\nIt is the \"i18n.logOutputEncoding = utf8\" that *sometimes* triggers the error \nabove.\n\nI know \"utf8\" is not an accepted denomination (\"UTF-8\" or \"utf-8\" should be \nused, according to IANA standards), but we have attenuating circumstances in \nthe fact that most things dealing with encoding accept the erroneous name. \nThat includes at least iconv(1) and python(1). Thus we ignored that a \ndistinction existed and, as self-respecting lazy typers, we preferred the (one \ntouch) shorter version.\n\nI wonder if it should be expected that git accepts these name variants (\"utf8\" \nand \"UTF8\") as valid and equivalent to the standard ones.\n\nOf course it is very easy for us to work around the error, since setting \n\"i18n.logOutputEncoding = utf-8\" or removing it altogether from the git config \nfile chases the error away. It's only that these kinds of things are bound to \nhappen and for a good proportion of git users it might be well opaque, \ndifficult to fix and, in drastic (user ignorance-induced) cases, a \nshowstopper.\n\nThanks for listening.\n\n-- \nCristian Tibirna\t\t\t\t(418) 656-2131 / 4340\n  Laval University - Québec, CAN ... http://www.giref.ulaval.ca/~ctibirna\n  Research professional - GIREF ... ctibirna@giref.ulaval.ca\n"},{"id":"201546","messageId":"7vy5j3j9hm.fsf@alter.siamese.dyndns.org","threadId":"31866","inReplyTo":"1532361.Y42DjGJIX1@leto","subject":"Re: fatal: cannot convert from utf8 to UTF-8","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-10-19T05:50:45Z","receivedAt":"2012-10-19T05:50:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Cristian Tibirna <ctibirna@giref.ulaval.ca> writes:\n\n> This error:\n>\n> fatal: cannot convert from utf8 to UTF-8\n> ...\n> This is in part our fault: during the standardisation of our git environment, \n> we (re)enforced UTF-8 encodings by setting \"i18n.commitenconding\" and \n> \"i18n.logOutputEncoding\" to \"utf8\".\n> ...\n> I know \"utf8\" is not an accepted denomination (\"UTF-8\" or \"utf-8\" should be \n> used, according to IANA standards),...\n\nPerhaps like this.\n\n-- >8 --\nSubject: [PATCH] reencode_string(): introduce and use same_encoding()\n\nCallers of reencode_string() that re-encodes a string from one\nencoding to another all used ad-hoc way to bypass the case where the\ninput and the output encodings are the same.  Some did strcmp(),\nsome did strcasecmp(), yet some others when converting to UTF-8 used\nis_encoding_utf8().\n\nIntroduce same_encoding() helper function to make these callers\nuse the same logic.  Notably, is_encoding_utf8() has a work-around\nfor common misconfiguration to use \"utf8\" to name UTF-8 encoding,\nwhich does not match \"UTF-8\" hence strcasecmp() would not consider\nthe same.  Make use of it in this helper function.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n builtin/mailinfo.c | 2 +-\n notes.c            | 2 +-\n pretty.c           | 2 +-\n sequencer.c        | 2 +-\n utf8.c             | 7 +++++++\n utf8.h             | 1 +\n 6 files changed, 12 insertions(+), 4 deletions(-)\n\ndiff --git c/builtin/mailinfo.c w/builtin/mailinfo.c\nindex da23140..e4e39d6 100644\n--- c/builtin/mailinfo.c\n+++ w/builtin/mailinfo.c\n@@ -483,7 +483,7 @@ static void convert_to_utf8(struct strbuf *line, const char *charset)\n \n \tif (!charset || !*charset)\n \t\treturn;\n-\tif (!strcasecmp(metainfo_charset, charset))\n+\tif (same_encoding(metainfo_charset, charset))\n \t\treturn;\n \tout = reencode_string(line->buf, metainfo_charset, charset);\n \tif (!out)\ndiff --git c/notes.c w/notes.c\nindex bc454e1..ee8f01f 100644\n--- c/notes.c\n+++ w/notes.c\n@@ -1231,7 +1231,7 @@ static void format_note(struct notes_tree *t, const unsigned char *object_sha1,\n \t}\n \n \tif (output_encoding && *output_encoding &&\n-\t\t\tstrcmp(utf8, output_encoding)) {\n+\t    !is_encoding_utf8(output_encoding)) {\n \t\tchar *reencoded = reencode_string(msg, output_encoding, utf8);\n \t\tif (reencoded) {\n \t\t\tfree(msg);\ndiff --git c/pretty.c w/pretty.c\nindex 8b1ea9f..e87fe9f 100644\n--- c/pretty.c\n+++ w/pretty.c\n@@ -504,7 +504,7 @@ char *logmsg_reencode(const struct commit *commit,\n \t\treturn NULL;\n \tencoding = get_header(commit, \"encoding\");\n \tuse_encoding = encoding ? encoding : utf8;\n-\tif (!strcmp(use_encoding, output_encoding))\n+\tif (same_encoding(use_encoding, output_encoding))\n \t\tif (encoding) /* we'll strip encoding header later */\n \t\t\tout = xstrdup(commit->buffer);\n \t\telse\ndiff --git c/sequencer.c w/sequencer.c\nindex e3723d2..73c396b 100644\n--- c/sequencer.c\n+++ w/sequencer.c\n@@ -60,7 +60,7 @@ static int get_message(struct commit *commit, struct commit_message *out)\n \n \tout->reencoded_message = NULL;\n \tout->message = commit->buffer;\n-\tif (strcmp(encoding, git_commit_encoding))\n+\tif (same_encoding(encoding, git_commit_encoding))\n \t\tout->reencoded_message = reencode_string(commit->buffer,\n \t\t\t\t\tgit_commit_encoding, encoding);\n \tif (out->reencoded_message)\ndiff --git c/utf8.c w/utf8.c\nindex a544f15..6a52834 100644\n--- c/utf8.c\n+++ w/utf8.c\n@@ -423,6 +423,13 @@ int is_encoding_utf8(const char *name)\n \treturn 0;\n }\n \n+int same_encoding(const char *src, const char *dst)\n+{\n+\tif (is_encoding_utf8(src) && is_encoding_utf8(dst))\n+\t\treturn 1;\n+\treturn !strcasecmp(src, dst);\n+}\n+\n /*\n  * Given a buffer and its encoding, return it re-encoded\n  * with iconv.  If the conversion fails, returns NULL.\ndiff --git c/utf8.h w/utf8.h\nindex 3c0ae76..93ef600 100644\n--- c/utf8.h\n+++ w/utf8.h\n@@ -7,6 +7,7 @@ int utf8_width(const char **start, size_t *remainder_p);\n int utf8_strwidth(const char *string);\n int is_utf8(const char *text);\n int is_encoding_utf8(const char *name);\n+int same_encoding(const char *, const char *);\n \n int strbuf_add_wrapped_text(struct strbuf *buf,\n \t\tconst char *text, int indent, int indent2, int width);\n"}]}