{"thread":{"id":"24023","subject":"[PATCH 1/2] [RFC] add --recode-patch option to git-mailinfo","startedAt":"2010-06-06T11:59:42Z","lastAt":"2010-06-07T01:44:36Z","messageCount":4,"participants":["Zhang Le","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"143082","messageId":"1275825583-23913-1-git-send-email-r0bertz@gentoo.org","threadId":"24023","inReplyTo":null,"subject":"[PATCH 1/2] [RFC] add --recode-patch option to git-mailinfo","fromName":"Zhang Le","fromEmail":"r0bertz@gentoo.org","sentAt":"2010-06-06T11:59:42Z","receivedAt":"2010-06-06T11:59:42Z","isPatch":true,"sender":{"key":"r0bertz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/29025?v=4"},"body":"I have a translation project which uses UTF-8 as charset.\nSo the patch must be encoded in UTF-8, not just the commit msg etc.\nAnd we use google group as our mailing list.\n\nRecently, due to unknown reason, mails saved from gmail are encoded using GB2312.\nThis never happened before. I guess google has did something.\nBut I haven't found how to change this behavior.\n\nSo I took another way, i.e. add this option to git-mailinfo.\nI hope this could benefit others as well.\n\nSigned-off-by: Zhang Le <r0bertz@gentoo.org>\n---\n builtin/mailinfo.c  |    8 +++++++-\n man1/git-mailinfo.1 |    7 ++++++-\n 2 files changed, 13 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/mailinfo.c b/builtin/mailinfo.c\nindex 4a9729b..73f51f3 100644\n--- a/builtin/mailinfo.c\n+++ b/builtin/mailinfo.c\n@@ -12,6 +12,7 @@ static FILE *cmitmsg, *patchfile, *fin, *fout;\n static int keep_subject;\n static int keep_non_patch_brackets_in_subject;\n static const char *metainfo_charset;\n+static int recode_patch;\n static struct strbuf line = STRBUF_INIT;\n static struct strbuf name = STRBUF_INIT;\n static struct strbuf email = STRBUF_INIT;\n@@ -830,6 +831,8 @@ static int handle_commit_msg(struct strbuf *line)\n \n static void handle_patch(const struct strbuf *line)\n {\n+\tif (recode_patch)\n+\t\tconvert_to_utf8(line, charset.buf);\n \tfwrite(line->buf, 1, line->len, patchfile);\n \tpatch_lines++;\n }\n@@ -1021,7 +1024,7 @@ static int git_mailinfo_config(const char *var, const char *value, void *unused)\n }\n \n static const char mailinfo_usage[] =\n-\t\"git mailinfo [-k|-b] [-u | --encoding=<encoding> | -n] [--scissors | --no-scissors] msg patch < mail >info\";\n+\t\"git mailinfo [-k|-b] [-u | --encoding=<encoding> | -n] [--recode-patch] [--scissors | --no-scissors] msg patch < mail >info\";\n \n int cmd_mailinfo(int argc, const char **argv, const char *prefix)\n {\n@@ -1034,6 +1037,7 @@ int cmd_mailinfo(int argc, const char **argv, const char *prefix)\n \n \tdef_charset = (git_commit_encoding ? git_commit_encoding : \"UTF-8\");\n \tmetainfo_charset = def_charset;\n+\trecode_patch = 0;\n \n \twhile (1 < argc && argv[1][0] == '-') {\n \t\tif (!strcmp(argv[1], \"-k\"))\n@@ -1046,6 +1050,8 @@ int cmd_mailinfo(int argc, const char **argv, const char *prefix)\n \t\t\tmetainfo_charset = NULL;\n \t\telse if (!prefixcmp(argv[1], \"--encoding=\"))\n \t\t\tmetainfo_charset = argv[1] + 11;\n+\t\telse if (!strcmp(argv[1], \"--recode-patch\"))\n+\t\t\trecode_patch = 1;\n \t\telse if (!strcmp(argv[1], \"--scissors\"))\n \t\t\tuse_scissors = 1;\n \t\telse if (!strcmp(argv[1], \"--no-scissors\"))\ndiff --git a/man1/git-mailinfo.1 b/man1/git-mailinfo.1\nindex 4d0e929..d52457f 100644\n--- a/man1/git-mailinfo.1\n+++ b/man1/git-mailinfo.1\n@@ -22,7 +22,7 @@\n git-mailinfo \\- Extracts patch and authorship from a single e\\-mail message\n .SH \"SYNOPSIS\"\n .sp\n-\\fIgit mailinfo\\fR [\\-k|\\-b] [\\-u | \\-\\-encoding=<encoding> | \\-n] [\\-\\-scissors] <msg> <patch>\n+\\fIgit mailinfo\\fR [\\-k|\\-b] [\\-u | \\-\\-encoding=<encoding> | \\-n] [\\-\\-recode\\-patch] [\\-\\-scissors] <msg> <patch>\n .SH \"DESCRIPTION\"\n .sp\n Reads a single e\\-mail message from the standard input, and writes the commit log message in <msg> file, and the patches in <patch> file\\&. The author name, e\\-mail and e\\-mail subject are written out to the standard output to be used by \\fIgit am\\fR to create a commit\\&. It is usually not necessary to use this command directly\\&. See \\fBgit-am\\fR(1) instead\\&.\n@@ -70,6 +70,11 @@ Similar to \\-u but if the local convention is different from what is specified b\n Disable all charset re\\-coding of the metadata\\&.\n .RE\n .PP\n+\\-\\-recode\\-patch\n+.RS 4\n+Re\\-code patch as well, using the same encoding as metadata\\&. The default is off\\&.\n+.RE\n+.PP\n \\-\\-scissors\n .RS 4\n Remove everything in body before a scissors line\\&. A line that mainly consists of scissors (either \">8\" or \"8<\") and perforation (dash \"\\-\") marks is called a scissors line, and is used to request the reader to cut the message at that line\\&. If such a line appears in the body of the message before the patch, everything before it (including the scissors line itself) is ignored when this option is used\\&.\n-- \n1.7.1\n"},{"id":"143083","messageId":"1275825583-23913-2-git-send-email-r0bertz@gentoo.org","threadId":"24023","inReplyTo":"1275825583-23913-1-git-send-email-r0bertz@gentoo.org","subject":"[PATCH 2/2] [RFC] add --recode-patch to git-am","fromName":"Zhang Le","fromEmail":"r0bertz@gentoo.org","sentAt":"2010-06-06T11:59:43Z","receivedAt":"2010-06-06T11:59:43Z","isPatch":true,"sender":{"key":"r0bertz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/29025?v=4"},"body":"The reason has been explained in the \"add --recode-patch to git-mailinfo\" patch\n\nSigned-off-by: Zhang Le <r0bertz@gentoo.org>\n---\n git-am.sh     |   13 +++++++++++--\n man1/git-am.1 |   12 +++++++++++-\n 2 files changed, 22 insertions(+), 3 deletions(-)\n\ndiff --git a/git-am.sh b/git-am.sh\nindex 1056075..62e7870 100755\n--- a/git-am.sh\n+++ b/git-am.sh\n@@ -14,6 +14,7 @@ b,binary*       (historical option -- no-op)\n q,quiet         be quiet\n s,signoff       add a Signed-off-by line to the commit message\n u,utf8          recode into utf8 (default)\n+recode-patch    pass --recode-patch flag to git-mailinfo\n k,keep          pass -k flag to git-mailinfo\n keep-cr         pass --keep-cr flag to git-mailsplit for mbox format\n no-keep-cr      do not pass --keep-cr flag to git-mailsplit independent of am.keepcr\n@@ -294,7 +295,7 @@ split_patches () {\n prec=4\n dotest=\"$GIT_DIR/rebase-apply\"\n sign= utf8=t keep= keepcr= skip= interactive= resolved= rebasing= abort=\n-resolvemsg= resume= scissors= no_inbody_headers=\n+resolvemsg= resume= scissors= no_inbody_headers= recode_patch=\n git_apply_opt=\n committer_date_is_author_date=\n ignore_date=\n@@ -320,6 +321,8 @@ do\n \t\tutf8=t ;; # this is now default\n \t--no-utf8)\n \t\tutf8= ;;\n+\t--recode-patch)\n+\t\trecode_patch=t ;;\n \t-k|--keep)\n \t\tkeep=t ;;\n \t-c|--scissors)\n@@ -463,6 +466,7 @@ else\n \techo \"$threeway\" >\"$dotest/threeway\"\n \techo \"$sign\" >\"$dotest/sign\"\n \techo \"$utf8\" >\"$dotest/utf8\"\n+\techo \"$recode_patch\" >\"$dotest/recode_patch\"\n \techo \"$keep\" >\"$dotest/keep\"\n \techo \"$keepcr\" >\"$dotest/keepcr\"\n \techo \"$scissors\" >\"$dotest/scissors\"\n@@ -504,6 +508,10 @@ then\n else\n \tutf8=-n\n fi\n+if test \"$(cat \"$dotest/recode_patch\")\" = t\n+then\n+\trecodepatch=--recode-patch\n+fi\n if test \"$(cat \"$dotest/keep\")\" = t\n then\n \tkeep=-k\n@@ -580,7 +588,8 @@ do\n \t# by the user, or the user can tell us to do so by --resolved flag.\n \tcase \"$resume\" in\n \t'')\n-\t\tgit mailinfo $keep $no_inbody_headers $scissors $utf8 \"$dotest/msg\" \"$dotest/patch\" \\\n+\t\tgit mailinfo $keep $no_inbody_headers $scissors $utf8 \\\n+\t\t$recodepatch \"$dotest/msg\" \"$dotest/patch\" \\\n \t\t\t<\"$dotest/$msgnum\" >\"$dotest/info\" ||\n \t\t\tstop_here $this\n \ndiff --git a/man1/git-am.1 b/man1/git-am.1\nindex c6a0d27..b5bc0e8 100644\n--- a/man1/git-am.1\n+++ b/man1/git-am.1\n@@ -24,7 +24,7 @@ git-am \\- Apply a series of patches from a mailbox\n .sp\n .nf\n \\fIgit am\\fR [\\-\\-signoff] [\\-\\-keep] [\\-\\-keep\\-cr | \\-\\-no\\-keep\\-cr] [\\-\\-utf8 | \\-\\-no\\-utf8]\n-         [\\-\\-3way] [\\-\\-interactive] [\\-\\-committer\\-date\\-is\\-author\\-date]\n+         [\\-\\-recode\\-patch] [\\-\\-3way] [\\-\\-interactive] [\\-\\-committer\\-date\\-is\\-author\\-date]\n          [\\-\\-ignore\\-date] [\\-\\-ignore\\-space\\-change | \\-\\-ignore\\-whitespace]\n          [\\-\\-whitespace=<option>] [\\-C<n>] [\\-p<n>] [\\-\\-directory=<dir>]\n          [\\-\\-reject] [\\-q | \\-\\-quiet] [\\-\\-scissors | \\-\\-no\\-scissors]\n@@ -116,6 +116,16 @@ flag to\n \\fBgit-mailinfo\\fR(1))\\&.\n .RE\n .PP\n+\\-\\-recode\\-patch\n+.RS 4\n+Pass\n+\\-\\-recode\\-patch\n+flag to\n+\\fIgit mailinfo\\fR\n+(see\n+\\fBgit-mailinfo\\fR(1))\\&.\n+.RE\n+.PP\n \\-3, \\-\\-3way\n .RS 4\n When the patch does not apply cleanly, fall back on 3\\-way merge if the patch records the identity of blobs it is supposed to apply to and we have those blobs available locally\\&.\n-- \n1.7.1\n"},{"id":"143102","messageId":"7vr5kkvshz.fsf@alter.siamese.dyndns.org","threadId":"24023","inReplyTo":"1275825583-23913-1-git-send-email-r0bertz@gentoo.org","subject":"Re: [PATCH 1/2] [RFC] add --recode-patch option to git-mailinfo","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-06-06T20:03:04Z","receivedAt":"2010-06-06T20:03:04Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Zhang Le <r0bertz@gentoo.org> writes:\n\n> I have a translation project which uses UTF-8 as charset.\n> So the patch must be encoded in UTF-8, not just the commit msg etc.\n> And we use google group as our mailing list.\n>\n> Recently, due to unknown reason, mails saved from gmail are encoded using GB2312.\n> This never happened before. I guess google has did something.\n> But I haven't found how to change this behavior.\n>\n> So I took another way, i.e. add this option to git-mailinfo.\n> I hope this could benefit others as well.\n>\n> Signed-off-by: Zhang Le <r0bertz@gentoo.org>\n> ---\n>  builtin/mailinfo.c  |    8 +++++++-\n>  man1/git-mailinfo.1 |    7 ++++++-\n\nDon't patch anything in man?/ as they are autogenerated files and not\nsource; patch the source file in Documentation/ directory instead.\n\nI take it that you recode from whatever encoding the mail message is in\n(probably stated in \"Content-type: ...; charset=xxx\" header) to the\nencoding specified with --encoding option (defaulting to UTF-8), but it\nwasn't very clear from the documentation.  We might want to improve \nthe descriptions of both this new option and --encoding option.\n\nAlso it might be useful to find out what that \"due to unknown reason\" is,\nat least to see if that is what Google did or what the user did.\n\nThanks.\n"},{"id":"143130","messageId":"20100607014434.GA24220@adriano","threadId":"24023","inReplyTo":"7vr5kkvshz.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 1/2] [RFC] add --recode-patch option to git-mailinfo","fromName":"Zhang Le","fromEmail":"r0bertz@gentoo.org","sentAt":"2010-06-07T01:44:36Z","receivedAt":"2010-06-07T01:44:36Z","isPatch":true,"sender":{"key":"r0bertz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/29025?v=4"},"body":"On 13:03 Sun 06 Jun     , Junio C Hamano wrote:\n> Zhang Le <r0bertz@gentoo.org> writes:\n> \n> > I have a translation project which uses UTF-8 as charset.\n> > So the patch must be encoded in UTF-8, not just the commit msg etc.\n> > And we use google group as our mailing list.\n> >\n> > Recently, due to unknown reason, mails saved from gmail are encoded using GB2312.\n> > This never happened before. I guess google has did something.\n> > But I haven't found how to change this behavior.\n> >\n> > So I took another way, i.e. add this option to git-mailinfo.\n> > I hope this could benefit others as well.\n> >\n> > Signed-off-by: Zhang Le <r0bertz@gentoo.org>\n> > ---\n> >  builtin/mailinfo.c  |    8 +++++++-\n> >  man1/git-mailinfo.1 |    7 ++++++-\n> \n> Don't patch anything in man?/ as they are autogenerated files and not\n> source; patch the source file in Documentation/ directory instead.\n\nThanks, will do it.\n\n> \n> I take it that you recode from whatever encoding the mail message is in\n> (probably stated in \"Content-type: ...; charset=xxx\" header) to the\n> encoding specified with --encoding option (defaulting to UTF-8), but it\n> wasn't very clear from the documentation.  We might want to improve \n> the descriptions of both this new option and --encoding option.\n\nThat's exactly what this patch's purpose is.\nI will try to improve the doc.\n\n> \n> Also it might be useful to find out what that \"due to unknown reason\" is,\n> at least to see if that is what Google did or what the user did.\n\nOne of my friend, Yang Xiaoguang, found that google tries to detect the\nlanguage of the email and recode it using the native charset.\nFor Simplified Chinese, it is GB2312.\nFor Traditional Chinese, it is Big5.\n\nIn the test, Yang sent all emails using UTF-8 charset.\nHe sent those mails to a google group and then checked the \"Content-type: ...;\ncharset=xxx\" in gmail.\n\nIf the mail is written in Simplified Chinese, the charset became GB2312.\nIf the mail is written in Traditional Chinese, the charset became Big5.\nIf the mail is mixed with Simplified and Traditional Chinese, the charset\nremains as UTF-8.\n\n-- \nZhang, Le\nGentoo/Loongson Developer\nhttp://zhangle.is-a-geek.org\n0260 C902 B8F8 6506 6586 2B90 BC51 C808 1E4E 2973\n"}]}