{"thread":{"id":"19718","subject":"[PATCH 1/2] git-send-email.perl: improve detection of MIME encoded-words","startedAt":"2009-06-07T01:12:30Z","lastAt":"2009-06-08T00:31:20Z","messageCount":5,"participants":["Brandon Casey"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"115680","messageId":"1244337151-6506-1-git-send-email-drafnel@gmail.com","threadId":"19718","inReplyTo":null,"subject":"[PATCH 1/2] git-send-email.perl: improve detection of MIME encoded-words","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2009-06-07T01:12:30Z","receivedAt":"2009-06-07T01:12:30Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"According to rfc2047, an encoded word has the following form:\n\n   encoded-word = \"=?\" charset \"?\" encoding \"?\" encoded-text \"?=\"\n\n   charset = token\n\n   encoding = token\n\n   token = <Any CHAR except SPACE, CTLs, and especials>\n\n   especials = \"(\" / \")\" / \"<\" / \">\" / \"@\" / \",\" / \";\" / \":\" / \"\n               <\"> / \"/\" / \"[\" / \"]\" / \"?\" / \".\" / \"=\"\n\n   encoded-text = <Any printable ASCII character other than \"?\"\n                     or SPACE>\n\nAnd rfc822 defines CTLs as:\n\n    CTL = <any ASCII control;  (  0- 37,  0.- 31.)\n           character and DEL>; (    177,     127.)\n\nThe original code only detected rfc2047 encoded strings when the charset\nwas UTF-8.  This patch generalizes the matching expression and breaks the\ncheck for an rfc2047 encoded string into its own function.  There's no real\nfunctional change, since any properly rfc2047 encoded string (the ones that\nweren't UTF-8) would have fallen through the remaining 'if' statements and\nbeen returned unchanged.\n\nSigned-off-by: Brandon Casey <drafnel@gmail.com>\n---\n git-send-email.perl |   10 +++++++++-\n 1 files changed, 9 insertions(+), 1 deletions(-)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 3d6a982..e735815 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -772,6 +772,14 @@ sub quote_rfc2047 {\n \treturn $_;\n }\n \n+sub is_rfc2047_quoted {\n+\tmy $s = shift;\n+\tmy $token = '[^][()<>@,;:\"\\/?.= \\000-\\037\\177]+';\n+\tmy $encoded_text = '[!->@-~]+';\n+\tlength($s) <= 75 &&\n+\t$s =~ m/^(?:\"[[:ascii:]]*\"|=\\?$token\\?$token\\?$encoded_text\\?=)$/o;\n+}\n+\n # use the simplest quoting being able to handle the recipient\n sub sanitize_address\n {\n@@ -783,7 +791,7 @@ sub sanitize_address\n \t}\n \n \t# if recipient_name is already quoted, do nothing\n-\tif ($recipient_name =~ /^(\"[[:ascii:]]*\"|=\\?utf-8\\?q\\?.*\\?=)$/) {\n+\tif (is_rfc2047_quoted($recipient_name)) {\n \t\treturn $recipient;\n \t}\n \n-- \n1.6.3.1.9.g95405b\n"},{"id":"115681","messageId":"1244337151-6506-2-git-send-email-drafnel@gmail.com","threadId":"19718","inReplyTo":"1244337151-6506-1-git-send-email-drafnel@gmail.com","subject":"[PATCH 2/2] send-email: use UTF-8 rather than utf-8 for consistency","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2009-06-07T01:12:31Z","receivedAt":"2009-06-07T01:12:31Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"The rest of the git source has been converted to use upper-case character\nencoding names to assist older platforms.  The charset attribute of MIME\nis defined to be case-insensitive, but older platforms may still have an\neasier time dealing with upper-case rather than lower-case.  So do so for\nsend-email too.\n\nUpdate t9001 to handle the changes.\n\nSigned-off-by: Brandon Casey <drafnel@gmail.com>\n---\n git-send-email.perl   |    4 ++--\n t/t9001-send-email.sh |    8 ++++----\n 2 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex e735815..1d48349 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -577,7 +577,7 @@ EOT\n \t\t\tif ($need_8bit_cte) {\n \t\t\t\tprint C2 \"MIME-Version: 1.0\\n\",\n \t\t\t\t\t \"Content-Type: text/plain; \",\n-\t\t\t\t\t   \"charset=utf-8\\n\",\n+\t\t\t\t\t   \"charset=UTF-8\\n\",\n \t\t\t\t\t \"Content-Transfer-Encoding: 8bit\\n\";\n \t\t\t}\n \t\t} elsif (/^MIME-Version:/i) {\n@@ -766,7 +766,7 @@ sub unquote_rfc2047 {\n \n sub quote_rfc2047 {\n \tlocal $_ = shift;\n-\tmy $encoding = shift || 'utf-8';\n+\tmy $encoding = shift || 'UTF-8';\n \ts/([^-a-zA-Z0-9!*+\\/])/sprintf(\"=%02X\", ord($1))/eg;\n \ts/(.*)/=\\?$encoding\\?q\\?$1\\?=/;\n \treturn $_;\ndiff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh\nindex ce26ea4..2ce24cd 100755\n--- a/t/t9001-send-email.sh\n+++ b/t/t9001-send-email.sh\n@@ -533,7 +533,7 @@ test_expect_success 'utf8 Cc is rfc2047 encoded' '\n \t--smtp-server=\"$(pwd)/fake.sendmail\" \\\n \toutdir/*.patch &&\n \tgrep \"^Cc:\" msgtxt1 |\n-\tgrep \"=?utf-8?q?=C3=A0=C3=A9=C3=AC=C3=B6=C3=BA?= <utf8@example.com>\"\n+\tgrep \"=?UTF-8?q?=C3=A0=C3=A9=C3=AC=C3=B6=C3=BA?= <utf8@example.com>\"\n '\n \n test_expect_success '--compose adds MIME for utf8 body' '\n@@ -550,7 +550,7 @@ test_expect_success '--compose adds MIME for utf8 body' '\n \t  --smtp-server=\"$(pwd)/fake.sendmail\" \\\n \t  $patches &&\n \tgrep \"^utf8 body\" msgtxt1 &&\n-\tgrep \"^Content-Type: text/plain; charset=utf-8\" msgtxt1\n+\tgrep \"^Content-Type: text/plain; charset=UTF-8\" msgtxt1\n '\n \n test_expect_success '--compose respects user mime type' '\n@@ -573,7 +573,7 @@ test_expect_success '--compose respects user mime type' '\n \t  $patches &&\n \tgrep \"^utf8 body\" msgtxt1 &&\n \tgrep \"^Content-Type: text/plain; charset=iso-8859-1\" msgtxt1 &&\n-\t! grep \"^Content-Type: text/plain; charset=utf-8\" msgtxt1\n+\t! grep \"^Content-Type: text/plain; charset=UTF-8\" msgtxt1\n '\n \n test_expect_success '--compose adds MIME for utf8 subject' '\n@@ -586,7 +586,7 @@ test_expect_success '--compose adds MIME for utf8 subject' '\n \t  --smtp-server=\"$(pwd)/fake.sendmail\" \\\n \t  $patches &&\n \tgrep \"^fake edit\" msgtxt1 &&\n-\tgrep \"^Subject: =?utf-8?q?utf8-s=C3=BCbj=C3=ABct?=\" msgtxt1\n+\tgrep \"^Subject: =?UTF-8?q?utf8-s=C3=BCbj=C3=ABct?=\" msgtxt1\n '\n \n test_expect_success 'detects ambiguous reference/file conflict' '\n-- \n1.6.3.1.9.g95405b\n"},{"id":"115713","messageId":"ee63ef30906070945g62e94313r8b2a7128bde16f9d@mail.gmail.com","threadId":"19718","inReplyTo":"1244337151-6506-1-git-send-email-drafnel@gmail.com","subject":"Re: [PATCH 1/2] git-send-email.perl: improve detection of MIME encoded-words","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2009-06-07T16:45:35Z","receivedAt":"2009-06-07T16:45:35Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"On Sat, Jun 6, 2009 at 8:12 PM, Brandon Casey <drafnel@gmail.com> wrote:\n\n>  git-send-email.perl |   10 +++++++++-\n>  1 files changed, 9 insertions(+), 1 deletions(-)\n>\n> diff --git a/git-send-email.perl b/git-send-email.perl\n> index 3d6a982..e735815 100755\n> --- a/git-send-email.perl\n> +++ b/git-send-email.perl\n> @@ -772,6 +772,14 @@ sub quote_rfc2047 {\n>        return $_;\n>  }\n>\n> +sub is_rfc2047_quoted {\n> +       my $s = shift;\n> +       my $token = '[^][()<>@,;:\"\\/?.= \\000-\\037\\177]+';\n\nThe last element of this regex should be changed to \\177-\\377 since\nnon-ascii characters are disallowed too.\n\nI won't be able to send a new patch for a while.  Home internet is\ndown at the moment, and I don't have an alternative right now.\n\n-brandon\n"},{"id":"115755","messageId":"1244420758-5604-1-git-send-email-drafnel@gmail.com","threadId":"19718","inReplyTo":"ee63ef30906070945g62e94313r8b2a7128bde16f9d@mail.gmail.com","subject":"[PATCH v2] git-send-email.perl: improve detection of MIME encoded-words","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2009-06-08T00:25:58Z","receivedAt":"2009-06-08T00:25:58Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"According to rfc2047, an encoded word has the following form:\n\n   encoded-word = \"=?\" charset \"?\" encoding \"?\" encoded-text \"?=\"\n\n   charset = token\n\n   encoding = token\n\n   token = <Any CHAR except SPACE, CTLs, and especials>\n\n   especials = \"(\" / \")\" / \"<\" / \">\" / \"@\" / \",\" / \";\" / \":\" / \"\n               <\"> / \"/\" / \"[\" / \"]\" / \"?\" / \".\" / \"=\"\n\n   encoded-text = <Any printable ASCII character other than \"?\"\n                     or SPACE>\n\nAnd rfc822 defines CHARs and CTLs as:\n\n    CHAR = <any ASCII character> ; (  0-177,  0.-127.)\n\n    CTL = <any ASCII control     ; (  0- 37,  0.- 31.)\n           character and DEL>    ; (    177,     127.)\n\nThe original code only detected rfc2047 encoded strings when the charset\nwas UTF-8.  This patch generalizes the matching expression and breaks the\ncheck for an rfc2047 encoded string into its own function.  There's no real\nfunctional change, since any properly rfc2047 encoded string would have\nfallen through the remaining 'if' statements and been returned unchanged.\n\nSigned-off-by: Brandon Casey <drafnel@gmail.com>\n---\n\n\nHere's a replacement patch which increases the range of excluded characters\nallowed as tokens so only ASCII characters are allowed (minus the other\nexclusions).\n\n-brandon\n\n\n git-send-email.perl |   10 +++++++++-\n 1 files changed, 9 insertions(+), 1 deletions(-)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 3d6a982..8a1a40d 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -772,6 +772,14 @@ sub quote_rfc2047 {\n \treturn $_;\n }\n \n+sub is_rfc2047_quoted {\n+\tmy $s = shift;\n+\tmy $token = '[^][()<>@,;:\"\\/?.= \\000-\\037\\177-\\377]+';\n+\tmy $encoded_text = '[!->@-~]+';\n+\tlength($s) <= 75 &&\n+\t$s =~ m/^(?:\"[[:ascii:]]*\"|=\\?$token\\?$token\\?$encoded_text\\?=)$/o;\n+}\n+\n # use the simplest quoting being able to handle the recipient\n sub sanitize_address\n {\n@@ -783,7 +791,7 @@ sub sanitize_address\n \t}\n \n \t# if recipient_name is already quoted, do nothing\n-\tif ($recipient_name =~ /^(\"[[:ascii:]]*\"|=\\?utf-8\\?q\\?.*\\?=)$/) {\n+\tif (is_rfc2047_quoted($recipient_name)) {\n \t\treturn $recipient;\n \t}\n \n-- \n1.6.3.1.9.g95405b\n"},{"id":"115756","messageId":"ee63ef30906071731j33c45b6dr3afcec2128023846@mail.gmail.com","threadId":"19718","inReplyTo":"1244420758-5604-1-git-send-email-drafnel@gmail.com","subject":"Re: [PATCH v2] git-send-email.perl: improve detection of MIME encoded-words","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2009-06-08T00:31:20Z","receivedAt":"2009-06-08T00:31:20Z","isPatch":true,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"On Sun, Jun 7, 2009 at 7:25 PM, Brandon Casey<drafnel@gmail.com> wrote:\n\n> Here's a replacement patch which increases the range of excluded characters\n> allowed as tokens so only ASCII characters are allowed (minus the other\n> exclusions).\n\nHere's the diff from the original patch:\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex e735815..8a1a40d 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -774,7 +774,7 @@ sub quote_rfc2047 {\n\n sub is_rfc2047_quoted {\n        my $s = shift;\n-       my $token = '[^][()<>@,;:\"\\/?.= \\000-\\037\\177]+';\n+       my $token = '[^][()<>@,;:\"\\/?.= \\000-\\037\\177-\\377]+';\n        my $encoded_text = '[!->@-~]+';\n        length($s) <= 75 &&\n        $s =~ m/^(?:\"[[:ascii:]]*\"|=\\?$token\\?$token\\?$encoded_text\\?=)$/o;\n\n-brandon\n"}]}