{"thread":{"id":"31140","subject":"[PATCH] send-email: improve RFC2047 quote parsing","startedAt":"2012-07-30T19:25:40Z","lastAt":"2012-07-31T08:09:05Z","messageCount":3,"participants":["Thomas Rast","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"196186","messageId":"cb7aa2a8fc08df4f1d12add603ffcbc104f67823.1343676034.git.trast@student.ethz.ch","threadId":"31140","inReplyTo":null,"subject":"[PATCH] send-email: improve RFC2047 quote parsing","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2012-07-30T19:25:40Z","receivedAt":"2012-07-30T19:25:40Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"The RFC2047 unquoting, used to parse email addresses in From and Cc\nheaders, is broken in several ways:\n\n* It erroneously substitutes ' ' for '_' in *the whole* header, even\n outside the quoted field. [Noticed by Christoph.]\n\n* It is too liberal in its matching, and happily matches the start of\n one quoted chunk against the end of another, or even just something\n that looks like such an end. [Noticed by Junio.]\n\n* It fundamentally cannot cope with encodings that are not a superset\n of ASCII, nor several (incompatible) encodings in the same header.\n\nThis patch fixes the first two by doing a more careful decoding of the\n=AB outer quoting. Fixing the fundamental issues is left for a\nfuture, more intrusive, patch.\n\nNoticed-by: Christoph Miebach \nHelped-by: Junio C Hamano \nSigned-off-by: Thomas Rast \n---\n\nThis is the easy part, fixed as per Junio's comment that it needs to\nuse a .*? match for the contents, and with a test.\n\n git-send-email.perl | 10 ++++++----\n t/t9001-send-email.sh | 13 +++++++++++++\n 2 files changed, 19 insertions(+), 4 deletions(-)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex ef30c55..6647137 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -862,11 +862,13 @@ sub make_message_id {\n sub unquote_rfc2047 {\n \tlocal ($_) = @_;\n \tmy $encoding;\n-\tif (s/=\\?([^?]+)\\?q\\?(.*)\\?=/$2/g) {\n+\ts{=\\?([^?]+)\\?q\\?(.*?)\\?=}{\n \t\t$encoding = $1;\n-\t\ts/_/ /g;\n-\t\ts/=([0-9A-F]{2})/chr(hex($1))/eg;\n-\t}\n+\t\tmy $e = $2;\n+\t\t$e =~ s/_/ /g;\n+\t\t$e =~ s/=([0-9A-F]{2})/chr(hex($1))/eg;\n+\t\t$e;\n+\t}eg;\n \treturn wantarray ? ($_, $encoding) : $_;\n }\n \ndiff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh\nindex 8c12c65..0351228 100755\n--- a/t/t9001-send-email.sh\n+++ b/t/t9001-send-email.sh\n@@ -841,6 +841,19 @@ test_expect_success $PREREQ '--compose adds MIME for utf8 subject' '\n \tgrep \"^Subject: =?UTF-8?q?utf8-s=C3=BCbj=C3=ABct?=\" msgtxt1\n '\n \n+test_expect_success $PREREQ 'utf8 author is correctly passed on' '\n+\tclean_fake_sendmail &&\n+\ttest_commit weird_author &&\n+\ttest_when_finished \"git reset --hard HEAD^\" &&\n+\tgit commit --amend --author \"Füñný Nâmé \" &&\n+\tgit format-patch --stdout -1 >funny_name.patch &&\n+\tgit send-email --from=\"Example \" \\\n+\t --to=nobody@example.com \\\n+\t --smtp-server=\"$(pwd)/fake.sendmail\" \\\n+\t funny_name.patch &&\n+\tgrep \"^From: Füñný Nâmé \" msgtxt1\n+'\n+\n test_expect_success $PREREQ 'detects ambiguous reference/file conflict' '\n \techo master > master &&\n \tgit add master &&\n-- \n1.7.12.rc0.434.gd809d0f\n"},{"id":"196195","messageId":"7vzk6hot8a.fsf@alter.siamese.dyndns.org","threadId":"31140","inReplyTo":"cb7aa2a8fc08df4f1d12add603ffcbc104f67823.1343676034.git.trast@student.ethz.ch","subject":"Re: [PATCH] send-email: improve RFC2047 quote parsing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-07-30T21:05:09Z","receivedAt":"2012-07-30T21:05:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast writes:\n\n> The RFC2047 unquoting, used to parse email addresses in From and Cc\n> headers, is broken in several ways:\n>\n> * It erroneously substitutes ' ' for '_' in *the whole* header, even\n> outside the quoted field. [Noticed by Christoph.]\n>\n> * It is too liberal in its matching, and happily matches the start of\n> one quoted chunk against the end of another, or even just something\n> that looks like such an end. [Noticed by Junio.]\n>\n> * It fundamentally cannot cope with encodings that are not a superset\n> of ASCII, nor several (incompatible) encodings in the same header.\n>\n> This patch fixes the first two by doing a more careful decoding of the\n> =AB outer quoting. Fixing the fundamental issues is left for a\n> future, more intrusive, patch.\n\nWhat is this =AB thing?\n\n>\n> Noticed-by: Christoph Miebach \n> Helped-by: Junio C Hamano \n> Signed-off-by: Thomas Rast \n> ---\n>\n> This is the easy part, fixed as per Junio's comment that it needs to\n> use a .*? match for the contents, and with a test.\n\nWhat's the hard part? Do you mean the \"fundamentally cannot\" part?\n\nThanks.\n\n> git-send-email.perl | 10 ++++++----\n> t/t9001-send-email.sh | 13 +++++++++++++\n> 2 files changed, 19 insertions(+), 4 deletions(-)\n>\n> diff --git a/git-send-email.perl b/git-send-email.perl\n> index ef30c55..6647137 100755\n> --- a/git-send-email.perl\n> +++ b/git-send-email.perl\n> @@ -862,11 +862,13 @@ sub make_message_id {\n> sub unquote_rfc2047 {\n> \tlocal ($_) = @_;\n> \tmy $encoding;\n> -\tif (s/=\\?([^?]+)\\?q\\?(.*)\\?=/$2/g) {\n> +\ts{=\\?([^?]+)\\?q\\?(.*?)\\?=}{\n> \t\t$encoding = $1;\n> -\t\ts/_/ /g;\n> -\t\ts/=([0-9A-F]{2})/chr(hex($1))/eg;\n> -\t}\n> +\t\tmy $e = $2;\n> +\t\t$e =~ s/_/ /g;\n> +\t\t$e =~ s/=([0-9A-F]{2})/chr(hex($1))/eg;\n> +\t\t$e;\n> +\t}eg;\n> \treturn wantarray ? ($_, $encoding) : $_;\n> }\n> \n> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh\n> index 8c12c65..0351228 100755\n> --- a/t/t9001-send-email.sh\n> +++ b/t/t9001-send-email.sh\n> @@ -841,6 +841,19 @@ test_expect_success $PREREQ '--compose adds MIME for utf8 subject' '\n> \tgrep \"^Subject: =?UTF-8?q?utf8-s=C3=BCbj=C3=ABct?=\" msgtxt1\n> '\n> \n> +test_expect_success $PREREQ 'utf8 author is correctly passed on' '\n> +\tclean_fake_sendmail &&\n> +\ttest_commit weird_author &&\n> +\ttest_when_finished \"git reset --hard HEAD^\" &&\n> +\tgit commit --amend --author \"Füñný Nâmé \" &&\n> +\tgit format-patch --stdout -1 >funny_name.patch &&\n> +\tgit send-email --from=\"Example \" \\\n> +\t --to=nobody@example.com \\\n> +\t --smtp-server=\"$(pwd)/fake.sendmail\" \\\n> +\t funny_name.patch &&\n> +\tgrep \"^From: Füñný Nâmé \" msgtxt1\n> +'\n> +\n> test_expect_success $PREREQ 'detects ambiguous reference/file conflict' '\n> \techo master > master &&\n> \tgit add master &&\n"},{"id":"196213","messageId":"87lii0gxni.fsf@thomas.inf.ethz.ch","threadId":"31140","inReplyTo":"7vzk6hot8a.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] send-email: improve RFC2047 quote parsing","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2012-07-31T08:09:05Z","receivedAt":"2012-07-31T08:09:05Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Junio C Hamano writes:\n\n> Thomas Rast writes:\n>\n>> This patch fixes the first two by doing a more careful decoding of the\n>> =AB outer quoting. Fixing the fundamental issues is left for a\n>> future, more intrusive, patch.\n>\n> What is this =AB thing?\n\nThe two-hex-digits quoting in the style of MIME quoted-printable. I\ncalled it the \"outer quoting\" (RFC2047: \"encoding\") because it serves to\nprotect the bytes from transport damage; there is another encoding\n(RFC2047: \"character set\") \"inside\" which is specified by the\n=?utf-8?...?= wrapper.\n\nBTW, note that we also only handle the Q outer quoting\n(quoted-printable). There is a B encoding, which your email in fact\nused in the Cc: header:\n\nCc: [...] =?utf-8?B?SsO8cmdlbiBSw7xobGU=?= \n\nB means Base64, as you can probably guess from the looks of it.\n\n>> This is the easy part, fixed as per Junio's comment that it needs to\n>> use a .*? match for the contents, and with a test.\n>\n> What's the hard part? Do you mean the \"fundamentally cannot\" part?\n\nYes, and by \"fundamentally\" I meant \"not without fixing something\noutside of this function, which I am too lazy to do at this time\". ;-)\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"}]}