{"thread":{"id":"45830","subject":"[PATCH] send-email: new option to walkaround email server limits","startedAt":"2017-04-29T12:28:38Z","lastAt":"2017-05-07T12:12:15Z","messageCount":9,"participants":["xiaoqiang zhao","Junio C Hamano","赵小强","Paolo Bonzini","Ævar Arnfjörð Bjarmason","Ramkumar Ramachandra"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"318234","messageId":"20170429122632.7122-1-zxq_yx_007@163.com","threadId":"45830","inReplyTo":null,"subject":"[PATCH] send-email: new option to walkaround email server limits","fromName":"xiaoqiang zhao","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-04-29T12:26:32Z","receivedAt":"2017-04-29T12:28:38Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"Some email server(e.g. smtp.163.com) limits a fixed number emails to be send per\nsession(connection) and this will lead to a send faliure.\nWith --split option, a auto reconnection will occur when number of sended\nemail reaches and the problem is solved.\n\nSigned-off-by: xiaoqiang zhao \n---\n git-send-email.perl | 22 +++++++++++++++++++++-\n 1 file changed, 21 insertions(+), 1 deletion(-)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex eea0a517f..0de9b7058 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -81,6 +81,8 @@ git send-email --dump-aliases\n This setting forces to use one of the listed mechanisms.\n --smtp-debug <0|1> * Disable, enable Net::SMTP debug.\n \n+ --split * send \\$num message per connection.\n+\n Automating:\n --identity * Use the sendemail. options.\n --to-cmd * Email To: via ` \\$patch_path`\n@@ -153,6 +155,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };\n my $have_mail_address = eval { require Mail::Address; 1 };\n my $smtp;\n my $auth;\n+my $send_count = 0;\n \n # Regexes for RFC 2047 productions.\n my $re_token = qr/[^][()<>@,;:\\\\\"\\/?.= \\000-\\037\\177-\\377]+/;\n@@ -186,6 +189,7 @@ my $format_patch;\n my $compose_filename;\n my $force = 0;\n my $dump_aliases = 0;\n+my $split = 0;\n \n # Handle interactive edition of files.\n my $multiedit;\n@@ -358,6 +362,7 @@ $rc = GetOptions(\n \t\t \"force\" => \\$force,\n \t\t \"xmailer!\" => \\$use_xmailer,\n \t\t \"no-xmailer\" => sub {$use_xmailer = 0},\n+\t\t \"split=i\" => \\$split,\n \t );\n \n usage() if $help;\n@@ -1158,10 +1163,15 @@ sub smtp_host_string {\n # (smtp_user was not specified), and 0 otherwise.\n \n sub smtp_auth_maybe {\n-\tif (!defined $smtp_authuser || $auth) {\n+\tif (!defined $smtp_authuser || $send_count != 0) {\n \t\treturn 1;\n \t}\n \n+\tif ($auth && $send_count == 0) {\n+\t\tprint \"Auth use saved password. \\n\";\n+\t\treturn !!$smtp->auth($smtp_authuser, $smtp_authpass);\n+\t}\n+\n \t# Workaround AUTH PLAIN/LOGIN interaction defect\n \t# with Authen::SASL::Cyrus\n \teval {\n@@ -1187,6 +1197,7 @@ sub smtp_auth_maybe {\n \t\t'password' => $smtp_authpass\n \t}, sub {\n \t\tmy $cred = shift;\n+\t\t$smtp_authpass = $cred->{'password'};\n \n \t\tif ($smtp_auth) {\n \t\t\tmy $sasl = Authen::SASL->new(\n@@ -1442,6 +1453,15 @@ EOF\n \t\t}\n \t}\n \n+\t$send_count++;\n+\tif ($send_count == $split) {\n+\t\t$smtp->quit;\n+\t\t$smtp = undef;\n+\t\t$send_count = 0;\n+\t\tprint \"Reconnect SMTP server required. \\n\";\n+\n+\t}\n+\n \treturn 1;\n }\n \n-- \n2.11.0\n\n\n"},{"id":"318284","messageId":"xmqqo9vdiboz.fsf@gitster.mtv.corp.google.com","threadId":"45830","inReplyTo":"20170429122632.7122-1-zxq_yx_007@163.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-01T01:54:20Z","receivedAt":"2017-05-01T01:54:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"xiaoqiang zhao writes:\n\n> Some email server(e.g. smtp.163.com) limits a fixed number emails to be send per\n> session(connection) and this will lead to a send faliure.\n\nThat sounds painful.\n\n> With --split option, a auto reconnection will occur when number of sended\n> email reaches and the problem is solved.\n\nTwo nits (other than s/sended/sent/ and s/$send_count/$num_sent/):\n\n - \"--split\" sounds as if you are splitting a single large message\n into multiple pieces, which of course is not what is going on\n here. We need to find a better name for the option. Perhaps\n \"--batch-size=\", \"--max-messages-per-connection=\" or\n something?\n\n - The code seems to do the \"logging out and the logging back in\"\n dance without any delay in between. Is it something we want to\n consider to add a small (possibly configurable) delay in between?\n\nThanks.\n"},{"id":"318353","messageId":"743D62FA-65B9-4DCC-978E-DAF442AE0B0D@163.com","threadId":"45830","inReplyTo":"xmqqo9vdiboz.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"赵小强","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-05-01T06:03:45Z","receivedAt":"2017-05-01T06:04:41Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"\nThanks for your reply , Junio !\n\n> 在 2017年5月1日，09:54，Junio C Hamano 写道：\n> \n> here. We need to find a better name for the option. Perhaps\n> \"--batch-size=\", \"--max-messages-per-connection=\" or\n> something?\n> \n\n--batch-size is ok with me\n\n> - The code seems to do the \"logging out and the logging back in\"\n> dance without any delay in between. Is it something we want to\n> consider to add a small (possibly configurable) delay in between?\n\nIt' s a good idea to have a configurable delay.\n\n\n\n"},{"id":"318478","messageId":"6bef306b-0bfb-2314-50fd-9c152888b344@redhat.com","threadId":"45830","inReplyTo":"20170429122632.7122-1-zxq_yx_007@163.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Paolo Bonzini","fromEmail":"pbonzini@redhat.com","sentAt":"2017-05-02T09:32:14Z","receivedAt":"2017-05-02T09:32:26Z","isPatch":true,"sender":{"key":"pbonzini@redhat.com","avatar":"https://avatars.githubusercontent.com/u/42082?v=4"},"body":"\n\nOn 29/04/2017 14:26, xiaoqiang zhao wrote:\n> Some email server(e.g. smtp.163.com) limits a fixed number emails to be send per\n> session(connection) and this will lead to a send faliure.\n> With --split option, a auto reconnection will occur when number of sended\n> email reaches and the problem is solved.\n> \n> Signed-off-by: xiaoqiang zhao \n\nI think you should also add a matching configuration option, or you are\ngoing to forget it on the command line sooner or later!\n\nPaolo\n\n> ---\n> git-send-email.perl | 22 +++++++++++++++++++++-\n> 1 file changed, 21 insertions(+), 1 deletion(-)\n> \n> diff --git a/git-send-email.perl b/git-send-email.perl\n> index eea0a517f..0de9b7058 100755\n> --- a/git-send-email.perl\n> +++ b/git-send-email.perl\n> @@ -81,6 +81,8 @@ git send-email --dump-aliases\n> This setting forces to use one of the listed mechanisms.\n> --smtp-debug <0|1> * Disable, enable Net::SMTP debug.\n> \n> + --split * send \\$num message per connection.\n> +\n> Automating:\n> --identity * Use the sendemail. options.\n> --to-cmd * Email To: via ` \\$patch_path`\n> @@ -153,6 +155,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };\n> my $have_mail_address = eval { require Mail::Address; 1 };\n> my $smtp;\n> my $auth;\n> +my $send_count = 0;\n> \n> # Regexes for RFC 2047 productions.\n> my $re_token = qr/[^][()<>@,;:\\\\\"\\/?.= \\000-\\037\\177-\\377]+/;\n> @@ -186,6 +189,7 @@ my $format_patch;\n> my $compose_filename;\n> my $force = 0;\n> my $dump_aliases = 0;\n> +my $split = 0;\n> \n> # Handle interactive edition of files.\n> my $multiedit;\n> @@ -358,6 +362,7 @@ $rc = GetOptions(\n> \t\t \"force\" => \\$force,\n> \t\t \"xmailer!\" => \\$use_xmailer,\n> \t\t \"no-xmailer\" => sub {$use_xmailer = 0},\n> +\t\t \"split=i\" => \\$split,\n> \t );\n> \n> usage() if $help;\n> @@ -1158,10 +1163,15 @@ sub smtp_host_string {\n> # (smtp_user was not specified), and 0 otherwise.\n> \n> sub smtp_auth_maybe {\n> -\tif (!defined $smtp_authuser || $auth) {\n> +\tif (!defined $smtp_authuser || $send_count != 0) {\n> \t\treturn 1;\n> \t}\n> \n> +\tif ($auth && $send_count == 0) {\n> +\t\tprint \"Auth use saved password. \\n\";\n> +\t\treturn !!$smtp->auth($smtp_authuser, $smtp_authpass);\n> +\t}\n> +\n> \t# Workaround AUTH PLAIN/LOGIN interaction defect\n> \t# with Authen::SASL::Cyrus\n> \teval {\n> @@ -1187,6 +1197,7 @@ sub smtp_auth_maybe {\n> \t\t'password' => $smtp_authpass\n> \t}, sub {\n> \t\tmy $cred = shift;\n> +\t\t$smtp_authpass = $cred->{'password'};\n> \n> \t\tif ($smtp_auth) {\n> \t\t\tmy $sasl = Authen::SASL->new(\n> @@ -1442,6 +1453,15 @@ EOF\n> \t\t}\n> \t}\n> \n> +\t$send_count++;\n> +\tif ($send_count == $split) {\n> +\t\t$smtp->quit;\n> +\t\t$smtp = undef;\n> +\t\t$send_count = 0;\n> +\t\tprint \"Reconnect SMTP server required. \\n\";\n> +\n> +\t}\n> +\n> \treturn 1;\n> }\n> \n> \n"},{"id":"318481","messageId":"CACBZZX6fursxEaBGr9raz+x8qBm0WZrmHCkMbtpajRzOM3B6qA@mail.gmail.com","threadId":"45830","inReplyTo":"743D62FA-65B9-4DCC-978E-DAF442AE0B0D@163.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-02T09:58:52Z","receivedAt":"2017-05-02T09:59:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Mon, May 1, 2017 at 8:03 AM, 赵小强 wrote:\n>\n> Thanks for your reply , Junio !\n>\n>> 在 2017年5月1日，09:54，Junio C Hamano 写道：\n>>\n>> here. We need to find a better name for the option. Perhaps\n>> \"--batch-size=\", \"--max-messages-per-connection=\" or\n>> something?\n>>\n>\n> --batch-size is ok with me\n>\n>> - The code seems to do the \"logging out and the logging back in\"\n>> dance without any delay in between. Is it something we want to\n>> consider to add a small (possibly configurable) delay in between?\n>\n> It' s a good idea to have a configurable delay.\n\nIt makes sense to have a configurable delay for git-send-email\nunrelated to this option, I'd use a facility like that.\n\nA lot of mail clients just sort based on date/msgid or whatever not\ndate/subject, so the rapid-fire output of send-email often arrives out\nof order because it's all sent at the same second. I'd use some option\nwhere I could send a series as \"all\" and have it sleep(N) in between\nsending mails.\n"},{"id":"318713","messageId":"xmqqk25xct1w.fsf@gitster.mtv.corp.google.com","threadId":"45830","inReplyTo":"CACBZZX6fursxEaBGr9raz+x8qBm0WZrmHCkMbtpajRzOM3B6qA@mail.gmail.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-04T01:24:59Z","receivedAt":"2017-05-04T01:25:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> It makes sense to have a configurable delay for git-send-email\n> unrelated to this option, I'd use a facility like that.\n>\n> A lot of mail clients just sort based on date/msgid or whatever not\n> date/subject, so the rapid-fire output of send-email often arrives out\n> of order because it's all sent at the same second. I'd use some option\n> where I could send a series as \"all\" and have it sleep(N) in between\n> sending mails.\n\nHmph. When sending many messages, send-email first grabs the\ncurrent time, counts backwards N seconds for N message series,\nand uses that timestamp that is N seconds in the past for the first\nmessage, incrementing the timestamp by 1 second per each subsequent\nones.\n\nI found that this trick is sufficient to cause receiving MUAs sort\nmessages based on date, as the Date: field will have the timestamps\nthat increases by 1 second in between messages in a batch. \n\nThere might be MUAs that do not use the value of the Date: field\nwhen told to sort by date (perhaps they use the timestamp of the\nmessage file they received at the final hop to them?), but it is\nhopeless to help such MUAs unless the mail path guarantees the order\nat the originator, which is not how \"store and forward\" e-mails\nwork.\n\nSo...\n"},{"id":"318716","messageId":"CALkWK0kceHMyjuKvTa=CQZsGnXKaUuG6=G-HxEFSHaN+dqe6nw@mail.gmail.com","threadId":"45830","inReplyTo":"xmqqk25xct1w.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Ramkumar Ramachandra","fromEmail":"artagnon@gmail.com","sentAt":"2017-05-04T02:18:50Z","receivedAt":"2017-05-04T02:19:38Z","isPatch":true,"sender":{"key":"r@artagnon.com","avatar":"https://avatars.githubusercontent.com/u/37226?v=4"},"body":"For the list, in plain text:\n\nIIUC, they use the date received to sort. I think this might stem from\na historical cruft: emails sometimes took non-trivial amounts of time\nto transit, back in the old days. MUAs (especially web-based ones)\nprobably did not want to violate user expectation by placing a new\nemail under several already-read emails. I would argue that this was\nreasonable behavior. Further, since email is near-instantaneous today,\nit really makes no difference which way the MUA sorts. Except for git\nsend-email.\n\nIt might be acceptable to put in a \"practical hack\" to help out MUAs\nin the context of \"near instant forwarding\". I would argue against it\non grounds of it being an ugly hack, for very little benefit. The\npatch that began this thread is also ugly, but has the important\nconsequence of enabling some people to use git send-email at all.\n\np.s- We should probably allow emails from mobile devices on the list.\nIt usually contains a HTML subpart.\n"},{"id":"318730","messageId":"CACBZZX4zgNXBxCUP3tWrB6KAvVftxQX65tF8L8YJaqm4VrVqcQ@mail.gmail.com","threadId":"45830","inReplyTo":"xmqqk25xct1w.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-04T07:48:14Z","receivedAt":"2017-05-04T07:48:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, May 4, 2017 at 3:24 AM, Junio C Hamano wrote:\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> It makes sense to have a configurable delay for git-send-email\n>> unrelated to this option, I'd use a facility like that.\n>>\n>> A lot of mail clients just sort based on date/msgid or whatever not\n>> date/subject, so the rapid-fire output of send-email often arrives out\n>> of order because it's all sent at the same second. I'd use some option\n>> where I could send a series as \"all\" and have it sleep(N) in between\n>> sending mails.\n>\n> Hmph. When sending many messages, send-email first grabs the\n> current time, counts backwards N seconds for N message series,\n> and uses that timestamp that is N seconds in the past for the first\n> message, incrementing the timestamp by 1 second per each subsequent\n> ones.\n>\n> I found that this trick is sufficient to cause receiving MUAs sort\n> messages based on date, as the Date: field will have the timestamps\n> that increases by 1 second in between messages in a batch.\n>\n> There might be MUAs that do not use the value of the Date: field\n> when told to sort by date (perhaps they use the timestamp of the\n> message file they received at the final hop to them?), but it is\n> hopeless to help such MUAs unless the mail path guarantees the order\n> at the originator, which is not how \"store and forward\" e-mails\n> work.\n\nAs Ramkumar points out many MUAs don't sort by Date because of mail\ndelays / inaccurate sender clocks. Fun fact: You happen to work for a\ncompany making one such MUA :)\n\nE.g. just to name one example in my GMail view (it's full of this sort\nof thing) Stefan Beller's recent cache.h series starts in the order\n01/03/02 (and continues out of order).\n\nThe Date headers on the messages themselves are incremented by 1\nsecond as you note, but on those first 3x the Recieved chain ends in\nthis for all 3:\n\n Received: by 10.28.48.210 with SMTP id w201csp570755wmw;\n Tue, 2 May 2017 15:24:50 -0700 (PDT)\n\nI.e. all Received on the same second, showing that the Date header is\nignored by GMail, just from observing it GMail's sort function seems\nto be (pseudocode):\n\n a['Received'] <=> b['Reiceved'] || a['GMailInternalID'] <=>\nb['GMailInternalID']\n\nNot:\n\n a['Received'] <=> b['Reiceved'] || a['MessageID'] <=>\nb['MessageID'] || a['GMailInternalID'] <=> b['GMailInternalID']\n\nOr:\n\n a['Received'] <=> b['Reiceved'] || a['Subject'] <=> b['Subject']\n|| a['MessageID'] <=> b['MessageID'] || a['GMailInternalID'] <=>\nb['GMailInternalID']\n\nOr:\n\n a['Received'] <=> b['Reiceved'] || a['Date'] <=> b['Date'] ....\n\nAnyway, you get the idea, but all of these would cause it to show\ngit-send-email list traffic in order, since while the Received field\nis the same both Date & MessageID is different & sortable (actually\nnot quite, but that's another matter).\n\nWhen I send my own patches with git-send-email I tend to sit there\nhitting \"y\" in succession instead of doing \"a\" because I like being\nable to browse my patches in order, which I guess tells you something\nabout my tolerance for tedium before submitting a patch to sleep().\n\nIf you look at GMail's web view you can effectively also see how close\nsomeone is to Google's servers by how out of order their patches are,\ne.g. your patches, Stefan's etc. & other Google employees working in\nSV are pretty much entirely shuffled, whereas Michael Haggerty's are\npretty much in order because his mailpath involves a transatlantic &\ntrans-US-costal route before being delivered.\n"},{"id":"319010","messageId":"7B2190B3-714D-4B22-857C-EBFC5F48A563@163.com","threadId":"45830","inReplyTo":"6bef306b-0bfb-2314-50fd-9c152888b344@redhat.com","subject":"Re: [PATCH] send-email: new option to walkaround email server limits","fromName":"赵小强","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-05-04T07:55:49Z","receivedAt":"2017-05-07T12:12:15Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"\n\n在 2017年5月2日，17:32，Paolo Bonzini 写道：\n\n>> On 29/04/2017 14:26, xiaoqiang zhao wrote:\n>> Some email server(e.g. smtp.163.com) limits a fixed number emails to be send per\n>> session(connection) and this will lead to a send faliure.\n>> With --split option, a auto reconnection will occur when number of sended\n>> email reaches and the problem is solved.\n>> \n>> Signed-off-by: xiaoqiang zhao \n> \n> I think you should also add a matching configuration option, or you are\n> going to forget it on the command line sooner or later!\n> \n> Paolo\n\nGood idea！\n\n"}]}