{"thread":{"id":"46016","subject":"[PATCH v5] send-email: --batch-size to work around some SMTP server limit","startedAt":"2017-05-21T13:00:34Z","lastAt":"2018-02-12T19:44:16Z","messageCount":15,"participants":["xiaoqiang zhao","Junio C Hamano","Zhaoxiangqiang","Ævar Arnfjörð Bjarmason","Jan Viktorin","赵小强","Stefan Beller","Eric Sunshine"],"isPatch":true,"patchVersion":5,"patchTotal":null},"messages":[{"id":"320375","messageId":"20170521125950.5524-1-zxq_yx_007@163.com","threadId":"46016","inReplyTo":null,"subject":"[PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"xiaoqiang zhao","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-05-21T12:59:50Z","receivedAt":"2017-05-21T13:00:34Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"Some email servers (e.g. smtp.163.com) limit the number emails to be\nsent per session(connection) and this will lead to a faliure when\nsending many messages.\n\nTeach send-email to disconnect after sending a number of messages\n(configurable via the --batch-size= option), wait for a few\nseconds (configurable via the --relogin-delay= option) and\nreconnect, to work around such a limit.\n\nAlso add this two configuration option.\n\nNote:\n Re-authentication will happen every $ messages, so it\nwill be much more acceptable if you use some form of credential helper\n(e.g. the 'sendemail.smtppass' config option), otherwise you will have\nto retype password every time when asked.\n\nSigned-off-by: xiaoqiang zhao \n---\n Documentation/config.txt | 8 ++++++++\n Documentation/git-send-email.txt | 11 +++++++++++\n contrib/completion/git-completion.bash | 2 ++\n git-send-email.perl | 18 ++++++++++++++++++\n 4 files changed, 39 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 96e9cf8b7..173ed63f6 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2917,6 +2917,14 @@ sendemail.xmailer::\n sendemail.signedoffcc (deprecated)::\n \tDeprecated alias for `sendemail.signedoffbycc`.\n \n+sendemail.smtpbatchsize::\n+\tNumber of messages to be sent per connection, after that a relogin\n+\twill happen. if the value is 0 or undefined, send all messages in\n+\tone connection.\n+\n+sendemail.smtprelogindelay::\n+\tSeconds wait before reconnecting to smtp server.\n+\n showbranch.default::\n \tThe default set of branches for linkgit:git-show-branch[1].\n \tSee linkgit:git-show-branch[1].\ndiff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt\nindex 9d66166f6..5380d8c95 100644\n--- a/Documentation/git-send-email.txt\n+++ b/Documentation/git-send-email.txt\n@@ -248,6 +248,17 @@ must be used for each option.\n \tcommands and replies will be printed. Useful to debug TLS\n \tconnection and authentication problems.\n \n+--batch-size=::\n+\tSome email servers (e.g. smtp.163.com) limit the number emails to be\n+\tsent per session(connection) and this will lead to a faliure when\n+\tsending many messages. With this option, send-email will disconnect after\n+\tsending $ messages and wait for a few seconds (see --relogin-delay)\n+\tand reconnect, to work around such a limit.\n+\n+--relogin-delay=::\n+\tWaiting $ seconds before reconnecting to smtp server. Used together\n+\twith --batch-size option.\n+\n Automating\n ~~~~~~~~~~\n \ndiff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash\nindex 1ed0a09fe..933e7badf 100644\n--- a/contrib/completion/git-completion.bash\n+++ b/contrib/completion/git-completion.bash\n@@ -2611,6 +2611,8 @@ _git_config ()\n \t\tsendemail.thread\n \t\tsendemail.to\n \t\tsendemail.validate\n+\t\tsendemail.smtpbatchsize\n+\t\tsendemail.smtprelogindelay\n \t\tshowbranch.default\n \t\tstatus.relativePaths\n \t\tstatus.showUntrackedFiles\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex eea0a517f..8a1ee0f0d 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -81,6 +81,10 @@ git send-email --dump-aliases\n This setting forces to use one of the listed mechanisms.\n --smtp-debug <0|1> * Disable, enable Net::SMTP debug.\n \n+ --batch-size * send max message per connection.\n+ --relogin-delay * delay seconds between two successive login.\n+ This option can only be used with --batch-size\n+\n Automating:\n --identity * Use the sendemail. options.\n --to-cmd * Email To: via ` \\$patch_path`\n@@ -153,6 +157,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };\n my $have_mail_address = eval { require Mail::Address; 1 };\n my $smtp;\n my $auth;\n+my $num_sent = 0;\n \n # Regexes for RFC 2047 productions.\n my $re_token = qr/[^][()<>@,;:\\\\\"\\/?.= \\000-\\037\\177-\\377]+/;\n@@ -216,6 +221,7 @@ my ($cover_cc, $cover_to);\n my ($to_cmd, $cc_cmd);\n my ($smtp_server, $smtp_server_port, @smtp_server_options);\n my ($smtp_authuser, $smtp_encryption, $smtp_ssl_cert_path);\n+my ($batch_size, $relogin_delay);\n my ($identity, $aliasfiletype, @alias_files, $smtp_domain, $smtp_auth);\n my ($validate, $confirm);\n my (@suppress_cc);\n@@ -247,6 +253,8 @@ my %config_settings = (\n \"smtppass\" => \\$smtp_authpass,\n \"smtpdomain\" => \\$smtp_domain,\n \"smtpauth\" => \\$smtp_auth,\n+ \"smtpbatchsize\" => \\$batch_size,\n+ \"smtprelogindelay\" => \\$relogin_delay,\n \"to\" => \\@initial_to,\n \"tocmd\" => \\$to_cmd,\n \"cc\" => \\@initial_cc,\n@@ -358,6 +366,8 @@ $rc = GetOptions(\n \t\t \"force\" => \\$force,\n \t\t \"xmailer!\" => \\$use_xmailer,\n \t\t \"no-xmailer\" => sub {$use_xmailer = 0},\n+\t\t \"batch-size=i\" => \\$batch_size,\n+\t\t \"relogin-delay=i\" => \\$relogin_delay,\n \t );\n \n usage() if $help;\n@@ -1664,6 +1674,14 @@ foreach my $t (@files) {\n \t\t}\n \t}\n \t$message_id = undef;\n+\t$num_sent++;\n+\tif (defined $batch_size && $num_sent == $batch_size) {\n+\t\t$num_sent = 0;\n+\t\t$smtp->quit if defined $smtp;\n+\t\tundef $smtp;\n+\t\tundef $auth;\n+\t\tsleep($relogin_delay) if defined $relogin_delay;\n+\t}\n }\n \n # Execute a command (e.g. $to_cmd) to get a list of email addresses\n-- \n2.13.0.rc2.116.g565bdd0fc.dirty\n\n\n"},{"id":"320388","messageId":"xmqqlgpppri7.fsf@gitster.mtv.corp.google.com","threadId":"46016","inReplyTo":"20170521125950.5524-1-zxq_yx_007@163.com","subject":"Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-22T02:14:24Z","receivedAt":"2017-05-22T02:14:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I think this is almost perfect.\n\nI'd propose squashing the patch below to\n\n - Add cross reference between config and option\n\n - Spell configuration variables in camelCase to mimic other\n sendemail.* variables\n\n - Spell SMTP in all caps to mimic other parts of the manual\n\n - Suggest use of credential helper in the document to help the end\n users, not in the proposed log message.\n\nIf you are fine with all of these changes, there is no need to\nresend (you can say so and I can locally squash these in). Of\ncourse, anyone is very welcome to point out documentation bugs I may\nbe introducing with this patch.\n\nThanks.\n\n-- >8 --\nSubject: fixup! send-email: --batch-size to work around some SMTP server limit\n\nSome email servers (e.g. smtp.163.com) limit the number emails to be\nsent per session (connection) and this will lead to a faliure when\nsending many messages.\n\nTeach send-email to disconnect after sending a number of messages\n(configurable via the --batch-size= option), wait for a few\nseconds (configurable via the --relogin-delay= option) and\nreconnect, to work around such a limit.\n\nAlso add two configuration variables to give these options the default.\n\nSigned-off-by: Junio C Hamano \n---\n Documentation/config.txt | 8 +++++---\n Documentation/git-send-email.txt | 12 ++++++++----\n 2 files changed, 13 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex a13315ed69..ee4a111878 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2914,13 +2914,15 @@ sendemail.xmailer::\n sendemail.signedoffcc (deprecated)::\n \tDeprecated alias for `sendemail.signedoffbycc`.\n \n-sendemail.smtpbatchsize::\n+sendemail.smtpBatchSize::\n \tNumber of messages to be sent per connection, after that a relogin\n-\twill happen. if the value is 0 or undefined, send all messages in\n+\twill happen. If the value is 0 or undefined, send all messages in\n \tone connection.\n+\tSee also the `--batch-size` option of linkgit:git-send-email[1].\n \n-sendemail.smtprelogindelay::\n+sendemail.smtpReloginDelay::\n \tSeconds wait before reconnecting to smtp server.\n+\tSee also the `--relogin-delay` option of linkgit:git-send-email[1].\n \n showbranch.default::\n \tThe default set of branches for linkgit:git-show-branch[1].\ndiff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt\nindex 5380d8c956..79b418bfa5 100644\n--- a/Documentation/git-send-email.txt\n+++ b/Documentation/git-send-email.txt\n@@ -250,14 +250,18 @@ must be used for each option.\n \n --batch-size=::\n \tSome email servers (e.g. smtp.163.com) limit the number emails to be\n-\tsent per session(connection) and this will lead to a faliure when\n+\tsent per session (connection) and this will lead to a faliure when\n \tsending many messages. With this option, send-email will disconnect after\n \tsending $ messages and wait for a few seconds (see --relogin-delay)\n-\tand reconnect, to work around such a limit.\n+\tand reconnect, to work around such a limit. You may want to\n+\tuse some form of credential helper to avoid having to retype\n+\tyour password every time this happens. Defaults to the\n+\t`sendemail.smtpBatchSize` configuration variable.\n \n --relogin-delay=::\n-\tWaiting $ seconds before reconnecting to smtp server. Used together\n-\twith --batch-size option.\n+\tWaiting $ seconds before reconnecting to SMTP server. Used together\n+\twith --batch-size option. Defaults to the `sendemail.smtpReloginDelay`\n+\tconfiguration variable.\n \n Automating\n ~~~~~~~~~~\n-- \n2.13.0-440-g3ce6d2d5b8\n\n"},{"id":"320393","messageId":"cfm22a.oqc3vr.1hgearw-qmf@mail","threadId":"46016","inReplyTo":"xmqqlgpppri7.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"Zhaoxiangqiang","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-05-22T03:19:02Z","receivedAt":"2017-05-22T03:19:38Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"\n\nJunio C Hamano 于 2017 年 5 月 22 日 星期一 写道:\n> I think this is almost perfect.\n> \n> I'd propose squashing the patch below to\n> \n> - Add cross reference between config and option\n> \n> - Spell configuration variables in camelCase to mimic other\n> sendemail.* variables\n> \n> - Spell SMTP in all caps to mimic other parts of the manual\n> \n> - Suggest use of credential helper in the document to help the end\n> users, not in the proposed log message.\n> \n> If you are fine with all of these changes, there is no need to\n> resend (you can say so and I can locally squash these in). Of\n> course, anyone is very welcome to point out documentation bugs I may\n> be introducing with this patch.\n> \n> Thanks.\n> \n> -- >8 --\n> Subject: fixup! send-email: --batch-size to work around some SMTP server limit\n> \n> Some email servers (e.g. smtp.163.com) limit the number emails to be\n> sent per session (connection) and this will lead to a faliure when\n> sending many messages.\n> \n> Teach send-email to disconnect after sending a number of messages\n> (configurable via the --batch-size= option), wait for a few\n> seconds (configurable via the --relogin-delay= option) and\n> reconnect, to work around such a limit.\n> \n> Also add two configuration variables to give these options the default.\n> \n> Signed-off-by: Junio C Hamano \n> ---\n> Documentation/config.txt | 8 +++++---\n> Documentation/git-send-email.txt | 12 ++++++++----\n> 2 files changed, 13 insertions(+), 7 deletions(-)\n>\n\nIt's fine with me, just go ahead.\nThank you Hamano!\n \n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index a13315ed69..ee4a111878 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -2914,13 +2914,15 @@ sendemail.xmailer::\n> sendemail.signedoffcc (deprecated)::\n> \tDeprecated alias for `sendemail.signedoffbycc`.\n> \n> -sendemail.smtpbatchsize::\n> +sendemail.smtpBatchSize::\n> \tNumber of messages to be sent per connection, after that a relogin\n> -\twill happen. if the value is 0 or undefined, send all messages in\n> +\twill happen. If the value is 0 or undefined, send all messages in\n> \tone connection.\n> +\tSee also the `--batch-size` option of linkgit:git-send-email[1].\n> \n> -sendemail.smtprelogindelay::\n> +sendemail.smtpReloginDelay::\n> \tSeconds wait before reconnecting to smtp server.\n> +\tSee also the `--relogin-delay` option of linkgit:git-send-email[1].\n> \n> showbranch.default::\n> \tThe default set of branches for linkgit:git-show-branch[1].\n> diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt\n> index 5380d8c956..79b418bfa5 100644\n> --- a/Documentation/git-send-email.txt\n> +++ b/Documentation/git-send-email.txt\n> @@ -250,14 +250,18 @@ must be used for each option.\n> \n> --batch-size=::\n> \tSome email servers (e.g. smtp.163.com) limit the number emails to be\n> -\tsent per session(connection) and this will lead to a faliure when\n> +\tsent per session (connection) and this will lead to a faliure when\n> \tsending many messages. With this option, send-email will disconnect after\n> \tsending $ messages and wait for a few seconds (see --relogin-delay)\n> -\tand reconnect, to work around such a limit.\n> +\tand reconnect, to work around such a limit. You may want to\n> +\tuse some form of credential helper to avoid having to retype\n> +\tyour password every time this happens. Defaults to the\n> +\t`sendemail.smtpBatchSize` configuration variable.\n> \n> --relogin-delay=::\n> -\tWaiting $ seconds before reconnecting to smtp server. Used together\n> -\twith --batch-size option.\n> +\tWaiting $ seconds before reconnecting to SMTP server. Used together\n> +\twith --batch-size option. Defaults to the `sendemail.smtpReloginDelay`\n> +\tconfiguration variable.\n> \n> Automating\n> ~~~~~~~~~~\n> -- \n> 2.13.0-440-g3ce6d2d5b8\n> \n>"},{"id":"320400","messageId":"CACBZZX5GYV50rjg9X602JHqFPaoofH9TwDf_-r_MDu8-rmNV6Q@mail.gmail.com","threadId":"46016","inReplyTo":"20170521125950.5524-1-zxq_yx_007@163.com","subject":"Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2017-05-22T09:26:41Z","receivedAt":"2017-05-22T09:27:11Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Sun, May 21, 2017 at 2:59 PM, xiaoqiang zhao wrote:\n> Some email servers (e.g. smtp.163.com) limit the number emails to be\n> sent per session(connection) and this will lead to a faliure when\n> sending many messages.\n\nThis OK to me, the nits I had are addressed by Junio's reply.\n\nLooking at this the Nth time now though I wonder about this approach\nin general. In all your E-Mails I don't think you ever said /what/\nsort of error you had from the SMTP server, you just said you had a\nfailure or an error, I assume you hit one of the die's in the\nsend_message() function. Can you paste the actual error you get\nwithout this patch?\n\nI wonder if something like this would Just Work for this case without\nany configuration or command-line options, with the added benefit of\njust working for anyone with transitory SMTP issues as well (patch\nposted with -w, full version at\nhttps://github.com/avar/git/commit/acb60c4bde50bdcb62b71ed46f49617e2caef84e.patch):\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 8a1ee0f0d4..c2d85236d1 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -1363,6 +1363,10 @@ EOF\n die __(\"The required SMTP server is not\nproperly defined.\")\n }\n\n+ my $num_tries = 0;\n+ my $max_tries = 5;\n+ smtp_again:\n+ eval {\n if ($smtp_encryption eq 'ssl') {\n $smtp_server_port ||= 465; # ssmtp\n require Net::SMTP::SSL;\n@@ -1429,6 +1433,22 @@ EOF\n }\n $smtp->dataend() or die $smtp->message;\n $smtp->code =~ /250|200/ or die\nsprintf(__(\"Failed to send %s\\n\"), $subject).$smtp->message;\n+ 1;\n+ } or do {\n+ my $error = $@ || \"Zombie Error\";\n+\n+ warn sprintf(__(\"Failed to send %s due to\nerror: %s\"), $subject, $error);\n+ if ($num_tries++ < $max_tries) {\n+ $smtp->quit if defined $smtp;\n+ $smtp = undef;\n+ $auth = undef;\n+ my $sleep = $num_tries * 3; # 3, 6, 9, ...\n+ warn sprintf(__(\"This is retry %d/%d.\nSleeping %d before trying again\"),\n+ $num_tries, $max_tries, $sleep);\n+ sleep($sleep);\n+ goto smtp_again;\n+ }\n+ };\n }\n if ($quiet) {\n printf($dry_run ? __(\"Dry-Sent %s\\n\") : __(\"Sent\n%s\\n\"), $subject);\n\nNow that's very much a WIP and I don't have a server like that to test against.\n\nHaving worked with SMTP a lot in a past life/job, I'd say it's *very*\nlikely that you're just getting a /^4/ error code from 163.com,\nprobably 421, which would make this logic even simpler. I.e. we could\njust adjust this to back-off for /^4/ instead of trying to handle\narbitrary errors.\n\nAnyway, I'm not interested in pursuing that WIP patch, and I don't\nthink perfect should be the enemy of the good here. Your patch works\nfor you, doesn't really damage anything else, so if you're not\ninterested in hacking up something like the above I think we should\njust take it.\n\nBut I do think it would be very good to get a reply to you / details\nin the commit message about what error you get exactly in this\nscenario, see if you get better details with --smtp-debug, and if so\npaste that (sans any secret info like user/password you don't want to\nshare).\n\nThen if we're poking at this code in the future we can maybe just fix\nthis in some more general fashion while keeping this use-case in mind.\n"},{"id":"320529","messageId":"xmqq60gskobw.fsf@gitster.mtv.corp.google.com","threadId":"46016","inReplyTo":"CACBZZX5GYV50rjg9X602JHqFPaoofH9TwDf_-r_MDu8-rmNV6Q@mail.gmail.com","subject":"Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-05-23T07:46:27Z","receivedAt":"2017-05-23T07:46:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> Looking at this the Nth time now though I wonder about this approach\n> in general. In all your E-Mails I don't think you ever said /what/\n> sort of error you had from the SMTP server, you just said you had a\n> failure or an error, I assume you hit one of the die's in the\n> send_message() function. Can you paste the actual error you get\n> without this patch?\n>\n> I wonder if something like this would Just Work for this case without\n> any configuration or command-line options, with the added benefit of\n> just working for anyone with transitory SMTP issues as well (patch\n> posted with -w, full version at\n> https://github.com/avar/git/commit/acb60c4bde50bdcb62b71ed46f49617e2caef84e.patch):\n\nYeah, if the issues users of 163.com are having can be resolved with\na more general approach like this, that would be very much preferred.\n\n> Now that's very much a WIP and I don't have a server like that to test against.\n>\n> Having worked with SMTP a lot in a past life/job, I'd say it's *very*\n> likely that you're just getting a /^4/ error code from 163.com,\n> probably 421, which would make this logic even simpler. I.e. we could\n> just adjust this to back-off for /^4/ instead of trying to handle\n> arbitrary errors.\n>\n> Anyway, I'm not interested in pursuing that WIP patch, and I don't\n> think perfect should be the enemy of the good here. Your patch works\n> for you, doesn't really damage anything else, so if you're not\n> interested in hacking up something like the above I think we should\n> just take it.\n>\n>\n> But I do think it would be very good to get a reply to you / details\n> in the commit message about what error you get exactly in this\n> scenario, see if you get better details with --smtp-debug, and if so\n> paste that (sans any secret info like user/password you don't want to\n> share).\n\nLet's wait for a few days to see if xiaoqiang wants to take your\noutline of more general approach and polish it. I do prefer the \"no\nconfig\" solution as xiaoqiang won't be the only 163.com user, but\nIndividual Contributors cannot be forced, so ...\n\nThanks.\n"},{"id":"320533","messageId":"20170523103050.1f7ab7e0@jvn","threadId":"46016","inReplyTo":"xmqq60gskobw.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"Jan Viktorin","fromEmail":"viktorin@rehivetech.com","sentAt":"2017-05-23T08:30:50Z","receivedAt":"2017-05-23T08:31:17Z","isPatch":true,"sender":{"key":"viktorin@rehivetech.com","avatar":null},"body":"On Tue, 23 May 2017 16:46:27 +0900\nJunio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n> \n> > Looking at this the Nth time now though I wonder about this approach\n> > in general. In all your E-Mails I don't think you ever said /what/\n> > sort of error you had from the SMTP server, you just said you had a\n> > failure or an error, I assume you hit one of the die's in the\n> > send_message() function. Can you paste the actual error you get\n> > without this patch?\n\nHello,\n\nI have issues with a company SMTP server that returns:\n\nNet::SMTP::SSL=GLOB(0x20d6510)<<< 451 4.3.0 Please try again later,\nrate limited.\n4.3.0 Please try again later, rate limited.\n\nUnfortunately, I didn't find out the exact properties of the limit yet.\nIt seems that sending more then 10 patches at once fails. Thus, I have\nto send longer patch sets in 2 rounds:\n\n1. normal git send-email\n2. git send-email --no-thread --in-reply-to=\"\" \\\n ...\n\nIt is not exactly the same as sending all the patches at once.\n\nThe xiaoqiang's solution sounds promising to me. However, probably a\nmore general solution would be to \"just\" enable sending a whole patch\nset in 2 rounds manually. But I didn't find any way how to do it right.\n\nRegards\nJan\n\n> >\n> > I wonder if something like this would Just Work for this case without\n> > any configuration or command-line options, with the added benefit of\n> > just working for anyone with transitory SMTP issues as well (patch\n> > posted with -w, full version at\n> > https://github.com/avar/git/commit/acb60c4bde50bdcb62b71ed46f49617e2caef84e.patch): \n> \n> Yeah, if the issues users of 163.com are having can be resolved with\n> a more general approach like this, that would be very much preferred.\n> \n> > Now that's very much a WIP and I don't have a server like that to test against.\n> >\n> > Having worked with SMTP a lot in a past life/job, I'd say it's *very*\n> > likely that you're just getting a /^4/ error code from 163.com,\n> > probably 421, which would make this logic even simpler. I.e. we could\n> > just adjust this to back-off for /^4/ instead of trying to handle\n> > arbitrary errors.\n> >\n> > Anyway, I'm not interested in pursuing that WIP patch, and I don't\n> > think perfect should be the enemy of the good here. Your patch works\n> > for you, doesn't really damage anything else, so if you're not\n> > interested in hacking up something like the above I think we should\n> > just take it.\n> >\n> >\n> > But I do think it would be very good to get a reply to you / details\n> > in the commit message about what error you get exactly in this\n> > scenario, see if you get better details with --smtp-debug, and if so\n> > paste that (sans any secret info like user/password you don't want to\n> > share). \n> \n> Let's wait for a few days to see if xiaoqiang wants to take your\n> outline of more general approach and polish it. I do prefer the \"no\n> config\" solution as xiaoqiang won't be the only 163.com user, but\n> Individual Contributors cannot be forced, so ...\n> \n> Thanks.\n"},{"id":"320558","messageId":"7993e188.d18d.15c3560bcaf.Coremail.zxq_yx_007@163.com","threadId":"46016","inReplyTo":"CACBZZX5GYV50rjg9X602JHqFPaoofH9TwDf_-r_MDu8-rmNV6Q@mail.gmail.com","subject":"Re:Re: [PATCH v5] send-email: --batch-size to work around some SMTP server limit","fromName":"赵小强","fromEmail":"zxq_yx_007@163.com","sentAt":"2017-05-23T12:55:51Z","receivedAt":"2017-05-23T13:12:34Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"\n\nAt 2017-05-22 17:26:41, \"Ævar Arnfjörð Bjarmason\" wrote:\n>On Sun, May 21, 2017 at 2:59 PM, xiaoqiang zhao wrote:\n>> Some email servers (e.g. smtp.163.com) limit the number emails to be\n>> sent per session(connection) and this will lead to a faliure when\n>> sending many messages.\n>\n>This OK to me, the nits I had are addressed by Junio's reply.\n>\n>Looking at this the Nth time now though I wonder about this approach\n>in general. In all your E-Mails I don't think you ever said /what/\n>sort of error you had from the SMTP server, you just said you had a\n>failure or an error, I assume you hit one of the die's in the\n>send_message() function. Can you paste the actual error you get\n>without this patch?\n>\n\nWhen I send a patch series which has 13 (plus cover) messages as a test, I got errors as follows and send-email quit when sending the 11th message:\n\nMI:DMC 163 smtp14,EsCowAD3o71TKyRZTBlJHw--.20496S13 1495542613 http://mail.163.com/help/help_spam_16.htm?ip=1.203.183.150&hostid=smtp14&time=1495542613\n\nFollow the link above, I find two error code:\n\n•450 MI:DMC 当前连接发送的邮件数量超出限制。请减少每次连接中投递的邮件数量\n•451 MI:DMC 当前连接发送的邮件数量超出限制。请控制每次连接中投递的邮件数量\n\nTranslate into English:\n•450 MI:DMC The number of messages sent execeeds the limits. Please reduce the number of messages to be sent per connection.\n•451 MI:DMC The number of messages sent execeeds the limits. Please control the number of messages to be sent per connection.\n\nAlthough has different error code, but says similar reason. Testing with --smtp-debug option produce the same error.\n\n>I wonder if something like this would Just Work for this case without\n>any configuration or command-line options, with the added benefit of\n>just working for anyone with transitory SMTP issues as well (patch\n>posted with -w, full version at\n>https://github.com/avar/git/commit/acb60c4bde50bdcb62b71ed46f49617e2caef84e.patch):\n>\n>diff --git a/git-send-email.perl b/git-send-email.perl\n>index 8a1ee0f0d4..c2d85236d1 100755\n>--- a/git-send-email.perl\n>+++ b/git-send-email.perl\n>@@ -1363,6 +1363,10 @@ EOF\n> die __(\"The required SMTP server is not\n>properly defined.\")\n> }\n>\n>+ my $num_tries = 0;\n>+ my $max_tries = 5;\n>+ smtp_again:\n>+ eval {\n> if ($smtp_encryption eq 'ssl') {\n> $smtp_server_port ||= 465; # ssmtp\n> require Net::SMTP::SSL;\n>@@ -1429,6 +1433,22 @@ EOF\n> }\n> $smtp->dataend() or die $smtp->message;\n> $smtp->code =~ /250|200/ or die\n>sprintf(__(\"Failed to send %s\\n\"), $subject).$smtp->message;\n>+ 1;\n>+ } or do {\n>+ my $error = $@ || \"Zombie Error\";\n>+\n>+ warn sprintf(__(\"Failed to send %s due to\n>error: %s\"), $subject, $error);\n>+ if ($num_tries++ < $max_tries) {\n>+ $smtp->quit if defined $smtp;\n>+ $smtp = undef;\n>+ $auth = undef;\n>+ my $sleep = $num_tries * 3; # 3, 6, 9, ...\n>+ warn sprintf(__(\"This is retry %d/%d.\n>Sleeping %d before trying again\"),\n>+ $num_tries, $max_tries, $sleep);\n>+ sleep($sleep);\n>+ goto smtp_again;\n>+ }\n>+ };\n> }\n> if ($quiet) {\n> printf($dry_run ? __(\"Dry-Sent %s\\n\") : __(\"Sent\n>%s\\n\"), $subject);\n>\n>Now that's very much a WIP and I don't have a server like that to test against.\n>\n>Having worked with SMTP a lot in a past life/job, I'd say it's *very*\n>likely that you're just getting a /^4/ error code from 163.com,\n>probably 421, which would make this logic even simpler. I.e. we could\n>just adjust this to back-off for /^4/ instead of trying to handle\n>arbitrary errors.\n>\n>Anyway, I'm not interested in pursuing that WIP patch, and I don't\n>think perfect should be the enemy of the good here. Your patch works\n>for you, doesn't really damage anything else, so if you're not\n>interested in hacking up something like the above I think we should\n>just take it.\n>\n>But I do think it would be very good to get a reply to you / details\n>in the commit message about what error you get exactly in this\n>scenario, see if you get better details with --smtp-debug, and if so\n>paste that (sans any secret info like user/password you don't want to\n>share).\n>\n>Then if we're poking at this code in the future we can maybe just fix\n>this in some more general fashion while keeping this use-case in mind.\n"},{"id":"338656","messageId":"20180207194547.1412-1-sbeller@google.com","threadId":"46016","inReplyTo":"xmqqlgpppri7.fsf@gitster.mtv.corp.google.com","subject":"[PATCH] send-email: have default batch size when relogin delay is given","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-07T19:45:47Z","receivedAt":"2018-02-07T19:46:00Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When the batch size is neither configured nor given on the command\nline, but the relogin delay is given, then the user is not using the\nthe feature as intended. But as the user gave a relogin delay, there is\nclearly the intention to delay sending out emails. Assume a batch size\nof 1 instead of silently ignoring the given relogin delay.\n\nSigned-off-by: Stefan Beller \n---\n git-send-email.perl | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 340b5c8482..5672e05b98 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -379,6 +379,12 @@ unless ($rc) {\n die __(\"Cannot run git format-patch from outside a repository\\n\")\n \tif $format_patch and not $repo;\n \n+if (defined $relogin_delay) {\n+\tif (not defined $batch_size) {\n+\t\t$batch_size = 1;\n+\t}\n+}\n+\n # Now, let's fill any that aren't set in with defaults:\n \n sub read_config {\n-- \n2.15.1.433.g936d1b9894.dirty\n\n"},{"id":"338658","messageId":"CAPig+cQZr=UN3KC3LnaKy=oUYd1inbtDgi_pDuhPXf9HgFOdNw@mail.gmail.com","threadId":"46016","inReplyTo":"20180207194547.1412-1-sbeller@google.com","subject":"Re: [PATCH] send-email: have default batch size when relogin delay is given","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-07T19:50:33Z","receivedAt":"2018-02-07T19:50:51Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Feb 7, 2018 at 2:45 PM, Stefan Beller wrote:\n> When the batch size is neither configured nor given on the command\n> line, but the relogin delay is given, then the user is not using the\n> the feature as intended. But as the user gave a relogin delay, there is\n> clearly the intention to delay sending out emails. Assume a batch size\n> of 1 instead of silently ignoring the given relogin delay.\n>\n> Signed-off-by: Stefan Beller \n> ---\n> diff --git a/git-send-email.perl b/git-send-email.perl\n> @@ -379,6 +379,12 @@ unless ($rc) {\n> +if (defined $relogin_delay) {\n> + if (not defined $batch_size) {\n> + $batch_size = 1;\n> + }\n> +}\n\nMaybe also print a message that this batch size has been used as\ndefault lest the user wonder why it's sending \"slowly\" without\napparently batching anything.\n\nAlternately, complain and die if both options are not specified.\n"},{"id":"338682","messageId":"20180207234359.31493-1-sbeller@google.com","threadId":"46016","inReplyTo":"CAPig+cQZr=UN3KC3LnaKy=oUYd1inbtDgi_pDuhPXf9HgFOdNw@mail.gmail.com","subject":"[PATCH] send-email: error out when relogin delay is missing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-07T23:43:59Z","receivedAt":"2018-02-07T23:44:10Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When the batch size is neither configured nor given on the command\nline, but the relogin delay is given, then the current code ignores\nthe relogin delay setting.\n\nThis is unsafe as there was some intention when setting the batch size.\nOne workaround would be to just assume a batch size of 1 as a default.\nThis however may be bad UX, as then the user may wonder why it is sending\nslowly without apparent batching.\n\nError out for now instead of potentially confusing the user.\nAs 5453b83bdf (send-email: --batch-size to work around some SMTP\nserver limit, 2017-05-21) lays out, we rather want to not have this\ninterface anyway and would rather want to react on the server throttling\ndynamically.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Stefan Beller \n---\n git-send-email.perl | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 340b5c8482..bc0d3ade16 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -379,6 +379,9 @@ unless ($rc) {\n die __(\"Cannot run git format-patch from outside a repository\\n\")\n \tif $format_patch and not $repo;\n \n+die __(\"When a batch size is given, the relogin delay must be set\\n\")\n+\tif defined $relogin_delay and not defined $batch_size;\n+\n # Now, let's fill any that aren't set in with defaults:\n \n sub read_config {\n-- \n2.16.0.rc1.238.g530d649a79-goog\n\n"},{"id":"338689","messageId":"E5979434-8F78-496C-9364-3663FC3670F1@163.com","threadId":"46016","inReplyTo":"20180207234359.31493-1-sbeller@google.com","subject":"Re: [PATCH] send-email: error out when relogin delay is missing","fromName":"xiaoqiang zhao","fromEmail":"zxq_yx_007@163.com","sentAt":"2018-02-08T05:45:40Z","receivedAt":"2018-02-08T05:46:56Z","isPatch":true,"sender":{"key":"zxq_yx_007@163.com","avatar":"https://avatars.githubusercontent.com/u/3981296?v=4"},"body":"\n> 在 2018年2月8日，上午7:43，Stefan Beller 写道：\n> \n> +die __(\"When a batch size is given, the relogin delay must be set\\n\")\n> + if defined $relogin_delay and not defined $batch_size;\n> +\n \nAccording the code, maybe you want to say “When relogin delay is given, a batch size must be set “ ?\n\n"},{"id":"338690","messageId":"CAPig+cQejz+1ZWep31hw0pjxUoewRt+-2WATtw6PEusfvhn8EA@mail.gmail.com","threadId":"46016","inReplyTo":"20180207234359.31493-1-sbeller@google.com","subject":"Re: [PATCH] send-email: error out when relogin delay is missing","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-08T08:08:11Z","receivedAt":"2018-02-08T08:08:18Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Feb 7, 2018 at 6:43 PM, Stefan Beller wrote:\n> [...]\n> Error out for now instead of potentially confusing the user.\n> As 5453b83bdf (send-email: --batch-size to work around some SMTP\n> server limit, 2017-05-21) lays out, we rather want to not have this\n> interface anyway and would rather want to react on the server throttling\n> dynamically.\n>\n> Signed-off-by: Stefan Beller \n> ---\n> diff --git a/git-send-email.perl b/git-send-email.perl\n> @@ -379,6 +379,9 @@ unless ($rc) {\n> +die __(\"When a batch size is given, the relogin delay must be set\\n\")\n> + if defined $relogin_delay and not defined $batch_size;\n\nThis only makes sense is 'batch-size' is specified but not 'relogin'.\nIf the other way around, then the error is confusing. How about this\ninstead?\n\n \"--batch-size and --relogin must be specified together\"\n\n...or something.\n"},{"id":"338745","messageId":"CAGZ79kZ1WA15cybgi9u5h2fh37TOjYmDpLdHgFfyLGBf8pUwpg@mail.gmail.com","threadId":"46016","inReplyTo":"CAPig+cQejz+1ZWep31hw0pjxUoewRt+-2WATtw6PEusfvhn8EA@mail.gmail.com","subject":"Re: [PATCH] send-email: error out when relogin delay is missing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-08T18:21:40Z","receivedAt":"2018-02-08T18:21:47Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Feb 8, 2018 at 12:08 AM, Eric Sunshine wrote:\n> On Wed, Feb 7, 2018 at 6:43 PM, Stefan Beller wrote:\n>> [...]\n>> Error out for now instead of potentially confusing the user.\n>> As 5453b83bdf (send-email: --batch-size to work around some SMTP\n>> server limit, 2017-05-21) lays out, we rather want to not have this\n>> interface anyway and would rather want to react on the server throttling\n>> dynamically.\n>>\n>> Signed-off-by: Stefan Beller \n>> ---\n>> diff --git a/git-send-email.perl b/git-send-email.perl\n>> @@ -379,6 +379,9 @@ unless ($rc) {\n>> +die __(\"When a batch size is given, the relogin delay must be set\\n\")\n>> + if defined $relogin_delay and not defined $batch_size;\n>\n> This only makes sense is 'batch-size' is specified but not 'relogin'.\n> If the other way around, then the error is confusing. How about this\n> instead?\n>\n> \"--batch-size and --relogin must be specified together\"\n>\n> ...or something.\n\nI like this for its expressiveness as it would have helped me a lot.\nI dislike this because it is incorrect when you use the config options\ninstead of command line arguments.\n\nStefan\n"},{"id":"338783","messageId":"CAPig+cRyx1KHJMn97mNy_BmX1D+cGEqVUT6x3Bw+tmsUOE3J+w@mail.gmail.com","threadId":"46016","inReplyTo":"CAGZ79kZ1WA15cybgi9u5h2fh37TOjYmDpLdHgFfyLGBf8pUwpg@mail.gmail.com","subject":"Re: [PATCH] send-email: error out when relogin delay is missing","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-02-08T22:00:18Z","receivedAt":"2018-02-08T22:00:26Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Feb 8, 2018 at 1:21 PM, Stefan Beller wrote:\n> On Thu, Feb 8, 2018 at 12:08 AM, Eric Sunshine wrote:\n>> On Wed, Feb 7, 2018 at 6:43 PM, Stefan Beller wrote:\n>>> +die __(\"When a batch size is given, the relogin delay must be set\\n\")\n>>> + if defined $relogin_delay and not defined $batch_size;\n>>\n>> This only makes sense is 'batch-size' is specified but not 'relogin'.\n>> If the other way around, then the error is confusing. How about this\n>> instead?\n>> \"--batch-size and --relogin must be specified together\"\n>> ...or something.\n>\n> I like this for its expressiveness as it would have helped me a lot.\n> I dislike this because it is incorrect when you use the config options\n> instead of command line arguments.\n\nPerhaps:\n\n \"`batch-size` and `relogin` must be specified together\n (via command-line or configuration option)\"\n"},{"id":"339098","messageId":"20180212194404.66863-1-sbeller@google.com","threadId":"46016","inReplyTo":"CAPig+cRyx1KHJMn97mNy_BmX1D+cGEqVUT6x3Bw+tmsUOE3J+w@mail.gmail.com","subject":"[PATCH] send-email: error out when relogin delay is missing","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-02-12T19:44:04Z","receivedAt":"2018-02-12T19:44:16Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"When the batch size is neither configured nor given on the command\nline, but the relogin delay is given, then the current code ignores\nthe relogin delay setting.\n\nThis is unsafe as there was some intention when setting the batch size.\nOne workaround would be to just assume a batch size of 1 as a default.\nThis however may be bad UX, as then the user may wonder why it is sending\nslowly without apparent batching.\n\nError out for now instead of potentially confusing the user.\nAs 5453b83bdf (send-email: --batch-size to work around some SMTP\nserver limit, 2017-05-21) lays out, we rather want to not have this\ninterface anyway and would rather want to react on the server throttling\ndynamically.\n\nHelped-by: Eric Sunshine \nSigned-off-by: Stefan Beller \n---\n git-send-email.perl | 4 ++++\n 1 file changed, 4 insertions(+)\n\ndiff --git a/git-send-email.perl b/git-send-email.perl\nindex 340b5c8482..f7913f7c2c 100755\n--- a/git-send-email.perl\n+++ b/git-send-email.perl\n@@ -379,6 +379,10 @@ unless ($rc) {\n die __(\"Cannot run git format-patch from outside a repository\\n\")\n \tif $format_patch and not $repo;\n \n+die __(\"`batch-size` and `relogin` must be specified together \" .\n+\t\"(via command-line or configuration option)\\n\")\n+\tif defined $relogin_delay and not defined $batch_size;\n+\n # Now, let's fill any that aren't set in with defaults:\n \n sub read_config {\n-- \n2.15.1.433.g936d1b9894.dirty\n\n"}]}