threads / patch / 40455

patchgit-send-email.perl: Fixed sending of many/huge changes/patches

Subject: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

## tl;dr

5 messages between Sep 30, 2015 and Nov 24, 2015. Diffs are folded; open one to read it.

replies: 4people: 4as markdown or json

Lars Wendler· Sep 30, 2015, 07:26 UTC · lore

From: Stefan Agner <stefan@agner.ch>

Sometimes sending huge patches/commits fail with

[Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email line 1320.

Running the command with --smtp-debug=1 yields to

Net::SMTP::SSL: Net::Cmd::datasend(): unexpected EOF on command channel: at /usr/lib/git-core/git-send-email line 1320. [Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email line 1320.

Stefan described it in his mail like this:

It seems to me that there is a size limit, after cutting down the patch to ~16K, sending started to work. I cut it twice, once by removing lines from the head and once from the bottom, in both cases at the size of around 16K I could send the patch.

See also original report: http://permalink.gmane.org/gmane.comp.version-control.git/274569

Reported-by: Juston Li <juston.h.li@gmail.com>
Tested-by: Markos Chandras <hwoarang@gentoo.org>
Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
---
 git-send-email.perl | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

Show changes to git-send-email.perl +5 −1

diff --git a/git-send-email.perl b/git-send-email.perl
index e3ff44b..e907e0ea 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1365,7 +1365,11 @@ Message-Id: $message_id
 		$smtp->mail( $raw_from ) or die $smtp->message;
 		$smtp->to( @recipients ) or die $smtp->message;
 		$smtp->data or die $smtp->message;
-		$smtp->datasend("$header\n$message") or die $smtp->message;
+		$smtp->datasend("$header\n") or die $smtp->message;
+		my @lines = split /^/, $message;
+		foreach my $line (@lines) {
+			$smtp->datasend("$line") or die $smtp->message;
+		}
 		$smtp->dataend() or die $smtp->message;
 		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
 	}

-- 
2.6.0

Johannes Schindelin· Sep 30, 2015, 08:32 UTC · re: Lars Wendler · lore

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

Hi Lars,

On 2015-09-30 09:26, Lars Wendler wrote:

Show 28 quoted lines

> From: Stefan Agner <stefan@agner.ch>
> 
> Sometimes sending huge patches/commits fail with
> 
> [Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
> line 1320.
> 
> Running the command with --smtp-debug=1 yields to
> 
> Net::SMTP::SSL: Net::Cmd::datasend(): unexpected EOF on command channel:
> at /usr/lib/git-core/git-send-email line 1320.
> [Net::SMTP::SSL] Connection closed at /usr/lib/git-core/git-send-email
> line 1320.
> 
> Stefan described it in his mail like this:
> 
> It seems to me that there is a size limit, after cutting down the patch
> to ~16K, sending started to work. I cut it twice, once by removing lines
> from the head and once from the bottom, in both cases at the size of
> around 16K I could send the patch.
> 
> See also original report:
> http://permalink.gmane.org/gmane.comp.version-control.git/274569
> 
> Reported-by: Juston Li <juston.h.li@gmail.com>
> Tested-by: Markos Chandras <hwoarang@gentoo.org>
> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
> ---

Very nice! Thank you, Dscho

Junio C Hamano· Sep 30, 2015, 17:51 UTC · re: Lars Wendler · lore

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

Lars Wendler <polynomial-c@gentoo.org> writes:

Show 29 quoted lines

> It seems to me that there is a size limit, after cutting down the patch
> to ~16K, sending started to work. I cut it twice, once by removing lines
> from the head and once from the bottom, in both cases at the size of
> around 16K I could send the patch.
>
> See also original report:
> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>
> Reported-by: Juston Li <juston.h.li@gmail.com>
> Tested-by: Markos Chandras <hwoarang@gentoo.org>
> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
> ---
>  git-send-email.perl | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index e3ff44b..e907e0ea 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>  		$smtp->mail( $raw_from ) or die $smtp->message;
>  		$smtp->to( @recipients ) or die $smtp->message;
>  		$smtp->data or die $smtp->message;
> -		$smtp->datasend("$header\n$message") or die $smtp->message;
> +		$smtp->datasend("$header\n") or die $smtp->message;
> +		my @lines = split /^/, $message;
> +		foreach my $line (@lines) {
> +			$smtp->datasend("$line") or die $smtp->message;
> +		}

Thanks.  One and a half comments.

 * If 16k is the limit, and smtp payload line limit is much much
   shorter than that, is it sensible to send data line by line?

 * Has this been reported to Net::Cmd::datasend() upstream?

>  		$smtp->dataend() or die $smtp->message;
>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>  	}

Stefan Agner· Oct 1, 2015, 23:43 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

On 2015-09-30 10:51, Junio C Hamano wrote:

Show 32 quoted lines

> Lars Wendler <polynomial-c@gentoo.org> writes:
> 
>> It seems to me that there is a size limit, after cutting down the patch
>> to ~16K, sending started to work. I cut it twice, once by removing lines
>> from the head and once from the bottom, in both cases at the size of
>> around 16K I could send the patch.
>>
>> See also original report:
>> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>>
>> Reported-by: Juston Li <juston.h.li@gmail.com>
>> Tested-by: Markos Chandras <hwoarang@gentoo.org>
>> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
>> ---
>>  git-send-email.perl | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/git-send-email.perl b/git-send-email.perl
>> index e3ff44b..e907e0ea 100755
>> --- a/git-send-email.perl
>> +++ b/git-send-email.perl
>> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>>  		$smtp->mail( $raw_from ) or die $smtp->message;
>>  		$smtp->to( @recipients ) or die $smtp->message;
>>  		$smtp->data or die $smtp->message;
>> -		$smtp->datasend("$header\n$message") or die $smtp->message;
>> +		$smtp->datasend("$header\n") or die $smtp->message;
>> +		my @lines = split /^/, $message;
>> +		foreach my $line (@lines) {
>> +			$smtp->datasend("$line") or die $smtp->message;
>> +		}
>

Hi,

> Thanks.  One and a half comments.
> 
>  * If 16k is the limit, and smtp payload line limit is much much
>    shorter than that, is it sensible to send data line by line?

It was a easy and quick way which popped into my mind. The 16k limit has been evaluated just by trying out some patch sizes, not sure whether that is really right and where that really comes from. I guess before using this as split size one would need to understand what causes that limit actually...

>  * Has this been reported to Net::Cmd::datasend() upstream?

Not by me. I also have to admit that I am not a perl hacker at all...

-- Stefan

> 
>>  		$smtp->dataend() or die $smtp->message;
>>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>>  	}

Stefan Agner· Nov 24, 2015, 00:38 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-send-email.perl: Fixed sending of many/huge changes/patches

On 2015-09-30 10:51, Junio C Hamano wrote:

Show 38 quoted lines

> Lars Wendler <polynomial-c@gentoo.org> writes:
> 
>> It seems to me that there is a size limit, after cutting down the patch
>> to ~16K, sending started to work. I cut it twice, once by removing lines
>> from the head and once from the bottom, in both cases at the size of
>> around 16K I could send the patch.
>>
>> See also original report:
>> http://permalink.gmane.org/gmane.comp.version-control.git/274569
>>
>> Reported-by: Juston Li <juston.h.li@gmail.com>
>> Tested-by: Markos Chandras <hwoarang@gentoo.org>
>> Signed-off-by: Lars Wendler <polynomial-c@gentoo.org>
>> ---
>>  git-send-email.perl | 6 +++++-
>>  1 file changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/git-send-email.perl b/git-send-email.perl
>> index e3ff44b..e907e0ea 100755
>> --- a/git-send-email.perl
>> +++ b/git-send-email.perl
>> @@ -1365,7 +1365,11 @@ Message-Id: $message_id
>>  		$smtp->mail( $raw_from ) or die $smtp->message;
>>  		$smtp->to( @recipients ) or die $smtp->message;
>>  		$smtp->data or die $smtp->message;
>> -		$smtp->datasend("$header\n$message") or die $smtp->message;
>> +		$smtp->datasend("$header\n") or die $smtp->message;
>> +		my @lines = split /^/, $message;
>> +		foreach my $line (@lines) {
>> +			$smtp->datasend("$line") or die $smtp->message;
>> +		}
> 
> Thanks.  One and a half comments.
> 
>  * If 16k is the limit, and smtp payload line limit is much much
>    shorter than that, is it sensible to send data line by line?
> 
>  * Has this been reported to Net::Cmd::datasend() upstream?

I still constantly run in to this issue. Fixing it locally, and next time git gets updated and I send a larger patch, it happens again.

Just dig a bit more into that, it seems that this is a documented Perl limitation: http://search.cpan.org/~sullr/IO-Socket-SSL-2.020/lib/IO/Socket/SSL.pod#syswrite

All examples use datasend line wise, so I guess it is not a all to bad choice...

-- Stefan

> 
>>  		$smtp->dataend() or die $smtp->message;
>>  		$smtp->code =~ /250|200/ or die "Failed to send $subject\n".$smtp->message;
>>  	}