threads / patch / 19773

patchgit-format-patch, git-send-email: generate/handle escaped >From

Subject: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

## tl;dr

4 messages between Jun 11, 2009 and Jun 28, 2009. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

Paolo Bonzini· Jun 11, 2009, 10:00 UTC · lore

I noticed that the mbox files generated by git-format-patch (especially with --stdout) are not proper in the sense that the lines starting with "From " are not escaped with a > sign. This is unlikely to cause problems with mail clients such as mutt, but many scripts designed to work on mbox files will fumble in this case.

This patch fixes it and dually unescapes the lines in git-send-email.
Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
---
 git-send-email.perl     |    1 +
 pretty.c                |    2 ++
 t/t4014-format-patch.sh |   13 +++++++++++++
 t/t9001-send-email.sh   |   15 +++++++++++++++
 4 files changed, 31 insertions(+), 0 deletions(-)
Show changes to 4 files +31 −0

git-send-email.perl, pretty.c, t/t4014-format-patch.sh, t/t9001-send-email.sh

diff --git a/git-send-email.perl b/git-send-email.perl
index 4c795a4..de57303 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1085,6 +1085,7 @@ foreach my $t (@files) {
 	}
 	# Now parse the message body
 	while(<F>) {
+		s/^>From /From /;
 		$message .=  $_;
 		if (/^(Signed-off-by|Cc): (.*)$/i) {
 			chomp;
diff --git a/pretty.c b/pretty.c
index e5328da..d7f8228 100644
--- a/pretty.c
+++ b/pretty.c
@@ -868,6 +868,8 @@ void pp_remainder(enum cmit_fmt fmt,
 			memset(sb->buf + sb->len, ' ', indent);
 			strbuf_setlen(sb, sb->len + indent);
 		}
+		if (fmt == CMIT_FMT_EMAIL && !prefixcmp(line, "From "))
+			strbuf_addch(sb, '>');
 		strbuf_add(sb, line, linelen);
 		strbuf_addch(sb, '\n');
 	}
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 922a894..8474718 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -28,6 +28,13 @@ test_expect_success setup '
 	git update-index file &&
 	git commit -m "Side changes #3 with \\n backslash-n in it." &&
 
+	git checkout -b more &&
+	for i in A B C; do echo "$i"; done >>file &&
+	git update-index file &&
+	git commit -m "More changes
+
+From this point on..." &&
+
 	git checkout master &&
 	git diff-tree -p C2 | git apply --index &&
 	git commit -m "Master accepts moral equivalent of #2"
@@ -516,4 +523,10 @@ test_expect_success 'format-patch --signoff' '
 	grep "^Signed-off-by: $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>"
 '
 
+test_expect_success 'format-patch escapes From' '
+	git format-patch more^..more --stdout > patch9 &&
+	grep "^>From this" patch9 &&
+	! grep "^From this" patch9
+'
+
 test_done
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 2ce24cd..5c6f0f4 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -168,6 +168,21 @@ test_expect_success 'no patch was sent' '
 	! test -e commandline1
 '
 
+test_expect_success 'fix >From lines' '
+	clean_fake_sendmail &&
+	cp $patches gt-from.patch &&
+	echo ">From abcdef" >>gt-from.patch &&
+	git send-email \
+		--from="Example <nobody@example.com>" \
+		--to=nobody@example.com \
+		--smtp-server="$(pwd)/fake.sendmail" \
+		gt-from.patch \
+		2>errors &&
+	test -e msgtxt1 &&
+	! grep "^>From abcdef" msgtxt1 &&
+	grep "^From abcdef" msgtxt1
+'
+
 test_expect_success 'Author From: in message body' '
 	clean_fake_sendmail &&
 	git send-email \

-- 
1.6.0.3
Jeff King· Jun 11, 2009, 10:55 UTC · re: Paolo Bonzini · lore

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

On Thu, Jun 11, 2009 at 12:00:34PM +0200, Paolo Bonzini wrote:
Show 7 quoted lines
> I noticed that the mbox files generated by git-format-patch (especially
> with --stdout) are not proper in the sense that the lines starting with
> "From " are not escaped with a > sign.  This is unlikely to cause problems
> with mail clients such as mutt, but many scripts designed to work on
> mbox files will fumble in this case.
> 
> This patch fixes it and dually unescapes the lines in git-send-email.

Ugh. Can we please at least make this optional? Many modern MTAs handle the current situation just fine by being much more strict in their "From" matching (i.e., you would need something that really looks like a valid "From" line to get a match), and do not do ">From" de-quoting at all (mutt is such an example).

Some even take it a step farther and use Content-Length headers, though I do not think that is a good idea here (we encourage people to munge the contents while stored as an mbox).

Show 7 quoted lines
> +++ b/pretty.c
> @@ -868,6 +868,8 @@ void pp_remainder(enum cmit_fmt fmt,
>  			memset(sb->buf + sb->len, ' ', indent);
>  			strbuf_setlen(sb, sb->len + indent);
>  		}
> +		if (fmt == CMIT_FMT_EMAIL && !prefixcmp(line, "From "))
> +			strbuf_addch(sb, '>');

This is the lossy "mboxo" conversion. A quoted From is now indistinguishable from an original ">From". To be reversible, you need to quote "s/^>*From />&/".

But of course, which conversion you want depends entirely on what you are going to feed it to, and which mbox they are expecting. Which is why it really should be configurable.

Your use case looks to be feeding it to send-email; I suspect you would be better served by improving the 'From ' detection in send-email, probably to something like:

  /^From \S+ \w{3} \w{3} \d+ \d\d:\d\d:\d\d \d+/

though that may be too strict. It would probably make sense to steal one from one of the many mbox-reading CPAN modules.

-Peff
Paolo Bonzini· Jun 11, 2009, 11:58 UTC · re: Jeff King · lore

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

Show 5 quoted lines
> But of course, which conversion you want depends entirely on what you
> are going to feed it to, and which mbox they are expecting. Which is why
> it really should be configurable.
> 
> Your use case looks to be feeding it to send-email

Almost, :-) because git-send-email does not support sending multiple messages out of one mbox file. I have an upcoming patch to add this support, and I was just "feeling the waters" before posting it.

Show 8 quoted lines
> I suspect you would
> be better served by improving the 'From ' detection in send-email,
> probably to something like:
> 
>   /^From \S+ \w{3} \w{3} \d+ \d\d:\d\d:\d\d \d+/
> 
> though that may be too strict. It would probably make sense to steal one
> from one of the many mbox-reading CPAN modules.

Good idea. I'll let others speak and can make this configurable (as well as use the s/^>*From />&/ conversion), but if no one speaks, I'll change my git-send-email patch to use the above regex or a similar one, and withdraw this patch.

Thanks for the prompt remark!
Paolo
Paolo Bonzini· Jun 28, 2009, 07:52 UTC · re: Paolo Bonzini · lore

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

Paolo Bonzini wrote:
Show 10 quoted lines
> 
>> But of course, which conversion you want depends entirely on what you
>> are going to feed it to, and which mbox they are expecting. Which is why
>> it really should be configurable.
>>
>> Your use case looks to be feeding it to send-email
> 
> Almost, :-) because git-send-email does not support sending multiple 
> messages out of one mbox file.  I have an upcoming patch to add this 
> support, and I was just "feeling the waters" before posting it.

I decided that it's easier to support sending a single mbox file with this alias instead:

send-mbox = "!bash -ec 'eval f=\\$$#; eval set -- `seq -f\"\\$%.0f\" 1 $(($#-1))`; if last=`mkdir .mboxsplit && git mailsplit -d4 -o.mboxsplit -b -- \"$f\"`; then echo Found $last messages in \"$f\"; git send-email \"$@\" .mboxsplit && rm -rf .mboxsplit; else rm -rf .mboxsplit; fi' -"

So, patch withdrawn. I don't think I'll post the patches in my mailsplit-with-sendemail branch.

Paolo

← back to recent threads