Re: [PATCH v2 2/5] mailinfo: warn if CR found in base64/quoted-printable email

From: Junio C Hamano <gitster@pobox.com>
Date: May 5, 2021, 03:41 UTC
Message-ID: <xmqqr1il4zk0.fsf@gitster.g>
In-Reply-To: <45d2c4ab58c4b0c6f0c7790890bbf75eb373f999.1620148732.git.congdanhqx@gmail.com>

Đoàn Trần Công Danh  <congdanhqx@gmail.com> writes:

> When an SMTP server receives an 8-bit email message, possibly with only
> LF as line ending, some of those servers decide to change said LF to
> CRLF.

s/an SMTP server receives/SMTP servers receive/ s/those servers/them/

> Some mailing list softwares, when receives an 8-bit email message,
> decide to encoding such message in base64 or quoted-printable.

s/encoding/encode/

So the issue is not about CRLF terminating the lines of base64 or QP (we should treat CRLF and LF terminated lines when unwrapping base64 or QP the same way). It is about seeing CRLF in the payload after unwrapping base64 or QP. It was unclear which one was at issue from the subject alone.

Show 7 quoted lines

> If an email is transfered through above mail servers, then distributed
> by such mailing list softwares, the recipients will receive an email
> contains a patch mungled with CRLF encoded inside another encoding.
> Thus, such CR couldn't be dropped by mailsplit. Hence, the mailed patch
> couldn't be applied cleanly. Such accidents have been observed in the wild [1].
>
> Let's give our users some warnings if such CR is found.

Hmph. It is unclear which one of the following we want our endgame to be:

 (1) strip silently and apply
 (2) warn but strip and apply
 (3) warn but do not strip, letting the application fail

but let's keep reading. I suspect (1) and (2) might be error prone, as the mailpath that may have caused this kind of breakage may not be under end-user's control.

Show 5 quoted lines

> +static void summarize_quoted_cr(struct mailinfo *mi, int have_quoted_cr)
> +{
> +	if (have_quoted_cr)
> +		warning("quoted CR detected");
> +}

At this step, it is unclear if it is easier to read to make it the responsibility of the caller to check for have_quoted_cr, but it will become clear as we add more condition for the warning in later steps to let callers unconditionally call this helper and decide when we want to be silent inside this function.

Have you considered adding a new have_quoted_cr member to "struct mailinfo"? After all, the mailinfo struct is not only about end user preference but contains all information we gleaned out of the incoming message.

Show 51 quoted lines

>  static void handle_body(struct mailinfo *mi, struct strbuf *line)
>  {
>  	struct strbuf prev = STRBUF_INIT;
> +	int have_quoted_cr = 0;
>  
>  	/* Skip up to the first boundary */
>  	if (*(mi->content_top)) {
> @@ -1051,6 +1063,8 @@ static void handle_body(struct mailinfo *mi, struct strbuf *line)
>  				handle_filter(mi, &prev);
>  				strbuf_reset(&prev);
>  			}
> +			summarize_quoted_cr(mi, have_quoted_cr);
> +			have_quoted_cr = 0;
>  			if (!handle_boundary(mi, line))
>  				goto handle_body_out;
>  		}
> @@ -1081,7 +1095,7 @@ static void handle_body(struct mailinfo *mi, struct strbuf *line)
>  						strbuf_addbuf(&prev, sb);
>  						break;
>  					}
> -				handle_filter_flowed(mi, sb, &prev);
> +				handle_filter_flowed(mi, sb, &prev, &have_quoted_cr);
>  			}
>  			/*
>  			 * The partial chunk is saved in "prev" and will be
> @@ -1091,7 +1105,7 @@ static void handle_body(struct mailinfo *mi, struct strbuf *line)
>  			break;
>  		}
>  		default:
> -			handle_filter_flowed(mi, line, &prev);
> +			handle_filter_flowed(mi, line, &prev, &have_quoted_cr);
>  		}
>  
>  		if (mi->input_error)
> @@ -1100,6 +1114,7 @@ static void handle_body(struct mailinfo *mi, struct strbuf *line)
>  
>  	if (prev.len)
>  		handle_filter(mi, &prev);
> +	summarize_quoted_cr(mi, have_quoted_cr);
>  
>  	flush_inbody_header_accum(mi);
>  
> diff --git a/t/t5100-mailinfo.sh b/t/t5100-mailinfo.sh
> index 147e616533..d8fdda6bea 100755
> --- a/t/t5100-mailinfo.sh
> +++ b/t/t5100-mailinfo.sh
> @@ -228,4 +228,19 @@ test_expect_success 'mailinfo handles unusual header whitespace' '
>  	test_cmp expect actual
>  '
>  
> +check_quoted_cr_mail() {

SP on both sides of (), i.e.

	check_quoted_cr_mail () {

Show 15 quoted lines

> +	git mailinfo -u "$@" quoted-cr-msg quoted-cr-patch \
> +		<"$DATA/quoted-cr.mbox" >quoted-cr-info 2>quoted-cr-err &&
> +	test_cmp "expect-cr-msg" quoted-cr-msg &&
> +	test_cmp "expect-cr-patch" quoted-cr-patch &&
> +	test_cmp "$DATA/quoted-cr-info" quoted-cr-info
> +}
> +
> +test_expect_success 'mailinfo warn CR in base64 encoded email' '
> +	sed "s/%%/$(printf \\015)/" "$DATA/quoted-cr-msg" >expect-cr-msg &&
> +	sed "s/%%/$(printf \\015)/" "$DATA/quoted-cr-patch" >expect-cr-patch &&
> +	check_quoted_cr_mail &&
> +	grep "quoted CR detected" quoted-cr-err
> +'
> +
>  test_done