threads / patch / 43588

patchRe: [PATCH] Add a MIME-Version header to e-mails

Subject: Re: [PATCH] Add a MIME-Version header to e-mails

## tl;dr

10 messages between Nov 5, 2006 and Nov 16, 2006. Diffs are folded; open one to read it.

replies: 9people: 3as markdown or json

Jakub Narebski· Nov 5, 2006, 19:29 UTC · lore

Re: [PATCH] Allow hand-editing of patches before sending

Karl Hasselström wrote:
Show 8 quoted lines
> These are the headers that StGIT uses without my patch -- that is, the
> headers used to get that error message:
> 
>   Content-Type: text/plain; charset=utf-8; format=fixed
>   Content-Transfer-Encoding: 8bit
> 
> Which is obviously not good enough for some picky part of the SMTP
> chain.
I think you are missing MIME-Version: header.
[...]
Show 7 quoted lines
> Well, it added these headers:
> 
>   MIME-Version: 1.0
>   Content-Transfer-Encoding: QUOTED-PRINTABLE
>   Content-Type: TEXT/PLAIN; charset=ISO-8859-1
> 
> Maybe MIME-Version was the only thing missing? I'll have to try.

MIME-Version _is_ important. I remember that Outlook Express sent news messages without this header, and those posts were either mangled by news server, or news readers didn't parse MIME headers without MIME-Version:

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
Karl Hasselström· Nov 6, 2006, 07:45 UTC · re: Jakub Narebski · lore

[PATCH] Add a MIME-Version header to e-mails

From: Karl Hasselström <kha@treskal.com>

This is required by some mail servers that want to change the transfer encoding of the mail.

Signed-off-by: Karl Hasselström <kha@treskal.com>
---

OK, this is a test to see if adding MIME-Version makes the problem go away. I don't have any other way to test this than sending a patch to the list. :-)

 stgit/commands/mail.py |   12 +++++-------
 1 files changed, 5 insertions(+), 7 deletions(-)
Show changes to stgit/commands/mail.py +5 −7
diff --git a/stgit/commands/mail.py b/stgit/commands/mail.py
index 45f7438..c428d67 100644
--- a/stgit/commands/mail.py
+++ b/stgit/commands/mail.py
@@ -271,13 +271,11 @@ def __get_signers_list(msg):
     return addr_list
 
 def __build_extra_headers():
-    """Build extra headers like content-type etc.
-    """
-    headers  = 'Content-Type: text/plain; charset=utf-8; format=fixed\n'
-    headers += 'Content-Transfer-Encoding: 8bit\n'
-    headers += 'User-Agent: StGIT/%s\n' % version.version
-
-    return headers
+    """Build extra headers like content-type etc."""
+    return ('MIME-Version: 1.0\n'
+            'Content-Type: text/plain; charset=utf-8; format=fixed\n'
+            'Content-Transfer-Encoding: 8bit\n'
+            'User-Agent: StGIT/%s\n' % version.version)
 
 def edit_message(msg):
Karl Hasselström· Nov 6, 2006, 07:56 UTC · re: Karl Hasselström · lore
On 2006-11-06 08:45:32 +0100, Karl Hasselström wrote:
> OK, this is a test to see if adding MIME-Version makes the problem
> go away. I don't have any other way to test this than sending a
> patch to the list. :-)

All right, it seems to have worked! When I send patches to myself, they arrive still 8bit-encoded, and when I get them through the list, they have been properly QP-encoded by whatever part of the chain didn't like 8bit.

Catalin, this should replace the QP-encoding patch. (But not the RFC2047-encoding patch -- the headers are required to always be ascii, if I've not misunderstood something.)

Thanks everyone!
-- 
Karl Hasselström, kha@treskal.com
Catalin Marinas· Nov 7, 2006, 09:53 UTC · re: Karl Hasselström · lore
Karl,
On 06/11/06, Karl Hasselström <kha@treskal.com> wrote:
> This is required by some mail servers that want to change the transfer
> encoding of the mail.

Thanks for digging into this problem and thanks to everyone else (especially Linus) for the detailed explanations.

I'll include these patches (maybe modified) and I'll try to fix "import" as well.

Catalin Marinas· Nov 7, 2006, 19:09 UTC · re: Catalin Marinas · lore
On 07/11/06, Catalin Marinas <catalin.marinas@gmail.com> wrote:
> I'll include these patches (maybe modified) and I'll try to fix
> "import" as well.

I re-implemented parts of the mail and import commands (inspiring from your patches). They now use the email Python package. The mail command encodes the body to 7 or 8bit depending on non-ascii characters. The headers are QP-encoded. The import command can decode messages properly and can also handle multipart e-mails.

I still have to write a test script but my simple tests showed that it works. Please let me know if there is anything wrong (especially with the QP-encoding of the mail headers).

Thanks.
Karl Hasselström· Nov 8, 2006, 08:40 UTC · re: Catalin Marinas · lore
On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote:
> I re-implemented parts of the mail and import commands (inspiring
> from your patches). They now use the email Python package. The mail
> command encodes the body to 7 or 8bit depending on non-ascii
> characters.
Even though it turned out to be possible to use 8bit always?
> The headers are QP-encoded.

Strictly speaking, it's rfc2047-encoded. I think the options are QP and base64.

> The import command can decode messages properly and can also handle
> multipart e-mails.
Goodie.
> I still have to write a test script but my simple tests showed that
> it works. Please let me know if there is anything wrong (especially
> with the QP-encoding of the mail headers).

I won't have time to check this out today (in fact, it may have to wait until the weekend). But when I do check it out, I might have time to write that test, so please holler if you start working on that.

-- 
Karl Hasselström, kha@treskal.com
Catalin Marinas· Nov 8, 2006, 09:41 UTC · re: Karl Hasselström · lore
On 08/11/06, Karl Hasselström <kha@treskal.com> wrote:
Show 8 quoted lines
> On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote:
>
> > I re-implemented parts of the mail and import commands (inspiring
> > from your patches). They now use the email Python package. The mail
> > command encodes the body to 7 or 8bit depending on non-ascii
> > characters.
>
> Even though it turned out to be possible to use 8bit always?

I just let the Python email package to the encoding and it first tries pure ascii and switches to 8bit if this fails. This package automatically adds the proper e-mail headers based on the encoding type. It can also do QP or base64 encodings (if someone complains of an old SMTP server, maybe I'll add a config option to force QP body).

Show 7 quoted lines
> > I still have to write a test script but my simple tests showed that
> > it works. Please let me know if there is anything wrong (especially
> > with the QP-encoding of the mail headers).
>
> I won't have time to check this out today (in fact, it may have to
> wait until the weekend). But when I do check it out, I might have time
> to write that test, so please holler if you start working on that.

It would be great if you can run a test script. I've been thinking more of an "import" test that also checks both the "export" and "mail" commands. BTW, I'll try to add support for mbox importing which is pretty easy with the email package.

Karl Hasselström· Nov 8, 2006, 10:01 UTC · re: Catalin Marinas · lore
On 2006-11-08 09:41:33 +0000, Catalin Marinas wrote:
Show 13 quoted lines
> On 08/11/06, Karl Hasselström <kha@treskal.com> wrote:
>
> > On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote:
> >
> > > I re-implemented parts of the mail and import commands
> > > (inspiring from your patches). They now use the email Python
> > > package. The mail command encodes the body to 7 or 8bit
> > > depending on non-ascii characters.
> >
> > Even though it turned out to be possible to use 8bit always?
>
> I just let the Python email package to the encoding and it first
> tries pure ascii and switches to 8bit if this fails.
Ah, OK, I misread you. This is precisely what I wanted.
> It can also do QP or base64 encodings (if someone complains of an
> old SMTP server, maybe I'll add a config option to force QP body).

Yes, but let's hope we don't have to. :-) And if we do have to, we should probably autodetect it from the SMTP dialog as Linus explained (or rather, we should hope that the library can do this for us).

Show 12 quoted lines
> > > I still have to write a test script but my simple tests showed
> > > that it works. Please let me know if there is anything wrong
> > > (especially with the QP-encoding of the mail headers).
> >
> > I won't have time to check this out today (in fact, it may have to
> > wait until the weekend). But when I do check it out, I might have
> > time to write that test, so please holler if you start working on
> > that.
>
> It would be great if you can run a test script. I've been thinking
> more of an "import" test that also checks both the "export" and "mail"
> commands.

I was thinking along these lines too; first make sure the importer works, by feeding it a number of hard-coded patches in different formats, and then make sure that the importer can read what the exporter produces. (With "exporter" meaning both "stg export" and "stg mail".)

I didn't originally plan to thest the actual e-mail sending, but now that I think of it, a simple script listening on a random port, pretending to be an SMTP daemon, should be quite simple to set up.

> BTW, I'll try to add support for mbox importing which is pretty easy
> with the email package.
Sounds good. If you do it in time, I'll write a test for that too. ;-)
-- 
Karl Hasselström, kha@treskal.com
Karl Hasselström· Nov 11, 2006, 12:24 UTC · re: Catalin Marinas · lore
On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote:
Show 9 quoted lines
> I re-implemented parts of the mail and import commands (inspiring
> from your patches). They now use the email Python package. The mail
> command encodes the body to 7 or 8bit depending on non-ascii
> characters. The headers are QP-encoded. The import command can
> decode messages properly and can also handle multipart e-mails.
>
> I still have to write a test script but my simple tests showed that
> it works. Please let me know if there is anything wrong (especially
> with the QP-encoding of the mail headers).
One potentially hazardous thing: you encode the mail before letting
the user edit it (with the -e and -E switches). This means that the
user can insert non-ascii characters in the body after you've already
decided it's safe to use 7bit encoding. It also means that the user
must be careful to rfc2047-encode any changes to the Subject: and
From: headers.
-- 
Karl Hasselström, kha@treskal.com
Catalin Marinas· Nov 16, 2006, 17:24 UTC · re: Karl Hasselström · lore
On 11/11/06, Karl Hasselström <kha@treskal.com> wrote:
Show 6 quoted lines
> One potentially hazardous thing: you encode the mail before letting
> the user edit it (with the -e and -E switches). This means that the
> user can insert non-ascii characters in the body after you've already
> decided it's safe to use 7bit encoding. It also means that the user
> must be careful to rfc2047-encode any changes to the Subject: and
> From: headers.

I changed this to edit the mail before encoding it. You can no longer see the full headers (those added by StGIT) but even in a normal e-mail client you can only see the whole headers after sending it (and if you receive it back).

← back to recent threads