# Re: [PATCH] Add a MIME-Version header to e-mails 10 messages from 2006-11-05 to 2006-11-16. Participants: Catalin Marinas, Karl Hasselström, Jakub Narebski. Thread: https://gitlist.dev/t/43588 ## Jakub Narebski, 2006-11-05 19:29 Subject: Re: [PATCH] Allow hand-editing of patches before sending Message-ID: URL: https://gitlist.dev/e/eile19%24p7r%241%40sea.gmane.org In-Reply-To: <20061105190400.GC25259@diana.vm.bytemark.co.uk> ``` Karl Hasselström wrote: > These are the headers that StGIT uses without my patch -- that is, the > headers used to get that error message: > > Content-Type: text/plain; charset=utf-8; format=fixed > Content-Transfer-Encoding: 8bit > > Which is obviously not good enough for some picky part of the SMTP > chain. I think you are missing MIME-Version: header. [...] > Well, it added these headers: > > MIME-Version: 1.0 > Content-Transfer-Encoding: QUOTED-PRINTABLE > Content-Type: TEXT/PLAIN; charset=ISO-8859-1 > > Maybe MIME-Version was the only thing missing? I'll have to try. MIME-Version _is_ important. I remember that Outlook Express sent news messages without this header, and those posts were either mangled by news server, or news readers didn't parse MIME headers without MIME-Version: -- Jakub Narebski Warsaw, Poland ShadeHawk on #git ``` ## Karl Hasselström, 2006-11-06 07:45 Subject: [PATCH] Add a MIME-Version header to e-mails Message-ID: <20061106074532.10376.60478.stgit@localhost> URL: https://gitlist.dev/e/20061106074532.10376.60478.stgit%40localhost In-Reply-To: ``` From: Karl Hasselström This is required by some mail servers that want to change the transfer encoding of the mail. Signed-off-by: Karl Hasselström --- OK, this is a test to see if adding MIME-Version makes the problem go away. I don't have any other way to test this than sending a patch to the list. :-) stgit/commands/mail.py | 12 +++++------- 1 files changed, 5 insertions(+), 7 deletions(-) diff --git a/stgit/commands/mail.py b/stgit/commands/mail.py index 45f7438..c428d67 100644 --- a/stgit/commands/mail.py +++ b/stgit/commands/mail.py @@ -271,13 +271,11 @@ def __get_signers_list(msg): return addr_list def __build_extra_headers(): - """Build extra headers like content-type etc. - """ - headers = 'Content-Type: text/plain; charset=utf-8; format=fixed\n' - headers += 'Content-Transfer-Encoding: 8bit\n' - headers += 'User-Agent: StGIT/%s\n' % version.version - - return headers + """Build extra headers like content-type etc.""" + return ('MIME-Version: 1.0\n' + 'Content-Type: text/plain; charset=utf-8; format=fixed\n' + 'Content-Transfer-Encoding: 8bit\n' + 'User-Agent: StGIT/%s\n' % version.version) def edit_message(msg): ``` ## Karl Hasselström, 2006-11-06 07:56 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: <20061106075634.GA4532@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20061106075634.GA4532%40diana.vm.bytemark.co.uk In-Reply-To: <20061106074532.10376.60478.stgit@localhost> ``` On 2006-11-06 08:45:32 +0100, Karl Hasselström wrote: > OK, this is a test to see if adding MIME-Version makes the problem > go away. I don't have any other way to test this than sending a > patch to the list. :-) All right, it seems to have worked! When I send patches to myself, they arrive still 8bit-encoded, and when I get them through the list, they have been properly QP-encoded by whatever part of the chain didn't like 8bit. Catalin, this should replace the QP-encoding patch. (But not the RFC2047-encoding patch -- the headers are required to always be ascii, if I've not misunderstood something.) Thanks everyone! -- Karl Hasselström, kha@treskal.com ``` ## Catalin Marinas, 2006-11-07 09:53 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: URL: https://gitlist.dev/e/b0943d9e0611070153s2a52f65k6ed4643e60a144b3%40mail.gmail.com In-Reply-To: <20061106074532.10376.60478.stgit@localhost> ``` Karl, On 06/11/06, Karl Hasselström wrote: > This is required by some mail servers that want to change the transfer > encoding of the mail. Thanks for digging into this problem and thanks to everyone else (especially Linus) for the detailed explanations. I'll include these patches (maybe modified) and I'll try to fix "import" as well. -- ``` ## Catalin Marinas, 2006-11-07 19:09 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: URL: https://gitlist.dev/e/b0943d9e0611071109w584f4f7fv3ba1b7dbd9413717%40mail.gmail.com In-Reply-To: ``` On 07/11/06, Catalin Marinas wrote: > I'll include these patches (maybe modified) and I'll try to fix > "import" as well. I re-implemented parts of the mail and import commands (inspiring from your patches). They now use the email Python package. The mail command encodes the body to 7 or 8bit depending on non-ascii characters. The headers are QP-encoded. The import command can decode messages properly and can also handle multipart e-mails. I still have to write a test script but my simple tests showed that it works. Please let me know if there is anything wrong (especially with the QP-encoding of the mail headers). Thanks. -- ``` ## Karl Hasselström, 2006-11-08 08:40 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: <20061108084038.GA3164@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20061108084038.GA3164%40diana.vm.bytemark.co.uk In-Reply-To: ``` On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote: > I re-implemented parts of the mail and import commands (inspiring > from your patches). They now use the email Python package. The mail > command encodes the body to 7 or 8bit depending on non-ascii > characters. Even though it turned out to be possible to use 8bit always? > The headers are QP-encoded. Strictly speaking, it's rfc2047-encoded. I think the options are QP and base64. > The import command can decode messages properly and can also handle > multipart e-mails. Goodie. > I still have to write a test script but my simple tests showed that > it works. Please let me know if there is anything wrong (especially > with the QP-encoding of the mail headers). I won't have time to check this out today (in fact, it may have to wait until the weekend). But when I do check it out, I might have time to write that test, so please holler if you start working on that. -- Karl Hasselström, kha@treskal.com ``` ## Catalin Marinas, 2006-11-08 09:41 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: URL: https://gitlist.dev/e/b0943d9e0611080141y3751039v37b0289147237e4f%40mail.gmail.com In-Reply-To: <20061108084038.GA3164@diana.vm.bytemark.co.uk> ``` On 08/11/06, Karl Hasselström wrote: > On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote: > > > I re-implemented parts of the mail and import commands (inspiring > > from your patches). They now use the email Python package. The mail > > command encodes the body to 7 or 8bit depending on non-ascii > > characters. > > Even though it turned out to be possible to use 8bit always? I just let the Python email package to the encoding and it first tries pure ascii and switches to 8bit if this fails. This package automatically adds the proper e-mail headers based on the encoding type. It can also do QP or base64 encodings (if someone complains of an old SMTP server, maybe I'll add a config option to force QP body). > > I still have to write a test script but my simple tests showed that > > it works. Please let me know if there is anything wrong (especially > > with the QP-encoding of the mail headers). > > I won't have time to check this out today (in fact, it may have to > wait until the weekend). But when I do check it out, I might have time > to write that test, so please holler if you start working on that. It would be great if you can run a test script. I've been thinking more of an "import" test that also checks both the "export" and "mail" commands. BTW, I'll try to add support for mbox importing which is pretty easy with the email package. -- ``` ## Karl Hasselström, 2006-11-08 10:01 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: <20061108100103.GA18117@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20061108100103.GA18117%40diana.vm.bytemark.co.uk In-Reply-To: ``` On 2006-11-08 09:41:33 +0000, Catalin Marinas wrote: > On 08/11/06, Karl Hasselström wrote: > > > On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote: > > > > > I re-implemented parts of the mail and import commands > > > (inspiring from your patches). They now use the email Python > > > package. The mail command encodes the body to 7 or 8bit > > > depending on non-ascii characters. > > > > Even though it turned out to be possible to use 8bit always? > > I just let the Python email package to the encoding and it first > tries pure ascii and switches to 8bit if this fails. Ah, OK, I misread you. This is precisely what I wanted. > It can also do QP or base64 encodings (if someone complains of an > old SMTP server, maybe I'll add a config option to force QP body). Yes, but let's hope we don't have to. :-) And if we do have to, we should probably autodetect it from the SMTP dialog as Linus explained (or rather, we should hope that the library can do this for us). > > > I still have to write a test script but my simple tests showed > > > that it works. Please let me know if there is anything wrong > > > (especially with the QP-encoding of the mail headers). > > > > I won't have time to check this out today (in fact, it may have to > > wait until the weekend). But when I do check it out, I might have > > time to write that test, so please holler if you start working on > > that. > > It would be great if you can run a test script. I've been thinking > more of an "import" test that also checks both the "export" and "mail" > commands. I was thinking along these lines too; first make sure the importer works, by feeding it a number of hard-coded patches in different formats, and then make sure that the importer can read what the exporter produces. (With "exporter" meaning both "stg export" and "stg mail".) I didn't originally plan to thest the actual e-mail sending, but now that I think of it, a simple script listening on a random port, pretending to be an SMTP daemon, should be quite simple to set up. > BTW, I'll try to add support for mbox importing which is pretty easy > with the email package. Sounds good. If you do it in time, I'll write a test for that too. ;-) -- Karl Hasselström, kha@treskal.com ``` ## Karl Hasselström, 2006-11-11 12:24 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: <20061111122403.GC11224@diana.vm.bytemark.co.uk> URL: https://gitlist.dev/e/20061111122403.GC11224%40diana.vm.bytemark.co.uk In-Reply-To: ``` On 2006-11-07 19:09:06 +0000, Catalin Marinas wrote: > I re-implemented parts of the mail and import commands (inspiring > from your patches). They now use the email Python package. The mail > command encodes the body to 7 or 8bit depending on non-ascii > characters. The headers are QP-encoded. The import command can > decode messages properly and can also handle multipart e-mails. > > I still have to write a test script but my simple tests showed that > it works. Please let me know if there is anything wrong (especially > with the QP-encoding of the mail headers). One potentially hazardous thing: you encode the mail before letting the user edit it (with the -e and -E switches). This means that the user can insert non-ascii characters in the body after you've already decided it's safe to use 7bit encoding. It also means that the user must be careful to rfc2047-encode any changes to the Subject: and From: headers. -- Karl Hasselström, kha@treskal.com ``` ## Catalin Marinas, 2006-11-16 17:24 Subject: Re: [PATCH] Add a MIME-Version header to e-mails Message-ID: URL: https://gitlist.dev/e/b0943d9e0611160924s55383480x68832ef0a6183780%40mail.gmail.com In-Reply-To: <20061111122403.GC11224@diana.vm.bytemark.co.uk> ``` On 11/11/06, Karl Hasselström wrote: > One potentially hazardous thing: you encode the mail before letting > the user edit it (with the -e and -E switches). This means that the > user can insert non-ascii characters in the body after you've already > decided it's safe to use 7bit encoding. It also means that the user > must be careful to rfc2047-encode any changes to the Subject: and > From: headers. I changed this to edit the mail before encoding it. You can no longer see the full headers (those added by StGIT) but even in a normal e-mail client you can only see the whole headers after sending it (and if you receive it back). -- ```