RFC patch, 3 partspost-receive-email: explicitly set Content-Type header

Hi all,

This is a revival of [1], which declares encoding in emails to make it more likely that they can be read. I like to think it avoids the mistakes of previous attempts, but I'll let you judge. :)

Sorry for the long delay.  Thoughts of all kinds welcome, as always.

Thanks, Jonathan

Gerrit Pape (1):
  hooks/post-receive-email: set declared encoding to utf-8

Jonathan Nieder (2):
  hooks/post-receive-email: use plumbing instead of 'git log' and 'git show'
  hooks/post-receive-email: force log messages in UTF-8

 contrib/hooks/post-receive-email | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

[1] http://thread.gmane.org/gmane.comp.version-control.git/181737/focus=183070

This way the hook doesn't have to keep being tweaked as porcelain learns new features like color and pagination.

While at it, replace the "git rev-list | git shortlog" idiom with plain "git shortlog" for simplicity.

Except for depending less on the value of settings like '[log] abbrevCommit', no change in output intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 contrib/hooks/post-receive-email | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

Git commands write commit messages in UTF-8 by default, but that default can be overridden by the [i18n] commitEncoding and logOutputEncoding settings. With such a setting, the emails written by the post-receive-email hook use a mixture of encodings:

 1. Log messages use the configured log output encoding, which is
    meant to be whatever encoding works best with local terminals
    (and does not have much to do with what encoding should be used
    for email)

 2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw
    port (which uses Unicode filesystem APIs), always UTF-8

 3. The "This is an automated email" preface uses a project description
    from .git/description, which is typically in UTF-8 to support
    gitweb.

So (1) is configurable, and (2) and (3) are unconfigurable and typically UTF-8. Override the log output encoding to always use UTF-8 when writing the email to get the best chance of a comprehensible single-encoding email.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
 contrib/hooks/post-receive-email | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

On Fri, Aug 02, 2013 at 04:23:38PM -0700, Jonathan Nieder wrote:

I cannot say exactly if it makes sense for THIS patch, but I'd like to remind about Cygwin port, which definitely does not use UTF-8 encoding (in my case it is Windows-1251) for filenames.

I cannot agree to receive e-mails in UTF-8 only for Windows projects which have non-UTF-8 encoding. I want to see and read correctly formed e-mail without any corrupted symbols instead of filenames (that is the main problem here as far as filenames are not converted unlike log messages)

Alexey Shumkin wrote:

Thanks for clarifying. So in the context you describe, (1) is configurable, (2) is Windows-1251, (3) is unconfigurably UTF-8, and there is no way with current git facilities to force the email to use a single encoding unless (3) happens to contain no special characters.

What is the value of the "[i18n] commitEncoding" setting in your project? What encoding do the raw commit messages (shown with "git log --format=raw") use for their text, and what do they declare with an in-commit 'encoding' header, if any?

Does everyone on this project use Cygwin? That should be fine, but I'd expect there to be problems as soon as someone wants to try the Mingw port ("Git for Windows").

I wonder if there should be an "[i18n] repositoryPathEncoding" configuration item to support this kind of repository. Then git could be aware of the intended encoding of paths, could recode them for display to a terminal, and at least on Linux and Mingw could recode them for use in filenames on disk. "repositoryPathEncoding = none" would mean the current behavior of treating paths as raw sequences of bytes.

What do you think? Jonathan

On Sun, Aug 04, 2013 at 11:14:40AM -0700, Jonathan Nieder wrote:

commitEncoding is equal to filenames' encoding, Windows-1251, of course.

Well, despite `git log --help` 
--8<--
raw
           The raw format shows the entire commit exactly as stored in
           the commit object"
--8<--
on a Linux box (UTF-8) I can see "readable" commit messages nevertheless
they are stored in 'Windows-1251' (so they are converted to UTF-8). To
be sure I've checked actual content of them with `git cat-file commit`
Actually, to be honest, I usually use modified version of Git (see
ecaee8050cec23eb4cf082512e907e3e52c20b57) in 'next' branch, that could
affect the results, so I've checked `git log --format=raw` with
unmodified v1.8.3.3 of Git.

But let's go back to the answer to your question. Commit encoding stored as a header in a raw commit messages is 'Windows-1251'.

This is a "closed" (commercial) project and every developer uses Cygwin, except me. I use a Linux box as a desktop (mail, IM, web-browsing; but development goes on Cygwin). And sometimes I run utility scripts included to that project on my desktop (as far as Linux works with files much faster than Cygwin does ;)) Also, a Git server is a coLinux box (http://www.colinux.org/) on a Windows Server 2003, but I guess, it does not much matter here.

Yep, one of our developers tried to use modern version of TortoiseGit with MinGW port of Git. That was a failure. As far as since v1.7.9 MinGW port transcodes filenames to store them internally in UTF-8. This problem could be solved with converting once that non-ASCII filenames to UTF-8, but I do not want to use MinGW port. I like Cygwin "infrastructure" that is more Linux-like than MinGW.

I'd be happy if such a setting exists. That could solve many problems with cross-platform projects with non-ASCII filenames. Indeed, MinGW port does resolve that problem somehow!

From: Gerrit Pape <pape@smarden.org>
Date: Thu, 11 Dec 2008 20:27:21 +0000

Some email clients (e.g., claws-mail) display the message body incorrectly when the charset is not defined explicitly in a Content-Type header. "git log" generates logs in UTF-8 encoding by default, so add a Content-Type header declaring that encoding to the emails the post-receive-email example hook sends.

[jn: also setting the Content-Transfer-Encoding so MTAs know what
 kind of mangling might be needed when sending to a non 8-bit clean
 SMTP host]

Requested-by: Alexander Gerasiov <gq@debian.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
That's the end of the series.  Thanks for reading.

 contrib/hooks/post-receive-email | 3 +++
 1 file changed, 3 insertions(+)