{"thread":{"id":"34592","subject":"[PATCH/RFC 0/3] post-receive-email: explicitly set Content-Type header","startedAt":"2013-08-02T23:21:18Z","lastAt":"2013-08-05T08:45:43Z","messageCount":7,"participants":["Jonathan Nieder","Alexey Shumkin"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"224470","messageId":"20130802232118.GB2963@elie.Belkin","threadId":"34592","inReplyTo":null,"subject":"[PATCH/RFC 0/3] post-receive-email: explicitly set Content-Type header","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-08-02T23:21:18Z","receivedAt":"2013-08-02T23:21:18Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi all,\n\nThis is a revival of [1], which declares encoding in emails to make it\nmore likely that they can be read.  I like to think it avoids the\nmistakes of previous attempts, but I'll let you judge. :)\n\nSorry for the long delay.  Thoughts of all kinds welcome, as always.\n\nThanks,\nJonathan\n\nGerrit Pape (1):\n  hooks/post-receive-email: set declared encoding to utf-8\n\nJonathan Nieder (2):\n  hooks/post-receive-email: use plumbing instead of 'git log' and 'git show'\n  hooks/post-receive-email: force log messages in UTF-8\n\n contrib/hooks/post-receive-email | 15 +++++++++------\n 1 file changed, 9 insertions(+), 6 deletions(-)\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/181737/focus=183070\n"},{"id":"224471","messageId":"20130802232208.GC2963@elie.Belkin","threadId":"34592","inReplyTo":"20130802232118.GB2963@elie.Belkin","subject":"[PATCH 1/3] hooks/post-receive-email: use plumbing instead of git log/show","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-08-02T23:22:08Z","receivedAt":"2013-08-02T23:22:08Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"This way the hook doesn't have to keep being tweaked as porcelain\nlearns new features like color and pagination.\n\nWhile at it, replace the \"git rev-list | git shortlog\" idiom with\nplain \"git shortlog\" for simplicity.\n\nExcept for depending less on the value of settings like '[log]\nabbrevCommit', no change in output intended.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n contrib/hooks/post-receive-email | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\nindex 15311502..72084511 100755\n--- a/contrib/hooks/post-receive-email\n+++ b/contrib/hooks/post-receive-email\n@@ -471,7 +471,7 @@ generate_delete_branch_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -547,11 +547,11 @@ generate_atag_email()\n \t\t# performed on them\n \t\tif [ -n \"$prevtag\" ]; then\n \t\t\t# Show changes since the previous release\n-\t\t\tgit rev-list --pretty=short \"$prevtag..$newrev\" | git shortlog\n+\t\t\tgit shortlog \"$prevtag..$newrev\"\n \t\telse\n \t\t\t# No previous tag, show all the changes since time\n \t\t\t# began\n-\t\t\tgit rev-list --pretty=short $newrev | git shortlog\n+\t\t\tgit shortlog $newrev\n \t\tfi\n \t\t;;\n \t*)\n@@ -571,7 +571,7 @@ generate_delete_atag_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -617,7 +617,7 @@ generate_general_email()\n \techo \"\"\n \tif [ \"$newrev_type\" = \"commit\" ]; then\n \t\techo $LOGBEGIN\n-\t\tgit show --no-color --root -s --pretty=medium $newrev\n+\t\tgit diff-tree -s --always --pretty=medium $newrev\n \t\techo $LOGEND\n \telse\n \t\t# What can we do here?  The tag marks an object that is not\n@@ -636,7 +636,7 @@ generate_delete_general_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n-- \n1.8.4.rc1\n"},{"id":"224472","messageId":"20130802232338.GD2963@elie.Belkin","threadId":"34592","inReplyTo":"20130802232118.GB2963@elie.Belkin","subject":"[PATCH 2/3] hooks/post-receive-email: force log messages in UTF-8","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-08-02T23:23:38Z","receivedAt":"2013-08-02T23:23:38Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Git commands write commit messages in UTF-8 by default, but that\ndefault can be overridden by the [i18n] commitEncoding and\nlogOutputEncoding settings.  With such a setting, the emails written\nby the post-receive-email hook use a mixture of encodings:\n\n 1. Log messages use the configured log output encoding, which is\n    meant to be whatever encoding works best with local terminals\n    (and does not have much to do with what encoding should be used\n    for email)\n\n 2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw\n    port (which uses Unicode filesystem APIs), always UTF-8\n\n 3. The \"This is an automated email\" preface uses a project description\n    from .git/description, which is typically in UTF-8 to support\n    gitweb.\n\nSo (1) is configurable, and (2) and (3) are unconfigurable and\ntypically UTF-8.  Override the log output encoding to always use UTF-8\nwhen writing the email to get the best chance of a comprehensible\nsingle-encoding email.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n contrib/hooks/post-receive-email | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\nindex 72084511..ba93a0d8 100755\n--- a/contrib/hooks/post-receive-email\n+++ b/contrib/hooks/post-receive-email\n@@ -471,7 +471,7 @@ generate_delete_branch_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit diff-tree -s --always --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -571,7 +571,7 @@ generate_delete_atag_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit diff-tree -s --always --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -617,7 +617,7 @@ generate_general_email()\n \techo \"\"\n \tif [ \"$newrev_type\" = \"commit\" ]; then\n \t\techo $LOGBEGIN\n-\t\tgit diff-tree -s --always --pretty=medium $newrev\n+\t\tgit diff-tree -s --always --encoding=UTF-8 --pretty=medium $newrev\n \t\techo $LOGEND\n \telse\n \t\t# What can we do here?  The tag marks an object that is not\n@@ -636,7 +636,7 @@ generate_delete_general_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit diff-tree -s --always --pretty=oneline $oldrev\n+\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n-- \n1.8.4.rc1\n"},{"id":"224473","messageId":"20130802232417.GE2963@elie.Belkin","threadId":"34592","inReplyTo":"20130802232118.GB2963@elie.Belkin","subject":"[PATCH 3/3] hooks/post-receive-email: set declared encoding to utf-8","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-08-02T23:24:17Z","receivedAt":"2013-08-02T23:24:17Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Gerrit Pape <pape@smarden.org>\nDate: Thu, 11 Dec 2008 20:27:21 +0000\n\nSome email clients (e.g., claws-mail) display the message body\nincorrectly when the charset is not defined explicitly in a\nContent-Type header.  \"git log\" generates logs in UTF-8 encoding by\ndefault, so add a Content-Type header declaring that encoding to\nthe emails the post-receive-email example hook sends.\n\n[jn: also setting the Content-Transfer-Encoding so MTAs know what\n kind of mangling might be needed when sending to a non 8-bit clean\n SMTP host]\n\nRequested-by: Alexander Gerasiov <gq@debian.org>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\nThat's the end of the series.  Thanks for reading.\n\n contrib/hooks/post-receive-email | 3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\nindex ba93a0d8..8ee410f8 100755\n--- a/contrib/hooks/post-receive-email\n+++ b/contrib/hooks/post-receive-email\n@@ -242,6 +242,9 @@ generate_email_header()\n \tcat <<-EOF\n \tTo: $recipients\n \tSubject: ${emailprefix}$projectdesc $refname_type $short_refname ${change_type}d. $describe\n+\tMIME-Version: 1.0\n+\tContent-Type: text/plain; charset=utf-8\n+\tContent-Transfer-Encoding: 8bit\n \tX-Git-Refname: $refname\n \tX-Git-Reftype: $refname_type\n \tX-Git-Oldrev: $oldrev\n-- \n1.8.4.rc1\n"},{"id":"224558","messageId":"20130804145448.GA15097@dell-note","threadId":"34592","inReplyTo":"20130802232338.GD2963@elie.Belkin","subject":"Re: [PATCH 2/3] hooks/post-receive-email: force log messages in UTF-8","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-08-04T14:54:48Z","receivedAt":"2013-08-04T14:54:48Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"On Fri, Aug 02, 2013 at 04:23:38PM -0700, Jonathan Nieder wrote:\n> Git commands write commit messages in UTF-8 by default, but that\n> default can be overridden by the [i18n] commitEncoding and\n> logOutputEncoding settings.  With such a setting, the emails written\n> by the post-receive-email hook use a mixture of encodings:\n> \n>  1. Log messages use the configured log output encoding, which is\n>     meant to be whatever encoding works best with local terminals\n>     (and does not have much to do with what encoding should be used\n>     for email)\n> \n>  2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw\n>     port (which uses Unicode filesystem APIs), always UTF-8\nI cannot say exactly if it makes sense for THIS patch, but I'd like to\nremind about Cygwin port, which definitely does not use UTF-8 encoding\n(in my case it is Windows-1251) for filenames.\n> \n>  3. The \"This is an automated email\" preface uses a project description\n>     from .git/description, which is typically in UTF-8 to support\n>     gitweb.\n> \n> So (1) is configurable, and (2) and (3) are unconfigurable and\n> typically UTF-8.  Override the log output encoding to always use UTF-8\n> when writing the email to get the best chance of a comprehensible\n> single-encoding email.\nI cannot agree to receive e-mails in UTF-8 only for Windows projects\nwhich have non-UTF-8 encoding. I want to see and read correctly formed\ne-mail without any corrupted symbols instead of filenames (that is the\nmain problem here as far as filenames are not converted unlike log\nmessages)\n> \n> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>\n> ---\n>  contrib/hooks/post-receive-email | 8 ++++----\n>  1 file changed, 4 insertions(+), 4 deletions(-)\n> \n> diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\n> index 72084511..ba93a0d8 100755\n> --- a/contrib/hooks/post-receive-email\n> +++ b/contrib/hooks/post-receive-email\n> @@ -471,7 +471,7 @@ generate_delete_branch_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit diff-tree -s --always --pretty=oneline $oldrev\n> +\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n> @@ -571,7 +571,7 @@ generate_delete_atag_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit diff-tree -s --always --pretty=oneline $oldrev\n> +\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n> @@ -617,7 +617,7 @@ generate_general_email()\n>  \techo \"\"\n>  \tif [ \"$newrev_type\" = \"commit\" ]; then\n>  \t\techo $LOGBEGIN\n> -\t\tgit diff-tree -s --always --pretty=medium $newrev\n> +\t\tgit diff-tree -s --always --encoding=UTF-8 --pretty=medium $newrev\n>  \t\techo $LOGEND\n>  \telse\n>  \t\t# What can we do here?  The tag marks an object that is not\n> @@ -636,7 +636,7 @@ generate_delete_general_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit diff-tree -s --always --pretty=oneline $oldrev\n> +\tgit diff-tree -s --always --encoding=UTF-8 --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n> -- \n> 1.8.4.rc1\n> \n\n-- \nAlexey Shumkin\n"},{"id":"224562","messageId":"20130804181440.GA2894@elie.Belkin","threadId":"34592","inReplyTo":"20130804145448.GA15097@dell-note","subject":"Re: [PATCH 2/3] hooks/post-receive-email: force log messages in UTF-8","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-08-04T18:14:40Z","receivedAt":"2013-08-04T18:14:40Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Alexey Shumkin wrote:\n> On Fri, Aug 02, 2013 at 04:23:38PM -0700, Jonathan Nieder wrote:\n\n>>  1. Log messages use the configured log output encoding, which is\n>>     meant to be whatever encoding works best with local terminals\n>>     (and does not have much to do with what encoding should be used\n>>     for email)\n>>\n>>  2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw\n>>     port (which uses Unicode filesystem APIs), always UTF-8\n>\n> I cannot say exactly if it makes sense for THIS patch, but I'd like to\n> remind about Cygwin port, which definitely does not use UTF-8 encoding\n> (in my case it is Windows-1251) for filenames.\n>\n>> \n>>  3. The \"This is an automated email\" preface uses a project description\n>>     from .git/description, which is typically in UTF-8 to support\n>>     gitweb.\n\nThanks for clarifying.  So in the context you describe, (1) is\nconfigurable, (2) is Windows-1251, (3) is unconfigurably UTF-8, and\nthere is no way with current git facilities to force the email to use\na single encoding unless (3) happens to contain no special characters.\n\nWhat is the value of the \"[i18n] commitEncoding\" setting in your\nproject?  What encoding do the raw commit messages (shown with\n\"git log --format=raw\") use for their text, and what do they declare\nwith an in-commit 'encoding' header, if any?\n\nDoes everyone on this project use Cygwin?  That should be fine, but\nI'd expect there to be problems as soon as someone wants to try the\nMingw port (\"Git for Windows\").\n\nI wonder if there should be an \"[i18n] repositoryPathEncoding\"\nconfiguration item to support this kind of repository.  Then git could\nbe aware of the intended encoding of paths, could recode them for\ndisplay to a terminal, and at least on Linux and Mingw could recode\nthem for use in filenames on disk.  \"repositoryPathEncoding = none\"\nwould mean the current behavior of treating paths as raw sequences of\nbytes.\n\nWhat do you think?\nJonathan\n"},{"id":"224566","messageId":"20130805084543.GA11270@ashu.dyn1.rarus.ru","threadId":"34592","inReplyTo":"20130804181440.GA2894@elie.Belkin","subject":"Re: [PATCH 2/3] hooks/post-receive-email: force log messages in UTF-8","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-08-05T08:45:43Z","receivedAt":"2013-08-05T08:45:43Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"On Sun, Aug 04, 2013 at 11:14:40AM -0700, Jonathan Nieder wrote:\n> Alexey Shumkin wrote:\n> > On Fri, Aug 02, 2013 at 04:23:38PM -0700, Jonathan Nieder wrote:\n> \n> >>  1. Log messages use the configured log output encoding, which is\n> >>     meant to be whatever encoding works best with local terminals\n> >>     (and does not have much to do with what encoding should be used\n> >>     for email)\n> >>\n> >>  2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw\n> >>     port (which uses Unicode filesystem APIs), always UTF-8\n> >\n> > I cannot say exactly if it makes sense for THIS patch, but I'd like to\n> > remind about Cygwin port, which definitely does not use UTF-8 encoding\n> > (in my case it is Windows-1251) for filenames.\n> >\n> >> \n> >>  3. The \"This is an automated email\" preface uses a project description\n> >>     from .git/description, which is typically in UTF-8 to support\n> >>     gitweb.\n> \n> Thanks for clarifying.  So in the context you describe, (1) is\n> configurable, (2) is Windows-1251, (3) is unconfigurably UTF-8, and\n> there is no way with current git facilities to force the email to use\n> a single encoding unless (3) happens to contain no special characters.\n> \n> What is the value of the \"[i18n] commitEncoding\" setting in your\n> project?\ncommitEncoding is equal to filenames' encoding, Windows-1251, of course.\n\n> What encoding do the raw commit messages (shown with\n> \"git log --format=raw\") use for their text, and what do they declare\n> with an in-commit 'encoding' header, if any?\nWell, despite `git log --help` \n--8<--\nraw\n           The raw format shows the entire commit exactly as stored in\n           the commit object\"\n--8<--\non a Linux box (UTF-8) I can see \"readable\" commit messages nevertheless\nthey are stored in 'Windows-1251' (so they are converted to UTF-8). To\nbe sure I've checked actual content of them with `git cat-file commit`\nActually, to be honest, I usually use modified version of Git (see\necaee8050cec23eb4cf082512e907e3e52c20b57) in 'next' branch, that could\naffect the results, so I've checked `git log --format=raw` with\nunmodified v1.8.3.3 of Git.\n\nBut let's go back to the answer to your question. Commit encoding stored\nas a header in a raw commit messages is 'Windows-1251'.\n> \n> Does everyone on this project use Cygwin?i\nThis is a \"closed\" (commercial) project and every developer uses Cygwin,\nexcept me. I use a Linux box as a desktop (mail, IM, web-browsing; but\ndevelopment goes on Cygwin). And sometimes I run utility scripts\nincluded to that project on my desktop (as far as Linux works with files\nmuch faster than Cygwin does ;))\nAlso, a Git server is a coLinux box (http://www.colinux.org/) on a\nWindows Server 2003, but I guess, it does not much matter here.\n>  That should be fine, but\n> I'd expect there to be problems as soon as someone wants to try the\n> Mingw port (\"Git for Windows\").\nYep, one of our developers tried to use modern version of TortoiseGit\nwith MinGW port of Git. That was a failure. As far as since v1.7.9 MinGW\nport transcodes filenames to store them internally in UTF-8. This\nproblem could be solved with converting once that non-ASCII filenames to\nUTF-8, but I do not want to use MinGW port. I like Cygwin\n\"infrastructure\" that is more Linux-like than MinGW.\n> \n> I wonder if there should be an \"[i18n] repositoryPathEncoding\"\n> configuration item to support this kind of repository.  Then git could\n> be aware of the intended encoding of paths, could recode them for\n> display to a terminal, and at least on Linux and Mingw could recode\n> them for use in filenames on disk.  \"repositoryPathEncoding = none\"\n> would mean the current behavior of treating paths as raw sequences of\n> bytes.\nI'd be happy if such a setting exists. That could solve many problems\nwith cross-platform projects with non-ASCII filenames.\nIndeed, MinGW port does resolve that problem somehow!\n> \n> What do you think?\n> Jonathan\n\n-- \nAlexey Shumkin\n"}]}