{"thread":{"id":"29883","subject":"[PATCH/RFC 0/2] post-receive-email: declaring and consistently using one output encoding","startedAt":"2012-03-08T11:51:19Z","lastAt":"2012-03-14T19:55:09Z","messageCount":9,"participants":["Jonathan Nieder","Jeff King","Junio C Hamano","Jon Jensen","Alexey Shumkin"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"186421","messageId":"20120308115119.GA2604@burratino","threadId":"29883","inReplyTo":null,"subject":"[PATCH/RFC 0/2] post-receive-email: declaring and consistently using one output encoding","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-03-08T11:51:19Z","receivedAt":"2012-03-08T11:51:19Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\nThese patches revisit the bug described at [1], where the sample\npost-receive script annoys receiving MUAs by not declaring what\nencoding it uses.  Worse, sometimes the mails have a mixture of\nencodings.\n\nThese patches standardize on UTF-8, but that is only for the sake of\nsimplicity.  A patch on top to make the choice of encoding\ncustomizable would probably not be too complicated, if someone is\ninterested.\n\nPatches are targetted at 1.7.11 unless there is overwhelming\ninterest in them landing sooner.  Thanks to Alexander Gerasiov\n<gq@cs.msu.su> for the writing a patch long ago to get this\nstarted[1].\n\nThoughts?\n\nGerrit Pape (1):\n  bug#506445: hooks/post-receive-email: set encoding to utf-8\n\nJonathan Nieder (1):\n  post-receive-email: defend against non-utf8 [i18n] logoutputencoding\n    setting\n\n contrib/hooks/post-receive-email |   15 +++++++++------\n 1 file changed, 9 insertions(+), 6 deletions(-)\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/181737/focus=181755\n[2] http://bugs.debian.org/506445\n"},{"id":"186426","messageId":"20120308115754.GA2750@burratino","threadId":"29883","inReplyTo":"20120308115119.GA2604@burratino","subject":"[PATCH 1/2] hooks/post-receive-email: set encoding to utf-8","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-03-08T11:57:54Z","receivedAt":"2012-03-08T11:57:54Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"From: Gerrit Pape <pape@smarden.org>\n\n\"git log\" generates logs in UTF-8 encoding by default, but the\npost-receive-email example hook does not declare any encoding in\nthe emails it sends.  So add a line there:\n\n+\tContent-Type: text/plain; charset=utf-8\n\n[jn: tweaked to also set the Content-Transfer-Encoding so MTAs know\nwhat kind of mangling might be needed when sending to a non 8-bit\nclean SMTP host]\n\nRequested-by: Alexander Gerasiov <gq@cs.msu.su>\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n contrib/hooks/post-receive-email |    3 +++\n 1 file changed, 3 insertions(+)\n\ndiff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\nindex 01af9df1..dc184d0b 100755\n--- a/contrib/hooks/post-receive-email\n+++ b/contrib/hooks/post-receive-email\n@@ -233,6 +233,9 @@ generate_email_header()\n \tcat <<-EOF\n \tTo: $recipients\n \tSubject: ${emailprefix}$projectdesc $refname_type $short_refname ${change_type}d. $describe\n+\tMIME-Version: 1.0\n+\tContent-Type: text/plain; charset=utf-8\n+\tContent-Transfer-Encoding: 8bit\n \tX-Git-Refname: $refname\n \tX-Git-Reftype: $refname_type\n \tX-Git-Oldrev: $oldrev\n-- \n1.7.9.2\n"},{"id":"186427","messageId":"20120308115957.GB2750@burratino","threadId":"29883","inReplyTo":"20120308115119.GA2604@burratino","subject":"[PATCH 2/2] post-receive-email: defend against non UTF-8 i18n.logoutputencoding setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-03-08T11:59:57Z","receivedAt":"2012-03-08T11:59:57Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Johannes Sixt wrote:\n> Am 9/20/2011 12:42, schrieb Shumkin Alexey:\n\n>> 1. post-send-mail uses description file of a repo\n>> 2. gitweb also uses this file and AFAIK it assumes one to be in UTF-8\n>>   (I do not know whether it can be changed there but I tested gitweb once long\n>>     time ago)\n>> 3. So if i18n.logoutputencoding is not UTF-8 we get a message composed\n>> \twith mixed encodings. This fact oblidge us to encode headers\n>> \t(as quoted printable at least) and synchronize body message that contain\n>> \trepo description (in UTF-8) and diffstat (in i18n.logoutputencoding).\n[...]\n> In this case, it may make sense to have a separate setting, but you should\n> call git like this:\n>\n>    git -c \"i18n.logoutputencoding=$emailcharset\" show ...\n>    git -c \"i18n.logoutputencoding=$emailcharset\" rev-list --pretty ...\n\nSomething like this, I suppose?\n\nThis teaches post-receive-email to use plumbing where possible and to\nexplicitly declare what encoding it expects output to use.\n\nBased on an advice from Alexey Shumkin and Johannes Sixt, but all bugs\nare mine.  Making the email charset configurable is left as an\nexercise for the interested reader.\n\nSigned-off-by: Jonathan Nieder <jrnieder@gmail.com>\n---\n contrib/hooks/post-receive-email |   12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\nindex dc184d0b..b59e03cd 100755\n--- a/contrib/hooks/post-receive-email\n+++ b/contrib/hooks/post-receive-email\n@@ -464,7 +464,7 @@ generate_delete_branch_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -540,11 +540,11 @@ generate_atag_email()\n \t\t# performed on them\n \t\tif [ -n \"$prevtag\" ]; then\n \t\t\t# Show changes since the previous release\n-\t\t\tgit rev-list --pretty=short \"$prevtag..$newrev\" | git shortlog\n+\t\t\tgit shortlog --encoding=UTF-8 \"$prevtag..$newrev\"\n \t\telse\n \t\t\t# No previous tag, show all the changes since time\n \t\t\t# began\n-\t\t\tgit rev-list --pretty=short $newrev | git shortlog\n+\t\t\tgit shortlog --encoding=UTF-8 \"$newrev\"\n \t\tfi\n \t\t;;\n \t*)\n@@ -564,7 +564,7 @@ generate_delete_atag_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n@@ -610,7 +610,7 @@ generate_general_email()\n \techo \"\"\n \tif [ \"$newrev_type\" = \"commit\" ]; then\n \t\techo $LOGBEGIN\n-\t\tgit show --no-color --root -s --pretty=medium $newrev\n+\t\tgit diff-tree --encoding=UTF-8 --root -s --pretty=oneline $newrev\n \t\techo $LOGEND\n \telse\n \t\t# What can we do here?  The tag marks an object that is not\n@@ -629,7 +629,7 @@ generate_delete_general_email()\n \techo \"       was  $oldrev\"\n \techo \"\"\n \techo $LOGBEGIN\n-\tgit show -s --pretty=oneline $oldrev\n+\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n \techo $LOGEND\n }\n \n-- \n1.7.9.2\n"},{"id":"186448","messageId":"20120308135030.GB30602@sigill.intra.peff.net","threadId":"29883","inReplyTo":"20120308115957.GB2750@burratino","subject":"Re: [PATCH 2/2] post-receive-email: defend against non UTF-8 i18n.logoutputencoding setting","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2012-03-08T13:50:30Z","receivedAt":"2012-03-08T13:50:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Mar 08, 2012 at 05:59:57AM -0600, Jonathan Nieder wrote:\n\n> diff --git a/contrib/hooks/post-receive-email b/contrib/hooks/post-receive-email\n> index dc184d0b..b59e03cd 100755\n> --- a/contrib/hooks/post-receive-email\n> +++ b/contrib/hooks/post-receive-email\n> @@ -464,7 +464,7 @@ generate_delete_branch_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit show -s --pretty=oneline $oldrev\n> +\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n>  \techo $LOGEND\n\nIf you are using \"-s\" to suppress diff output, why are you using\ndiff-tree? Wouldn't \"rev-list -1\" (or \"rev-list --no-walk\") work equally\nwell and be a little more obvious?\n\n-Peff\n"},{"id":"186449","messageId":"20120308135237.GA14918@burratino","threadId":"29883","inReplyTo":"20120308135030.GB30602@sigill.intra.peff.net","subject":"Re: [PATCH 2/2] post-receive-email: defend against non UTF-8 i18n.logoutputencoding setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-03-08T13:52:37Z","receivedAt":"2012-03-08T13:52:37Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jeff King wrote:\n\n> If you are using \"-s\" to suppress diff output, why are you using\n> diff-tree? Wouldn't \"rev-list -1\" (or \"rev-list --no-walk\") work equally\n> well and be a little more obvious?\n\nJust a habit --- some part of me is wired to think \"diff-tree is the\ncommand to show a commit\".  Maybe rev-list didn't support --pretty in\nolden days?\n"},{"id":"186478","messageId":"7v62eeewl8.fsf@alter.siamese.dyndns.org","threadId":"29883","inReplyTo":"20120308115119.GA2604@burratino","subject":"Re: [PATCH/RFC 0/2] post-receive-email: declaring and consistently using one output encoding","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-03-08T19:12:03Z","receivedAt":"2012-03-08T19:12:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jonathan Nieder <jrnieder@gmail.com> writes:\n\n> Patches are targetted at 1.7.11 unless there is overwhelming\n> interest in them landing sooner.\n\nEven if there is none, I wouldn't mind taking it for 1.7.10, if only\nto reduce the chance that the patch gets forgotten again, provided\nthat there is a clear concensus that this is a good change that won't\nregress people who have been actually using this sample in real life.\n\n> <gq@cs.msu.su> for the writing a patch long ago to get this\n> started[1].\n>\n> Thoughts?\n>\n> Gerrit Pape (1):\n>   bug#506445: hooks/post-receive-email: set encoding to utf-8\n>\n> Jonathan Nieder (1):\n>   post-receive-email: defend against non-utf8 [i18n] logoutputencoding\n>     setting\n>\n>  contrib/hooks/post-receive-email |   15 +++++++++------\n>  1 file changed, 9 insertions(+), 6 deletions(-)\n>\n> [1] http://thread.gmane.org/gmane.comp.version-control.git/181737/focus=181755\n> [2] http://bugs.debian.org/506445\n"},{"id":"186973","messageId":"alpine.LFD.2.02.1203141153460.4355@fnzzl","threadId":"29883","inReplyTo":"7v62eeewl8.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH/RFC 0/2] post-receive-email: declaring and consistently using one output encoding","fromName":"Jon Jensen","fromEmail":"jon@endpoint.com","sentAt":"2012-03-14T17:54:50Z","receivedAt":"2012-03-14T17:54:50Z","isPatch":true,"sender":{"key":"jon@endpoint.com","avatar":"https://avatars.githubusercontent.com/u/3811?v=4"},"body":"On Thu, 8 Mar 2012, Junio C Hamano wrote:\n\n>> Patches are targetted at 1.7.11 unless there is overwhelming interest \n>> in them landing sooner.\n>\n> Even if there is none, I wouldn't mind taking it for 1.7.10, if only to \n> reduce the chance that the patch gets forgotten again, provided that \n> there is a clear concensus that this is a good change that won't regress \n> people who have been actually using this sample in real life.\n\n+1 from me. This would solve an occasional problem we have because \nencodings aren't set in post-receive-email output.\n\nJon\n"},{"id":"186987","messageId":"20120314233609.04a2a37b@zappedws","threadId":"29883","inReplyTo":"20120308115957.GB2750@burratino","subject":"Re: [PATCH 2/2] post-receive-email: defend against non UTF-8 i18n.logoutputencoding setting","fromName":"Alexey Shumkin","fromEmail":"zapped@mail.ru","sentAt":"2012-03-14T19:36:09Z","receivedAt":"2012-03-14T19:36:09Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"I'd like to remind then following aspect\n> My proposition was in to send email message in explicitly defined\n> custom encoding. Why? In development process under Windows non-UTF-8\n> encoding is used (cp1251 in my case). So, filenames have this\n> encoding, and as we know Git stores their names as is - in cp1251 -\n> without a conversion. And filenames are also used in diff-stat (with\n> core.quotepath= false, BTW, I did not take into account this config)\n> without any conversion. So if we'll make all text in UTF-8 but\n> filenames are still non-UTF-8, email would look corrupted.\nthis is citation from \nhttp://thread.gmane.org/gmane.comp.version-control.git/181737/focus=183076\n\n> Johannes Sixt wrote:\n> > Am 9/20/2011 12:42, schrieb Shumkin Alexey:\n> \n> >> 1. post-send-mail uses description file of a repo\n> >> 2. gitweb also uses this file and AFAIK it assumes one to be in\n> >> UTF-8 (I do not know whether it can be changed there but I tested\n> >> gitweb once long time ago)\n> >> 3. So if i18n.logoutputencoding is not UTF-8 we get a message\n> >> composed with mixed encodings. This fact oblidge us to encode\n> >> headers (as quoted printable at least) and synchronize body\n> >> message that contain repo description (in UTF-8) and diffstat (in\n> >> i18n.logoutputencoding).\n> [...]\n> > In this case, it may make sense to have a separate setting, but you\n> > should call git like this:\n> >\n> >    git -c \"i18n.logoutputencoding=$emailcharset\" show ...\n> >    git -c \"i18n.logoutputencoding=$emailcharset\" rev-list\n> > --pretty ...\n> \n> Something like this, I suppose?\n> \n> This teaches post-receive-email to use plumbing where possible and to\n> explicitly declare what encoding it expects output to use.\n> \n> Based on an advice from Alexey Shumkin and Johannes Sixt, but all bugs\n> are mine.  Making the email charset configurable is left as an\n> exercise for the interested reader.\n> \n> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>\n> ---\n>  contrib/hooks/post-receive-email |   12 ++++++------\n>  1 file changed, 6 insertions(+), 6 deletions(-)\n> \n> diff --git a/contrib/hooks/post-receive-email\n> b/contrib/hooks/post-receive-email index dc184d0b..b59e03cd 100755\n> --- a/contrib/hooks/post-receive-email\n> +++ b/contrib/hooks/post-receive-email\n> @@ -464,7 +464,7 @@ generate_delete_branch_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit show -s --pretty=oneline $oldrev\n> +\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n> @@ -540,11 +540,11 @@ generate_atag_email()\n>  \t\t# performed on them\n>  \t\tif [ -n \"$prevtag\" ]; then\n>  \t\t\t# Show changes since the previous release\n> -\t\t\tgit rev-list --pretty=short\n> \"$prevtag..$newrev\" | git shortlog\n> +\t\t\tgit shortlog --encoding=UTF-8\n> \"$prevtag..$newrev\" else\n>  \t\t\t# No previous tag, show all the changes\n> since time # began\n> -\t\t\tgit rev-list --pretty=short $newrev | git\n> shortlog\n> +\t\t\tgit shortlog --encoding=UTF-8 \"$newrev\"\n>  \t\tfi\n>  \t\t;;\n>  \t*)\n> @@ -564,7 +564,7 @@ generate_delete_atag_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit show -s --pretty=oneline $oldrev\n> +\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n> @@ -610,7 +610,7 @@ generate_general_email()\n>  \techo \"\"\n>  \tif [ \"$newrev_type\" = \"commit\" ]; then\n>  \t\techo $LOGBEGIN\n> -\t\tgit show --no-color --root -s --pretty=medium $newrev\n> +\t\tgit diff-tree --encoding=UTF-8 --root -s\n> --pretty=oneline $newrev echo $LOGEND\n>  \telse\n>  \t\t# What can we do here?  The tag marks an object that\n> is not @@ -629,7 +629,7 @@ generate_delete_general_email()\n>  \techo \"       was  $oldrev\"\n>  \techo \"\"\n>  \techo $LOGBEGIN\n> -\tgit show -s --pretty=oneline $oldrev\n> +\tgit diff-tree --encoding=UTF-8 -s --pretty=oneline $oldrev\n>  \techo $LOGEND\n>  }\n>  \n"},{"id":"186989","messageId":"20120314195509.GA31686@burratino","threadId":"29883","inReplyTo":"20120314233609.04a2a37b@zappedws","subject":"Re: [PATCH 2/2] post-receive-email: defend against non UTF-8 i18n.logoutputencoding setting","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-03-14T19:55:09Z","receivedAt":"2012-03-14T19:55:09Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Alexey Shumkin wrote:\n\n> I'd like to remind then following aspect\n>\n>> My proposition was in to send email message in explicitly defined\n>> custom encoding. Why? In development process under Windows non-UTF-8\n>> encoding is used (cp1251 in my case). So, filenames have this\n>> encoding, and as we know Git stores their names as is\n[...]\n>>                                                                 with\n>> core.quotepath= false\n\nSure.  Do you think this patch makes that problem worse, and if so, do\nyou have any ideas about how that could be prevented?  Otherwise:\n\n>>            Making the email charset configurable is left as an\n>> exercise for the interested reader.\n\nI did not want to do that part because I do not trust myself to\nunderstand the needs of people using non-utf8 and test it\nappropriately, but I tried to make sure the patch was structured in a\nway that would make it easy.\n\nHoping clarifies a little,\nJonathan\n"}]}