{"thread":{"id":"34337","subject":"[PATCH] t4205: don't rely on en_US.UTF-8 locale existing","startedAt":"2013-07-03T20:18:08Z","lastAt":"2013-07-03T22:27:29Z","messageCount":7,"participants":["John Keeping","Alexey Shumkin","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"222494","messageId":"f607decdc65b86b1759438e375ddf77fd5b91042.1372882590.git.john@keeping.me.uk","threadId":"34337","inReplyTo":null,"subject":"[PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-07-03T20:18:08Z","receivedAt":"2013-07-03T20:18:08Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\ncauses t4205 to fail by counting bytes instead of UTF-8 codepoints.\n\nInstead of using sed for this, use Perl which behaves predictably\nwhatever locale is in use.\n\nSigned-off-by: John Keeping <john@keeping.me.uk>\n---\nThis patch is on top of 'as/log-output-encoding-in-user-format'.\n\n t/t4205-log-pretty-formats.sh | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\nindex 3cfb744..5864f5b 100755\n--- a/t/t4205-log-pretty-formats.sh\n+++ b/t/t4205-log-pretty-formats.sh\n@@ -20,9 +20,7 @@ commit_msg () {\n \t\t# cut string, replace cut part with two dots\n \t\t# $2 - chars count from the beginning of the string\n \t\t# $3 - \"trailing\" chars\n-\t\t# LC_ALL is set to make `sed` interpret \".\" as a UTF-8 char not a byte\n-\t\t# as it does with C locale\n-\t\tmsg=$(echo $msg | LC_ALL=en_US.UTF-8 sed -e \"s/^\\(.\\{$2\\}\\)$3/\\1../\")\n+\t\tmsg=$(echo $msg | \"$PERL_PATH\" -CIO -pe \"s/^(.{$2})$3/\\1../\")\n \tfi\n \techo $msg\n }\n@@ -205,7 +203,7 @@ test_expect_success 'left alignment formatting with ltrunc' \"\n ..sage two\n ..sage one\n add bar  Z\n-$(commit_msg \"\" \"0\" \".\\{11\\}\")\n+$(commit_msg \"\" \"0\" \".{11}\")\n EOF\n \ttest_cmp expected actual\n \"\n@@ -218,7 +216,7 @@ test_expect_success 'left alignment formatting with mtrunc' \"\n mess.. two\n mess.. one\n add bar  Z\n-$(commit_msg \"\" \"4\" \".\\{11\\}\")\n+$(commit_msg \"\" \"4\" \".{11}\")\n EOF\n \ttest_cmp expected actual\n \"\n-- \n1.8.3.1.747.g77f7d3a\n"},{"id":"222500","messageId":"20130703204024.GA6148@dell-note","threadId":"34337","inReplyTo":"f607decdc65b86b1759438e375ddf77fd5b91042.1372882590.git.john@keeping.me.uk","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-07-03T20:40:24Z","receivedAt":"2013-07-03T20:40:24Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"CC this to Johannes Sixt\n\nOn Wed, Jul 03, 2013 at 09:18:08PM +0100, John Keeping wrote:\n> My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n> \n> Instead of using sed for this, use Perl which behaves predictably\n> whatever locale is in use.\n> \n> Signed-off-by: John Keeping <john@keeping.me.uk>\n> ---\n> This patch is on top of 'as/log-output-encoding-in-user-format'.\n> \n>  t/t4205-log-pretty-formats.sh | 8 +++-----\n>  1 file changed, 3 insertions(+), 5 deletions(-)\n> \n> diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> index 3cfb744..5864f5b 100755\n> --- a/t/t4205-log-pretty-formats.sh\n> +++ b/t/t4205-log-pretty-formats.sh\n> @@ -20,9 +20,7 @@ commit_msg () {\n>  \t\t# cut string, replace cut part with two dots\n>  \t\t# $2 - chars count from the beginning of the string\n>  \t\t# $3 - \"trailing\" chars\n> -\t\t# LC_ALL is set to make `sed` interpret \".\" as a UTF-8 char not a byte\n> -\t\t# as it does with C locale\n> -\t\tmsg=$(echo $msg | LC_ALL=en_US.UTF-8 sed -e \"s/^\\(.\\{$2\\}\\)$3/\\1../\")\n> +\t\tmsg=$(echo $msg | \"$PERL_PATH\" -CIO -pe \"s/^(.{$2})$3/\\1../\")\n>  \tfi\n>  \techo $msg\n>  }\n> @@ -205,7 +203,7 @@ test_expect_success 'left alignment formatting with ltrunc' \"\n>  ..sage two\n>  ..sage one\n>  add bar  Z\n> -$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> +$(commit_msg \"\" \"0\" \".{11}\")\n>  EOF\n>  \ttest_cmp expected actual\n>  \"\n> @@ -218,7 +216,7 @@ test_expect_success 'left alignment formatting with mtrunc' \"\n>  mess.. two\n>  mess.. one\n>  add bar  Z\n> -$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> +$(commit_msg \"\" \"4\" \".{11}\")\n>  EOF\n>  \ttest_cmp expected actual\n>  \"\n> -- \n> 1.8.3.1.747.g77f7d3a\n> \n"},{"id":"222501","messageId":"20130703204702.GB6148@dell-note","threadId":"34337","inReplyTo":"20130703204024.GA6148@dell-note","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-07-03T20:47:02Z","receivedAt":"2013-07-03T20:47:02Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"http://thread.gmane.org/gmane.comp.version-control.git/229291\n\nthis is why CCed\n> CC this to Johannes Sixt\n> \n> On Wed, Jul 03, 2013 at 09:18:08PM +0100, John Keeping wrote:\n> > My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> > causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n> > \n> > Instead of using sed for this, use Perl which behaves predictably\n> > whatever locale is in use.\n> > \n> > Signed-off-by: John Keeping <john@keeping.me.uk>\n> > ---\n> > This patch is on top of 'as/log-output-encoding-in-user-format'.\n> > \n> >  t/t4205-log-pretty-formats.sh | 8 +++-----\n> >  1 file changed, 3 insertions(+), 5 deletions(-)\n> > \n> > diff --git a/t/t4205-log-pretty-formats.sh b/t/t4205-log-pretty-formats.sh\n> > index 3cfb744..5864f5b 100755\n> > --- a/t/t4205-log-pretty-formats.sh\n> > +++ b/t/t4205-log-pretty-formats.sh\n> > @@ -20,9 +20,7 @@ commit_msg () {\n> >  \t\t# cut string, replace cut part with two dots\n> >  \t\t# $2 - chars count from the beginning of the string\n> >  \t\t# $3 - \"trailing\" chars\n> > -\t\t# LC_ALL is set to make `sed` interpret \".\" as a UTF-8 char not a byte\n> > -\t\t# as it does with C locale\n> > -\t\tmsg=$(echo $msg | LC_ALL=en_US.UTF-8 sed -e \"s/^\\(.\\{$2\\}\\)$3/\\1../\")\n> > +\t\tmsg=$(echo $msg | \"$PERL_PATH\" -CIO -pe \"s/^(.{$2})$3/\\1../\")\n> >  \tfi\n> >  \techo $msg\n> >  }\n> > @@ -205,7 +203,7 @@ test_expect_success 'left alignment formatting with ltrunc' \"\n> >  ..sage two\n> >  ..sage one\n> >  add bar  Z\n> > -$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> > +$(commit_msg \"\" \"0\" \".{11}\")\n> >  EOF\n> >  \ttest_cmp expected actual\n> >  \"\n> > @@ -218,7 +216,7 @@ test_expect_success 'left alignment formatting with mtrunc' \"\n> >  mess.. two\n> >  mess.. one\n> >  add bar  Z\n> > -$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> > +$(commit_msg \"\" \"4\" \".{11}\")\n> >  EOF\n> >  \ttest_cmp expected actual\n> >  \"\n> > -- \n> > 1.8.3.1.747.g77f7d3a\n> > \n"},{"id":"222507","messageId":"7vr4ffcoel.fsf@alter.siamese.dyndns.org","threadId":"34337","inReplyTo":"f607decdc65b86b1759438e375ddf77fd5b91042.1372882590.git.john@keeping.me.uk","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-07-03T21:41:06Z","receivedAt":"2013-07-03T21:41:06Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"John Keeping <john@keeping.me.uk> writes:\n\n> My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n>\n> Instead of using sed for this, use Perl which behaves predictably\n> whatever locale is in use.\n>\n> Signed-off-by: John Keeping <john@keeping.me.uk>\n> ---\n> This patch is on top of 'as/log-output-encoding-in-user-format'.\n\nThanks.  I think Alexey is going to send incremental updates to the\ntopic so I won't interfere by applying this patch on top of the\nversion I have in my tree.\n\nBut I do agree that using Perl may be a workable solution.\n\nAn alternative might be not to use this cryptic 3-arg form of\ncommit_msg at all.  They are used only for these three:\n\n\t$(commit_msg \"\" \"8\" \"..*$\")\n\t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n\t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n\nI somehow find them simply not readable, in order to figure out what\nis going on.\n\nJust using three variables to hold what are expected would be far\nmore portable and readable.\n\n# \"anfänglich\" whatever it means.\nsample_utf8_part=$(printf \"anf\\303\\244ng\")\n\ncommit_msg () {\n\tmsg=\"initial. ${sample_utf8_part}lich\";\n\tif test -n \"$1\"\n\tthen\n\t\techo \"$msg\" | iconv -f utf-8 -t \"$1\"\n\telse\n\t\techo \"$msg\"\n        fi\n}\n\nAnd then instead of writing in the expected test output.\n\n\t$(commit_msg \"\" \"8\" \"..*$\")\n\t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n\t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n\nwe can just say\n\n\tinitial...\n        ..an${sample_utf8_part}lich\n\tinit..lich\n\nIt is no worse than those cryptic 0, 4, 8 and 11 magic numbers we\nsee in the test, no?\n"},{"id":"222509","messageId":"20130703215303.GG9161@serenity.lan","threadId":"34337","inReplyTo":"7vr4ffcoel.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"John Keeping","fromEmail":"john@keeping.me.uk","sentAt":"2013-07-03T21:53:03Z","receivedAt":"2013-07-03T21:53:03Z","isPatch":true,"sender":{"key":"john@keeping.me.uk","avatar":"https://avatars.githubusercontent.com/u/1702081?v=4"},"body":"On Wed, Jul 03, 2013 at 02:41:06PM -0700, Junio C Hamano wrote:\n> John Keeping <john@keeping.me.uk> writes:\n> \n> > My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> > causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n> >\n> > Instead of using sed for this, use Perl which behaves predictably\n> > whatever locale is in use.\n> >\n> > Signed-off-by: John Keeping <john@keeping.me.uk>\n> > ---\n> > This patch is on top of 'as/log-output-encoding-in-user-format'.\n> \n> Thanks.  I think Alexey is going to send incremental updates to the\n> topic so I won't interfere by applying this patch on top of the\n> version I have in my tree.\n> \n> But I do agree that using Perl may be a workable solution.\n> \n> An alternative might be not to use this cryptic 3-arg form of\n> commit_msg at all.  They are used only for these three:\n> \n> \t$(commit_msg \"\" \"8\" \"..*$\")\n> \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> \n> I somehow find them simply not readable, in order to figure out what\n> is going on.\n> \n> Just using three variables to hold what are expected would be far\n> more portable and readable.\n> \n> # \"anfänglich\" whatever it means.\n> sample_utf8_part=$(printf \"anf\\303\\244ng\")\n> \n> commit_msg () {\n> \tmsg=\"initial. ${sample_utf8_part}lich\";\n> \tif test -n \"$1\"\n> \tthen\n> \t\techo \"$msg\" | iconv -f utf-8 -t \"$1\"\n> \telse\n> \t\techo \"$msg\"\n>         fi\n> }\n> \n> And then instead of writing in the expected test output.\n> \n> \t$(commit_msg \"\" \"8\" \"..*$\")\n> \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> \n> we can just say\n> \n> \tinitial...\n>         ..an${sample_utf8_part}lich\n> \tinit..lich\n> \n> It is no worse than those cryptic 0, 4, 8 and 11 magic numbers we\n> see in the test, no?\n\nThat's probably better since we don't need to rely on some other tool\ngetting it right.\n\nAlexey, will you incorporate this change in your incremental updates?\n"},{"id":"222512","messageId":"20130703222508.GC6148@dell-note","threadId":"34337","inReplyTo":"7vr4ffcoel.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-07-03T22:25:08Z","receivedAt":"2013-07-03T22:25:08Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"On Wed, Jul 03, 2013 at 02:41:06PM -0700, Junio C Hamano wrote:\n> John Keeping <john@keeping.me.uk> writes:\n> \n> > My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> > causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n> >\n> > Instead of using sed for this, use Perl which behaves predictably\n> > whatever locale is in use.\n> >\n> > Signed-off-by: John Keeping <john@keeping.me.uk>\n> > ---\n> > This patch is on top of 'as/log-output-encoding-in-user-format'.\n> \n> Thanks.  I think Alexey is going to send incremental updates to the\n> topic so I won't interfere by applying this patch on top of the\n> version I have in my tree.\n> \n> But I do agree that using Perl may be a workable solution.\n> \n> An alternative might be not to use this cryptic 3-arg form of\n> commit_msg at all.  They are used only for these three:\n> \n> \t$(commit_msg \"\" \"8\" \"..*$\")\n> \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> \n> I somehow find them simply not readable, in order to figure out what\n> is going on.\n> \n> Just using three variables to hold what are expected would be far\n> more portable and readable.\n> \n> # \"anfänglich\" whatever it means.\n> sample_utf8_part=$(printf \"anf\\303\\244ng\")\n> \n> commit_msg () {\n> \tmsg=\"initial. ${sample_utf8_part}lich\";\n> \tif test -n \"$1\"\n> \tthen\n> \t\techo \"$msg\" | iconv -f utf-8 -t \"$1\"\n> \telse\n> \t\techo \"$msg\"\n>         fi\n> }\n> \n> And then instead of writing in the expected test output.\n> \n> \t$(commit_msg \"\" \"8\" \"..*$\")\n> \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> \n> we can just say\n> \n> \tinitial...\n>         ..an${sample_utf8_part}lich\n> \tinit..lich\n> \n> It is no worse than those cryptic 0, 4, 8 and 11 magic numbers we\n> see in the test, no?\nYep!\nwhen I was thinking about Johannes's suggestions, I finally came to the decision\nalike yours.\n"},{"id":"222513","messageId":"20130703222729.GD6148@dell-note","threadId":"34337","inReplyTo":"20130703215303.GG9161@serenity.lan","subject":"Re: [PATCH] t4205: don't rely on en_US.UTF-8 locale existing","fromName":"Alexey Shumkin","fromEmail":"alex.crezoff@gmail.com","sentAt":"2013-07-03T22:27:29Z","receivedAt":"2013-07-03T22:27:29Z","isPatch":true,"sender":{"key":"alex.crezoff@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1183752?v=4"},"body":"On Wed, Jul 03, 2013 at 10:53:03PM +0100, John Keeping wrote:\n> On Wed, Jul 03, 2013 at 02:41:06PM -0700, Junio C Hamano wrote:\n> > John Keeping <john@keeping.me.uk> writes:\n> > \n> > > My system doesn't have the en_US.UTF-8 locale (or plain en_US), which\n> > > causes t4205 to fail by counting bytes instead of UTF-8 codepoints.\n> > >\n> > > Instead of using sed for this, use Perl which behaves predictably\n> > > whatever locale is in use.\n> > >\n> > > Signed-off-by: John Keeping <john@keeping.me.uk>\n> > > ---\n> > > This patch is on top of 'as/log-output-encoding-in-user-format'.\n> > \n> > Thanks.  I think Alexey is going to send incremental updates to the\n> > topic so I won't interfere by applying this patch on top of the\n> > version I have in my tree.\n> > \n> > But I do agree that using Perl may be a workable solution.\n> > \n> > An alternative might be not to use this cryptic 3-arg form of\n> > commit_msg at all.  They are used only for these three:\n> > \n> > \t$(commit_msg \"\" \"8\" \"..*$\")\n> > \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> > \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> > \n> > I somehow find them simply not readable, in order to figure out what\n> > is going on.\n> > \n> > Just using three variables to hold what are expected would be far\n> > more portable and readable.\n> > \n> > # \"anfänglich\" whatever it means.\n> > sample_utf8_part=$(printf \"anf\\303\\244ng\")\n> > \n> > commit_msg () {\n> > \tmsg=\"initial. ${sample_utf8_part}lich\";\n> > \tif test -n \"$1\"\n> > \tthen\n> > \t\techo \"$msg\" | iconv -f utf-8 -t \"$1\"\n> > \telse\n> > \t\techo \"$msg\"\n> >         fi\n> > }\n> > \n> > And then instead of writing in the expected test output.\n> > \n> > \t$(commit_msg \"\" \"8\" \"..*$\")\n> > \t$(commit_msg \"\" \"0\" \".\\{11\\}\")\n> > \t$(commit_msg \"\" \"4\" \".\\{11\\}\")\n> > \n> > we can just say\n> > \n> > \tinitial...\n> >         ..an${sample_utf8_part}lich\n> > \tinit..lich\n> > \n> > It is no worse than those cryptic 0, 4, 8 and 11 magic numbers we\n> > see in the test, no?\n> \n> That's probably better since we don't need to rely on some other tool\n> getting it right.\n> \n> Alexey, will you incorporate this change in your incremental updates?\nYes, of course!\nThank you for your additions\n"}]}