{"thread":{"id":"31347","subject":"[PATCH] branch -v: align even when the first column is in UTF-8","startedAt":"2012-08-24T14:17:52Z","lastAt":"2012-08-26T18:28:54Z","messageCount":7,"participants":["Nguyễn Thái Ngọc Duy","Junio C Hamano","Erik Faye-Lund","Nguyen Thai Ngoc Duy"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"197773","messageId":"1345817872-7943-1-git-send-email-pclouds@gmail.com","threadId":"31347","inReplyTo":null,"subject":"[PATCH] branch -v: align even when the first column is in UTF-8","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-08-24T14:17:52Z","receivedAt":"2012-08-24T14:17:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Branch names are usually in ASCII so they are not the problem. The\nproblem most likely comes from \"(no branch)\" translation, which is in\nUTF-8 and makes length calculation just wrong.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n So far all git translations are utf-8 compatible. Branch names may\n use filesystem encoding, but then packed-refs specifies no encoding.\n Anyway branch names should be in utf-8.. at least internally, imo.\n\n builtin/branch.c | 8 +++++---\n 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)\n\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex 0e060f2..7c1ffa8 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -17,6 +17,7 @@\n #include \"revision.h\"\n #include \"string-list.h\"\n #include \"column.h\"\n+#include \"utf8.h\"\n \n static const char * const builtin_branch_usage[] = {\n \t\"git branch [options] [-r | -a] [--merged | --no-merged]\",\n@@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,\n \t}\n \n \tstrbuf_addf(&name, \"%s%s\", prefix, item->name);\n-\tif (verbose)\n+\tif (verbose) {\n+\t\tint utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);\n \t\tstrbuf_addf(&out, \"%c %s%-*s%s\", c, branch_get_color(color),\n-\t\t\t maxwidth, name.buf,\n+\t\t\t maxwidth + utf8_compensation, name.buf,\n \t\t\t branch_get_color(BRANCH_COLOR_RESET));\n-\telse\n+\t} else\n \t\tstrbuf_addf(&out, \"%c %s%s%s\", c, branch_get_color(color),\n \t\t\t name.buf, branch_get_color(BRANCH_COLOR_RESET));\n \n-- \n1.7.12.rc2.18.g61b472e\n"},{"id":"197784","messageId":"7vvcg8yzgx.fsf@alter.siamese.dyndns.org","threadId":"31347","inReplyTo":"1345817872-7943-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] branch -v: align even when the first column is in UTF-8","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-24T17:25:02Z","receivedAt":"2012-08-24T17:25:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy writes:\n\n> Branch names are usually in ASCII so they are not the problem. The\n> problem most likely comes from \"(no branch)\" translation, which is in\n> UTF-8 and makes length calculation just wrong.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> So far all git translations are utf-8 compatible. Branch names may\n> use filesystem encoding, but then packed-refs specifies no encoding.\n> Anyway branch names should be in utf-8.. at least internally, imo.\n\nI agree with all of the above, but shouldn't you be computing the\n\"maxwidth\" based on the strwidth in the first place? The use of\nmaxwidth in strbuf_addf() here clearly wants \"we know N columns is\nsufficient to show all output items, so pad the string to N columns\"\nhere. Looking for assignment \"item.len = xxx\" in the same file\nshows these are computed as byte length, so you are offsetting off\nof an incorrectly computed value.\n\nGiving fewer padding bytes when showing a string that will occupy\nfewer columns than it has bytes is independently necessary, once we\nhave the correct maxwidth that is computed in terms of the strwidth,\nso this patch is not wrong per-se, but it is incomplete without a\ncorrect maxwidth, no?\n\n> builtin/branch.c | 8 +++++---\n> 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)\n>\n> diff --git a/builtin/branch.c b/builtin/branch.c\n> index 0e060f2..7c1ffa8 100644\n> --- a/builtin/branch.c\n> +++ b/builtin/branch.c\n> @@ -17,6 +17,7 @@\n> #include \"revision.h\"\n> #include \"string-list.h\"\n> #include \"column.h\"\n> +#include \"utf8.h\"\n> \n> static const char * const builtin_branch_usage[] = {\n> \t\"git branch [options] [-r | -a] [--merged | --no-merged]\",\n> @@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,\n> \t}\n> \n> \tstrbuf_addf(&name, \"%s%s\", prefix, item->name);\n> -\tif (verbose)\n> +\tif (verbose) {\n> +\t\tint utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);\n> \t\tstrbuf_addf(&out, \"%c %s%-*s%s\", c, branch_get_color(color),\n> -\t\t\t maxwidth, name.buf,\n> +\t\t\t maxwidth + utf8_compensation, name.buf,\n> \t\t\t branch_get_color(BRANCH_COLOR_RESET));\n> -\telse\n> +\t} else\n> \t\tstrbuf_addf(&out, \"%c %s%s%s\", c, branch_get_color(color),\n> \t\t\t name.buf, branch_get_color(BRANCH_COLOR_RESET));\n"},{"id":"197833","messageId":"CABPQNSYabHGzmEfukt1PzT=j3qa99fScDHRZqLZvLNv9cJKXag@mail.gmail.com","threadId":"31347","inReplyTo":"1345817872-7943-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH] branch -v: align even when the first column is in UTF-8","fromName":"Erik Faye-Lund","fromEmail":"kusmabite@gmail.com","sentAt":"2012-08-25T10:48:40Z","receivedAt":"2012-08-25T10:48:40Z","isPatch":true,"sender":{"key":"kusmabite@gmail.com","avatar":"https://avatars.githubusercontent.com/u/47073?v=4"},"body":"On Fri, Aug 24, 2012 at 4:17 PM, Nguyễn Thái Ngọc Duy wrote:\n> 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)\n\nHuh?\n"},{"id":"197834","messageId":"CACsJy8BZ77JY=7UhOU_irWm0e4p2incbEpgU5=_oq_WNXGzj7w@mail.gmail.com","threadId":"31347","inReplyTo":"CABPQNSYabHGzmEfukt1PzT=j3qa99fScDHRZqLZvLNv9cJKXag@mail.gmail.com","subject":"Re: [PATCH] branch -v: align even when the first column is in UTF-8","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-08-25T11:19:13Z","receivedAt":"2012-08-25T11:19:13Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"2012/8/25 Erik Faye-Lund :\n> On Fri, Aug 24, 2012 at 4:17 PM, Nguyễn Thái Ngọc Duy wrote:\n>> 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)\n>\n> Huh?\n\nOh, that's something we should fix soon, too. I suggested a \"project\nlanguage\" config key some time ago, where commands like format-patch\nshould follow. It's hard because only part of the format-patch will be\nin this language while error messages for example are still in $LANG.\nI've been thinking, but I haven't found anything worth mentioning yet.\n-- \nDuy\n"},{"id":"197841","messageId":"1345918632-17756-1-git-send-email-pclouds@gmail.com","threadId":"31347","inReplyTo":"7vvcg8yzgx.fsf@alter.siamese.dyndns.org","subject":"[PATCH v2] branch -v: align even when branch names are in UTF-8","fromName":"Nguyễn Thái Ngọc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-08-25T18:17:12Z","receivedAt":"2012-08-25T18:17:12Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"Branch names are usually in ASCII so they are not the problem. The\nproblem most likely comes from \"(no branch)\" translation, which is in\nUTF-8 and makes length calculation just wrong.\n\nUpdate document to mention the fact that we may want ref names in\nUTF-8. Encodings that produce invalid UTF-8 are safe as utf8_strwidth()\nfalls back to strlen(). The ones that incidentally produce valid UTF-8\nsequences will cause misalignment.\n\nSigned-off-by: Nguyễn Thái Ngọc Duy \n---\n On Sat, Aug 25, 2012 at 12:25 AM, Junio C Hamano wrote:\n > I agree with all of the above, but shouldn't you be computing the\n > \"maxwidth\" based on the strwidth in the first place? The use of\n > maxwidth in strbuf_addf() here clearly wants \"we know N columns is\n > sufficient to show all output items, so pad the string to N columns\"\n > here. Looking for assignment \"item.len = xxx\" in the same file\n > shows these are computed as byte length, so you are offsetting off\n > of an incorrectly computed value.\n >\n > Giving fewer padding bytes when showing a string that will occupy\n > fewer columns than it has bytes is independently necessary, once we\n > have the correct maxwidth that is computed in terms of the strwidth,\n > so this patch is not wrong per-se, but it is incomplete without a\n > correct maxwidth, no?\n\n Yes. This fixes that and also mentions about ref names in utf-8.\n\n Documentation/revisions.txt | 2 ++\n builtin/branch.c | 12 +++++++-----\n 2 files changed, 9 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/revisions.txt b/Documentation/revisions.txt\nindex dc0070b..175d397 100644\n--- a/Documentation/revisions.txt\n+++ b/Documentation/revisions.txt\n@@ -55,6 +55,8 @@ when you run `git cherry-pick`.\n +\n Note that any of the 'refs/*' cases above may come either from\n the '$GIT_DIR/refs' directory or from the '$GIT_DIR/packed-refs' file.\n+While the ref name encoding is unspecified, UTF-8 is prefered as\n+some output processing may assume ref names in UTF-8.\n \n '@\\{\\}', e.g. 'master@\\{yesterday\\}', 'HEAD@\\{5 minutes ago\\}'::\n A ref followed by the suffix '@' with a date specification\ndiff --git a/builtin/branch.c b/builtin/branch.c\nindex 0e060f2..73ff7e7 100644\n--- a/builtin/branch.c\n+++ b/builtin/branch.c\n@@ -17,6 +17,7 @@\n #include \"revision.h\"\n #include \"string-list.h\"\n #include \"column.h\"\n+#include \"utf8.h\"\n \n static const char * const builtin_branch_usage[] = {\n \t\"git branch [options] [-r | -a] [--merged | --no-merged]\",\n@@ -354,7 +355,7 @@ static int append_ref(const char *refname, const unsigned char *sha1, int flags,\n \tnewitem->name = xstrdup(refname);\n \tnewitem->kind = kind;\n \tnewitem->commit = commit;\n-\tnewitem->len = strlen(refname);\n+\tnewitem->len = utf8_strwidth(refname);\n \tnewitem->dest = resolve_symref(orig_refname, prefix);\n \t/* adjust for \"remotes/\" */\n \tif (newitem->kind == REF_REMOTE_BRANCH &&\n@@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,\n \t}\n \n \tstrbuf_addf(&name, \"%s%s\", prefix, item->name);\n-\tif (verbose)\n+\tif (verbose) {\n+\t\tint utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);\n \t\tstrbuf_addf(&out, \"%c %s%-*s%s\", c, branch_get_color(color),\n-\t\t\t maxwidth, name.buf,\n+\t\t\t maxwidth + utf8_compensation, name.buf,\n \t\t\t branch_get_color(BRANCH_COLOR_RESET));\n-\telse\n+\t} else\n \t\tstrbuf_addf(&out, \"%c %s%s%s\", c, branch_get_color(color),\n \t\t\t name.buf, branch_get_color(BRANCH_COLOR_RESET));\n \n@@ -533,7 +535,7 @@ static void show_detached(struct ref_list *ref_list)\n \tif (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {\n \t\tstruct ref_item item;\n \t\titem.name = xstrdup(_(\"(no branch)\"));\n-\t\titem.len = strlen(item.name);\n+\t\titem.len = utf8_strwidth(item.name);\n \t\titem.kind = REF_LOCAL_BRANCH;\n \t\titem.dest = NULL;\n \t\titem.commit = head_commit;\n-- \n1.7.12.rc2.18.g61b472e\n"},{"id":"197865","messageId":"7vlih1v8ak.fsf@alter.siamese.dyndns.org","threadId":"31347","inReplyTo":"1345918632-17756-1-git-send-email-pclouds@gmail.com","subject":"Re: [PATCH v2] branch -v: align even when branch names are in UTF-8","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-26T18:04:51Z","receivedAt":"2012-08-26T18:04:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Nguyễn Thái Ngọc Duy writes:\n\n> Branch names are usually in ASCII so they are not the problem. The\n> problem most likely comes from \"(no branch)\" translation, which is in\n> UTF-8 and makes length calculation just wrong.\n>\n> Update document to mention the fact that we may want ref names in\n> UTF-8. Encodings that produce invalid UTF-8 are safe as utf8_strwidth()\n> falls back to strlen(). The ones that incidentally produce valid UTF-8\n> sequences will cause misalignment.\n>\n> Signed-off-by: Nguyễn Thái Ngọc Duy \n> ---\n> ...\n> @@ -533,7 +535,7 @@ static void show_detached(struct ref_list *ref_list)\n> \tif (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {\n> \t\tstruct ref_item item;\n> \t\titem.name = xstrdup(_(\"(no branch)\"));\n> -\t\titem.len = strlen(item.name);\n> +\t\titem.len = utf8_strwidth(item.name);\n> \t\titem.kind = REF_LOCAL_BRANCH;\n> \t\titem.dest = NULL;\n> \t\titem.commit = head_commit;\n\nWe should probably rename the \"len\" field, as it is no longer about\nthe length (i.e. that which strlen() returns); it is the display\nwidth, and is better called \"cols\", \"width\" or somesuch.\n\nI'll squash-in the following.\n\nThanks.\n\n builtin/branch.c | 20 ++++++++++----------\n 1 file changed, 10 insertions(+), 10 deletions(-)\n\ndiff --git c/builtin/branch.c w/builtin/branch.c\nindex 73ff7e7..4ec556f 100644\n--- c/builtin/branch.c\n+++ w/builtin/branch.c\n@@ -250,7 +250,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,\n struct ref_item {\n \tchar *name;\n \tchar *dest;\n-\tunsigned int kind, len;\n+\tunsigned int kind, width;\n \tstruct commit *commit;\n };\n \n@@ -355,14 +355,14 @@ static int append_ref(const char *refname, const unsigned char *sha1, int flags,\n \tnewitem->name = xstrdup(refname);\n \tnewitem->kind = kind;\n \tnewitem->commit = commit;\n-\tnewitem->len = utf8_strwidth(refname);\n+\tnewitem->width = utf8_strwidth(refname);\n \tnewitem->dest = resolve_symref(orig_refname, prefix);\n \t/* adjust for \"remotes/\" */\n \tif (newitem->kind == REF_REMOTE_BRANCH &&\n \t ref_list->kinds != REF_REMOTE_BRANCH)\n-\t\tnewitem->len += 8;\n-\tif (newitem->len > ref_list->maxwidth)\n-\t\tref_list->maxwidth = newitem->len;\n+\t\tnewitem->width += 8;\n+\tif (newitem->width > ref_list->maxwidth)\n+\t\tref_list->maxwidth = newitem->width;\n \n \treturn 0;\n }\n@@ -521,8 +521,8 @@ static int calc_maxwidth(struct ref_list *refs)\n \tfor (i = 0; i < refs->index; i++) {\n \t\tif (!matches_merge_filter(refs->list[i].commit))\n \t\t\tcontinue;\n-\t\tif (refs->list[i].len > w)\n-\t\t\tw = refs->list[i].len;\n+\t\tif (refs->list[i].width > w)\n+\t\t\tw = refs->list[i].width;\n \t}\n \treturn w;\n }\n@@ -535,12 +535,12 @@ static void show_detached(struct ref_list *ref_list)\n \tif (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {\n \t\tstruct ref_item item;\n \t\titem.name = xstrdup(_(\"(no branch)\"));\n-\t\titem.len = utf8_strwidth(item.name);\n+\t\titem.width = utf8_strwidth(item.name);\n \t\titem.kind = REF_LOCAL_BRANCH;\n \t\titem.dest = NULL;\n \t\titem.commit = head_commit;\n-\t\tif (item.len > ref_list->maxwidth)\n-\t\t\tref_list->maxwidth = item.len;\n+\t\tif (item.width > ref_list->maxwidth)\n+\t\t\tref_list->maxwidth = item.width;\n \t\tprint_ref_item(&item, ref_list->maxwidth, ref_list->verbose, ref_list->abbrev, 1, \"\");\n \t\tfree(item.name);\n \t}\n"},{"id":"197868","messageId":"7vd32dv76h.fsf@alter.siamese.dyndns.org","threadId":"31347","inReplyTo":"CABPQNSYabHGzmEfukt1PzT=j3qa99fScDHRZqLZvLNv9cJKXag@mail.gmail.com","subject":"Re: [PATCH] branch -v: align even when the first column is in UTF-8","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2012-08-26T18:28:54Z","receivedAt":"2012-08-26T18:28:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Erik Faye-Lund writes:\n\n> On Fri, Aug 24, 2012 at 4:17 PM, Nguyễn Thái Ngọc Duy wrote:\n>> 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)\n>\n> Huh?\n\nPerhaps format-patch should always use C locale.\n"}]}