threads / patch / 31347

patchbranch -v: align even when the first column is in UTF-8

Subject: [PATCH] branch -v: align even when the first column is in UTF-8

## tl;dr

7 messages between Aug 24, 2012 and Aug 26, 2012. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Nguyễn Thái Ngọc Duy· Aug 24, 2012, 14:17 UTC · lore

Branch names are usually in ASCII so they are not the problem. The problem most likely comes from "(no branch)" translation, which is in UTF-8 and makes length calculation just wrong.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 So far all git translations are utf-8 compatible. Branch names may
 use filesystem encoding, but then packed-refs specifies no encoding.
 Anyway branch names should be in utf-8.. at least internally, imo.
 builtin/branch.c | 8 +++++---
 1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)
Show changes to builtin/branch.c +5 −3
diff --git a/builtin/branch.c b/builtin/branch.c
index 0e060f2..7c1ffa8 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -17,6 +17,7 @@
 #include "revision.h"
 #include "string-list.h"
 #include "column.h"
+#include "utf8.h"
 
 static const char * const builtin_branch_usage[] = {
 	"git branch [options] [-r | -a] [--merged | --no-merged]",
@@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,
 	}
 
 	strbuf_addf(&name, "%s%s", prefix, item->name);
-	if (verbose)
+	if (verbose) {
+		int utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);
 		strbuf_addf(&out, "%c %s%-*s%s", c, branch_get_color(color),
-			    maxwidth, name.buf,
+			    maxwidth + utf8_compensation, name.buf,
 			    branch_get_color(BRANCH_COLOR_RESET));
-	else
+	} else
 		strbuf_addf(&out, "%c %s%s%s", c, branch_get_color(color),
 			    name.buf, branch_get_color(BRANCH_COLOR_RESET));
 
-- 
1.7.12.rc2.18.g61b472e
Junio C Hamano· Aug 24, 2012, 17:25 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH] branch -v: align even when the first column is in UTF-8

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 9 quoted lines
> Branch names are usually in ASCII so they are not the problem. The
> problem most likely comes from "(no branch)" translation, which is in
> UTF-8 and makes length calculation just wrong.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  So far all git translations are utf-8 compatible. Branch names may
>  use filesystem encoding, but then packed-refs specifies no encoding.
>  Anyway branch names should be in utf-8.. at least internally, imo.

I agree with all of the above, but shouldn't you be computing the "maxwidth" based on the strwidth in the first place? The use of maxwidth in strbuf_addf() here clearly wants "we know N columns is sufficient to show all output items, so pad the string to N columns" here. Looking for assignment "item.len = xxx" in the same file shows these are computed as byte length, so you are offsetting off of an incorrectly computed value.

Giving fewer padding bytes when showing a string that will occupy fewer columns than it has bytes is independently necessary, once we have the correct maxwidth that is computed in terms of the strwidth, so this patch is not wrong per-se, but it is incomplete without a correct maxwidth, no?

Show 30 quoted lines
>  builtin/branch.c | 8 +++++---
>  1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)
>
> diff --git a/builtin/branch.c b/builtin/branch.c
> index 0e060f2..7c1ffa8 100644
> --- a/builtin/branch.c
> +++ b/builtin/branch.c
> @@ -17,6 +17,7 @@
>  #include "revision.h"
>  #include "string-list.h"
>  #include "column.h"
> +#include "utf8.h"
>  
>  static const char * const builtin_branch_usage[] = {
>  	"git branch [options] [-r | -a] [--merged | --no-merged]",
> @@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,
>  	}
>  
>  	strbuf_addf(&name, "%s%s", prefix, item->name);
> -	if (verbose)
> +	if (verbose) {
> +		int utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);
>  		strbuf_addf(&out, "%c %s%-*s%s", c, branch_get_color(color),
> -			    maxwidth, name.buf,
> +			    maxwidth + utf8_compensation, name.buf,
>  			    branch_get_color(BRANCH_COLOR_RESET));
> -	else
> +	} else
>  		strbuf_addf(&out, "%c %s%s%s", c, branch_get_color(color),
>  			    name.buf, branch_get_color(BRANCH_COLOR_RESET));
Nguyễn Thái Ngọc Duy· Aug 25, 2012, 18:17 UTC · re: Junio C Hamano · lore

[PATCH v2] branch -v: align even when branch names are in UTF-8

Branch names are usually in ASCII so they are not the problem. The problem most likely comes from "(no branch)" translation, which is in UTF-8 and makes length calculation just wrong.

Update document to mention the fact that we may want ref names in UTF-8. Encodings that produce invalid UTF-8 are safe as utf8_strwidth() falls back to strlen(). The ones that incidentally produce valid UTF-8 sequences will cause misalignment.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 On Sat, Aug 25, 2012 at 12:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
 > I agree with all of the above, but shouldn't you be computing the
 > "maxwidth" based on the strwidth in the first place?  The use of
 > maxwidth in strbuf_addf() here clearly wants "we know N columns is
 > sufficient to show all output items, so pad the string to N columns"
 > here.  Looking for assignment "item.len = xxx" in the same file
 > shows these are computed as byte length, so you are offsetting off
 > of an incorrectly computed value.
 >
 > Giving fewer padding bytes when showing a string that will occupy
 > fewer columns than it has bytes is independently necessary, once we
 > have the correct maxwidth that is computed in terms of the strwidth,
 > so this patch is not wrong per-se, but it is incomplete without a
 > correct maxwidth, no?
 Yes. This fixes that and also mentions about ref names in utf-8.
 Documentation/revisions.txt |  2 ++
 builtin/branch.c            | 12 +++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)
Show changes to 2 files +9 −5

Documentation/revisions.txt, builtin/branch.c

diff --git a/Documentation/revisions.txt b/Documentation/revisions.txt
index dc0070b..175d397 100644
--- a/Documentation/revisions.txt
+++ b/Documentation/revisions.txt
@@ -55,6 +55,8 @@ when you run `git cherry-pick`.
 +
 Note that any of the 'refs/*' cases above may come either from
 the '$GIT_DIR/refs' directory or from the '$GIT_DIR/packed-refs' file.
+While the ref name encoding is unspecified, UTF-8 is prefered as
+some output processing may assume ref names in UTF-8.
 
 '<refname>@\{<date>\}', e.g. 'master@\{yesterday\}', 'HEAD@\{5 minutes ago\}'::
   A ref followed by the suffix '@' with a date specification
diff --git a/builtin/branch.c b/builtin/branch.c
index 0e060f2..73ff7e7 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -17,6 +17,7 @@
 #include "revision.h"
 #include "string-list.h"
 #include "column.h"
+#include "utf8.h"
 
 static const char * const builtin_branch_usage[] = {
 	"git branch [options] [-r | -a] [--merged | --no-merged]",
@@ -354,7 +355,7 @@ static int append_ref(const char *refname, const unsigned char *sha1, int flags,
 	newitem->name = xstrdup(refname);
 	newitem->kind = kind;
 	newitem->commit = commit;
-	newitem->len = strlen(refname);
+	newitem->len = utf8_strwidth(refname);
 	newitem->dest = resolve_symref(orig_refname, prefix);
 	/* adjust for "remotes/" */
 	if (newitem->kind == REF_REMOTE_BRANCH &&
@@ -490,11 +491,12 @@ static void print_ref_item(struct ref_item *item, int maxwidth, int verbose,
 	}
 
 	strbuf_addf(&name, "%s%s", prefix, item->name);
-	if (verbose)
+	if (verbose) {
+		int utf8_compensation = strlen(name.buf) - utf8_strwidth(name.buf);
 		strbuf_addf(&out, "%c %s%-*s%s", c, branch_get_color(color),
-			    maxwidth, name.buf,
+			    maxwidth + utf8_compensation, name.buf,
 			    branch_get_color(BRANCH_COLOR_RESET));
-	else
+	} else
 		strbuf_addf(&out, "%c %s%s%s", c, branch_get_color(color),
 			    name.buf, branch_get_color(BRANCH_COLOR_RESET));
 
@@ -533,7 +535,7 @@ static void show_detached(struct ref_list *ref_list)
 	if (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {
 		struct ref_item item;
 		item.name = xstrdup(_("(no branch)"));
-		item.len = strlen(item.name);
+		item.len = utf8_strwidth(item.name);
 		item.kind = REF_LOCAL_BRANCH;
 		item.dest = NULL;
 		item.commit = head_commit;
-- 
1.7.12.rc2.18.g61b472e
Junio C Hamano· Aug 26, 2012, 18:04 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v2] branch -v: align even when branch names are in UTF-8

Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
Show 21 quoted lines
> Branch names are usually in ASCII so they are not the problem. The
> problem most likely comes from "(no branch)" translation, which is in
> UTF-8 and makes length calculation just wrong.
>
> Update document to mention the fact that we may want ref names in
> UTF-8. Encodings that produce invalid UTF-8 are safe as utf8_strwidth()
> falls back to strlen(). The ones that incidentally produce valid UTF-8
> sequences will cause misalignment.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> ...
> @@ -533,7 +535,7 @@ static void show_detached(struct ref_list *ref_list)
>  	if (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {
>  		struct ref_item item;
>  		item.name = xstrdup(_("(no branch)"));
> -		item.len = strlen(item.name);
> +		item.len = utf8_strwidth(item.name);
>  		item.kind = REF_LOCAL_BRANCH;
>  		item.dest = NULL;
>  		item.commit = head_commit;

We should probably rename the "len" field, as it is no longer about the length (i.e. that which strlen() returns); it is the display width, and is better called "cols", "width" or somesuch.

I'll squash-in the following.
Thanks.
 builtin/branch.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
Show changes to diff +10 −10
diff --git c/builtin/branch.c w/builtin/branch.c
index 73ff7e7..4ec556f 100644
--- c/builtin/branch.c
+++ w/builtin/branch.c
@@ -250,7 +250,7 @@ static int delete_branches(int argc, const char **argv, int force, int kinds,
 struct ref_item {
 	char *name;
 	char *dest;
-	unsigned int kind, len;
+	unsigned int kind, width;
 	struct commit *commit;
 };
 
@@ -355,14 +355,14 @@ static int append_ref(const char *refname, const unsigned char *sha1, int flags,
 	newitem->name = xstrdup(refname);
 	newitem->kind = kind;
 	newitem->commit = commit;
-	newitem->len = utf8_strwidth(refname);
+	newitem->width = utf8_strwidth(refname);
 	newitem->dest = resolve_symref(orig_refname, prefix);
 	/* adjust for "remotes/" */
 	if (newitem->kind == REF_REMOTE_BRANCH &&
 	    ref_list->kinds != REF_REMOTE_BRANCH)
-		newitem->len += 8;
-	if (newitem->len > ref_list->maxwidth)
-		ref_list->maxwidth = newitem->len;
+		newitem->width += 8;
+	if (newitem->width > ref_list->maxwidth)
+		ref_list->maxwidth = newitem->width;
 
 	return 0;
 }
@@ -521,8 +521,8 @@ static int calc_maxwidth(struct ref_list *refs)
 	for (i = 0; i < refs->index; i++) {
 		if (!matches_merge_filter(refs->list[i].commit))
 			continue;
-		if (refs->list[i].len > w)
-			w = refs->list[i].len;
+		if (refs->list[i].width > w)
+			w = refs->list[i].width;
 	}
 	return w;
 }
@@ -535,12 +535,12 @@ static void show_detached(struct ref_list *ref_list)
 	if (head_commit && is_descendant_of(head_commit, ref_list->with_commit)) {
 		struct ref_item item;
 		item.name = xstrdup(_("(no branch)"));
-		item.len = utf8_strwidth(item.name);
+		item.width = utf8_strwidth(item.name);
 		item.kind = REF_LOCAL_BRANCH;
 		item.dest = NULL;
 		item.commit = head_commit;
-		if (item.len > ref_list->maxwidth)
-			ref_list->maxwidth = item.len;
+		if (item.width > ref_list->maxwidth)
+			ref_list->maxwidth = item.width;
 		print_ref_item(&item, ref_list->maxwidth, ref_list->verbose, ref_list->abbrev, 1, "");
 		free(item.name);
 	}
Nguyen Thai Ngoc Duy· Aug 25, 2012, 11:19 UTC · re: Erik Faye-Lund · lore

Re: [PATCH] branch -v: align even when the first column is in UTF-8

2012/8/25 Erik Faye-Lund <kusmabite@gmail.com>:
> On Fri, Aug 24, 2012 at 4:17 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
>>  1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)
>
> Huh?

Oh, that's something we should fix soon, too. I suggested a "project language" config key some time ago, where commands like format-patch should follow. It's hard because only part of the format-patch will be in this language while error messages for example are still in $LANG. I've been thinking, but I haven't found anything worth mentioning yet.

-- 
Duy
Junio C Hamano· Aug 26, 2012, 18:28 UTC · re: Erik Faye-Lund · lore

Re: [PATCH] branch -v: align even when the first column is in UTF-8

Erik Faye-Lund <kusmabite@gmail.com> writes:
> On Fri, Aug 24, 2012 at 4:17 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
>>  1 tập tin đã bị thay đổi, 5 được thêm vào(+), 3 bị xóa(-)
>
> Huh?
Perhaps format-patch should always use C locale.

← back to recent threads