threads / patch / 2658

patchgitk: UTF-8 support

Subject: [PATCH] gitk: UTF-8 support

## tl;dr

8 messages between Nov 23, 2005 and Nov 28, 2005. Diffs are folded; open one to read it.

replies: 7people: 3as markdown or json

Pavel Roskin· Nov 23, 2005, 04:15 UTC · lore

Add gitencoding variable and set it to "utf-8". Use it for converting git-rev-list output.

Signed-off-by: Pavel Roskin <proski@gnu.org>
Show changes to gitk +3 −2
diff --git a/gitk b/gitk
index 3dd97e2..e53b609 100755
--- a/gitk
+++ b/gitk
@@ -19,7 +19,7 @@ proc gitdir {} {
 proc getcommits {rargs} {
     global commits commfd phase canv mainfont env
     global startmsecs nextupdate ncmupdate
-    global ctext maincursor textcursor leftover
+    global ctext maincursor textcursor leftover gitencoding
 
     # check that we can find a .git directory somewhere...
     set gitdir [gitdir]
@@ -49,7 +49,7 @@ proc getcommits {rargs} {
 	exit 1
     }
     set leftover {}
-    fconfigure $commfd -blocking 0 -translation lf
+    fconfigure $commfd -blocking 0 -translation lf -encoding $gitencoding
     fileevent $commfd readable [list getcommitlines $commfd]
     $canv delete all
     $canv create text 3 3 -anchor nw -text "Reading commits..." \
@@ -3657,6 +3657,7 @@ set datemode 0
 set boldnames 0
 set diffopts "-U 5 -p"
 set wrcomcmd "git-diff-tree --stdin -p --pretty"
+set gitencoding "utf-8"
 
 set mainfont {Helvetica 9}
 set textfont {Courier 9}

-- 
Regards,
Pavel Roskin
Junio C Hamano· Nov 24, 2005, 00:47 UTC · re: Pavel Roskin · lore

Re: [PATCH] gitk: UTF-8 support

Pavel Roskin <proski@gnu.org> writes:
> Add gitencoding variable and set it to "utf-8".  Use it for converting
> git-rev-list output.

Sounds good, but is it necessary? Unless I am grossly mistaken, I am opposed to this patch.

When I run gitk with LANG and/or LC_CTYPE set to ja_JP.utf8 (I suspect *whatever*.utf8 would work the same way) on git.git repository, I see Lukas's name (originally in iso8859-1 but my commit objects have it in utf8) and Yoshifuji-san's name (iso2022 converted to utf8) just fine.

And when I run gitk with LANG and/or LC_CTYPE set to ja_JP.ujis (that is another name for EUC-JP) on a toy repository I have commit log messages in EUC-JP (I am not recommending that, just pointing out a possibility), I can see them just fine. In that test repository, setting locale to *.utf8 would not work.

So I suspect your change breaks projects that use local encodings, without fixing or adding anything new.

Paul Mackerras· Nov 24, 2005, 01:11 UTC · re: Junio C Hamano · lore

Re: [PATCH] gitk: UTF-8 support

Junio C Hamano writes:
Show 7 quoted lines
> Pavel Roskin <proski@gnu.org> writes:
> 
> > Add gitencoding variable and set it to "utf-8".  Use it for converting
> > git-rev-list output.
> 
> Sounds good, but is it necessary?  Unless I am grossly mistaken,
> I am opposed to this patch.

I already put this into my gitk.git repository, so you might want to hold off pulling that into git.git until we work this out.

Being clueless about i18n and locales, I tend to just take whatever people tell me is needed in that area. :)

Regards, Paul.

Pavel Roskin· Nov 24, 2005, 04:53 UTC · re: Junio C Hamano · lore

Re: [PATCH] gitk: UTF-8 support

Hello!
Quoting Junio C Hamano <junkio@cox.net>:
Show 13 quoted lines
> Pavel Roskin <proski@gnu.org> writes:
>
> > Add gitencoding variable and set it to "utf-8".  Use it for converting
> > git-rev-list output.
>
> Sounds good, but is it necessary?  Unless I am grossly mistaken,
> I am opposed to this patch.
>
> When I run gitk with LANG and/or LC_CTYPE set to ja_JP.utf8 (I
> suspect *whatever*.utf8 would work the same way) on git.git
> repository, I see Lukas's name (originally in iso8859-1 but my
> commit objects have it in utf8) and Yoshifuji-san's name
> (iso2022 converted to utf8) just fine.
I see.  I always use C locale.
Show 5 quoted lines
> And when I run gitk with LANG and/or LC_CTYPE set to ja_JP.ujis
> (that is another name for EUC-JP) on a toy repository I have
> commit log messages in EUC-JP (I am not recommending that, just
> pointing out a possibility), I can see them just fine.  In that
> test repository, setting locale to *.utf8 would not work.

Then what would you do to work with a repository using utf-8 if the current locale is not utf-8?

> So I suspect your change breaks projects that use local
> encodings, without fixing or adding anything new.

I'll be away from any sane OS until Monday, but I assume my patch should help those whose locale is set to an encoding other than utf-8 if they want to use a repository using utf-8.

Anyway, I see your point. Not ever git repository uses utf-8. It is not enforced by git.

So we have two solutions. One is to enforce utf-8 locale in git and force all "offenders" to convert. That's probably too intrusive for now.

The other solution is to have a publicly available file under .git that would keep the encoding name for the metadata (user names, logs etc). gitk could use that file now. git could implement conversions a bit later. Maybe git could warn about mismatching encoding as the first step.

-- Regards, Pavel Roskin

Junio C Hamano· Nov 24, 2005, 06:23 UTC · re: Pavel Roskin · lore

Re: [PATCH] gitk: UTF-8 support

Pavel Roskin <proski@gnu.org> writes:
> Then what would you do to work with a repository using utf-8 if the current
> locale is not utf-8?
Obviously the same way as I did to try things out:
	$ LANG=en_US.utf8 DISPLAY=:0 gitk blah
> Anyway, I see your point.  Not ever git repository uses utf-8.  It is not
> enforced by git.

That is not the point. Point is that I think the user can use LANG and LC_ALL (I suspect LC_CTYPE is what matters) to get what you want, and I suspect hardcoding utf8 robs users the possibility to deal with a repository that uses something else.

And as I suggested in another message (in the died-out thread about gitweb), we could have i18n.commitEncoding in the configuration to help gitk and gitweb. I think that is the same as your "other option".

Pavel Roskin· Nov 24, 2005, 07:12 UTC · re: Junio C Hamano · lore

Re: [PATCH] gitk: UTF-8 support

Quoting Junio C Hamano <junkio@cox.net>:
> That is not the point.  Point is that I think the user can use
> LANG and LC_ALL (I suspect LC_CTYPE is what matters) to get what
> you want, and I suspect hardcoding utf8 robs users the
> possibility to deal with a repository that uses something else.

Not to argue with you, but it's worth pointing out that git is heavily multiuser software, and interoperability should not be ranked below local configurability.

> And as I suggested in another message (in the died-out thread
> about gitweb), we could have i18n.commitEncoding in the
> configuration to help gitk and gitweb.  I think that is the same
> as your "other option".

Yes. Then my patch needs to be changed to set encoding to that setting and only if it's present.

-- Regards, Pavel Roskin

Junio C Hamano· Nov 28, 2005, 00:12 UTC · re: Pavel Roskin · lore

Re: [PATCH] gitk: UTF-8 support

Pavel Roskin <proski@gnu.org> writes:
Show 7 quoted lines
>> And as I suggested in another message (in the died-out thread
>> about gitweb), we could have i18n.commitEncoding in the
>> configuration to help gitk and gitweb.  I think that is the same
>> as your "other option".
>
> Yes.  Then my patch needs to be changed to set encoding to that setting and only
> if it's present.

The following patch on top of your patch has seen only very light testing, but it seems to do the right thing for my repository with utf-8 commit messages and another with euc-jp commit messages. For the latter, I needed to do:

	$ git-repo-config i18n.commitencoding euc-jp

-- >8 -- [PATCH] gitk: Use i18n.commitencoding configuration item.

Hardcoding "utf-8" in the script breaks projects that use local encoding, so allow setting i18n.commitEncoding.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---
Show changes to gitk +8 −1
diff --git a/gitk b/gitk
index b53a5c5..2242216 100755
--- a/gitk
+++ b/gitk
@@ -3669,7 +3669,14 @@ set datemode 0
 set boldnames 0
 set diffopts "-U 5 -p"
 set wrcomcmd "git-diff-tree --stdin -p --pretty"
-set gitencoding "utf-8"
+
+set gitencoding ""
+catch {
+    set gitencoding [exec git-repo-config --get i18n.commitencoding]
+}
+if {$gitencoding == ""} {
+	set gitencoding "utf-8"
+}
 
 set mainfont {Helvetica 9}
 set textfont {Courier 9}
Pavel Roskin· Nov 28, 2005, 21:55 UTC · re: Junio C Hamano · lore

Re: [PATCH] gitk: UTF-8 support

On Sun, 2005-11-27 at 16:12 -0800, Junio C Hamano wrote:
Show 6 quoted lines
> The following patch on top of your patch has seen only very
> light testing, but it seems to do the right thing for my
> repository with utf-8 commit messages and another with euc-jp
> commit messages.  For the latter, I needed to do:
> 
> 	$ git-repo-config i18n.commitencoding euc-jp
Thank you!  I'm glad my code was actually useful.

Now we may want to make git recode commit data to the repository encoding. Lots of iconv/glibc/portability fun on the horizon. At very least git should protest if both i18n.commitencoding and the locale encoding are set to different values.

Another issue is support for non-ASCII characters in the filenames.
-- 
Regards,
Pavel Roskin

← back to recent threads