Re: git-gui: diff/console part does not show UTF-8 Japanese

From: Shawn O. Pearce <spearce@spearce.org>
Date: Aug 1, 2007, 13:38 UTC
Message-ID: <20070801133842.GG20052@spearce.org>
In-Reply-To: <7vvebz602q.fsf@assigned-by-dhcp.cox.net>

Junio C Hamano <gitster@pobox.com> wrote:

Show 27 quoted lines

> Nanako Shiraishi <nanako3@bluebottle.com> writes:
> 
> > 	I was testing git-gui (i18n branch).  I can see the translated 
> > message in the UI buttons and menus, and also commit message part shows 
> > Japanese strings, but I cannot get the diff part at the bottom to show 
> > Japanese.  Because I did set the two fonts to the same font from the 
> > Edit/Option menu, I do not think it is the problem with the font 
> > settings.
> >
> > Can anybody help?
> 
> I can see the same breakage in regular git-gui without i18n.
> Curious.  The commit message shows Japanese but diff part is
> bogus.
> 
> And the breakage is not just Japanese.  I can see this with
> Kristian's name by resetting a branch to 12ace0b2 and then
> attempting to amend it.  Interestingly enough, gitk shows the
> patch as expected.  I do not quite know how it is arranged
> there, though.
> 
> Now, I freely admit I do not quite know what I am doing, but
> this patch that tries to mimick the handling of commit log
> message seems to help it for git-gui.
> 
> I do think using i18n.commitencoding for payload is wrong,
> though.

Yea, it is wrong. The patch is logically fine as-is, there's nothing wrong with it except this part. I think the reason that gitk shows the patch as expected is because it is using i18n.commitencoding for the payload, and that's only happening as a side-effect of how it used it for the commit message itself. Call it dumb luck the thing even displays right.

To really Do The Right Thing(tm) here I think we need to store an encoding for each file path so we know what we are looking at when we see the file content. E.g. a gitattributes enc attribute?

For some files we might be able to guess. But I have to admit, I'm not up on my encoding guessing algorithms. ;-)

For XML files we can probably guess its XML ("starts with <?xml") and we can guess the encoding from that header line. But not everything that contains Japanese characters is an XML file. Like .po files for instance.

Where it gets even more interesting is how do I show a patch where the encoding changed? E.g. reencode a file from ISO-8859-1 into UTF-8? A number of lines would probably be identical, except for their encoding. So the user would want to see that they are the same. So we really should be able to show the old line using its old encoding, and the new line using its new encoding. Hmmph.

As a stop-gap I guess I could include a "Change Encoding" option for the diff pane and let you set the encoding to a specific name. But I really think git-gui should be able to do better than that.

-- 
Shawn.