threads / discuss / 11256

git gui blame utf-8 bugs

Subject: git gui blame utf-8 bugs

## tl;dr

4 messages between Dec 12, 2007 and Dec 17, 2007.

replies: 3people: 3as markdown or json

Finn Arne Gangstad· Dec 12, 2007, 09:17 UTC · lore

git gui has some utf-8 bugs:

If you do git gui blame <file>, and the file contains utf-8 text, the lines are not parsed as utf-8, but seemingly as iso-8859-1 instead.

Also, the hovering comment is INITIALLY shown garbled (both Author and commit message), but if you click on a line, so that the commit message is shown in the bottom window, the hovering message is magically corrected to utf-8.

The text in the lower window (showing specific commits) seems to always be handled correctly.

To reproduce: Set your author name to include some utf-8 tokens, add a line with some utf-8 tokens to a file, commit it with a commit message including some utf-8 tokens, and do git gui blame on the file. The line will be garbled in the top window, and the hovering message will be garbled until you click on the line.

Verified with git-gui.git master

- Finn Arne

Shawn O. Pearce· Dec 14, 2007, 06:47 UTC · re: Finn Arne Gangstad · lore

Re: git gui blame utf-8 bugs

Finn Arne Gangstad <finnag@pvv.org> wrote:

> git gui has some utf-8 bugs:

It has several.  :-)

> If you do git gui blame <file>, and the file contains utf-8 text,
> the lines are not parsed as utf-8, but seemingly as iso-8859-1 instead.

Right. git-gui is keying off the environment setting for LANG, so I guess its set to iso-8859-1 on your system but you are working with a utf-8 file. We've talked about using something like .gitattributes to store encoding hints, or to just put a global gui setting in ~/.gitconfig but neither has had any patches written for it.

UTF-8 is seemingly the most common encoding that git-gui is mangling so maybe we should be defaulting to utf-8 until someone codes a more intelligent patch.

Show 7 quoted lines

> Also, the hovering comment is INITIALLY shown garbled (both Author and
> commit message), but if you click on a line, so that the commit
> message is shown in the bottom window, the hovering message is
> magically corrected to utf-8.
> 
> The text in the lower window (showing specific commits) seems to
> always be handled correctly.

That's a "feature".  :-)

What's happening here is the initial hovering message is obtained from the machine formatted output from `git blame --incremental` and in that format there is no encoding header so I'm just ignoring any encoding problems.

Later when you click on a line it does `git cat-file commit $sha1` and gets the proper encoding, and corrects the strings it originally had gotten from git-blame. So the hovering message "fixes" itself later on.

Maybe here too we should be defaulting to utf-8 instead of the native encoding.

-- 
Shawn.

Jakub Narebski· Dec 14, 2007, 12:39 UTC · re: Shawn O. Pearce · lore

Re: git gui blame utf-8 bugs

"Shawn O. Pearce" <spearce@spearce.org> writes:

Show 17 quoted lines

> Finn Arne Gangstad <finnag@pvv.org> wrote:
> > git gui has some utf-8 bugs:
> 
> It has several.  :-)
>  
> > If you do git gui blame <file>, and the file contains utf-8 text,
> > the lines are not parsed as utf-8, but seemingly as iso-8859-1 instead.
> 
> Right.  git-gui is keying off the environment setting for LANG, so I
> guess its set to iso-8859-1 on your system but you are working with a
> utf-8 file.  We've talked about using something like .gitattributes
> to store encoding hints, or to just put a global gui setting in
> ~/.gitconfig but neither has had any patches written for it.
> 
> UTF-8 is seemingly the most common encoding that git-gui is mangling
> so maybe we should be defaulting to utf-8 until someone codes a
> more intelligent patch.

Currently there is no config variable for default encoding of file
contents (of blobs) and of filenames (of trees) because those do not
matter for core git.  But they do matter for GUI.

Show 14 quoted lines

> > Also, the hovering comment is INITIALLY shown garbled (both Author and
> > commit message), but if you click on a line, so that the commit
> > message is shown in the bottom window, the hovering message is
> > magically corrected to utf-8.
> > 
> > The text in the lower window (showing specific commits) seems to
> > always be handled correctly.
> 
> That's a "feature".  :-)
> 
> What's happening here is the initial hovering message is obtained
> from the machine formatted output from `git blame --incremental`
> and in that format there is no encoding header so I'm just ignoring
> any encoding problems.

So the correct solution would be to enhance "git blame --incremental" to output 'encoding' header when needed (when commit has encoding header and it is different from log output encoding).

Show 7 quoted lines

> Later when you click on a line it does `git cat-file commit $sha1`
> and gets the proper encoding, and corrects the strings it originally
> had gotten from git-blame.  So the hovering message "fixes" itself
> later on.
> 
> Maybe here too we should be defaulting to utf-8 instead of the
> native encoding.

I think this is a good idea, as git repositories are meant to be cross-operating system (which means cross-delault-encodings) compatible.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Finn Arne Gangstad· Dec 17, 2007, 07:50 UTC · re: Jakub Narebski · lore

Re: git gui blame utf-8 bugs

On Fri, Dec 14, 2007 at 04:39:59AM -0800, Jakub Narebski wrote:

Show 13 quoted lines

> "Shawn O. Pearce" <spearce@spearce.org> writes:
> 
> > Finn Arne Gangstad <finnag@pvv.org> wrote:
> > > git gui has some utf-8 bugs:
> > 
> > It has several.  :-)
> >  
> > > If you do git gui blame <file>, and the file contains utf-8 text,
> > > the lines are not parsed as utf-8, but seemingly as iso-8859-1 instead.
> > 
> > Right.  git-gui is keying off the environment setting for LANG, so I
> > guess its set to iso-8859-1 on your system but you are working with a
> > [...]

Setting LANG does not seem to have any effect at all (neiher for file contents, aurhor or commit messages).

LANG=en_US.UTF-8 git gui blame dir.c -> same LANG=utf-8 git gui blame dir.c -> same

- Finn Arne