threads / discuss / 9771

HFS+ Unicode weirdness

Subject: HFS+ Unicode weirdness

## tl;dr

4 messages between Sep 4, 2007 and Sep 4, 2007.

replies: 3people: 3as markdown or json

Wincent Colaiuta· Sep 4, 2007, 12:30 UTC · lore

On a brand new clone of git.git the file "gitweb/test/Märchen" is provoking some weird behaviour running on Mac OS X and the toy HFS+ filesystem. Note how the unmodified checkout of the file is shown as "untracked" by "git st", but on deleting the file it's shown as "deleted". If I build a copy of Git based on the clean working tree then the resulting build has a version number of "1.5.x-dirty".

Any suggestions on where to start investigating the cause of this? About the only lead I have is that if I create a file with that name by typing it's name it's encoded as "Ma\314\210rchen", but the file in the git.git repo is encoded as "M\303\244rchen".

Cheers, Wincent

$ git clone git://git.kernel.org/pub/scm/git/git.git
Initialized empty Git repository in /tmp/git/.git/
remote: Generating pack...
remote: Counting objects: 12259
Done counting 58277 objects.
remote: Deltifying 58277 objects...
remote:  100% (58277/58277) done
Indexing 58277 objects...
remote: Total 58277 (delta 40699), reused 58034 (delta 40521)
  100% (58277/58277) done
Resolving 40699 deltas...
  100% (40699/40699) done
$ cd git
/tmp/git
$ git st
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       gitweb/test/Märchen
nothing added to commit but untracked files present (use "git add" to  
track)
$ ls -laF gitweb/test
total 24
drwxr-xr-x   5 wincent  wheel  170 Sep  4 14:18 ./
drwxr-xr-x   9 wincent  wheel  306 Sep  4 14:18 ../
-rw-r--r--   1 wincent  wheel   17 Sep  4 14:18 Ma??rchen
-rw-r--r--   1 wincent  wheel   31 Sep  4 14:18 file with spaces
-rw-r--r--   1 wincent  wheel   37 Sep  4 14:18 file+plus+sign
$ rm gitweb/test/Märchen
$ git st
# On branch master
# Changed but not updated:
#   (use "git add/rm <file>..." to update what will be committed)
#
#       deleted:    gitweb/test/Märchen
#
no changes added to commit (use "git add" and/or "git commit -a")
Johannes Schindelin· Sep 4, 2007, 13:00 UTC · re: Wincent Colaiuta · lore

Re: HFS+ Unicode weirdness

Hi,
On Tue, 4 Sep 2007, Wincent Colaiuta wrote:
> On a brand new clone of git.git the file "gitweb/test/M?rchen" is provoking
> some weird behaviour running on Mac OS X and the toy HFS+ filesystem.

Please search the mail archives. This has come up quite a couple of times.

Ciao, Dscho

Wincent Colaiuta· Sep 4, 2007, 13:59 UTC · re: Johannes Schindelin · lore

Re: HFS+ Unicode weirdness

El 4/9/2007, a las 15:00, Johannes Schindelin escribió:
Show 10 quoted lines
> Hi,
>
> On Tue, 4 Sep 2007, Wincent Colaiuta wrote:
>
>> On a brand new clone of git.git the file "gitweb/test/M?rchen" is  
>> provoking
>> some weird behaviour running on Mac OS X and the toy HFS+ filesystem.
>
> Please search the mail archives.  This has come up quite a couple of
> times.
Finally found it: for the record this thread seems to be relevant:
<http://marc.info/?l=git&m=117053359117549&w=2>

For what it's worth Subversion has exactly the same kind of problem when only some developers in a team work on HFS. For example:

<http://svn.haxx.se/users/archive-2005-12/0373.shtml> <http://subversion.tigris.org/issues/show_bug.cgi?id=2464>

Hopefully the wait for Apple to release a non-broken filesystem won't be too long, ZFS being the most likely candidate at this stage.

Cheers, Wincent

David Symonds· Sep 4, 2007, 13:07 UTC · re: Wincent Colaiuta · lore

Re: HFS+ Unicode weirdness

On 04/09/07, Wincent Colaiuta <win@wincent.com> wrote:
Show 11 quoted lines
> On a brand new clone of git.git the file "gitweb/test/Märchen" is
> provoking some weird behaviour running on Mac OS X and the toy HFS+
> filesystem. Note how the unmodified checkout of the file is shown as
> "untracked" by "git st", but on deleting the file it's shown as
> "deleted". If I build a copy of Git based on the clean working tree
> then the resulting build has a version number of "1.5.x-dirty".
>
> Any suggestions on where to start investigating the cause of this?
> About the only lead I have is that if I create a file with that name
> by typing it's name  it's encoded as "Ma\314\210rchen", but the file
> in the git.git repo is encoded as "M\303\244rchen".

It's because OS X is using composite Unicode encoding (storing "ä" as "a" + umlaut). Git doesn't canonicalise things, since OS X does its jiggery-pokery of that kind of stuff in CoreFramework, which I believe Git doesn't use.

Dave.

← back to recent threads