threads / discuss / 46366

Git on macOS shows committed files as untracked

Subject: Git on macOS shows committed files as untracked

## tl;dr

4 messages between Jul 12, 2017 and Jul 13, 2017.

replies: 3people: 4as markdown or json

roeder.git@mailnull.com· Jul 12, 2017, 22:21 UTC · lore
In Git on macOS (git version 2.13.2 | brew install git) the status command will show folders as untracked even though they are committed and checked out from the repository. Does not reproduce on Windows and Ubuntu.
Repro steps:
1. Download https://www.dropbox.com/s/0q5pbpqpckwzj7b/gitstatusrepro.zip?dl=0
2. unzip gitstatusrepro.zip && cd gitstatusrepro
3. git reset --hard
HEAD is now at 95fcd7e add folder with unicode name
4. git status
On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	"d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/"
nothing added to commit but untracked files present (use "git add" to track)

---------- This message was sent from a MailNull anti-spam account. You can get your free account and take control over your email by visiting the following URL.

   http://mailnull.com/
Jeff King· Jul 12, 2017, 23:15 UTC · re: roeder.git@mailnull.com · lore

Re: Git on macOS shows committed files as untracked

On Wed, Jul 12, 2017 at 06:21:28PM -0400, roeder.git@mailnull.com wrote:
Show 7 quoted lines
> In Git on macOS (git version 2.13.2 | brew install git) the status
> command will show folders as untracked even though they are committed
> and checked out from the repository. Does not reproduce on Windows and
> Ubuntu.
> [...]
> 
> 	"d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/"

Probably the issue has to do with Unicode normalization, and you have files in your repository that can't be represented on your filesystem. For example, the first two code-points above are "d" followed by U+0307, "COMBINING DOT ABOVE". That pair can also be represented as U+1E0B, "LATIN SMALL LETTER D WITH DOT ABOVE".

I don't recall which form HFS+ normalizes to, but basically what happens is that Git opens the file with some name, and the filesystem quietly rewrites that under the hood to a different, normalized name. Then when Git walks the directory later to ask which files are present, it sees this other filename that it has no clue about.

Generally the solution is to commit the normalized name. There's some logic inside Git to "precompose" names to the right normalization, but I think that only affects new files you add. Existing committed files with the wrong normalization run into this issue.

-Peff
Torsten Bögershausen· Jul 13, 2017, 08:42 UTC · re: Jeff King · lore

Re: Git on macOS shows committed files as untracked

On 13/07/17 01:15, Jeff King wrote:
Show 29 quoted lines
> On Wed, Jul 12, 2017 at 06:21:28PM -0400, roeder.git@mailnull.com wrote:
> 
>> In Git on macOS (git version 2.13.2 | brew install git) the status
>> command will show folders as untracked even though they are committed
>> and checked out from the repository. Does not reproduce on Windows and
>> Ubuntu.
>> [...]
>>
>> 	"d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/"
> 
> Probably the issue has to do with Unicode normalization, and you have
> files in your repository that can't be represented on your filesystem.
> For example, the first two code-points above are "d" followed by U+0307,
> "COMBINING DOT ABOVE". That pair can also be represented as U+1E0B,
> "LATIN SMALL LETTER D WITH DOT ABOVE".
> 
> I don't recall which form HFS+ normalizes to, but basically what happens
> is that Git opens the file with some name, and the filesystem quietly
> rewrites that under the hood to a different, normalized name. Then when
> Git walks the directory later to ask which files are present, it sees
> this other filename that it has no clue about.
> 
> Generally the solution is to commit the normalized name. There's some
> logic inside Git to "precompose" names to the right normalization, but I
> think that only affects new files you add. Existing committed files with
> the wrong normalization run into this issue.
> 
> -Peff
>

Thanks for the fast analyzes - in short: what does git -c core.precomposeunicode=true status say ?

The easiest thing may be to set git config --global core.precomposeunicode true

Junio C Hamano· Jul 13, 2017, 19:35 UTC · re: Torsten Bögershausen · lore

Re: Git on macOS shows committed files as untracked

Torsten Bögershausen <tboegi@web.de> writes:
Show 8 quoted lines
> Thanks for the fast analyzes -
> in short:
> what does
> git -c core.precomposeunicode=true status
> say ?
>
> The easiest thing may be to set
> git config --global core.precomposeunicode true
Good suggestion.

I learned a new thing today. I somehow thought that precompose trick was only about argv[] when a program starts up and did not apply to paths readdir(3) finds through dir.c, e.g.

    $ git add .

But apparently there is replacement readdir() used in compat/ for MacOSX so the paths from the system are also covered by the configuration.

← back to recent threads