# Git on macOS shows committed files as untracked 4 messages from 2017-07-12 to 2017-07-13. Participants: roeder.git@mailnull.com, Jeff King, Torsten Bögershausen, Junio C Hamano. Thread: https://gitlist.dev/t/46366 ## roeder.git@mailnull.com, 2017-07-12 22:21 Subject: Git on macOS shows committed files as untracked Message-ID: <20170712222128.A2F7A246E3@outside.256stuff.com> URL: https://gitlist.dev/e/20170712222128.A2F7A246E3%40outside.256stuff.com ``` In Git on macOS (git version 2.13.2 | brew install git) the status command will show folders as untracked even though they are committed and checked out from the repository. Does not reproduce on Windows and Ubuntu. Repro steps: 1. Download https://www.dropbox.com/s/0q5pbpqpckwzj7b/gitstatusrepro.zip?dl=0 2. unzip gitstatusrepro.zip && cd gitstatusrepro 3. git reset --hard HEAD is now at 95fcd7e add folder with unicode name 4. git status On branch master Untracked files: (use "git add ..." to include in what will be committed) "d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/" nothing added to commit but untracked files present (use "git add" to track) ---------- This message was sent from a MailNull anti-spam account. You can get your free account and take control over your email by visiting the following URL. http://mailnull.com/ ``` ## Jeff King, 2017-07-12 23:15 Subject: Re: Git on macOS shows committed files as untracked Message-ID: <20170712231557.lu7ppj3bric2fahi@sigill.intra.peff.net> URL: https://gitlist.dev/e/20170712231557.lu7ppj3bric2fahi%40sigill.intra.peff.net In-Reply-To: <20170712222128.A2F7A246E3@outside.256stuff.com> ``` On Wed, Jul 12, 2017 at 06:21:28PM -0400, roeder.git@mailnull.com wrote: > In Git on macOS (git version 2.13.2 | brew install git) the status > command will show folders as untracked even though they are committed > and checked out from the repository. Does not reproduce on Windows and > Ubuntu. > [...] > > "d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/" Probably the issue has to do with Unicode normalization, and you have files in your repository that can't be represented on your filesystem. For example, the first two code-points above are "d" followed by U+0307, "COMBINING DOT ABOVE". That pair can also be represented as U+1E0B, "LATIN SMALL LETTER D WITH DOT ABOVE". I don't recall which form HFS+ normalizes to, but basically what happens is that Git opens the file with some name, and the filesystem quietly rewrites that under the hood to a different, normalized name. Then when Git walks the directory later to ask which files are present, it sees this other filename that it has no clue about. Generally the solution is to commit the normalized name. There's some logic inside Git to "precompose" names to the right normalization, but I think that only affects new files you add. Existing committed files with the wrong normalization run into this issue. -Peff ``` ## Torsten Bögershausen, 2017-07-13 08:42 Subject: Re: Git on macOS shows committed files as untracked Message-ID: <130b338b-2e77-65d6-a3fa-f272ac43b81c@web.de> URL: https://gitlist.dev/e/130b338b-2e77-65d6-a3fa-f272ac43b81c%40web.de In-Reply-To: <20170712231557.lu7ppj3bric2fahi@sigill.intra.peff.net> ``` On 13/07/17 01:15, Jeff King wrote: > On Wed, Jul 12, 2017 at 06:21:28PM -0400, roeder.git@mailnull.com wrote: > >> In Git on macOS (git version 2.13.2 | brew install git) the status >> command will show folders as untracked even though they are committed >> and checked out from the repository. Does not reproduce on Windows and >> Ubuntu. >> [...] >> >> "d\314\207\316\271\314\223\314\200\342\225\223\316\265\357\256\257\360\222\221\217\342\227\213\342\225\223\320\243\314\213/" > > Probably the issue has to do with Unicode normalization, and you have > files in your repository that can't be represented on your filesystem. > For example, the first two code-points above are "d" followed by U+0307, > "COMBINING DOT ABOVE". That pair can also be represented as U+1E0B, > "LATIN SMALL LETTER D WITH DOT ABOVE". > > I don't recall which form HFS+ normalizes to, but basically what happens > is that Git opens the file with some name, and the filesystem quietly > rewrites that under the hood to a different, normalized name. Then when > Git walks the directory later to ask which files are present, it sees > this other filename that it has no clue about. > > Generally the solution is to commit the normalized name. There's some > logic inside Git to "precompose" names to the right normalization, but I > think that only affects new files you add. Existing committed files with > the wrong normalization run into this issue. > > -Peff > Thanks for the fast analyzes - in short: what does git -c core.precomposeunicode=true status say ? The easiest thing may be to set git config --global core.precomposeunicode true ``` ## Junio C Hamano, 2017-07-13 19:35 Subject: Re: Git on macOS shows committed files as untracked Message-ID: URL: https://gitlist.dev/e/xmqq60eww2nd.fsf%40gitster.mtv.corp.google.com In-Reply-To: <130b338b-2e77-65d6-a3fa-f272ac43b81c@web.de> ``` Torsten Bögershausen writes: > Thanks for the fast analyzes - > in short: > what does > git -c core.precomposeunicode=true status > say ? > > The easiest thing may be to set > git config --global core.precomposeunicode true Good suggestion. I learned a new thing today. I somehow thought that precompose trick was only about argv[] when a program starts up and did not apply to paths readdir(3) finds through dir.c, e.g. $ git add . But apparently there is replacement readdir() used in compat/ for MacOSX so the paths from the system are also covered by the configuration. ```