{"thread":{"id":"61396","subject":"[BUG] 'ls-files path' fails if absolute path of workdir contains NFD (macOS)","startedAt":"2024-04-30T03:32:42Z","lastAt":"2024-04-30T16:58:48Z","messageCount":4,"participants":["Jun T","Torsten Bögershausen","Jun. T"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"493721","messageId":"55D75808-70E0-46FA-906D-C00CD612A262@kba.biglobe.ne.jp","threadId":"61396","inReplyTo":null,"subject":"[BUG] 'ls-files path' fails if absolute path of workdir contains NFD (macOS)","fromName":"Jun T","fromEmail":"takimoto-j@kba.biglobe.ne.jp","sentAt":"2024-04-30T03:27:02Z","receivedAt":"2024-04-30T03:32:42Z","isPatch":false,"sender":{"key":"takimoto-j@kba.biglobe.ne.jp","avatar":null},"body":"On macOS, 'git ls-files path' does not work (gives an error)\nif the absolute 'path' contains characters in NFD (decomposed).\nI guess this is a (minor) bug of git.\n\n[1] How to reproduce the problem\n\nOn macOS, git 2.39.3 or 2.45.0.219.gb9fe23f5ca(next branch),\nfile system can be APFS or HFS.\n\nwith zsh or bash:\n% cd /somewhere         # some safe place, /tmp or ~/tmp etc.\n% mkdir $'u\\xcc\\x88'    # ü in NFD\n% cd ü                  # or cd $'u\\xcc\\x88' or cd $'\\xc3\\xbc'\n% git init\n% git ls-files $'/somewhere/u\\xcc\\x88'   # NFD\nfatal: /somewhere/ü: '/somewhere/ü' is outside repository at '/somewhere/ü'\n% git ls-files $'/somewhere/\\xc3\\xbc'    # NFC\n(the same error as above)\n\nIn the 'fatal:' error message, there are three ü;\nthe 1st and 2nd are in NFC, the 3rd is in NFD.\n\n[2] Some analysis\n\nThe path on the command line $'/somewhere/u\\xcc\\x88'\nis converted to NFC by precompose_argv_prefix(),\ncalled at git.c:451 in run_builtin().\n\nBut get_git_work_tree() (called at setup.c:50, in\nabspath_part_inside_repo()) returns the work_tree in NFD,\nand comparing it with the path in NFC (setup.c:92) fails.\n\nI'm not familiar with git internals, but maybe\nget_git_work_tree() should return NFC (on macoS)?\n\n--\nJun (Jun-ichi Takimoto)"},{"id":"493736","messageId":"20240430071247.GB1504@tb-raspi4","threadId":"61396","inReplyTo":"55D75808-70E0-46FA-906D-C00CD612A262@kba.biglobe.ne.jp","subject":"Re: [BUG] 'ls-files path' fails if absolute path of workdir contains NFD (macOS)","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2024-04-30T07:12:47Z","receivedAt":"2024-04-30T07:13:03Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Tue, Apr 30, 2024 at 12:27:02PM +0900, Jun T wrote:\n> On macOS, 'git ls-files path' does not work (gives an error)\n> if the absolute 'path' contains characters in NFD (decomposed).\n> I guess this is a (minor) bug of git.\n>\n> [1] How to reproduce the problem\n>\n> On macOS, git 2.39.3 or 2.45.0.219.gb9fe23f5ca(next branch),\n> file system can be APFS or HFS.\n>\n> with zsh or bash:\n> % cd /somewhere         # some safe place, /tmp or ~/tmp etc.\n> % mkdir $'u\\xcc\\x88'    # ü in NFD\n> % cd ü                  # or cd $'u\\xcc\\x88' or cd $'\\xc3\\xbc'\n> % git init\n> % git ls-files $'/somewhere/u\\xcc\\x88'   # NFD\n> fatal: /somewhere/ü: '/somewhere/ü' is outside repository at '/somewhere/ü'\n> % git ls-files $'/somewhere/\\xc3\\xbc'    # NFC\n> (the same error as above)\n>\n> In the 'fatal:' error message, there are three ü;\n> the 1st and 2nd are in NFC, the 3rd is in NFD.\n>\n> [2] Some analysis\n>\n> The path on the command line $'/somewhere/u\\xcc\\x88'\n> is converted to NFC by precompose_argv_prefix(),\n> called at git.c:451 in run_builtin().\n>\n> But get_git_work_tree() (called at setup.c:50, in\n> abspath_part_inside_repo()) returns the work_tree in NFD,\n> and comparing it with the path in NFC (setup.c:92) fails.\n>\n> I'm not familiar with git internals, but maybe\n> get_git_work_tree() should return NFC (on macoS)?\n>\n> --\n> Jun (Jun-ichi Takimoto)\n\nThanks for an excellent bug report and analyzes.\nI am familar with the NFC/NFD stuff, but not with get_git_work_tree(),\nat least not yet.\n\nIf you have a suggestion for a patch, would you like to share it ?\nA 'git diff' may be a good start, I am happy to review things.\n\n/Torsten\n\n\n\n"},{"id":"493789","messageId":"44EB4C2D-5C86-4448-B8D8-3E90F75DB5CB@kba.biglobe.ne.jp","threadId":"61396","inReplyTo":"20240430071247.GB1504@tb-raspi4","subject":"Re: [BUG] 'ls-files path' fails if absolute path of workdir contains NFD (macOS)","fromName":"Jun. T","fromEmail":"takimoto-j@kba.biglobe.ne.jp","sentAt":"2024-04-30T15:52:38Z","receivedAt":"2024-04-30T15:52:52Z","isPatch":false,"sender":{"key":"takimoto-j@kba.biglobe.ne.jp","avatar":null},"body":"\n> 2024/04/30 16:12, Torsten Bögershausen <tboegi@web.de> wrote:\n> \n> I am familar with the NFC/NFD stuff, but not with get_git_work_tree(),\n> at least not yet.\n> \n> If you have a suggestion for a patch, would you like to share it ?\n\nWell, the only thing I can tell is the patch below _seems_ to fix\nthe _current_ problem. There may be other problems, it may introduce\nnew problem(s) (such as memory leak), etc.\n\nBut I don't know anything about the git internals and have no time\nnow to investigate further. Sorry.\n\nget_git_work_tree() just returns the value of the_repository->worktree\nbut I have no idea where this variable is set.\n\nget_git_work_tree() and the_repository->worktree are used in\nmany places, and I'm not sure changing the function/variable has\nno bad side effects or not.\n\nAnd how to convert into NFC? By precompose_string_if_needed()?\nDoes this function allocate a new memory for NFC? If so, do we\nneed to free it at some point?\n\n\ndiff --git a/setup.c b/setup.c\nindex f4b32f76e3..3f2f3ed016 100644\n--- a/setup.c\n+++ b/setup.c\n@@ -47,7 +47,7 @@ static int abspath_part_inside_repo(char *path)\n \tsize_t wtlen;\n \tchar *path0;\n \tint off;\n-\tconst char *work_tree = get_git_work_tree();\n+\tconst char *work_tree = precompose_string_if_needed(get_git_work_tree());\n \tstruct strbuf realpath = STRBUF_INIT;\n \n \tif (!work_tree)\n\n"},{"id":"493796","messageId":"20240430165831.GA3546@tb-raspi4","threadId":"61396","inReplyTo":"44EB4C2D-5C86-4448-B8D8-3E90F75DB5CB@kba.biglobe.ne.jp","subject":"Re: [BUG] 'ls-files path' fails if absolute path of workdir contains NFD (macOS)","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2024-04-30T16:58:31Z","receivedAt":"2024-04-30T16:58:48Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Wed, May 01, 2024 at 12:52:38AM +0900, Jun. T wrote:\n>\n> > 2024/04/30 16:12, Torsten Bögershausen <tboegi@web.de> wrote:\n> >\n> > I am familar with the NFC/NFD stuff, but not with get_git_work_tree(),\n> > at least not yet.\n> >\n> > If you have a suggestion for a patch, would you like to share it ?\n>\n> Well, the only thing I can tell is the patch below _seems_ to fix\n> the _current_ problem. There may be other problems, it may introduce\n> new problem(s) (such as memory leak), etc.\n>\n> But I don't know anything about the git internals and have no time\n> now to investigate further. Sorry.\n>\n> get_git_work_tree() just returns the value of the_repository->worktree\n> but I have no idea where this variable is set.\n>\n> get_git_work_tree() and the_repository->worktree are used in\n> many places, and I'm not sure changing the function/variable has\n> no bad side effects or not.\n>\n> And how to convert into NFC? By precompose_string_if_needed()?\n> Does this function allocate a new memory for NFC? If so, do we\n> need to free it at some point?\n>\n>\n> diff --git a/setup.c b/setup.c\n> index f4b32f76e3..3f2f3ed016 100644\n> --- a/setup.c\n> +++ b/setup.c\n> @@ -47,7 +47,7 @@ static int abspath_part_inside_repo(char *path)\n>  \tsize_t wtlen;\n>  \tchar *path0;\n>  \tint off;\n> -\tconst char *work_tree = get_git_work_tree();\n> +\tconst char *work_tree = precompose_string_if_needed(get_git_work_tree());\n>  \tstruct strbuf realpath = STRBUF_INIT;\n>\n>  \tif (!work_tree)\n>\n\nThanks for digging - I have spend some time to find the cause,\nbut no success yet.\nThere is even a set_git_work_tree() in environment.c, and that may\nneed some treatment - or some other place.\nTo be continued - I will continue digging, where the NFD comes into Git.\n\n"}]}