{"thread":{"id":"48364","subject":"BUG report: unicode normalization on APFS (Mac OS High Sierra)","startedAt":"2018-04-26T16:48:08Z","lastAt":"2018-04-30T15:48:00Z","messageCount":11,"participants":["Elijah Newren","Torsten Bögershausen","Totsten Bögershausen","tboegi@web.de","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"345902","messageId":"CABPp-BEe+BK6Ew0ZLnkgCn=1J6kyJa5VTaibuJOEVeDdgLd_Ow@mail.gmail.com","threadId":"48364","inReplyTo":null,"subject":"BUG report: unicode normalization on APFS (Mac OS High Sierra)","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2018-04-26T16:48:03Z","receivedAt":"2018-04-26T16:48:08Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On HFS (which appears to be the default Mac filesystem prior to High\nSierra), unicode names are \"normalized\" before recording. Thus with a\nscript like:\n\n mkdir tmp\n cd tmp\n\n auml=$(printf \"\\303\\244\")\n aumlcdiar=$(printf \"\\141\\314\\210\")\n >\"$auml\"\n\n echo \"auml: \" $(echo -n \"$auml\" | xxd)\n echo \"aumlcdiar: \" $(echo -n \"$aumlcdiar\" | xxd)\n echo \"Dir contents: \" $(echo -n * | xxd)\n\n echo \"Stat auml: \" \"$(stat -f \"%i %Sm %Su %N\" \"$auml\")\"\n echo \"Stat aumlcdiar:\" \"$(stat -f \"%i %Sm %Su %N\" \"$aumlcdiar\")\"\n\nWe see output like:\n\n auml: 00000000: c3a4 ..\n aumlcdiar: 00000000: 61cc 88 a..\n Dir contents: 00000000: 61cc 88 a..\n Stat auml: 857473 Apr 26 09:40:40 2018 newren ä\n Stat aumlcdiar: 857473 Apr 26 09:40:40 2018 newren ä\n\nOn APFS, which appears to be the new default filesystem in Mac OS High\nSierra, we instead see:\n\n auml: 00000000: c3a4 ..\n aumlcdiar: 00000000: 61cc 88 a..\n Dir contents: 00000000: c3a4 ..\n Stat auml: 8591766636 Apr 26 09:40:59 2018 newren ä\n Stat aumlcdiar: 8591766636 Apr 26 09:40:59 2018 newren ä\n\ni.e. APFS appears to record the filename as specified by the user, but\ncontinues to allow the user to access it via any name that normalizes\nto the same thing. This difference causes t0050-filesystem.sh to fail\nthe final two tests. I could change the \"UTF8_NFD_TO_NFC\" flag\nchecking in test-lib.sh to instead test the exit code of stat to make\nit pass these two tests, but I have no idea if there are problems\nelsewhere that this would just be papering over.\n\nI dislike Mac OS and avoid it, so I'd prefer to find someone else\nmotivated to fix this. If no one is, I may eventually try to fix this\nup...in a year or three from now. But is someone else interested?\nWould this serve as a good microproject for our microprojects list (or\nare the internals hairy enough that this is too big of a project for\nthat list)?\n\n\nElijah\n"},{"id":"345903","messageId":"94c6fa4d-afe0-5a08-f844-85d3c091d3b2@web.de","threadId":"48364","inReplyTo":"CABPp-BEe+BK6Ew0ZLnkgCn=1J6kyJa5VTaibuJOEVeDdgLd_Ow@mail.gmail.com","subject":"Re: BUG report: unicode normalization on APFS (Mac OS High Sierra)","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2018-04-26T17:13:46Z","receivedAt":"2018-04-26T17:14:01Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 26.04.18 18:48, Elijah Newren wrote:\n> On HFS (which appears to be the default Mac filesystem prior to High\n> Sierra), unicode names are \"normalized\" before recording. Thus with a\n> script like:\n> \n> mkdir tmp\n> cd tmp\n> \n> auml=$(printf \"\\303\\244\")\n> aumlcdiar=$(printf \"\\141\\314\\210\")\n> >\"$auml\"\n> \n> echo \"auml: \" $(echo -n \"$auml\" | xxd)\n> echo \"aumlcdiar: \" $(echo -n \"$aumlcdiar\" | xxd)\n> echo \"Dir contents: \" $(echo -n * | xxd)\n> \n> echo \"Stat auml: \" \"$(stat -f \"%i %Sm %Su %N\" \"$auml\")\"\n> echo \"Stat aumlcdiar:\" \"$(stat -f \"%i %Sm %Su %N\" \"$aumlcdiar\")\"\n> \n> We see output like:\n> \n> auml: 00000000: c3a4 ..\n> aumlcdiar: 00000000: 61cc 88 a..\n> Dir contents: 00000000: 61cc 88 a..\n> Stat auml: 857473 Apr 26 09:40:40 2018 newren ä\n> Stat aumlcdiar: 857473 Apr 26 09:40:40 2018 newren ä\n> \n> On APFS, which appears to be the new default filesystem in Mac OS High\n> Sierra, we instead see:\n> \n> auml: 00000000: c3a4 ..\n> aumlcdiar: 00000000: 61cc 88 a..\n> Dir contents: 00000000: c3a4 ..\n> Stat auml: 8591766636 Apr 26 09:40:59 2018 newren ä\n> Stat aumlcdiar: 8591766636 Apr 26 09:40:59 2018 newren ä\n> \n> i.e. APFS appears to record the filename as specified by the user, but\n> continues to allow the user to access it via any name that normalizes\n> to the same thing. This difference causes t0050-filesystem.sh to fail\n> the final two tests. I could change the \"UTF8_NFD_TO_NFC\" flag\n> checking in test-lib.sh to instead test the exit code of stat to make\n> it pass these two tests, but I have no idea if there are problems\n> elsewhere that this would just be papering over.\n> \n> I dislike Mac OS and avoid it, so I'd prefer to find someone else\n> motivated to fix this. If no one is, I may eventually try to fix this\n> up...in a year or three from now. But is someone else interested?\n> Would this serve as a good microproject for our microprojects list (or\n> are the internals hairy enough that this is too big of a project for\n> that list)?\n> \n> \n> Elijah\n> \n\nHm,\nthanks for the report.\nI don't have a high sierra box, but I can probably get one.\nt0050 -should- pass automagically, so I feel that I can do something.\nUnless someone is faster of course.\n\nIs it possible that you run\ndebug=t verbose=t ./t0050-filesystem.sh \nand send the output to me ?\n\n\n\n"},{"id":"345905","messageId":"CABPp-BHitvta8we8di-tFiNdVV7vXnMNAhiAs2=CrQc-gGuSJw@mail.gmail.com","threadId":"48364","inReplyTo":"94c6fa4d-afe0-5a08-f844-85d3c091d3b2@web.de","subject":"Re: BUG report: unicode normalization on APFS (Mac OS High Sierra)","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2018-04-26T17:23:02Z","receivedAt":"2018-04-26T17:23:07Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Apr 26, 2018 at 10:13 AM, Torsten Bögershausen wrote:\n> Hm,\n> thanks for the report.\n> I don't have a high sierra box, but I can probably get one.\n> t0050 -should- pass automagically, so I feel that I can do something.\n> Unless someone is faster of course.\n\nSweet, thanks for taking a look.\n\n> Is it possible that you run\n> debug=t verbose=t ./t0050-filesystem.sh\n> and send the output to me ?\n\nSure. First, though, note that I can make it pass (or at least \"not\nok...TODO known breakage\") with the following patch (may be\nwhitespace-damaged by gmail):\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex 483c8d6d7..770b91f8c 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n auml=$(printf \"\\303\\244\")\n aumlcdiar=$(printf \"\\141\\314\\210\")\n >\"$auml\" &&\n- case \"$(echo *)\" in\n- \"$aumlcdiar\")\n- true ;;\n- *)\n- false ;;\n- esac\n+ stat \"$aumlcdiar\" >/dev/null 2>/dev/null\n '\n\n test_lazy_prereq AUTOIDENT '\n\n\nI'm just worried there are bugs elsewhere in dealing with filesystems\nlike this that would need to be fixed and that this papers over them.\n\nAnyway, the output you requested, at least for the last two failing tests, is:\n\n\nexpecting success:\ngit mv \"$aumlcdiar\" \"$auml\" &&\ngit commit -m rename\n\nfatal: destination exists, source=ä, destination=ä\nnot ok 9 - rename (silent unicode normalization)\n\n#\n# git mv \"$aumlcdiar\" \"$auml\" &&\n# git commit -m rename\n#\n\nexpecting success:\ngit reset --hard initial &&\ngit merge topic\n\nHEAD is now at 1b3caf6 initial\nUpdating 1b3caf6..2db1bf9\nerror: The following untracked working tree files would be overwritten by merge:\nä\nPlease move or remove them before you merge.\nAborting\nnot ok 10 - merge (silent unicode normalization)\n\n#\n# git reset --hard initial &&\n# git merge topic\n#\n\n# still have 1 known breakage(s)\n# failed 2 among remaining 9 test(s)\n"},{"id":"346009","messageId":"7f9feab0-3638-56cd-18bb-4de0122aabad@web.de","threadId":"48364","inReplyTo":"CABPp-BHitvta8we8di-tFiNdVV7vXnMNAhiAs2=CrQc-gGuSJw@mail.gmail.com","subject":"Re: BUG report: unicode normalization on APFS (Mac OS High Sierra)","fromName":"Totsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2018-04-27T21:45:42Z","receivedAt":"2018-04-27T21:45:53Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\n\nOn 2018-04-26 19:23, Elijah Newren wrote:\n> On Thu, Apr 26, 2018 at 10:13 AM, Torsten Bögershausen wrote:\n>> Hm,\n>> thanks for the report.\n>> I don't have a high sierra box, but I can probably get one.\n>> t0050 -should- pass automagically, so I feel that I can do something.\n>> Unless someone is faster of course.\n> \n> Sweet, thanks for taking a look.\n> \n>> Is it possible that you run\n>> debug=t verbose=t ./t0050-filesystem.sh\n>> and send the output to me ?\n> \n> Sure. First, though, note that I can make it pass (or at least \"not\n> ok...TODO known breakage\") with the following patch (may be\n> whitespace-damaged by gmail):\n> \n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index 483c8d6d7..770b91f8c 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n> auml=$(printf \"\\303\\244\")\n> aumlcdiar=$(printf \"\\141\\314\\210\")\n> >\"$auml\" &&\n> - case \"$(echo *)\" in\n> - \"$aumlcdiar\")\n> - true ;;\n> - *)\n> - false ;;\n> - esac\n> + stat \"$aumlcdiar\" >/dev/null 2>/dev/null\n\nNicely analyzed and improved.\n\nThe \"stat\" statement is technically correct.\nI think that a more git-style fix would be\n[] ---\n+ test -r \"$aumlcdiar\"\n\ninstead of the stat.\n\nI looked into the 2 known breakages.\nIn short: they test use cases which are not sooo important for a user in \npractice, but do a good test if the code is broken.\nIOW: I can't see a need for immediate action.\n\nAs you already did all the analyzes:\nDo you want to send a patch ?\n"},{"id":"346124","messageId":"20180430063519.27122-1-tboegi@web.de","threadId":"48364","inReplyTo":"CABPp-BHitvta8we8di-tFiNdVV7vXnMNAhiAs2=CrQc-gGuSJw@mail.gmail.com","subject":"[PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2018-04-30T06:35:19Z","receivedAt":"2018-04-30T06:35:27Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nOn HFS (which is the default Mac filesystem prior to High Sierra),\nunicode names are \"decomposed\" before recording.\nOn APFS, which appears to be the new default filesystem in Mac OS High\nSierra, filenames are recorded as specified by the user.\n\nAPFS continues to allow the user to access it via any name\nthat normalizes to the same thing.\n\nThis difference causes t0050-filesystem.sh to fail two tests.\n\nImprove the test for a NFD/NFC in test-lib.sh:\nTest if the same file can be reached in pre- and decomposed unicode.\n\nReported-By: Elijah Newren \nSigned-off-by: Torsten Bögershausen \n---\n t/test-lib.sh | 7 +------\n 1 file changed, 1 insertion(+), 6 deletions(-)\n\ndiff --git a/t/test-lib.sh b/t/test-lib.sh\nindex ea2bbaaa7a..e206250d1b 100644\n--- a/t/test-lib.sh\n+++ b/t/test-lib.sh\n@@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n \tauml=$(printf \"\\303\\244\")\n \taumlcdiar=$(printf \"\\141\\314\\210\")\n \t>\"$auml\" &&\n-\tcase \"$(echo *)\" in\n-\t\"$aumlcdiar\")\n-\t\ttrue ;;\n-\t*)\n-\t\tfalse ;;\n-\tesac\n+\ttest -r \"$aumlcdiar\"\n '\n \n test_lazy_prereq AUTOIDENT '\n-- \n2.16.0.rc0.8.g5497051b43\n\n"},{"id":"346125","messageId":"xmqqvac9xh4d.fsf@gitster-ct.c.googlers.com","threadId":"48364","inReplyTo":"20180430063519.27122-1-tboegi@web.de","subject":"Re: [PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-04-30T07:56:02Z","receivedAt":"2018-04-30T07:56:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> On HFS (which is the default Mac filesystem prior to High Sierra),\n> unicode names are \"decomposed\" before recording.\n> On APFS, which appears to be the new default filesystem in Mac OS High\n> Sierra, filenames are recorded as specified by the user.\n>\n> APFS continues to allow the user to access it via any name\n> that normalizes to the same thing.\n>\n> This difference causes t0050-filesystem.sh to fail two tests.\n>\n> Improve the test for a NFD/NFC in test-lib.sh:\n> Test if the same file can be reached in pre- and decomposed unicode.\n>\n> Reported-By: Elijah Newren \n> Signed-off-by: Torsten Bögershausen \n> ---\n> t/test-lib.sh | 7 +------\n> 1 file changed, 1 insertion(+), 6 deletions(-)\n\nThanks. \n\nWouldn't it logically make more sense to check for the target being\nan existing file with \"-f\"? It is not an essential part of the test\nfor the target to be \"readable\", but \"can be stat(2)ed with the\nother UTF-8 representation\" is.\n\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index ea2bbaaa7a..e206250d1b 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n> \tauml=$(printf \"\\303\\244\")\n> \taumlcdiar=$(printf \"\\141\\314\\210\")\n> \t>\"$auml\" &&\n> -\tcase \"$(echo *)\" in\n> -\t\"$aumlcdiar\")\n> -\t\ttrue ;;\n> -\t*)\n> -\t\tfalse ;;\n> -\tesac\n> +\ttest -r \"$aumlcdiar\"\n> '\n> \n> test_lazy_prereq AUTOIDENT '\n"},{"id":"346128","messageId":"255708be-3cf2-0285-b5e7-4fbc9e0e8ddd@web.de","threadId":"48364","inReplyTo":"xmqqvac9xh4d.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2018-04-30T08:55:50Z","receivedAt":"2018-04-30T08:56:03Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 30.04.18 09:56, Junio C Hamano wrote:\n> tboegi@web.de writes:\n> \n>> From: Torsten Bögershausen \n>>\n>> On HFS (which is the default Mac filesystem prior to High Sierra),\n>> unicode names are \"decomposed\" before recording.\n>> On APFS, which appears to be the new default filesystem in Mac OS High\n>> Sierra, filenames are recorded as specified by the user.\n>>\n>> APFS continues to allow the user to access it via any name\n>> that normalizes to the same thing.\n>>\n>> This difference causes t0050-filesystem.sh to fail two tests.\n>>\n>> Improve the test for a NFD/NFC in test-lib.sh:\n>> Test if the same file can be reached in pre- and decomposed unicode.\n>>\n>> Reported-By: Elijah Newren \n>> Signed-off-by: Torsten Bögershausen \n>> ---\n>> t/test-lib.sh | 7 +------\n>> 1 file changed, 1 insertion(+), 6 deletions(-)\n> \n> Thanks. \n> \n> Wouldn't it logically make more sense to check for the target being\n> an existing file with \"-f\"? It is not an essential part of the test\n> for the target to be \"readable\", but \"can be stat(2)ed with the\n> other UTF-8 representation\" is.\n\nThat make sense.\nWould you like to amend the patch ?\n\n\n"},{"id":"346154","messageId":"CABPp-BG_uYuZuKzARRKgoKmihEVuibyG8E-QZ-XB79Yz0+GHAw@mail.gmail.com","threadId":"48364","inReplyTo":"7f9feab0-3638-56cd-18bb-4de0122aabad@web.de","subject":"Re: BUG report: unicode normalization on APFS (Mac OS High Sierra)","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2018-04-30T15:29:04Z","receivedAt":"2018-04-30T15:29:09Z","isPatch":false,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Fri, Apr 27, 2018 at 2:45 PM, Totsten Bögershausen wrote:\n> On 2018-04-26 19:23, Elijah Newren wrote:\n\n>> Sure. First, though, note that I can make it pass (or at least \"not\n>> ok...TODO known breakage\") with the following patch (may be\n>> whitespace-damaged by gmail):\n>>\n>> diff --git a/t/test-lib.sh b/t/test-lib.sh\n>> index 483c8d6d7..770b91f8c 100644\n>> --- a/t/test-lib.sh\n>> +++ b/t/test-lib.sh\n>> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n>> auml=$(printf \"\\303\\244\")\n>> aumlcdiar=$(printf \"\\141\\314\\210\")\n>> >\"$auml\" &&\n>> - case \"$(echo *)\" in\n>> - \"$aumlcdiar\")\n>> - true ;;\n>> - *)\n>> - false ;;\n>> - esac\n>> + stat \"$aumlcdiar\" >/dev/null 2>/dev/null\n>\n>\n> Nicely analyzed and improved.\n>\n> The \"stat\" statement is technically correct.\n> I think that a more git-style fix would be\n> [] ---\n> + test -r \"$aumlcdiar\"\n>\n> instead of the stat.\n>\n> I looked into the 2 known breakages.\n> In short: they test use cases which are not sooo important for a user in\n> practice, but do a good test if the code is broken.\n> IOW: I can't see a need for immediate action.\n>\n> As you already did all the analyzes:\n> Do you want to send a patch ?\n\nYou know, despite seeing the \"test_expect_failure\" and \"TODO...known\nbreakage\" with these tests and even mentioning them, it somehow didn't\nsink in and I was still thinking that there might be some kind of\nunicode normalization handling in the codebase somewhere (similar to\nthe case insensitivy handling that I've seen in a place or two) that\nnow needed to be extended. I should have realized that\ntest_expect_failure meant there wasn't, and thus all we needed to do\nwas to mark it as continuing to fail with the new filesystem, Should\nhave realized, but didn't. Oops.\n\nAnyway, it looks like you've already submitted a patch and marked it\nas having been reported by me, which is just fine. Thanks!\n\nElijah\n"},{"id":"346161","messageId":"CABPp-BHa7y8ggoCoLY0z4_=3njH=UV8GscXaz_YWvfNbfmPxhA@mail.gmail.com","threadId":"48364","inReplyTo":"20180430063519.27122-1-tboegi@web.de","subject":"Re: [PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2018-04-30T15:33:05Z","receivedAt":"2018-04-30T15:33:09Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Sun, Apr 29, 2018 at 11:35 PM, wrote:\n> From: Torsten Bögershausen \n>\n> On HFS (which is the default Mac filesystem prior to High Sierra),\n> unicode names are \"decomposed\" before recording.\n> On APFS, which appears to be the new default filesystem in Mac OS High\n> Sierra, filenames are recorded as specified by the user.\n>\n> APFS continues to allow the user to access it via any name\n> that normalizes to the same thing.\n>\n> This difference causes t0050-filesystem.sh to fail two tests.\n>\n> Improve the test for a NFD/NFC in test-lib.sh:\n> Test if the same file can be reached in pre- and decomposed unicode.\n>\n> Reported-By: Elijah Newren \n> Signed-off-by: Torsten Bögershausen \n> ---\n> t/test-lib.sh | 7 +------\n> 1 file changed, 1 insertion(+), 6 deletions(-)\n>\n> diff --git a/t/test-lib.sh b/t/test-lib.sh\n> index ea2bbaaa7a..e206250d1b 100644\n> --- a/t/test-lib.sh\n> +++ b/t/test-lib.sh\n> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n\nI'm not sure what \"NFD\" and \"NFC\" stand for, but I suspect the test\nprerequisite name may be specific to how HFS handled things. If so,\nshould it be renamed from UTF8_NFD_TO_NFC to something else, such as\nUTF8_NORMALIZATION?\n\n> auml=$(printf \"\\303\\244\")\n> aumlcdiar=$(printf \"\\141\\314\\210\")\n> >\"$auml\" &&\n> - case \"$(echo *)\" in\n> - \"$aumlcdiar\")\n> - true ;;\n> - *)\n> - false ;;\n> - esac\n> + test -r \"$aumlcdiar\"\n> '\n>\n> test_lazy_prereq AUTOIDENT '\n> --\n> 2.16.0.rc0.8.g5497051b43\n"},{"id":"346164","messageId":"86b1fc05-b22c-cb9a-b5b5-d5e819d97c1e@web.de","threadId":"48364","inReplyTo":"CABPp-BHa7y8ggoCoLY0z4_=3njH=UV8GscXaz_YWvfNbfmPxhA@mail.gmail.com","subject":"Re: [PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2018-04-30T15:41:35Z","receivedAt":"2018-04-30T15:41:40Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 30.04.18 17:33, Elijah Newren wrote:\n> Hi,\n> \n> On Sun, Apr 29, 2018 at 11:35 PM, wrote:\n>> From: Torsten Bögershausen \n>>\n>> On HFS (which is the default Mac filesystem prior to High Sierra),\n>> unicode names are \"decomposed\" before recording.\n>> On APFS, which appears to be the new default filesystem in Mac OS High\n>> Sierra, filenames are recorded as specified by the user.\n>>\n>> APFS continues to allow the user to access it via any name\n>> that normalizes to the same thing.\n>>\n>> This difference causes t0050-filesystem.sh to fail two tests.\n>>\n>> Improve the test for a NFD/NFC in test-lib.sh:\n>> Test if the same file can be reached in pre- and decomposed unicode.\n>>\n>> Reported-By: Elijah Newren \n>> Signed-off-by: Torsten Bögershausen \n>> ---\n>> t/test-lib.sh | 7 +------\n>> 1 file changed, 1 insertion(+), 6 deletions(-)\n>>\n>> diff --git a/t/test-lib.sh b/t/test-lib.sh\n>> index ea2bbaaa7a..e206250d1b 100644\n>> --- a/t/test-lib.sh\n>> +++ b/t/test-lib.sh\n>> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n> \n> I'm not sure what \"NFD\" and \"NFC\" stand for, but I suspect the test\n> prerequisite name may be specific to how HFS handled things. If so,\n> should it be renamed from UTF8_NFD_TO_NFC to something else, such as\n> UTF8_NORMALIZATION?\n\nNFD and NFC both come from the unicode standard, and are just taken\n\"as is\" into the Git world:\nhttps://en.wikipedia.org/wiki/Unicode_equivalence\n\nIf you are otherwise happy with the patch, would it be possible\nto run it on your system ?\n(I don't have a High Sierra box, but I am confident that the test work\n for you).\n\nThe other comments may be addressed later, may be.\nIn any case, they should go into a different commit.\n \n"},{"id":"346166","messageId":"CABPp-BH54kVUnHR3r=XH9b6RhnQ2NpFSqVUnNyQU6M9b1tywMg@mail.gmail.com","threadId":"48364","inReplyTo":"86b1fc05-b22c-cb9a-b5b5-d5e819d97c1e@web.de","subject":"Re: [PATCH v1 1/1] test: Correct detection of UTF8_NFD_TO_NFC for APFS","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2018-04-30T15:47:55Z","receivedAt":"2018-04-30T15:48:00Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Apr 30, 2018 at 8:41 AM, Torsten Bögershausen wrote:\n> On 30.04.18 17:33, Elijah Newren wrote:\n>\n>> On Sun, Apr 29, 2018 at 11:35 PM, wrote:\n>>> From: Torsten Bögershausen \n>>>\n\n>>> @@ -1106,12 +1106,7 @@ test_lazy_prereq UTF8_NFD_TO_NFC '\n>>\n>> I'm not sure what \"NFD\" and \"NFC\" stand for, but I suspect the test\n>> prerequisite name may be specific to how HFS handled things. If so,\n>> should it be renamed from UTF8_NFD_TO_NFC to something else, such as\n>> UTF8_NORMALIZATION?\n>\n> NFD and NFC both come from the unicode standard, and are just taken\n> \"as is\" into the Git world:\n> https://en.wikipedia.org/wiki/Unicode_equivalence\n>\n> If you are otherwise happy with the patch, would it be possible\n> to run it on your system ?\n> (I don't have a High Sierra box, but I am confident that the test work\n> for you).\n\nSure, feel free to add a\n\nTested-by: Elijah Newren \n\nif you like. (Works with both 'test -r', as in your patch, and 'test\n-f', as suggested by Junio.)\n"}]}