{"thread":{"id":"34549","subject":"[PATCH/RFC] core.precomposeunicode is true by default","startedAt":"2013-07-27T01:21:31Z","lastAt":"2013-08-27T19:34:09Z","messageCount":10,"participants":["Torsten Bögershausen","Duy Nguyen","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"224162","messageId":"201307270321.32398.tboegi@web.de","threadId":"34549","inReplyTo":null,"subject":"[PATCH/RFC] core.precomposeunicode is true by default","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-07-27T01:21:31Z","receivedAt":"2013-07-27T01:21:31Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"When core.precomposeunicode was introduced, it was set to false\nby default, to be compatible with older versions of Git.\n\nWhenever UTF-8 file names are used in a mixed environment,\nthe Mac OS users need to find out that this configuration exist\nand set it to true manually.\n\nThere is no measurable performance impact between false and true.\nA smoother workflow can be achieved for new Git users,\nso change the default to true:\n\n- Remove the auto-sensing\n- Rename the internal variable into precompose_unicode,\n  and set it to 1 meaning true.\n- Adjust and clean up test cases\n\nThe configuration core.precomposeunicode is still supported.\n\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n Documentation/config.txt     |  2 +-\n builtin/init-db.c            |  1 -\n cache.h                      |  2 +-\n compat/precompose_utf8.c     | 28 ++--------------------------\n compat/precompose_utf8.h     |  1 -\n config.c                     |  2 +-\n environment.c                |  2 +-\n git-compat-util.h            |  1 -\n t/t0050-filesystem.sh        |  1 +\n t/t3910-mac-os-precompose.sh | 14 --------------\n t/t7400-submodule-basic.sh   |  1 -\n 11 files changed, 7 insertions(+), 48 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex e0b923f..abe4cfa 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -240,7 +240,7 @@ core.precomposeunicode::\n \tThis option is only used by Mac OS implementation of Git.\n \tWhen core.precomposeunicode=true, Git reverts the unicode decomposition\n \tof filenames done by Mac OS. This is useful when sharing a repository\n-\tbetween Mac OS and Linux or Windows.\n+\tbetween Mac OS and Linux or Windows. True by default.\n \t(Git for Windows 1.7.10 or higher is needed, or Git under cygwin 1.7).\n \tWhen false, file names are handled fully transparent by Git,\n \twhich is backward compatible with older versions of Git.\ndiff --git a/builtin/init-db.c b/builtin/init-db.c\nindex 78aa387..08854d5 100644\n--- a/builtin/init-db.c\n+++ b/builtin/init-db.c\n@@ -290,7 +290,6 @@ static int create_default_files(const char *template_path)\n \t\tstrcpy(path + len, \"CoNfIg\");\n \t\tif (!access(path, F_OK))\n \t\t\tgit_config_set(\"core.ignorecase\", \"true\");\n-\t\tprobe_utf8_pathname_composition(path, len);\n \t}\n \n \treturn reinit;\ndiff --git a/cache.h b/cache.h\nindex 4c606ce..abb2cad 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -593,7 +593,7 @@ extern int read_replace_refs;\n extern int fsync_object_files;\n extern int core_preload_index;\n extern int core_apply_sparse_checkout;\n-extern int precomposed_unicode;\n+extern int precompose_unicode;\n \n /*\n  * The character that begins a commented line in user-editable file\ndiff --git a/compat/precompose_utf8.c b/compat/precompose_utf8.c\nindex 7980abd..5396b91 100644\n--- a/compat/precompose_utf8.c\n+++ b/compat/precompose_utf8.c\n@@ -36,30 +36,6 @@ static size_t has_non_ascii(const char *s, size_t maxlen, size_t *strlen_c)\n }\n \n \n-void probe_utf8_pathname_composition(char *path, int len)\n-{\n-\tstatic const char *auml_nfc = \"\\xc3\\xa4\";\n-\tstatic const char *auml_nfd = \"\\x61\\xcc\\x88\";\n-\tint output_fd;\n-\tif (precomposed_unicode != -1)\n-\t\treturn; /* We found it defined in the global config, respect it */\n-\tstrcpy(path + len, auml_nfc);\n-\toutput_fd = open(path, O_CREAT|O_EXCL|O_RDWR, 0600);\n-\tif (output_fd >= 0) {\n-\t\tclose(output_fd);\n-\t\tstrcpy(path + len, auml_nfd);\n-\t\t/* Indicate to the user, that we can configure it to true */\n-\t\tif (!access(path, R_OK))\n-\t\t\tgit_config_set(\"core.precomposeunicode\", \"false\");\n-\t\t/* To be backward compatible, set precomposed_unicode to 0 */\n-\t\tprecomposed_unicode = 0;\n-\t\tstrcpy(path + len, auml_nfc);\n-\t\tif (unlink(path))\n-\t\t\tdie_errno(_(\"failed to unlink '%s'\"), path);\n-\t}\n-}\n-\n-\n void precompose_argv(int argc, const char **argv)\n {\n \tint i = 0;\n@@ -67,7 +43,7 @@ void precompose_argv(int argc, const char **argv)\n \tchar *newarg;\n \ticonv_t ic_precompose;\n \n-\tif (precomposed_unicode != 1)\n+\tif (!precompose_unicode)\n \t\treturn;\n \n \tic_precompose = iconv_open(repo_encoding, path_encoding);\n@@ -130,7 +106,7 @@ struct dirent_prec_psx *precompose_utf8_readdir(PREC_DIR *prec_dir)\n \t\tprec_dir->dirent_nfc->d_ino  = res->d_ino;\n \t\tprec_dir->dirent_nfc->d_type = res->d_type;\n \n-\t\tif ((precomposed_unicode == 1) && has_non_ascii(res->d_name, (size_t)-1, NULL)) {\n+\t\tif (precompose_unicode && has_non_ascii(res->d_name, (size_t)-1, NULL)) {\n \t\t\tif (prec_dir->ic_precompose == (iconv_t)-1) {\n \t\t\t\tdie(\"iconv_open(%s,%s) failed, but needed:\\n\"\n \t\t\t\t\t\t\"    precomposed unicode is not supported.\\n\"\ndiff --git a/compat/precompose_utf8.h b/compat/precompose_utf8.h\nindex 3b73585..488f81d 100644\n--- a/compat/precompose_utf8.h\n+++ b/compat/precompose_utf8.h\n@@ -27,7 +27,6 @@ typedef struct {\n } PREC_DIR;\n \n void precompose_argv(int argc, const char **argv);\n-void probe_utf8_pathname_composition(char *, int);\n \n PREC_DIR *precompose_utf8_opendir(const char *dirname);\n struct dirent_prec_psx *precompose_utf8_readdir(PREC_DIR *dirp);\ndiff --git a/config.c b/config.c\nindex e13a7b6..c181767 100644\n--- a/config.c\n+++ b/config.c\n@@ -827,7 +827,7 @@ static int git_default_core_config(const char *var, const char *value)\n \t}\n \n \tif (!strcmp(var, \"core.precomposeunicode\")) {\n-\t\tprecomposed_unicode = git_config_bool(var, value);\n+\t\tprecompose_unicode = git_config_bool(var, value);\n \t\treturn 0;\n \t}\n \ndiff --git a/environment.c b/environment.c\nindex 0cb67b2..b98f14b 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -58,7 +58,7 @@ char *notes_ref_name;\n int grafts_replace_parents = 1;\n int core_apply_sparse_checkout;\n int merge_log_config = -1;\n-int precomposed_unicode = -1; /* see probe_utf8_pathname_composition() */\n+int precompose_unicode = 1; /* default: true */\n struct startup_info *startup_info;\n unsigned long pack_size_limit_cfg;\n \ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex cc4ba4d..3f6f27c 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -185,7 +185,6 @@ typedef unsigned long uintptr_t;\n #else\n #define precompose_str(in,i_nfd2nfc)\n #define precompose_argv(c,v)\n-#define probe_utf8_pathname_composition(a,b)\n #endif\n \n #ifdef NEEDS_CLIPPED_WRITE\ndiff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\nindex 05d78d2..6b3cedc 100755\n--- a/t/t0050-filesystem.sh\n+++ b/t/t0050-filesystem.sh\n@@ -91,6 +91,7 @@ test_expect_failure CASE_INSENSITIVE_FS 'add (with different case)' '\n test_expect_success \"setup unicode normalization tests\" '\n \ttest_create_repo unicode &&\n \tcd unicode &&\n+\tgit config core.precomposeunicode false &&\n \ttouch \"$aumlcdiar\" &&\n \tgit add \"$aumlcdiar\" &&\n \tgit commit -m initial &&\ndiff --git a/t/t3910-mac-os-precompose.sh b/t/t3910-mac-os-precompose.sh\nindex 5fe57c5..b853d6a 100755\n--- a/t/t3910-mac-os-precompose.sh\n+++ b/t/t3910-mac-os-precompose.sh\n@@ -34,11 +34,6 @@ Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc           #50 Byte\n Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc           #250 Byte\n Alongc=$Alongc$AEligatu$AEligatu                     #254 Byte\n \n-test_expect_success \"detect if nfd needed\" '\n-\tprecomposeunicode=`git config core.precomposeunicode` &&\n-\ttest \"$precomposeunicode\" = false &&\n-\tgit config core.precomposeunicode true\n-'\n test_expect_success \"setup\" '\n \t>x &&\n \tgit add x &&\n@@ -140,14 +135,5 @@ test_expect_success \"Add long precomposed filename\" '\n \tgit add * &&\n \tgit commit -m \"Long filename\"\n '\n-# Test if the global core.precomposeunicode stops autosensing\n-# Must be the last test case\n-test_expect_success \"respect git config --global core.precomposeunicode\" '\n-\tgit config --global core.precomposeunicode true &&\n-\trm -rf .git &&\n-\tgit init &&\n-\tprecomposeunicode=`git config core.precomposeunicode` &&\n-\ttest \"$precomposeunicode\" = \"true\"\n-'\n \n test_done\ndiff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh\nindex 5ee97b0..f0f8cde 100755\n--- a/t/t7400-submodule-basic.sh\n+++ b/t/t7400-submodule-basic.sh\n@@ -958,7 +958,6 @@ test_expect_success 'submodule with UTF-8 name' '\n \t\tgit add sub &&\n \t\tgit commit -m \"init sub\"\n \t) &&\n-\ttest_config core.precomposeunicode true &&\n \tgit submodule add ./\"$svname\" &&\n \tgit submodule >&2 &&\n \ttest -n \"$(git submodule | grep \"$svname\")\"\n-- \n1.8.3\n"},{"id":"224186","messageId":"CACsJy8AT--3Lus-3F4-8EHL_3--a-98RkJxaMMALaSjgV+D3WQ@mail.gmail.com","threadId":"34549","inReplyTo":"201307270321.32398.tboegi@web.de","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-07-27T15:23:52Z","receivedAt":"2013-07-27T15:23:52Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sat, Jul 27, 2013 at 8:21 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n> When core.precomposeunicode was introduced, it was set to false\n> by default, to be compatible with older versions of Git.\n>\n> Whenever UTF-8 file names are used in a mixed environment,\n> the Mac OS users need to find out that this configuration exist\n> and set it to true manually.\n>\n> There is no measurable performance impact between false and true.\n> A smoother workflow can be achieved for new Git users,\n> so change the default to true:\n>\n> - Remove the auto-sensing\n> - Rename the internal variable into precompose_unicode,\n>   and set it to 1 meaning true.\n> - Adjust and clean up test cases\n>\n> The configuration core.precomposeunicode is still supported.\n\nDoes this have any effects on non-utf8 users? I'm on utf-8, so this is\nnot really my concern, that is unless it changes something on LANG=C..\n-- \nDuy\n"},{"id":"224191","messageId":"51F44F4D.7030106@web.de","threadId":"34549","inReplyTo":"CACsJy8AT--3Lus-3F4-8EHL_3--a-98RkJxaMMALaSjgV+D3WQ@mail.gmail.com","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-07-27T22:53:01Z","receivedAt":"2013-07-27T22:53:01Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2013-07-27 17.23, Duy Nguyen wrote:\n> On Sat, Jul 27, 2013 at 8:21 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n>> When core.precomposeunicode was introduced, it was set to false\n>> by default, to be compatible with older versions of Git.\n>>\n>> Whenever UTF-8 file names are used in a mixed environment,\n>> the Mac OS users need to find out that this configuration exist\n>> and set it to true manually.\n>>\n>> There is no measurable performance impact between false and true.\n>> A smoother workflow can be achieved for new Git users,\n>> so change the default to true:\n>>\n>> - Remove the auto-sensing\n>> - Rename the internal variable into precompose_unicode,\n>>   and set it to 1 meaning true.\n>> - Adjust and clean up test cases\n>>\n>> The configuration core.precomposeunicode is still supported.\n> \n> Does this have any effects on non-utf8 users? I'm on utf-8, so this is\n> not really my concern, that is unless it changes something on LANG=C..\n> \nNot sure if I fully understand the question.\n\nMac OS will always use UTF-8, and we can choose between\nprecomposesd and decomposed.\n\nWindows (\"Git for Windows\" == msysgit) uses UTF-8 (precomposed)\nGit under cygwin 1.7 uses UTF-8, precomposed.\nGit under cygwin 1.5 or git compiled under mingw does not use\nUTF-8, but a Windows code page\"\n\nLinux may use UTF-8 or ISO-8859 or whatever you configure.\n\nThis change affects only Mac OS, \n(should this be stated better in the commit MSG?)\n\nAnd if somebody wants to change a repo between Linux, Windows\nand/or Mac OS, everybody should use UTF-8 (precomposed) for filenames,\ndirectories and branches.\n(or stick to ASCII)\nDoes this answer the question?\n/Torsten\n\n\n \n"},{"id":"224196","messageId":"CACsJy8Dyaf0f5rk-NS_H-M8_t7WMGJyL+z_RaEYraGqbuZ+JiA@mail.gmail.com","threadId":"34549","inReplyTo":"51F44F4D.7030106@web.de","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-07-28T04:45:47Z","receivedAt":"2013-07-28T04:45:47Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Sun, Jul 28, 2013 at 5:53 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n>> Does this have any effects on non-utf8 users? I'm on utf-8, so this is\n>> not really my concern, that is unless it changes something on LANG=C..\n>>\n> Not sure if I fully understand the question.\n>\n> Mac OS will always use UTF-8, and we can choose between\n> precomposesd and decomposed.\n>\n> Windows (\"Git for Windows\" == msysgit) uses UTF-8 (precomposed)\n> Git under cygwin 1.7 uses UTF-8, precomposed.\n> Git under cygwin 1.5 or git compiled under mingw does not use\n> UTF-8, but a Windows code page\"\n>\n> Linux may use UTF-8 or ISO-8859 or whatever you configure.\n>\n> This change affects only Mac OS,\n> (should this be stated better in the commit MSG?)\n\nAh yes, I see precompose_unicode changes default value from -1 to 1,\nbut I did not see that the whole precompose unicode code is only\nactive in Mac OS.\n\n>\n> And if somebody wants to change a repo between Linux, Windows\n> and/or Mac OS, everybody should use UTF-8 (precomposed) for filenames,\n> directories and branches.\n> (or stick to ASCII)\n> Does this answer the question?\n\nIt does, thanks.\n-- \nDuy\n"},{"id":"224237","messageId":"7vmwp5z3iu.fsf@alter.siamese.dyndns.org","threadId":"34549","inReplyTo":"201307270321.32398.tboegi@web.de","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-07-29T17:20:41Z","receivedAt":"2013-07-29T17:20:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen <tboegi@web.de> writes:\n\n> When core.precomposeunicode was introduced, it was set to false\n> by default, to be compatible with older versions of Git.\n>\n> Whenever UTF-8 file names are used in a mixed environment,\n> the Mac OS users need to find out that this configuration exist\n> and set it to true manually.\n>\n> There is no measurable performance impact between false and true.\n\nThe real reason we default it to auto-sensing in the current code is\nfor correctness, I think. the new precompose code could be buggy,\nand by auto-sensing, we hoped that we would enable it only on\nfilesystems that the codepath matters.\n\n> A smoother workflow can be achieved for new Git users,\n> so change the default to true:\n>\n> - Remove the auto-sensing\n\nWhy?\n\n> - Rename the internal variable into precompose_unicode,\n>   and set it to 1 meaning true.\n\nWhy the rename?\n\n> - Adjust and clean up test cases\n>\n> The configuration core.precomposeunicode is still supported.\n\nSorry, but I do not quite understand the change.  Is this because\nthe auto-sensing is not working, or after auto-sensing we do a wrong\nthing?  If that is the case, perhaps that is what we should fix?\n\n> diff --git a/compat/precompose_utf8.c b/compat/precompose_utf8.c\n> index 7980abd..5396b91 100644\n> --- a/compat/precompose_utf8.c\n> +++ b/compat/precompose_utf8.c\n> @@ -36,30 +36,6 @@ static size_t has_non_ascii(const char *s, size_t maxlen, size_t *strlen_c)\n>  }\n>  \n>  \n> -void probe_utf8_pathname_composition(char *path, int len)\n> -{\n> -\tstatic const char *auml_nfc = \"\\xc3\\xa4\";\n> -\tstatic const char *auml_nfd = \"\\x61\\xcc\\x88\";\n> -\tint output_fd;\n> -\tif (precomposed_unicode != -1)\n> -\t\treturn; /* We found it defined in the global config, respect it */\n> -\tstrcpy(path + len, auml_nfc);\n> -\toutput_fd = open(path, O_CREAT|O_EXCL|O_RDWR, 0600);\n\nSo we try to create a path under one name, and ...\n\n> -\tif (output_fd >= 0) {\n> -\t\tclose(output_fd);\n> -\t\tstrcpy(path + len, auml_nfd);\n> -\t\t/* Indicate to the user, that we can configure it to true */\n> -\t\tif (!access(path, R_OK))\n> -\t\t\tgit_config_set(\"core.precomposeunicode\", \"false\");\n\n... see if that path can be seen under its alias.  Why do we set it\nto \"false\"?  Isn't this the true culprit?\n\nAfter all, this is not in the \"reinit\" codepath, so we know we are\ndealing with a repository that was created afresh.\n"},{"id":"225989","messageId":"521CAD88.4080609@web.de","threadId":"34549","inReplyTo":"7vmwp5z3iu.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-08-27T13:45:44Z","receivedAt":"2013-08-27T13:45:44Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"(Sorry for the somewhat late reply, thanks for review)\n>Torsten Bögershausen <tboegi@web.de> writes:\n>\n>> When core.precomposeunicode was introduced, it was set to false\n>> by default, to be compatible with older versions of Git.\n>>\n>> Whenever UTF-8 file names are used in a mixed environment,\n>> the Mac OS users need to find out that this configuration exist\n>> and set it to true manually.\n>>\n>> There is no measurable performance impact between false and true.\n>\n>The real reason we default it to auto-sensing in the current code is\n>for correctness, I think. the new precompose code could be buggy,\n>and by auto-sensing, we hoped that we would enable it only on\n>filesystems that the codepath matters.\n>\n>> A smoother workflow can be achieved for new Git users,\n>> so change the default to true:\n>>\n>> - Remove the auto-sensing\n>\n>Why?\n>\n>> - Rename the internal variable into precompose_unicode,\n>>   and set it to 1 meaning true.\n>\n>Why the rename?\n>\n>> - Adjust and clean up test cases\n>>\n>> The configuration core.precomposeunicode is still supported.\n>\n>Sorry, but I do not quite understand the change.  Is this because\n>the auto-sensing is not working, or after auto-sensing we do a wrong\n>thing?  If that is the case, perhaps that is what we should fix?\n>\n>> diff --git a/compat/precompose_utf8.c b/compat/precompose_utf8.c\n>> index 7980abd..5396b91 100644\n>> --- a/compat/precompose_utf8.c\n>> +++ b/compat/precompose_utf8.c\n>> @@ -36,30 +36,6 @@ static size_t has_non_ascii(const char *s, size_t maxlen, size_t *strlen_c)\n>>  }\n>>  \n>>  \n>> -void probe_utf8_pathname_composition(char *path, int len)\n>> -{\n>> -\tstatic const char *auml_nfc = \"\\xc3\\xa4\";\n>> -\tstatic const char *auml_nfd = \"\\x61\\xcc\\x88\";\n>> -\tint output_fd;\n>> -\tif (precomposed_unicode != -1)\n>> -\t\treturn; /* We found it defined in the global config, respect it */\n>> -\tstrcpy(path + len, auml_nfc);\n>> -\toutput_fd = open(path, O_CREAT|O_EXCL|O_RDWR, 0600);\n>\n>So we try to create a path under one name, and ...\n>\n>> -\tif (output_fd >= 0) {\n>> -\t\tclose(output_fd);\n>> -\t\tstrcpy(path + len, auml_nfd);\n>> -\t\t/* Indicate to the user, that we can configure it to true */\n>> -\t\tif (!access(path, R_OK))\n>> -\t\t\tgit_config_set(\"core.precomposeunicode\", \"false\");\n>\n>... see if that path can be seen under its alias.  Why do we set it\n>to \"false\"?  Isn't this the true culprit?\n>\n>After all, this is not in the \"reinit\" codepath, so we know we are\n>dealing with a repository that was created afresh.\n>\n\nThere is nothing wrong with the auto-sensing as such.\nThe problem for many users today is that we set core.precomposeunicode\nto false, when it should be true.\n\nA patch for that comes out in a minute. But first look back and \ncollect some experience with core.precomposeunicode.\n\nLets have a look at the variable \"precomposed_unicode\",\n(the one I wanted to rename to be more consistant).\nIt is controlled by the git config files and\ndepending on the config it is set like this:\ncore.precomposeuinicode false -> precomposed_unicode = 0\ncore.precomposeuinicode true  -> precomposed_unicode = 1\ncore.precomposeuinicode <not set> -> precomposed_unicode = -1.\n\nLet's look what precomposed_unicode does and go through a couple\nof git operations.\n\n1)\nWhen we create a repo under Mac OS using HFS+,\nwe want to have precomposed_unicode = 1\n\n2)\nWhen we access a repo from Windows/Linux using SAMBA,\nreaddir() will return decomposed.\nWhen the repo is created by nonMacOS, core.precomposeunicode is undefined.\nThe precomposition is off, but should be on, \nprecomposed_unicode = -1, but should be = 1\n\n3)\nWhen we access a repo from another Mac OS system using \nSAMBA, NFS or AFP readdir() will return decomposed.\nAs the repo is created under Mac OS, we have the same case as (1)\n\n4)\nWhen we access a repo from Linux using NFS we can have\nprecomposed_unicode = 0 (which is technically more correct).\nIf Linux users do not use decomposed unicode in their file names,\n(according to my understanding this is the case), we can use 1\nas well as 0:\nprecomposing an already precomposed string is a no-op, so it doesn't\nharm.\n\n\n5)\nWhen we create a repo under Linux/Windows on a USB-drive,\nand run \"git status\" under Mac OS, we want precomposed_unicode = 1.\n\nThere are few cases where we want to use precomposed_unicode=0:\na) To work around bugs. This may be a short term solution,\n  I would rather see bugs to be fixed.\n  I'm not aware of any bugs, so please remind me if I missed something.\n\nb) Working with foreign vcs:  E.g. bzr and hg use decomposed unicode,\n   so it may be better to use decomposed unicode in git as well.\n\nThe simplified V2 patch looks like this (I send it in a seperate mail):\n\ndiff --git a/compat/precompose_utf8.c b/compat/precompose_utf8.c\nindex 7980abd..95fe849 100644\n--- a/compat/precompose_utf8.c\n+++ b/compat/precompose_utf8.c\n@@ -48,11 +48,8 @@ void probe_utf8_pathname_composition(char *path, int len)\n \tif (output_fd >= 0) {\n \t\tclose(output_fd);\n \t\tstrcpy(path + len, auml_nfd);\n-\t\t/* Indicate to the user, that we can configure it to true */\n-\t\tif (!access(path, R_OK))\n-\t\t\tgit_config_set(\"core.precomposeunicode\", \"false\");\n-\t\t/* To be backward compatible, set precomposed_unicode to 0 */\n-\t\tprecomposed_unicode = 0;\n+\t\tprecomposed_unicode = access(path, R_OK) ? 0 : 1;\n+\t\tgit_config_set(\"core.precomposeunicode\", precomposed_unicode ? \"true\" : \"false\");\n\nThis will not affect existing repos, as they should have\ncore.precomposeunicode either true or false.\nWhen a new repo is created, users having core.precomposeunicode in the\nglobal config are not effected,\nand the global setting switches off the auto-sensing.\n\nThe users which are new to git are affected,\nand hopefully we do the right thing for them.\nAt least, according to my understanding, we do the best for majority\nof users and use cases.\n\nThoughts are welcome, and arguments for and against V1 or V2.\n\nAnybody who uses Mac OS and has experience with decomposed unicode?\nI have core.precomposeunicode true in my global config.\n \n"},{"id":"225996","messageId":"xmqqvc2rfau9.fsf@gitster.dls.corp.google.com","threadId":"34549","inReplyTo":"521CAD88.4080609@web.de","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-08-27T14:49:02Z","receivedAt":"2013-08-27T14:49:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen <tboegi@web.de> writes:\n\n>>... see if that path can be seen under its alias.  Why do we set it\n>>to \"false\"?  Isn't this the true culprit?\n>>\n>>After all, this is not in the \"reinit\" codepath, so we know we are\n>>dealing with a repository that was created afresh.\n>\n> There is nothing wrong with the auto-sensing as such.\n> The problem for many users today is that we set core.precomposeunicode\n> to false, when it should be true.\n\nI think we are in agreement then.\n\nThe code detects a broken filesystem just fine, but what it does\nwhen it finds the filesystem is broken is wrong---it sets the\nvariable to false.  That makes the whole auto-sensing wrong, and I\nthink it makes sense to correct that behaviour.\n\n> Let's look what precomposed_unicode does and go through a couple\n> of git operations.\n>\n> 1)\n> When we create a repo under Mac OS using HFS+,\n> we want to have precomposed_unicode = 1\n\nYes.\n\n> 2)\n> When we access a repo from Windows/Linux using SAMBA,\n\nYou mean s/repo/repository that resides on HFS+/?\n\n> readdir() will return decomposed.\n> When the repo is created by nonMacOS, core.precomposeunicode is undefined.\n> The precomposition is off, but should be on, \n> precomposed_unicode = -1, but should be = 1\n\nI do not think UTF-8-MAC is widely available; even if you flip the\nbit on, would it help much?\n"},{"id":"226000","messageId":"521CC07B.4000504@web.de","threadId":"34549","inReplyTo":"xmqqvc2rfau9.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-08-27T15:06:35Z","receivedAt":"2013-08-27T15:06:35Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 27.08.13 16:49, Junio C Hamano wrote:\n> Torsten Bögershausen <tboegi@web.de> writes:\n>\n>>> ... see if that path can be seen under its alias.  Why do we set it\n>>> to \"false\"?  Isn't this the true culprit?\n>>>\n>>> After all, this is not in the \"reinit\" codepath, so we know we are\n>>> dealing with a repository that was created afresh.\n>> There is nothing wrong with the auto-sensing as such.\n>> The problem for many users today is that we set core.precomposeunicode\n>> to false, when it should be true.\n> I think we are in agreement then.\n>\n> The code detects a broken filesystem just fine, but what it does\n> when it finds the filesystem is broken is wrong---it sets the\n> variable to false.  That makes the whole auto-sensing wrong, and I\n> think it makes sense to correct that behaviour.\n>\n>> Let's look what precomposed_unicode does and go through a couple\n>> of git operations.\n>>\n>> 1)\n>> When we create a repo under Mac OS using HFS+,\n>> we want to have precomposed_unicode = 1\n> Yes.\n>\n>> 2)\n>> When we access a repo from Windows/Linux using SAMBA,\n> You mean s/repo/repository that resides on HFS+/?\nSorry being unclear here, trying being clearer with an example:\nI have a /data/Docs on my linux box, which is handled by git\n\nI export /data/Docs via SAMBA, and use the Finder under Mac OS to have it\nmounted on my Mac OS X box:\n//tb@Linux/Docs on /Volumes/Docs (smbfs, nodev, nosuid, mounted by tb)\n>> readdir() will return decomposed.\n>> When the repo is created by nonMacOS, core.precomposeunicode is undefined.\n>> The precomposition is off, but should be on, \n>> precomposed_unicode = -1, but should be = 1\n> I do not think UTF-8-MAC is widely available; even if you flip the\n> bit on, would it help much?\nIn the above example\n/data/Docs/.git/config was created by Linux, so it does not have\ncore.precomposeunicode set, neither false nor true.\nThe Linux box does not have  UTF-8-MAC under iconv,\nbut will ignore core.precomposeunicode anyway (since the code is not compiled here)\n\nThe Mac OS machine sees it under /Volumes/Docs/.git/config\nAnd here we want the precomposition, even if core.precomposeunicode\nis not present in the config.\n\n \n"},{"id":"226013","messageId":"xmqqhaebdrp6.fsf@gitster.dls.corp.google.com","threadId":"34549","inReplyTo":"521CC07B.4000504@web.de","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-08-27T16:27:49Z","receivedAt":"2013-08-27T16:27:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen <tboegi@web.de> writes:\n\n>>> 2)\n>>> When we access a repo from Windows/Linux using SAMBA,\n>> You mean s/repo/repository that resides on HFS+/?\n> Sorry being unclear here, trying being clearer with an example:\n> I have a /data/Docs on my linux box, which is handled by git\n>\n> I export /data/Docs via SAMBA, and use the Finder under Mac OS to have it\n> mounted on my Mac OS X box:\n> //tb@Linux/Docs on /Volumes/Docs (smbfs, nodev, nosuid, mounted by tb)\n>>> readdir() will return decomposed.\n>>> When the repo is created by nonMacOS, core.precomposeunicode is undefined.\n>>> The precomposition is off, but should be on, \n>>> precomposed_unicode = -1, but should be = 1\n>> I do not think UTF-8-MAC is widely available; even if you flip the\n>> bit on, would it help much?\n> In the above example\n> /data/Docs/.git/config was created by Linux, so it does not have\n> core.precomposeunicode set, neither false nor true.\n> The Linux box does not have  UTF-8-MAC under iconv,\n> but will ignore core.precomposeunicode anyway (since the code is not compiled here)\n>\n> The Mac OS machine sees it under /Volumes/Docs/.git/config\n> And here we want the precomposition, even if core.precomposeunicode\n> is not present in the config.\n\nIt almost makes me wonder if you want not a per-repository but a\nper-machine configuration, i.e. \"Whichever repository I am\naccessing, either on a local filesystem or shared over the network,\nreaddir() on my box reports wrong paths, and I need correction.\"\n\nThat, or \"if it hurts, don't do the remote mount then.\"\n"},{"id":"226048","messageId":"521CFF31.9070607@web.de","threadId":"34549","inReplyTo":"xmqqhaebdrp6.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH/RFC] core.precomposeunicode is true by default","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-08-27T19:34:09Z","receivedAt":"2013-08-27T19:34:09Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2013-08-27 18.27, Junio C Hamano wrote:\n> Torsten Bögershausen <tboegi@web.de> writes:\n> \n>>>> 2)\n>>>> When we access a repo from Windows/Linux using SAMBA,\n>>> You mean s/repo/repository that resides on HFS+/?\n>> Sorry being unclear here, trying being clearer with an example:\n>> I have a /data/Docs on my linux box, which is handled by git\n>>\n>> I export /data/Docs via SAMBA, and use the Finder under Mac OS to have it\n>> mounted on my Mac OS X box:\n>> //tb@Linux/Docs on /Volumes/Docs (smbfs, nodev, nosuid, mounted by tb)\n>>>> readdir() will return decomposed.\n>>>> When the repo is created by nonMacOS, core.precomposeunicode is undefined.\n>>>> The precomposition is off, but should be on, \n>>>> precomposed_unicode = -1, but should be = 1\n>>> I do not think UTF-8-MAC is widely available; even if you flip the\n>>> bit on, would it help much?\n>> In the above example\n>> /data/Docs/.git/config was created by Linux, so it does not have\n>> core.precomposeunicode set, neither false nor true.\n>> The Linux box does not have  UTF-8-MAC under iconv,\n>> but will ignore core.precomposeunicode anyway (since the code is not compiled here)\n>>\n>> The Mac OS machine sees it under /Volumes/Docs/.git/config\n>> And here we want the precomposition, even if core.precomposeunicode\n>> is not present in the config.\n> \n> It almost makes me wonder if you want not a per-repository but a\n> per-machine configuration, i.e. \"Whichever repository I am\n> accessing, either on a local filesystem or shared over the network,\n> readdir() on my box reports wrong paths, and I need correction.\"\n> \n> That, or \"if it hurts, don't do the remote mount then.\"\nNo, we don't need to be that restrictive.\nWe already have repository/user/system wide configuration files,\nallowing tweeks and this is a good thing.\n\nRe-Re-reading $gmane/188940: \nI am happy having the V2 patch from today being queued, thanks.\n\nAs a next step I will have a look into the advice machine.\n \n"}]}