{"thread":{"id":"29362","subject":"[RFC][PATCH v3] git on Mac OS and precomposed unicode","startedAt":"2012-01-13T21:53:00Z","lastAt":"2012-01-13T22:56:00Z","messageCount":2,"participants":["tb","Jonathan Nieder"],"isPatch":true,"patchVersion":3,"patchTotal":null},"messages":[{"id":"182524","messageId":"201201132253.00799.tboegi@web.de","threadId":"29362","inReplyTo":null,"subject":"[RFC][PATCH v3] git on Mac OS and precomposed unicode","fromName":"tb","fromEmail":"tboegi@web.de","sentAt":"2012-01-13T21:53:00Z","receivedAt":"2012-01-13T21:53:00Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"Changes since V2:\n- renamed compat/darwin.[ch] into compat/precomposed_utf8.[ch]\n- builtin/init-db.c: use probe_utf8_pathname_composition()\n- fix in reencode_string_iconv(): Don't call iconv_close()\n- Allways convert all arguments, no special handling for \"git commit\"\nChanges since last version:\n- Improved testcase t/t3910-mac-os-precompose.sh:\n  test \"git commit -- pathspec\" (Thanks Junio)\n- Improved the converting of argv[] for \"git commit\"\n\n===============\nPurpose:\nThis patch is a suggestion to work around the unpleasenties\nwhen Mac OS is decomposing unicode filenames.\n\nThe suggested change:\na) is only used under Mac OS\nb) can be switched off by a configuration variable\nc) is optimized to handle ASCII only filename\nd) will improve the interwork between Mac OS, Linux and Windows*\n   via git push/pull, using USB sticks (technically speaking VFAT)\n   or mounted network shares using samba.\n\n* (Not all Windows versions support UTF-8 yet:\n   Msysgit needs the unicode branch, cygwin supports UTF-8 since 1.7)\n\n\nRuntime configuration:\nA new confguration variable is added: \"core.precomposedunicode\"\nThis variable is only used on Mac OS.\nIf set to false, git behaves exactly as older versions of git.\nWhen a new git version is installed and there is a repository\nwhere the configuration \"core.precomposedunicode\" is not present,\nthe new git is backward compatible.\n\nWhen core.precomposedunicode=true, all filenames are stored in precomposed\nunicode in the index (technically speaking precomposed UTF-8).\nEven when readdir() under Mac OS returns filenames as decomposed.\n\nImplementation:\nTwo files are added to the \"compat\" directory, darwin.h and darwin.c.\nThey implement basically 4 new functions:\nprecomposed_utf8_opendir(), precomposed_utf8_readdir(),\nprecomposed_utf8_closedir() argv_precompose()\n\n\n\nCompile time configuration:\nA new compiler option PRECOMPOSED_UNICODE is introduced in the Makefile,\nso that the patch can be switched off completely at compile time.\n\nNo decomposed file names in a git repository:\nIn order to prevent that ever a file name in decomposed unicode is entering\nthe index, a \"brute force\" attempt is taken:\nall arguments into git (technically argv[1]..argv[n]) are converted into\nprecomposed unicode.\nThis is done in git.c by calling argv_precompose().\nThis function is actually a #define, and it is only defined under Mac OS.\nNothing is converted on any other OS.\n\nImplementation details:\nThe main work is done in precomposed_utf8_readdir() and argv_precompose().\nThe conversion into precomposed unicode is done by using iconv,\nwhere decomposed is denoted by \"UTF-8-MAC\" and precomposed is \"UTF-8\".\nWhen already precomposed unicode is precomposed, the string is returned\nunchanged.\n\nThread save:\nSince there is no need for argv_precompose()to be thread-save, one iconv\ninstance is created at the beginning and kept for all conversions.\nEven readdir() is not thread-save, so that precomposed_utf8_opendir() will call\niconv_open() once and keep the instance for all calls of precomposed_utf8_readdir()\nuntil precomposed_utf8_close() is called.\n\nAuto sensing:\nWhen creating a new git repository with \"git init\" or \"git clone\", the\n\"core.precomposedunicode\" will be set automatically to \"true\" or \"false\".\n\nTypically core.precomposedunicode is \"true\" on HFS and VFAT.\nIt is even true for file systems mounted via SAMBA onto a Linux box,\nand \"false\" for drives mounted via NFS onto a Linux box.\n\n\nNew test case:\nThe new t3910-mac-os-precompose.sh is added to check if a filename\ncan be reached either in precomposed or decomposed unicode (NFC or NFD).\n\n\n\nTorsten Bögershausen (1):\n  git on Mac OS and precomposed unicode\n\n Documentation/config.txt     |    9 ++\n Makefile                     |    3 +\n builtin/init-db.c            |    2 +\n compat/precomposed_utf8.c    |  213 ++++++++++++++++++++++++++++++++++++++++++\n compat/precomposed_utf8.h    |   39 ++++++++\n git-compat-util.h            |    9 ++\n git.c                        |    1 +\n t/t0050-filesystem.sh        |    1 +\n t/t3910-mac-os-precompose.sh |  117 +++++++++++++++++++++++\n 9 files changed, 394 insertions(+), 0 deletions(-)\n create mode 100644 compat/precomposed_utf8.c\n create mode 100644 compat/precomposed_utf8.h\n create mode 100755 t/t3910-mac-os-precompose.sh\n\n-- \n1.7.8.rc0.43.gb49a8\n"},{"id":"182526","messageId":"20120113225559.GA7343@burratino","threadId":"29362","inReplyTo":"201201132253.00799.tboegi@web.de","subject":"Re: [RFC][PATCH v3] git on Mac OS and precomposed unicode","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2012-01-13T22:56:00Z","receivedAt":"2012-01-13T22:56:00Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi,\n\ntb wrote:\n\n> Purpose:\n[...]\n> Runtime configuration:\n[...]\n> Implementation:\n[...]\n> Compile time configuration:\n[...]\n> Implementation details:\n[....]\n> Thread safety:\n[...]\n> Auto sensing:\n[...]\n> New test case:\n\nThis information, to the extent that it is useful at all, belongs in\nthe commit log.  That is, the commit message should concisely say\neverything a person would want to know when reading a patch, whether\nreading it to review it for inclusion, to make sure it still works\nwhen making a related change, to consider whether it is safe to\nupgrade to a version including the change, to understand what is\nhappening when a bug is tracked down to be caused by that commit, or\nfor some other reason.\n\nSo please do not use a cover letter that separates this information\nwhen sending a single patch.\n\n> Changes since [...]\n\nThis kind of information that does not belong in the commit message\ncan go after the \"---\" in the same message as the patch.\n\nI haven't read the patch yet, except to glance at it and see some\nnitpicks I can mention later (e.g., source files should not #include\nanything else before git-compat-util.h or cache.h), and the approach\nseems likely to be sane; I'm mentioning this to help you present the\ninformation in a way that can save myself and other reviewers some\ntrouble for the next round.\n\nThanks much for your work, and hope that helps.\n\nRegards,\nJonathan\n"}]}