{"thread":{"id":"11966","subject":"[PATCH] [WIP] git on MacOSX and files with decomposed utf-8 file names","startedAt":"2008-02-08T07:09:35Z","lastAt":"2008-02-26T00:28:20Z","messageCount":7,"participants":["Steffen Prohaska","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"67923","messageId":"12024545763364-git-send-email-prohaska@zib.de","threadId":"11966","inReplyTo":null,"subject":"Add case-challenged file system tests","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-02-08T07:09:35Z","receivedAt":"2008-02-08T07:09:35Z","isPatch":false,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"There have been discussions lately on case-challenging file\nsystems and UTF normalization on Mac OS X.  I like to see\nthese problems fixed and would like to start working on a\nresolution.  But I did not follow the recent discussions closely.\n\nIs anyone actively orking on these issues?\nWhat is the current status?\n\nThe patch below adds two simple tests that currently fail on Mac\nand Windows.  I also collected Mitch's test on utf-8 and will\nsend it as a reply to this mail.  These two patches contain test\ncases that should pass, but do currently fail.\n\n    Steffen\n\n-- >8 --\nGit behaves strangely (from a user's point of view) on\nfilesystems that preserve case but do not distinguish filenames\nthat only differ by case.  The two major examples are Windows and\nMac OS X.  Simple operations such as \"git mv\" or \"git merge\" can\nfail unexpectedly.\n\nThis commit adds two simple tests.  Both tests currently fail on\nWindows and Mac, although they pass on Linux.\n\nSigned-off-by: Steffen Prohaska <prohaska@zib.de>\n---\n t/t0050-filesystems.sh |   36 ++++++++++++++++++++++++++++++++++++\n 1 files changed, 36 insertions(+), 0 deletions(-)\n create mode 100755 t/t0050-filesystems.sh\n\ndiff --git a/t/t0050-filesystems.sh b/t/t0050-filesystems.sh\nnew file mode 100755\nindex 0000000..953b02b\n--- /dev/null\n+++ b/t/t0050-filesystems.sh\n@@ -0,0 +1,36 @@\n+#!/bin/sh\n+\n+test_description='Various filesystems issues'\n+\n+. ./test-lib.sh\n+\n+test_expect_success setup '\n+\n+\ttouch camelcase &&\n+\tgit add camelcase &&\n+\tgit commit -m \"initial\" &&\n+\tgit tag initial &&\n+\tgit checkout -b topic &&\n+\tgit mv camelcase tmp &&\n+\tgit mv tmp CamelCase &&\n+\tgit commit -m \"rename\" &&\n+\tgit checkout -f master\n+\n+'\n+\n+test_expect_success 'rename (case change)' '\n+\n+\tgit mv camelcase CamelCase &&\n+\tgit commit -m \"rename\"\n+\n+'\n+\n+test_expect_success 'merge (case change)' '\n+\n+\tgit reset --hard initial &&\n+\tgit merge topic\n+\n+'\n+\n+\n+test_done\n-- \n1.5.4.40.g4a680\n"},{"id":"67920","messageId":"12024545841553-git-send-email-prohaska@zib.de","threadId":"11966","inReplyTo":"12024545763364-git-send-email-prohaska@zib.de","subject":"[PATCH] [WIP] git on MacOSX and files with decomposed utf-8 file names","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-02-08T07:09:36Z","receivedAt":"2008-02-08T07:09:36Z","isPatch":true,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"From: Mitch Tishmack <mitcht.git@gmail.com>\n\n[ spr: This patch contains Mitch's work with his original email\n       as the commit message. ]\n\nApologies Steffen, I grabbed your CamelCase test and did a search/replace,\nwasn't sure what to call it though... But I am on lunch and wanted to be\nuseful. Rip it apart all you want.\n\nFails on hfs* on OSX, works on ufs. I will bother with zfs when it can be used\nagain.\n\nOn UFS:\n$ /bin/sh ./t0060-normalization.sh\n*   ok 1: setup\n*   ok 2: rename (silent normalization)\n*   ok 3: merge (silent normalization)\n* passed all 3 test(s)\n\nOn HFS:\n$ /bin/sh t0060-normalization.sh\n*   ok 1: setup\n* FAIL 2: rename (silent normalization)\n\n     git mv ä ä &&\n     git commit -m \"rename\"\n\n* FAIL 3: merge (silent normalization)\n\n     git reset --hard initial &&\n     git merge topic\n\n* failed 2 among 3 test(s)\n\nThe test case, it uses perl, assuming only 5.6.1+ will work with this:\n\nSigned-off-by: Steffen Prohaska <prohaska@zib.de>\n---\n t/t0060-normalization.sh |   36 ++++++++++++++++++++++++++++++++++++\n 1 files changed, 36 insertions(+), 0 deletions(-)\n create mode 100755 t/t0060-normalization.sh\n\ndiff --git a/t/t0060-normalization.sh b/t/t0060-normalization.sh\nnew file mode 100755\nindex 0000000..f754fe4\n--- /dev/null\n+++ b/t/t0060-normalization.sh\n@@ -0,0 +1,36 @@\n+#!/bin/sh\n+\n+test_description='Test for silent normalization issues'\n+\n+. ./test-lib.sh\n+\n+auml=`perl -CO -e 'print pack(\"U\",0x00E4)'`\n+aumlcdiar=`perl -CO -e 'print pack(\"U\",0x0061).pack(\"U\",0x0308)'`\n+test_expect_success setup \"\n+  touch $aumlcdiar &&\n+  git add $aumlcdiar &&\n+  git commit -m \\\"initial\\\"\n+  git tag initial &&\n+  git checkout -b topic &&\n+  git mv $aumlcdiar tmp &&\n+  git mv tmp $auml &&\n+  git commit -m \\\"rename\\\" &&\n+  git checkout -f master\n+\n+\"\n+\n+test_expect_success 'rename (silent normalization)' \"\n+\n+ git mv $aumlcdiar $auml &&\n+ git commit -m \\\"rename\\\"\n+\n+\"\n+\n+test_expect_success 'merge (silent normalization)' '\n+\n+ git reset --hard initial &&\n+ git merge topic\n+\n+'\n+\n+test_done\n-- \n1.5.4.40.g4a680\n"},{"id":"67975","messageId":"7vlk5vjpi4.fsf@gitster.siamese.dyndns.org","threadId":"11966","inReplyTo":"12024545763364-git-send-email-prohaska@zib.de","subject":"Re: Add case-challenged file system tests","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-08T18:02:11Z","receivedAt":"2008-02-08T18:02:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steffen Prohaska <prohaska@zib.de> writes:\n\n> Is anyone actively orking on these issues?\n> What is the current status?\n\nNot that I know of.\n\n> The patch below adds two simple tests that currently fail on Mac\n> and Windows.  I also collected Mitch's test on utf-8 and will\n> send it as a reply to this mail.  These two patches contain test\n> cases that should pass, but do currently fail.\n\nCould you fold them into one?  They are about the same topic.\n\nAlso, mark them with test_expect_failure if these are meant to\nbe applied.\n"},{"id":"69802","messageId":"1203873549507-git-send-email-prohaska@zib.de","threadId":"11966","inReplyTo":"7vlk5vjpi4.fsf@gitster.siamese.dyndns.org","subject":"[PATCH] Add tests for filesystem challenges (case and unicode normalization)","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-02-24T17:19:09Z","receivedAt":"2008-02-24T17:19:09Z","isPatch":true,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"Unfortunately, I had no time to start the real work of fixing the\nissues that are tested below.  But at least the tests should be in\ngood shape now and could be applied.\n\n    Steffen\n\n-- 8< --\nGit has difficulties on file systems that do not properly\ndistinguish case or modify filenames in unexpected ways.  The two\nmajor examples are Windows and Mac OS X.  Both systems preserve\ncase of file names but do not distinguish between filenames that\ndiffer only by case.  Simple operations such as \"git mv\" or\n\"git merge\" can fail unexpectedly.  In addition, Mac OS X normalizes\nunicode, which make git's life even harder.\n\nThis commit adds tests that currently fail but should pass if\nfile system as decribed above are fully supported.  The test need\nto be run on Windows and Mac X as they already pass on Linux.\n\nMitch Tishmack is the original author of the tests for unicode\nnormalization.\n\nSigned-off-by: Steffen Prohaska <prohaska@zib.de>\n---\n t/t0050-filesystem.sh |   67 +++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 67 insertions(+), 0 deletions(-)\n create mode 100755 t/t0050-filesystem.sh\n\ndiff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\nnew file mode 100755\nindex 0000000..a0ab02e\n--- /dev/null\n+++ b/t/t0050-filesystem.sh\n@@ -0,0 +1,67 @@\n+#!/bin/sh\n+\n+test_description='Various filesystem issues'\n+\n+. ./test-lib.sh\n+\n+test_expect_success \"setup case tests\" '\n+\n+\ttouch camelcase &&\n+\tgit add camelcase &&\n+\tgit commit -m \"initial\" &&\n+\tgit tag initial &&\n+\tgit checkout -b topic &&\n+\tgit mv camelcase tmp &&\n+\tgit mv tmp CamelCase &&\n+\tgit commit -m \"rename\" &&\n+\tgit checkout -f master\n+\n+'\n+\n+test_expect_failure 'rename (case change)' '\n+\n+\tgit mv camelcase CamelCase &&\n+\tgit commit -m \"rename\"\n+\n+'\n+\n+test_expect_failure 'merge (case change)' '\n+\n+\tgit reset --hard initial &&\n+\tgit merge topic\n+\n+'\n+\n+auml=`perl -CO -e 'print pack(\"U\",0x00E4)'`\n+aumlcdiar=`perl -CO -e 'print pack(\"U\",0x0061).pack(\"U\",0x0308)'`\n+test_expect_success \"setup unicode normalization tests\" \"\n+\n+  test_create_repo unicode &&\n+  cd unicode &&\n+  touch $aumlcdiar &&\n+  git add $aumlcdiar &&\n+  git commit -m \\\"initial\\\"\n+  git tag initial &&\n+  git checkout -b topic &&\n+  git mv $aumlcdiar tmp &&\n+  git mv tmp $auml &&\n+  git commit -m \\\"rename\\\" &&\n+  git checkout -f master\n+\n+\"\n+\n+test_expect_failure 'rename (silent unicode normalization)' \"\n+\n+ git mv $aumlcdiar $auml &&\n+ git commit -m \\\"rename\\\"\n+\n+\"\n+\n+test_expect_failure 'merge (silent unicode normalization)' '\n+\n+ git reset --hard initial &&\n+ git merge topic\n+\n+'\n+\n+test_done\n-- \n1.5.4.3.310.g78af\n"},{"id":"69921","messageId":"7voda4hikx.fsf@gitster.siamese.dyndns.org","threadId":"11966","inReplyTo":"1203873549507-git-send-email-prohaska@zib.de","subject":"Re: [PATCH] Add tests for filesystem challenges (case and unicode normalization)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-25T20:44:30Z","receivedAt":"2008-02-25T20:44:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steffen Prohaska <prohaska@zib.de> writes:\n\n> Unfortunately, I had no time to start the real work of fixing the\n> issues that are tested below.\n\nThat's fine.  We are not in a hurry.\n\n> But at least the tests should be in\n> good shape now and could be applied.\n\nI do not think we would want to see \"FIXED\" on systems that\nalready behave sanely, so we would want a fix-up like this on\ntop of your patch, and it would be a good to go.\n\nI do not have a handy way to test this, though, so can you try\nit out and make sure test_case and test_unicode are set to\ntest_expect_failure correctly on problematic filesystems?\n\nThanks.\n\n--\n\n t/t0050-filesystem.sh |   38 ++++++++++++++++++++++++++++++++------\n 1 files changed, 32 insertions(+), 6 deletions(-)\n\ndiff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\nindex a0ab02e..b395c22 100755\n--- a/t/t0050-filesystem.sh\n+++ b/t/t0050-filesystem.sh\n@@ -4,6 +4,34 @@ test_description='Various filesystem issues'\n \n . ./test-lib.sh\n \n+auml=`perl -CO -e 'print pack(\"U\",0x00E4)'`\n+aumlcdiar=`perl -CO -e 'print pack(\"U\",0x0061).pack(\"U\",0x0308)'`\n+\n+test_expect_success 'see if we expect ' '\n+\n+\ttest_case=test_expect_success\n+\ttest_unicode=test_expect_success\n+\tmkdir junk &&\n+\techo good >junk/CamelCase &&\n+\techo bad >junk/camelcase &&\n+\tif test \"$(cat junk/CamelCase)\" != good\n+\tthen\n+\t\ttest_camel=test_expect_failure\n+\t\tsay \"will test on a case insensitive filesystem\"\n+\tfi &&\n+\trm -fr junk &&\n+\tmkdir junk &&\n+\t>junk/\"$auml\" &&\n+\tcase \"$(cd junk && echo *)\" in\n+\t\"$aumlcdiar\")\n+\t\ttest_unicode=test_expect_failure\n+\t\tsay \"will test on a unicode corrupting filesystem\"\n+\t\t;;\n+\t*)\t;;\n+\tesac &&\n+\trm -fr junk\n+'\n+\n test_expect_success \"setup case tests\" '\n \n \ttouch camelcase &&\n@@ -18,22 +46,20 @@ test_expect_success \"setup case tests\" '\n \n '\n \n-test_expect_failure 'rename (case change)' '\n+$test_case 'rename (case change)' '\n \n \tgit mv camelcase CamelCase &&\n \tgit commit -m \"rename\"\n \n '\n \n-test_expect_failure 'merge (case change)' '\n+$test_case 'merge (case change)' '\n \n \tgit reset --hard initial &&\n \tgit merge topic\n \n '\n \n-auml=`perl -CO -e 'print pack(\"U\",0x00E4)'`\n-aumlcdiar=`perl -CO -e 'print pack(\"U\",0x0061).pack(\"U\",0x0308)'`\n test_expect_success \"setup unicode normalization tests\" \"\n \n   test_create_repo unicode &&\n@@ -50,14 +76,14 @@ test_expect_success \"setup unicode normalization tests\" \"\n \n \"\n \n-test_expect_failure 'rename (silent unicode normalization)' \"\n+$test_unicode 'rename (silent unicode normalization)' \"\n \n  git mv $aumlcdiar $auml &&\n  git commit -m \\\"rename\\\"\n \n \"\n \n-test_expect_failure 'merge (silent unicode normalization)' '\n+$test_unicode 'merge (silent unicode normalization)' '\n \n  git reset --hard initial &&\n  git merge topic\n"},{"id":"69948","messageId":"FBFBA30A-0D44-4A2A-A693-EFF79218FCEB@zib.de","threadId":"11966","inReplyTo":"7voda4hikx.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Add tests for filesystem challenges (case and unicode normalization)","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-02-25T23:34:56Z","receivedAt":"2008-02-25T23:34:56Z","isPatch":true,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"\nOn Feb 25, 2008, at 9:44 PM, Junio C Hamano wrote:\n\n> Steffen Prohaska <prohaska@zib.de> writes:\n>\n>> Unfortunately, I had no time to start the real work of fixing the\n>> issues that are tested below.\n>\n> That's fine.  We are not in a hurry.\n\nWell, I am, ... kind of.  git constantly stumbles over a filename\nwhose case changed.  But fortunately I do not need to wait much\nlonger until history will eventually have grown sufficiently such\nthat the rename will no longer surface ;-)\n\n\n>> But at least the tests should be in\n>> good shape now and could be applied.\n>\n> I do not think we would want to see \"FIXED\" on systems that\n> already behave sanely, so we would want a fix-up like this on\n> top of your patch, and it would be a good to go.\n>\n> I do not have a handy way to test this, though, so can you try\n> it out and make sure test_case and test_unicode are set to\n> test_expect_failure correctly on problematic filesystems?\n\nThis is a sensible way of handling this.  It works for me,\nexcept for ...\n\n\n> --\n>\n>  t/t0050-filesystem.sh |   38 ++++++++++++++++++++++++++++++++------\n>  1 files changed, 32 insertions(+), 6 deletions(-)\n>\n> diff --git a/t/t0050-filesystem.sh b/t/t0050-filesystem.sh\n> index a0ab02e..b395c22 100755\n> --- a/t/t0050-filesystem.sh\n> +++ b/t/t0050-filesystem.sh\n> @@ -4,6 +4,34 @@ test_description='Various filesystem issues'\n>\n>  . ./test-lib.sh\n>\n> +auml=`perl -CO -e 'print pack(\"U\",0x00E4)'`\n> +aumlcdiar=`perl -CO -e 'print pack(\"U\",0x0061).pack(\"U\",0x0308)'`\n> +\n> +test_expect_success 'see if we expect ' '\n> +\n> +\ttest_case=test_expect_success\n> +\ttest_unicode=test_expect_success\n> +\tmkdir junk &&\n> +\techo good >junk/CamelCase &&\n> +\techo bad >junk/camelcase &&\n> +\tif test \"$(cat junk/CamelCase)\" != good\n> +\tthen\n> +\t\ttest_camel=test_expect_failure\n\ntest_case (instead of test_camel).\n\n\t\tSteffen\n"},{"id":"69952","messageId":"7vlk58ftnf.fsf@gitster.siamese.dyndns.org","threadId":"11966","inReplyTo":"FBFBA30A-0D44-4A2A-A693-EFF79218FCEB@zib.de","subject":"Re: [PATCH] Add tests for filesystem challenges (case and unicode normalization)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-26T00:28:20Z","receivedAt":"2008-02-26T00:28:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Steffen Prohaska <prohaska@zib.de> writes:\n\n> This is a sensible way of handling this.  It works for me,\n> except for ...\n> ...\n> test_case (instead of test_camel).\n\nThanks for testing.  Fixed.\n"}]}