{"thread":{"id":"6639","subject":"t9200 still failing...","startedAt":"2007-02-03T15:56:13Z","lastAt":"2007-02-06T15:17:00Z","messageCount":17,"participants":["Brian Gernhardt","Wolfgang Fischer","Junio C Hamano","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"33448","messageId":"DBACB5AE-7C40-4D44-85BB-76F30BE96E36@silverinsanity.com","threadId":"6639","inReplyTo":null,"subject":"t9200 still failing...","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T15:56:13Z","receivedAt":"2007-02-03T15:56:13Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"I got sidetracked by work and buying a house, so I didn't get to \nmention it at the time the attempted fix was committed, but t/t9200- \ngit-cvsexportcommit.sh still is failing on test 8, even in latest \nmaster. I'm still getting the message about a file being in \nmy .gitignore:\n\n> The following paths are ignored by one of your .gitignore files:\n> Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/ \n> gårdetsågårdet.txt\n> Use -f if you really want to add them.\n\nWhat I don't understand is that there are no .gitignore files in the \ntest repo and the .git/info/exclude file only has comments. This \nseems like it might actually be a subtle bug in git-add, although I'm \nbetting it's more OS X filesystem oddness. I'm trying to look into \nit, but thought that maybe someone with more experience with the code \nmight catch it first.\n\nThings I've discovered:\n- A \"git status\" appropriately shows Å as untracked.\n- Terminal.app seems to ignore attempts to type Å.\n- Using \"git add . && git commit\" by hand in the test repo seems to \nwork (using git version 1.5.0.rc2.g73a2)\n\n~~ Brian\n"},{"id":"33452","messageId":"13848C22-6C0B-46F0-ABC5-1A94D0DB6B06@wf227.com","threadId":"6639","inReplyTo":"DBACB5AE-7C40-4D44-85BB-76F30BE96E36@silverinsanity.com","subject":"Re: t9200 still failing...","fromName":"Wolfgang Fischer","fromEmail":"wf@wf227.com","sentAt":"2007-02-03T16:17:50Z","receivedAt":"2007-02-03T16:17:50Z","isPatch":false,"sender":{"key":"wf@wf227.com","avatar":null},"body":"On 03.02.2007, at 16:56, Brian Gernhardt wrote:\n> What I don't understand is that there are no .gitignore files in \n> the test repo and the .git/info/exclude file only has comments. \n> This seems like it might actually be a subtle bug in git-add, \n> although I'm betting it's more OS X filesystem oddness. I'm trying \n> to look into it, but thought that maybe someone with more \n> experience with the code might catch it first.\n\nThat was already discussed a lot. Any filename test on OSX with a HFS \n+ filesystem containing characters with a different UTF-8-NFC and \nUTF-8-NFD will make such a test fail. If you are using OSX, you might \nwant to use UnicodeChecker to see the encoding difference for such \ncharacters. If you want to make such tests pass, either use \ncharacters with only one UTF-8 encoding or use a UFS partition to run \nsuch tests.\n\n\tWolfgang\n"},{"id":"33460","messageId":"1AAD2384-C52D-4D60-A948-4F14C9515B53@silverinsanity.com","threadId":"6639","inReplyTo":"13848C22-6C0B-46F0-ABC5-1A94D0DB6B06@wf227.com","subject":"Re: t9200 still failing...","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T17:41:36Z","receivedAt":"2007-02-03T17:41:36Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 3, 2007, at 11:17 AM, Wolfgang Fischer wrote:\n\n> That was already discussed a lot. Any filename test on OSX with a \n> HFS+ filesystem containing characters with a different UTF-8-NFC \n> and UTF-8-NFD will make such a test fail. If you are using OSX, you \n> might want to use UnicodeChecker to see the encoding difference for \n> such characters. If you want to make such tests pass, either use \n> characters with only one UTF-8 encoding or use a UFS partition to \n> run such tests.\n\nI understand why that causes git-status to continually display \n\"gitweb/test/Märchen\" as an untracked file, and I'm fine with that \n(for the most part). But I don't understand why that causes \"git add \nÅ/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/ \ngårdetsågårdet.txt\" to show a message about gitignore, especially \nwhen a \"git add .\" will pick it up just fine. I'd argue that it's a \nbug in git, personally, and I can't figure out why it happens.\n\nIn any case, to get the test to pass we should probably just put a \"- \nf\" on the \"git add\", at which point it works just fine.\n\n~~ Brian\n"},{"id":"33462","messageId":"20070203181329.GA10192@179.242.249.10.in-addr.arpa","threadId":"6639","inReplyTo":"1AAD2384-C52D-4D60-A948-4F14C9515B53@silverinsanity.com","subject":"[PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T18:13:29Z","receivedAt":"2007-02-03T18:13:29Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"Without -f, the UTF-8 named files are ignored (supposedly via\n.gitignore) on systems with a different UTF-8 encoding.\nThis patch makes the test pass on OS X on HFS+, in particular.\n\nSigned-off-by: Brian Gernhardt \n---\n \n I still think that the behavior of git-add here is an error. The file\n is NOT in any ignore file and so should not be marked as such. However,\n this allows me to finally use \"make test && sudo make install\".\n\n t/t9200-git-cvsexportcommit.sh | 4 ++--\n 1 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/t/t9200-git-cvsexportcommit.sh b/t/t9200-git-cvsexportcommit.sh\nindex c443f32..6955cdd 100755\n--- a/t/t9200-git-cvsexportcommit.sh\n+++ b/t/t9200-git-cvsexportcommit.sh\n@@ -174,9 +174,9 @@ test_expect_success \\\n 'File with non-ascii file name' \\\n 'mkdir -p Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö &&\n echo Foo >Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.txt &&\n- git add Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.txt &&\n+ git add -f Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.txt &&\n cp ../test9200a.png Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.png &&\n- git add Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.png &&\n+ git add -f Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/gårdetsågårdet.png &&\n git commit -a -m \"Går det så går det\" && \\\n id=$(git rev-list --max-count=1 HEAD) &&\n (cd \"$CVSWORK\" &&\n-- \n1.5.0.rc3.22.g5057\n"},{"id":"33468","messageId":"E7A51B21-ECE4-4B0A-8CA2-538727AA6BFF@silverinsanity.com","threadId":"6639","inReplyTo":"82643737-7D5F-4A73-83BA-EB5AD9BBA5EA@wf227.com","subject":"Re: t9200 still failing...","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T19:41:45Z","receivedAt":"2007-02-03T19:41:45Z","isPatch":false,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 3, 2007, at 2:06 PM, Wolfgang Fischer wrote:\n\n> A simple test would be to create a UFS image, mount it somewhere, \n> and run the tests inside the mounted UFS image. If the test \n> succeeds, there is a getdirent() somewhere in the codepath, where \n> you are not expecting it. OTOH we can assume already, that there is \n> a getdirents() somewhere, so you might just start with a debugger \n> with an appropriate breakpoint.\n\nThe true solution was to read the code more completely. The way git \ntests for things that are ignored is to read the directory and \ncompare it to the pathspecs from the command line. If we don't find \na match, it's obviously because the file was in an ignore file. \nUnfortunately, in OS X it's not obvious because of the mismatch \nbetween NFC and NFD on different file systems.\n\nThe solution, as I see it, is to use a Unicode-aware comparison in \ndir.c:match_one(). ICU appears to both be available on OS X \n(although without headers) and have the needed functions. I'm \nunfamiliar with both ICU and this chunk of the git internals, but \nI'll work on a proof of concept patch.\n\n~~ Brian\n"},{"id":"33471","messageId":"7v4pq358q4.fsf@assigned-by-dhcp.cox.net","threadId":"6639","inReplyTo":"20070203181329.GA10192@179.242.249.10.in-addr.arpa","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-03T19:42:11Z","receivedAt":"2007-02-03T19:42:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brian Gernhardt writes:\n\n> Without -f, the UTF-8 named files are ignored (supposedly via\n> .gitignore) on systems with a different UTF-8 encoding.\n> This patch makes the test pass on OS X on HFS+, in particular.\n>\n> Signed-off-by: Brian Gernhardt \n> ---\n> \n> I still think that the behavior of git-add here is an error. The file\n> is NOT in any ignore file and so should not be marked as such.\n\nCan you describe why git-add finds the (presumably mangled by\nHFS) path in \".gitignore\"? If we had some default pattern in\ninfo/exclude that is installed in the trash test repository I\nwould understand that a mangled path could happen to match it,\nbut I do not think we do not have any exclude pattern by\ndefault.\n\nUnless/until you know why git-add thinks it is ignored,...\n\n> However,\n> this allows me to finally use \"make test && sudo make install\".\n\n... I think this change means you are installing something the\nexisting test knows to be broken, which is not very pretty.\n"},{"id":"33475","messageId":"F16FE131-CF33-4DD0-9A06-4A24C59C7623@silverinsanity.com","threadId":"6639","inReplyTo":"7v4pq358q4.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T20:12:09Z","receivedAt":"2007-02-03T20:12:09Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 3, 2007, at 2:42 PM, Junio C Hamano wrote:\n\n> Can you describe why git-add finds the (presumably mangled by\n> HFS) path in \".gitignore\"? If we had some default pattern in\n> info/exclude that is installed in the trash test repository I\n> would understand that a mangled path could happen to match it,\n> but I do not think we do not have any exclude pattern by\n> default.\n>\n> Unless/until you know why git-add thinks it is ignored,...\n\nIt occurs because of the normalization issue on HFS+. git-add \ncompares the pathspecs given on the command line to a directory tree \nread from disk. The pathspec is in NFC, and the tree is in NFD. \nWhen it tries to find the pathspec in the tree, it fails because of \nthat. When it checks to see if the file exists, HFS+ converts the \npathspec to NFD transparently. And since the file exists, but wasn't \nread by read_directory, git thinks it was because of an ignore file.\n\n> ... I think this change means you are installing something the\n> existing test knows to be broken, which is not very pretty.\n\nThe test finds that the ignore code is slightly broken on HFS+, which \nis not what it thinks it's testing. And since the ignore files are \nempty, it should not adversely affect any other platforms.\n\nThe only solution I can think of is to make dir.c:match_one() unicode- \naware. Which I'm working on to see if it will work, but don't know \nif you want to include that complication.\n\n~~ Brian\n"},{"id":"33479","messageId":"7vabzv3qxi.fsf@assigned-by-dhcp.cox.net","threadId":"6639","inReplyTo":"F16FE131-CF33-4DD0-9A06-4A24C59C7623@silverinsanity.com","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-03T20:51:53Z","receivedAt":"2007-02-03T20:51:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brian Gernhardt writes:\n\n> The only solution I can think of is to make dir.c:match_one() unicode-\n> aware. Which I'm working on to see if it will work, but don't know if\n> you want to include that complication.\n\nA filesystem that reports success on creat(path) and then does\nnot return that path to later readdir() from that directory is\nbroken from git's point of view. At least that has been the\ndefinition so far.\n\nI am not sure offhand the implications of the change you propose\nto make to match_one(), but it could also be the place to handle\ncase-challenged filesystems.\n\nBut even if you do that, I wonder what your plan would be to\nhandle something like \"git add .\". read_directory_recursive()\nasks readdir() for existing pathnames, and we expect that can be\nused as parameter to add_file_to_index and then eventually we\ncan call creat() or symlink() with it, so it also needs to be\ntaught to behave consistently with respect to what your updated\nmatch_one() does.\n"},{"id":"33488","messageId":"A9623793-111E-47F7-9709-1D569333C40C@silverinsanity.com","threadId":"6639","inReplyTo":"7vabzv3qxi.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-03T21:31:27Z","receivedAt":"2007-02-03T21:31:27Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 3, 2007, at 3:51 PM, Junio C Hamano wrote:\n\n> A filesystem that reports success on creat(path) and then does\n> not return that path to later readdir() from that directory is\n> broken from git's point of view. At least that has been the\n> definition so far.\n\nI agree, it's fairly idiotic and obtuse. By using the great Google, \nI've seen lots of other people complaining about it as well.\n\n> I am not sure offhand the implications of the change you propose\n> to make to match_one(), but it could also be the place to handle\n> case-challenged filesystems.\n\nI would simply like match_one to agree with the file system as to \nwhat is a match. On OS X that would mean that both normalized forms \nof UTF-8 match. I'm trying it as a proof-of-concept mostly to see \nwhat kind of effects it would have. match_one only seems to be used \nin match_pathspec, which is only used by add and rm... And I think \nthat getting it to match the behavior of the OS would make things \nless surprising for the user.\n\n> But even if you do that, I wonder what your plan would be to\n> handle something like \"git add .\". read_directory_recursive()\n> asks readdir() for existing pathnames, and we expect that can be\n> used as parameter to add_file_to_index and then eventually we\n> can call creat() or symlink() with it, so it also needs to be\n> taught to behave consistently with respect to what your updated\n> match_one() does.\n\n\"git add .\" works just fine, as it reads the name of the file from \ndisk which is already in the form HFS+ accepts. The only confusion \nexists when comparing data from the user to data from disk.\n\nIn any case, I don't appear to understand the proper incantations to \nbend ICU to my will, as it either doesn't change anything or breaks \ngit completely. Perhaps further attempts will help, but my lack of \nexperience with the API is a serious hinderance, and I can't find any \nother way to do the normalization.\n\n~~ Brian\n"},{"id":"33501","messageId":"7vlkje243u.fsf@assigned-by-dhcp.cox.net","threadId":"6639","inReplyTo":"A9623793-111E-47F7-9709-1D569333C40C@silverinsanity.com","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-03T23:50:13Z","receivedAt":"2007-02-03T23:50:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brian Gernhardt writes:\n\n> On Feb 3, 2007, at 3:51 PM, Junio C Hamano wrote:\n>\n>> A filesystem that reports success on creat(path) and then does\n>> not return that path to later readdir() from that directory is\n>> broken from git's point of view. At least that has been the\n>> definition so far.\n>\n> I agree, it's fairly idiotic and obtuse. By using the great Google,\n> I've seen lots of other people complaining about it as well.\n\nI am not sure if we are really agreeing, but I'll let it pass.\n\n> \"git add .\" works just fine, as it reads the name of the file from\n> disk which is already in the form HFS+ accepts. The only confusion\n> exists when comparing data from the user to data from disk.\n\nI wonder even if that is true.\n\nLuckily or unluckily, I do not have access to a system with\nbroken readdir() vs creat() confusion, so I cannot test it\nmyself, but I suspect that this sequence may not work as\nexpected:\n\n\t#!/bin/sh\n\n\tpathname='a pathname that canonicalize differently from original'\n\trm -fr testrepo\n\tmkdir testrepo\n cd testrepo\n git init-db\n\techo hello >\"$pathname\"\n git add -f .\n git ls-files -s \"$pathname\"\n\nIf my reading of what you said is correct, then in the above\nsequence:\n\n (1) The shell creates, via redirection of output of echo, a file\n but using canonicalized string, which is different from\n what the user gave;\n\n (2) \"git add\" will ask readdir(2) to get inventory of files in\n the working directory, and grabs canonicalized string;\n\n (3) \"git add\" uses that canonicalized string, open(2)s, mmap(2)s and\n hashes the blob contents and registers that object name\n under the canonicazlied string in the index;\n\n (4) \"git ls-files\" tries to look up the index with the string\n user used to create the file in (1), which is without the\n canonicalization.\n\nI think it is fairly idiotic and obtuse for a filesystem to\ntreat pathnames anything but a random sequence of bytes that is\nslash separated and NUL terminated. I would need a really hard\nconvincing to buy any path munging on the git side to match\nwhatever a stupid filesystem does, especially because we do not\nlive in the ideal Unicode/utf-8 only world.\n"},{"id":"33509","messageId":"7vr6t6zq2d.fsf@assigned-by-dhcp.cox.net","threadId":"6639","inReplyTo":"7vlkje243u.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-04T01:09:30Z","receivedAt":"2007-02-04T01:09:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> ... I suspect that this sequence may not work as\n> expected:\n>\n> \t#!/bin/sh\n>\n> \tpathname='a pathname that canonicalize differently from original'\n> \trm -fr testrepo\n> \tmkdir testrepo\n> cd testrepo\n> git init-db\n> \techo hello >\"$pathname\"\n> git add -f .\n> git ls-files -s \"$pathname\"\n\nAddendum.\n\nThis is not to make any point but purely for my own education,\nbut I wonder what would happen on HFS+ to the following\nsequence, which does not involve any git operation:\n\n #!/bin/sh\n LANG=C LC_ALL=C\n export LANG LC_ALL\n\n pathname='a pathname that canonicalize differently from original'\n rm -fr testrepo\n mkdir testrepo\n cd testrepo\n echo hello >\"$pathname\"\n /bin/ls | fgrep -e \"$pathname\"\n\nDoesn't the grep fail to find the file that was successfully\ncreated in the previous step?\n"},{"id":"33523","messageId":"20070204034722.GA16766@spearce.org","threadId":"6639","inReplyTo":"7vr6t6zq2d.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-02-04T03:47:22Z","receivedAt":"2007-02-04T03:47:22Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano wrote:\n> This is not to make any point but purely for my own education,\n> but I wonder what would happen on HFS+ to the following\n> sequence, which does not involve any git operation:\n> \n> Doesn't the grep fail to find the file that was successfully\n> created in the previous step?\n\nIndeed. I stole the path \"gitweb/test/M\\303\\244rchen\" from git.git\nand ran your script on Mac OS X:\n\n $ cat jt.sh; echo; sh jt.sh \n #!/bin/sh\n LANG=C LC_ALL=C\n export LANG LC_ALL\n \n rm -fr testrepo\n mkdir testrepo\n cd testrepo\n \n pathname='Märchen'\n echo $pathname\n echo hello >\"$pathname\"\n /bin/ls | fgrep -e \"$pathname\" | wc -l\n \n pathname='Mrchen'\n echo $pathname\n echo hello >\"$pathname\"\n /bin/ls | fgrep -e \"$pathname\" | wc -l\n Märchen\n 0\n Mrchen\n 1\n\nThe first pathname= line in the above script is the *exact* byte\nsequence which appears in gitweb/test's tree. I apologize if my\nemail chain screws up the line.\n\n-- \nShawn.\n"},{"id":"33576","messageId":"155A5E6A-9EC5-424B-A231-02EF1FE0FD2B@silverinsanity.com","threadId":"6639","inReplyTo":"7vlkje243u.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-04T16:35:36Z","receivedAt":"2007-02-04T16:35:36Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"Shawn has already provided an example of what happens to a UTF8 \nstring on OS X, so I won't provide a duplicate.\n\nOn Feb 3, 2007, at 6:50 PM, Junio C Hamano wrote:\n\n> I think it is fairly idiotic and obtuse for a filesystem to\n> treat pathnames anything but a random sequence of bytes that is\n> slash separated and NUL terminated. I would need a really hard\n> convincing to buy any path munging on the git side to match\n> whatever a stupid filesystem does, especially because we do not\n> live in the ideal Unicode/utf-8 only world.\n\nThat's what I was agreeing with. It drives me crazy that HFS+ is \ncase-insensitive and now does bad things to UTF8 strings. And I \ndidn't think that doing munging similar to the FS in git is a good \nidea for Git in general, but might be useful for a few people like \nmyself to have. And it wouldn't break anything that isn't already \nbroken.\n\nUnfortunately my lack of understanding of ICU means I can't actually \nget the code to do anything useful, so it's a moot point.\n\n~~ Brian\n"},{"id":"33577","messageId":"D7716B9E-0F00-4AEF-95BC-699933B7FFD8@silverinsanity.com","threadId":"6639","inReplyTo":"155A5E6A-9EC5-424B-A231-02EF1FE0FD2B@silverinsanity.com","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-04T16:50:33Z","receivedAt":"2007-02-04T16:50:33Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"And I forgot to re-iterate that the \"-f\" in the test only bypasses \nthe .gitignore machinery which is perfectly safe in a test \nenvironment where the ignore files are empty, and makes the git- \ncvsexportcommit test only git-cvsexportcommit. If you wish to add a \ntest for filesystem stupidity separately, that's a different issue.\n\nI really think the patch is useful and allows the test to pass even \non systems with idiotic filesystems and doesn't make it vulnerable to \nany bugs in cvsexportcommit. (Bugs in git-add, maybe, but that's not \nwhat it's testing.)\n\n~~ Brian\n"},{"id":"33597","messageId":"7vwt2xpezx.fsf@assigned-by-dhcp.cox.net","threadId":"6639","inReplyTo":"D7716B9E-0F00-4AEF-95BC-699933B7FFD8@silverinsanity.com","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2007-02-05T01:30:58Z","receivedAt":"2007-02-05T01:30:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brian Gernhardt writes:\n\n> I really think the patch is useful and allows the test to pass even on\n> systems with idiotic filesystems and doesn't make it vulnerable to\n> any bugs in cvsexportcommit. (Bugs in git-add, maybe, but that's not\n> what it's testing.)\n\nIn reality, I find highly valuable that our tests find new bugs\nintroduced in unexpected places. I agree that the person who\nwrote t9200 did not _mean_ to test \"git add\", but it ended up\nhelping us identify the problematic behaviour between creat()\nand readdir() on HFS+ affects \"git add\".\n\nWe at least know that the test as written has a problem in an\nenvironment where \"touch '$p'; ls | fgrep '$p'\" fails, and have\na clear understand why it fails. I think that knowledge is\nvaluable -- adding '-f' to the test may hide potential problems\nwe might have with \"git add\" in other environments.\n\nSo how about doing this instead? It tests if the filesystem has\nthat particular issue we know \"git add\" has a problem with, and\nskips the test in such an environment.\n\n---\n\n t/t9200-git-cvsexportcommit.sh | 14 ++++++++++++++\n 1 files changed, 14 insertions(+), 0 deletions(-)\n\ndiff --git a/t/t9200-git-cvsexportcommit.sh b/t/t9200-git-cvsexportcommit.sh\nindex c443f32..4efa0c9 100755\n--- a/t/t9200-git-cvsexportcommit.sh\n+++ b/t/t9200-git-cvsexportcommit.sh\n@@ -169,6 +169,16 @@ test_expect_success \\\n test \"$(echo $(sort \"G g/CVS/Entries\"|cut -d/ -f2,3,5))\" = \"with spaces.png/1.2/-kb with spaces.txt/1.2/\"\n )'\n \n+# Some filesystems mangle pathnames with UTF-8 characters --\n+# check and skip\n+if p=\"Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö\" &&\n+\tmkdir -p \"tst/$p\" &&\n+\tdate >\"tst/$p/day\" &&\n+\tfound=$(find tst -type f -print) &&\n+\ttest \"z$found\" = \"ztst/$p/day\" &&\n+\trm -fr tst\n+then\n+\n # This test contains UTF-8 characters\n test_expect_success \\\n 'File with non-ascii file name' \\\n@@ -184,6 +194,10 @@ test_expect_success \\\n test \"$(echo $(sort Å/goo/a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/w/x/y/z/å/ä/ö/CVS/Entries|cut -d/ -f2,3,5))\" = \"gårdetsågårdet.png/1.1/-kb gårdetsågårdet.txt/1.1/\"\n )'\n \n+fi\n+\n+rm -fr tst\n+\n test_expect_success \\\n 'Mismatching patch should fail' \\\n 'date >>\"E/newfile5.txt\" &&\n"},{"id":"33599","messageId":"EF3945DB-1CCB-4D8A-8300-FD4EB31CD583@silverinsanity.com","threadId":"6639","inReplyTo":"7vwt2xpezx.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-05T01:55:02Z","receivedAt":"2007-02-05T01:55:02Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 4, 2007, at 8:30 PM, Junio C Hamano wrote:\n\n> In reality, I find highly valuable that our tests find new bugs\n> introduced in unexpected places. I agree that the person who\n> wrote t9200 did not _mean_ to test \"git add\", but it ended up\n> helping us identify the problematic behaviour between creat()\n> and readdir() on HFS+ affects \"git add\".\n\nAlright, I understand that. I just meant that since we've identified \nit, the workaround didn't matter much.\n\n> So how about doing this instead? It tests if the filesystem has\n> that particular issue we know \"git add\" has a problem with, and\n> skips the test in such an environment.\n\nYes, something like this is fine. The patch looks good, but I \ncouldn't get it to apply and don't have the time to fix it at the \nmoment. If you *need* me to test it, I'll do so tomorrow or Tuesday.\n\n~~ Brian\n"},{"id":"33729","messageId":"981DBEC8-99B6-4921-81DE-CE739EAB3196@silverinsanity.com","threadId":"6639","inReplyTo":"7vwt2xpezx.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Use \"-f\" when adding files with odd names in t9200.","fromName":"Brian Gernhardt","fromEmail":"benji@silverinsanity.com","sentAt":"2007-02-06T15:17:00Z","receivedAt":"2007-02-06T15:17:00Z","isPatch":true,"sender":{"key":"benji@silverinsanity.com","avatar":"https://gravatar.com/avatar/e06c101dbc25c68114d859b4a9ec7cf8a2c52fd2b0270ef0eac0e2e63ff22311?d=mp&s=160"},"body":"\nOn Feb 4, 2007, at 8:30 PM, Junio C Hamano wrote:\n\n> So how about doing this instead? It tests if the filesystem has\n> that particular issue we know \"git add\" has a problem with, and\n> skips the test in such an environment.\n\nI'd be tempted to add a \"test_expect_success 'skipping UTF8 test on \nbroken filesystem'\", but I finally tested it (Sunday and Monday are \nmy weekends) and it works as expected. Thanks.\n\n~~ Brian\n"}]}