{"thread":{"id":"4269","subject":"file name case-sensitivity issues","startedAt":"2006-05-23T21:06:15Z","lastAt":"2006-05-26T03:59:25Z","messageCount":10,"participants":["Alex Riesen","Linus Torvalds","Ben Clifford","Junio C Hamano","Christopher Faylor"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"20581","messageId":"20060523210615.GB5869@steel.home","threadId":"4269","inReplyTo":null,"subject":"file name case-sensitivity issues","fromName":"Alex Riesen","fromEmail":"fork0@t-online.de","sentAt":"2006-05-23T21:06:15Z","receivedAt":"2006-05-23T21:06:15Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"Very simple to reproduce on FAT and NTFS, and under Windows, as usual,\nwhen a problem is especially annoying. I seem to have no chance to\nget my hands on this myself, so I at least let everyone know about the\nproblem.\n\nThe case goes as follows:\n\n  $ mkdir case-sensitivity-test\n  $ cd case-sensitivity-test\n  $ git init-db\n  defaulting to local storage area\n  $ echo foo > foo\n  $ echo bar > bar\n  $ git add foo bar\n  $ git commit -m initial\\ commit\n  Committing initial tree 89ff1a2aefcbff0f09197f0fd8beeb19a7b6e51c\n  $ git checkout -b side\n  $ echo bar-side >> bar\n  $ git commit -m side\\ commit -o bar\n  $ git checkout master\n  $ rm foo\n  $ git update-index --remove foo\n  $ echo FOO > FOO # note case change\n  $ git add FOO\n# this is on linux, vfat  on an usbstick (mounted with default case\n# conversion, which is \"lower\". That's why the file can't be found).\n# Have no Windows at home. On Windows the FOO is created and \"git add\"\n# just passes. We just assume it did add the file, as it would there.\n  git-ls-files: error: pathspec 'FOO' did not match any.\n  Maybe you misspelled it?\n  $ git commit -m case\\ change\n  $ git pull . side\n  Trying really trivial in-index merge...\n  git-read-tree: fatal: Untracked working tree file 'foo' would be overwritten by merge.\n  Nope. Really trivial in-index merge is not possible.\n  Merging HEAD with 7b0cad3a104487fa92afa06736294338acb84281\n  Merging:\n  7f6a8ba3e41683ef5b55921d050092e766aad4a5 case change\n  7b0cad3a104487fa92afa06736294338acb84281 side commit\n  found 1 common ancestor(s):\n  f857aaf5f1d3716d25ca7751f12de30420d9b2aa initial commit\n  git-read-tree: git-read-tree: fatal: Untracked working tree file 'foo' would be overwritten by merge.\n\n  No merge strategy handled the merge.\n\nWell, what now?\n\nWhat I did was to replace that die() with error() in\nread-tree.c:verify_absent, which if cause is not acceptable.\nI'll try to find a solution sometime later, but I really hope\nsomeone will find it sooner (because it'll take some time for me).\nHope it didn't bit anyone yet...\n"},{"id":"20584","messageId":"Pine.LNX.4.64.0605231412350.5623@g5.osdl.org","threadId":"4269","inReplyTo":"20060523210615.GB5869@steel.home","subject":"Re: file name case-sensitivity issues","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-23T21:16:25Z","receivedAt":"2006-05-23T21:16:25Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 23 May 2006, Alex Riesen wrote:\n>\n> Very simple to reproduce on FAT and NTFS, and under Windows, as usual,\n> when a problem is especially annoying. I seem to have no chance to\n> get my hands on this myself, so I at least let everyone know about the\n> problem.\n\nI don't think we can fix it.\n\nAt least not in the short term.\n\nThe closest I can imagine is to add a config option like \"core.lowercase\", \nand that would make us always add files to the index in lower case. That, \ntogether with making sure that \"setup_pathspec()\" &co always also \nlower-case their arguments might get things limping along with minimal \ntrouble.\n\nBut it won't ever do things _well_. Anything non-ascii would be just a \nnightmare.\n\n\t\tLinus\n"},{"id":"20585","messageId":"Pine.LNX.4.64.0605231426520.5623@g5.osdl.org","threadId":"4269","inReplyTo":"Pine.LNX.4.64.0605231412350.5623@g5.osdl.org","subject":"Re: file name case-sensitivity issues","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-05-23T21:30:42Z","receivedAt":"2006-05-23T21:30:42Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 23 May 2006, Linus Torvalds wrote:\n> \n> The closest I can imagine is to add a config option like \"core.lowercase\", \n> and that would make us always add files to the index in lower case.\n\nSide note: doing it by just changing the name compare functions to ignore \ncase is _not_ a good things to do, because that would generate tree \nobjects that simply don't work (or fsck) correctly on any other machine. \n\nThe index and tree objects are all sorted by pathname, and thus the \nsorting order has to be something that everybody agrees on, and any locale \ndependencies are not appropriate.\n\nIt might be worth asking the monotone guys what they do - they've worked \non Windows for a long time.\n\n\t\tLinus\n"},{"id":"20588","messageId":"Pine.LNX.4.64.0605232239070.15915@dildano.hawaga.org.uk","threadId":"4269","inReplyTo":"20060523210615.GB5869@steel.home","subject":"Re: file name case-sensitivity issues","fromName":"Ben Clifford","fromEmail":"benc@hawaga.org.uk","sentAt":"2006-05-23T22:43:15Z","receivedAt":"2006-05-23T22:43:15Z","isPatch":false,"sender":{"key":"benc@hawaga.org.uk","avatar":"https://gravatar.com/avatar/c7ce083471287f8e77b69dd147f757799d1efdd740727fd7e3003f33e88be898?d=mp&s=160"},"body":"\nOn OS X using whatever filesystem it comes with by default, I get the \nfollowing, which doesn't seem right (but in a different way).\n\n$ mkdir case-sensitivity-test\n$ cd case-sensitivity-test\n$ git init-db\ndefaulting to local storage area\n$ echo foo > foo\n$ echo bar > bar\n$ git add foo bar\n$ git commit -m initial\\ commit\nCommitting initial tree 89ff1a2aefcbff0f09197f0fd8beeb19a7b6e51c\n$ git checkout -b side\n$ echo bar-side >> bar\n$ git commit -m side\\ commit -o bar\n$ git checkout master\n$ rm foo\n$ git update-index --remove foo\n$ echo FOO > FOO\n$ git add FOO\n$ git commit -m case\\ change\n$ ls\nFOO bar\n$ git pull . side\nTrying really trivial in-index merge...\nfatal: Merge requires file-level merging\nNope.\nMerging HEAD with e1f1e78035b099fad2bbfb82af7ec31864d8e4c1\nMerging: \n5d70969775bf595dd5144a2bacc25d32cc288352 case change \ne1f1e78035b099fad2bbfb82af7ec31864d8e4c1 side commit \nfound 1 common ancestor(s): \ne35c42fad4f08c2ccf61d93409a0208e92028a51 initial commit \n\nMerge 98bf1cae75776c141ad3b61dc2cb938c71c303ef, made by recursive.\n bar |    1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n$ \n$ ls\nbar\n$ git ls-files -d\nFOO\n$ git ls-tree HEAD\n100644 blob b7d6715e2df11b9c32b2341423273c6b3ad9ae8a    FOO\n100644 blob 5f8b81e197a2cb27816112fb5a6b86b7031ffde8    bar\n\nThe checkout is losing the FOO file but the merged tree object has the \nmerged FOO in it.\n\n-- \n"},{"id":"20589","messageId":"7v7j4c4af3.fsf@assigned-by-dhcp.cox.net","threadId":"4269","inReplyTo":"20060523210615.GB5869@steel.home","subject":"Re: file name case-sensitivity issues","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-05-23T22:57:04Z","receivedAt":"2006-05-23T22:57:04Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"fork0@t-online.de (Alex Riesen) writes:\n\n> Very simple to reproduce on FAT and NTFS, and under Windows, as usual,\n> when a problem is especially annoying. I seem to have no chance to\n> get my hands on this myself, so I at least let everyone know about the\n> problem.\n\nIsn't it like complaining that the following sequence loses your\nprecious file on a case-challenged filesystem?\n\n\t$ echo precious contents >foo\n        $ rm -f FOO\n\nIs it a problem for the user?  Certainly yes.  You lost your\nprecious file.\n\nIs it a bug in the operating system and/or the filesystem?\nProbably not; it is doing what it is asked to do -- its\ndefinition of what string matches what file on the filesystem is\ndubious, but that is how it sees the world and you accept that\nview while you are on such a system.  Is it a bug in \"rm\"?\nProbably not; it is doing what it is asked to do within the\ncontext that you gave it.\n\nI'd call that a PEBCAK.\n\nIf you _know_ you are working on a case challenged filesystem, I\nthink the best thing you can do is not to work on a project that\nhas files in different cases on such a filesystem.\n"},{"id":"20598","messageId":"7vd5e4xkrh.fsf@assigned-by-dhcp.cox.net","threadId":"4269","inReplyTo":"Pine.LNX.4.64.0605232239070.15915@dildano.hawaga.org.uk","subject":"Re: file name case-sensitivity issues","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-05-24T01:40:50Z","receivedAt":"2006-05-24T01:40:50Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ben Clifford <benc@hawaga.org.uk> writes:\n\n> $ ls\n> bar\n> $ git ls-files -d\n> FOO\n> $ git ls-tree HEAD\n> 100644 blob b7d6715e2df11b9c32b2341423273c6b3ad9ae8a    FOO\n> 100644 blob 5f8b81e197a2cb27816112fb5a6b86b7031ffde8    bar\n>\n> The checkout is losing the FOO file but the merged tree object has the \n> merged FOO in it.\n\nThat's interesting.  I wonder how...  Does this sequence remove FOO\non that filesystem?\n\n\t$ date >FOO\n        $ rm -f foo\n        $ ls\n\nAlso if you do the final \"git pull\" using resolve strategy, does\nit change the result (say \"git pull -s resolve . side\" instead)?\n"},{"id":"20619","messageId":"Pine.LNX.4.64.0605240949470.29399@dildano.hawaga.org.uk","threadId":"4269","inReplyTo":"7vd5e4xkrh.fsf@assigned-by-dhcp.cox.net","subject":"Re: file name case-sensitivity issues","fromName":"Ben Clifford","fromEmail":"benc@hawaga.org.uk","sentAt":"2006-05-24T09:55:14Z","receivedAt":"2006-05-24T09:55:14Z","isPatch":false,"sender":{"key":"benc@hawaga.org.uk","avatar":"https://gravatar.com/avatar/c7ce083471287f8e77b69dd147f757799d1efdd740727fd7e3003f33e88be898?d=mp&s=160"},"body":"\n\nOn Tue, 23 May 2006, Junio C Hamano wrote:\n\n> That's interesting.  I wonder how...  Does this sequence remove FOO\n> on that filesystem?\n> \n> \t$ date >FOO\n>         $ rm -f foo\n>         $ ls\n\nyes.\n\n$ ls\n$ date >FOO\n$ ls\nFOO\n$ rm -f foo\n$ ls\n\n\n\n> Also if you do the final \"git pull\" using resolve strategy, does\n> it change the result (say \"git pull -s resolve . side\" instead)?\n\nDifferent result:\n\n$ mkdir case-sensitivity-test\n$ cd case-sensitivity-test\n$ git init-db\ndefaulting to local storage area\n$ echo foo > foo\n$ echo bar > bar\n$ git add foo bar\n$ git commit -m initial\\ commit\nCommitting initial tree 89ff1a2aefcbff0f09197f0fd8beeb19a7b6e51c\n$ git checkout -b side\n$ echo bar-side >> bar\n$ git commit -m side\\ commit -o bar\n$ git checkout master\n$ rm foo\n$ git update-index --remove foo\n$ echo FOO > FOO\n$ git add FOO\n$ git commit -m case\\ change\n$ ls\nFOO bar\n$ git pull -s resolve . side\nTrying really trivial in-index merge...\nfatal: Merge requires file-level merging\nNope.\nTrying simple merge.\nMerge 06c11eeb08edefba8178b091287ec6d951d1ef1d, made by resolve.\n bar |    1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n$ ls\nFOO bar\n$ \n\n\n-- \n"},{"id":"20685","messageId":"20060525154735.GA6119@steel.home","threadId":"4269","inReplyTo":"7v7j4c4af3.fsf@assigned-by-dhcp.cox.net","subject":"Re: file name case-sensitivity issues","fromName":"Alex Riesen","fromEmail":"fork0@t-online.de","sentAt":"2006-05-25T15:47:35Z","receivedAt":"2006-05-25T15:47:35Z","isPatch":false,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"Junio C Hamano, Wed, May 24, 2006 00:57:04 +0200:\n> I'd call that a PEBCAK.\n\nIt is not solvable there though.\n\n> If you _know_ you are working on a case challenged filesystem, I\n> think the best thing you can do is not to work on a project that\n> has files in different cases on such a filesystem.\n\nThat is seldom an acceptable suggestion. Besides, how about when you\ndon't _know_, like when cloning onto an usb-stick mounted with\nauto-detection? Will the files with case-different names just\noverwrite each other?\n"},{"id":"20695","messageId":"7vac96ufxv.fsf@assigned-by-dhcp.cox.net","threadId":"4269","inReplyTo":"20060525154735.GA6119@steel.home","subject":"Re: file name case-sensitivity issues","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-05-25T18:17:48Z","receivedAt":"2006-05-25T18:17:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"fork0@t-online.de (Alex Riesen) writes:\n\n> ... Besides, how about when you\n> don't _know_, like when cloning onto an usb-stick mounted with\n> auto-detection? Will the files with case-different names just\n> overwrite each other?\n\nYou _do_ realize that example is bogus, don't you?  At least I\nhope you did after you sent it.\n\nYou are cloning a project that has mixed cases (say foo and FOO)\nonto a case challenged filesystem but unfortunately you did not\nknow the filesystem was case challenged in advance.  So after\nthe cloning, your checkout results in only one file either foo\nor FOO but not both, because you cannot have two files whose\nnames are different only in case on such a filesystem.\n\nTough.\n\nThere are some other problems on case challenged filesystems\nthat we _could_ solve but we probably don't right now.  You\ncould concentrate on fixing those, instead of talking about\nunfixable.\n\n\nThere are probably 2 kinds of case-challenged-ness.  On non\ncase-challenged filesystems, if I say \"rm -f foo Foo; echo >foo;\necho >Foo\", \"ls\" says \"foo Foo\".  On case-challenged systems,\none of the following would happen:\n\n * \"ls\" says \"foo\".  If I swap the order of the \"echo\", it says\n   \"Foo\".  The filesystem does record the case but does not\n   allow two names with only case difference.\n\n * \"ls\" says ef oh oh in a case different from either \"foo\" nor\n   \"Foo\".  Or it says \"foo\" but if I swap the order of the\n   \"echo\", it still says \"foo\".  The filesystem does not record\n   the case, and does not allow two names with only case\n   difference.  readdir() may do some heuristics such as\n   lowercasing the name, but the point is the returned string is\n   unrealiable.\n\nI have git installed on a Cygwin on NTFS at work, and I think it\nis in the former category.  git seems to work as expected,\nmodulo that you obviously cannot have two files \"foo\" and \"Foo\"\nin your git-managed project.  Probably a patch to delete \"Foo\"\nand create \"foo\" (to make your project friendlier to Windows)\nand a merge to do the same would work well, though I haven't\ntried.\n\nWhat breaks on filesystems in the latter category?  I suspect\nnot many.\n\nupdate-index records the names given by the user (I am assuming\nthat at least the shell is case sensitive), uses that name to\nstat() and open() to update and/or refresh the cache entry, so\nthat codepath should be OK.  Anything that goes from index to\nfind names and then goes to the filesystem with those names\n(diff family, checkout-index and read-tree -u) should be fine.\n\nls-files -o/-i would have a hard time, since they need to work\nwith strings read from readdir(), as you found out.  That means\n\"git add\" and \"git clean\" may not work.\n\nI do not think of anything else that is affected by readdir()\nbreakage offhand; the core is doing pretty fine as it is (I do\nnot consider ls-files -o/-i a core -- that is more Porcelainish\npart of the whole package).\n\nI honestly think that on Windows people would not even want to\nuse the core Porcelainish nor even Cogito.  The would want a\nnative Window-ish UI that drives the core.  I do not think such\na program would internally call \"git add\" nor read from\n\"ls-files -o/-i\".  It would instead do its own Folder hierarchy\ntraversal, and use \"update-index --add --remove\" to implement\nits own \"git add/rm\" UI, and read from \"ls-files\" (not -o nor\n-i) so that it can show tracked and untracked files differently\nin its Explorer view.\n\nSo in that sense, I think ls-files -o/-i issue is quite low\npriority.  It does not matter on sane filesystems, and in the\nplace where it matters the most, the desired solution does not\ninvolve ls-files -o/-i working well there.\n\nHaving said that, I think you _could_ have a repository\nconfiguration that says \"this repository sits on a case\nchallenged filesystem\", and update ls-files to munge what it\ngets from readdir() by comparing them against what you have in\nthe index.  If your readdir() gives \"foo\" when you have \"FOO\" in\nthe index on such a filesystem, you do not say that \"foo\" is an\nuntracked file -- you just say you found \"FOO\" as you expected.\n"},{"id":"20734","messageId":"20060526035925.GA17618@trixie.casa.cgf.cx","threadId":"4269","inReplyTo":"7vac96ufxv.fsf@assigned-by-dhcp.cox.net","subject":"Re: file name case-sensitivity issues","fromName":"Christopher Faylor","fromEmail":"me@cgf.cx","sentAt":"2006-05-26T03:59:25Z","receivedAt":"2006-05-26T03:59:25Z","isPatch":false,"sender":{"key":"me@cgf.cx","avatar":null},"body":"On Thu, May 25, 2006 at 11:17:48AM -0700, Junio C Hamano wrote:\n>I have git installed on a Cygwin on NTFS at work...\n\nMaybe this has been mentioned already but I wanted to point out that\nCygwin's mount has a \"managed\" option: \"mount -o managed c:/foo /foo\"\nwhich causes cygwin to encode \"problem\" characters into the filename.\n\nThis means that there is a possibility that you'll run into the Windows\n260 character max filename limit sooner so many people don't like to use\nthis option.  However, since only uppercase characters and characters\nlike \">\", \":\", etc.  are encoded, in practice you wouldn't see path\nlength problems *from this* very often.  There is, of course, some\nprocessing overhead involved in this, too, so using managed mode\nwill slow things down slightly.\n\nWe've been contemplating using Unicode functions in cygwin for a while\nsince those allow much longer path lengths but this is a massive change\nand would potentially cause problems on Windows 9x.  There has also been\nsome discussion of using native NT calls which, I believe, allow case\npreservation like linux.  However, those have a similar set of problems.\n\nFYI,\ncgf\n"}]}