{"thread":{"id":"11593","subject":"Re-casing directories on case-insensitive systems","startedAt":"2008-01-11T20:19:25Z","lastAt":"2008-01-12T19:29:03Z","messageCount":29,"participants":["Kevin Ballard","Linus Torvalds","Johannes Schindelin","Robin Rosenberg","David Kastrup","Junio C Hamano","Dmitry Potapov"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"65056","messageId":"579DF776-4F4E-464C-88DB-B22C2EC291BD@sb.org","threadId":"11593","inReplyTo":null,"subject":"Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-11T20:19:25Z","receivedAt":"2008-01-11T20:19:25Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"Somehow I managed to change the case of a directory without git  \nrealizing it. I thought I issued `git mv CS4536 cs4536` but since that  \nwon't work in my efforts to reproduce the problem, I must have simply  \nissued the `mv` outside of git and then re-added it.\n\nAnyway, here's the state of my directory:\n\nkevin@KBALLARD:~/Documents/School/C07> git ls-tree HEAD\n040000 tree b47c8103e2e01fcf145bdc237c0e56ffc61f1c47\tCS4536\n040000 tree dbf7fc51ef3effebdf9b4e9172e4c86cae52b163\tcs4536\n040000 tree 15834a7b6534a285bf6930be4e5404b37e1dc718\tece3601\n040000 tree 62d229b8c4a389b550df20a3752d666c48c767a4\tma2071\n\nNote that I have both versions of the directory present.  \nUnfortunately, only one of them can be present on the filesystem. If I  \nrun `mv cs4536 CS4536; git reset --hard` I end up with a different  \nworking tree.\n\nGit should be able to detect this sort of conflict on a case- \ninsensitive system. I didn't even realize what I'd done until I pushed  \nback to the master repo and ran `git reset --hard` there, then  \nwondered why the new file I added to cs4536/ was missing and why my  \ndirectory was still named CS4536.\n\n-Kevin Ballard\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65060","messageId":"CCDF6298-4F17-4F70-AF00-E63F2C2A2816@sb.org","threadId":"11593","inReplyTo":"579DF776-4F4E-464C-88DB-B22C2EC291BD@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-11T21:09:12Z","receivedAt":"2008-01-11T21:09:12Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"Wow, it's even worse. I made a tmp branch and used git-filter-branch  \nto remove the commit that introduced CS4536, leaving only the cs4536  \ndirectory. But now if I try and run `git co master` it refuses, as it  \nthinks it's going to overwrite the untracked file CS4536/ \nintroduction.txt. I believe it's actually seeing the tracked file  \ncs4536/introduction.txt.\n\n-Kevin Ballard\n\nOn Jan 11, 2008, at 3:19 PM, Kevin Ballard wrote:\n\n> Somehow I managed to change the case of a directory without git  \n> realizing it. I thought I issued `git mv CS4536 cs4536` but since  \n> that won't work in my efforts to reproduce the problem, I must have  \n> simply issued the `mv` outside of git and then re-added it.\n>\n> Anyway, here's the state of my directory:\n>\n> kevin@KBALLARD:~/Documents/School/C07> git ls-tree HEAD\n> 040000 tree b47c8103e2e01fcf145bdc237c0e56ffc61f1c47\tCS4536\n> 040000 tree dbf7fc51ef3effebdf9b4e9172e4c86cae52b163\tcs4536\n> 040000 tree 15834a7b6534a285bf6930be4e5404b37e1dc718\tece3601\n> 040000 tree 62d229b8c4a389b550df20a3752d666c48c767a4\tma2071\n>\n> Note that I have both versions of the directory present.  \n> Unfortunately, only one of them can be present on the filesystem. If  \n> I run `mv cs4536 CS4536; git reset --hard` I end up with a different  \n> working tree.\n>\n> Git should be able to detect this sort of conflict on a case- \n> insensitive system. I didn't even realize what I'd done until I  \n> pushed back to the master repo and ran `git reset --hard` there,  \n> then wondered why the new file I added to cs4536/ was missing and  \n> why my directory was still named CS4536.\n>\n> -Kevin Ballard\n>\n> -- \n> Kevin Ballard\n> http://kevin.sb.org\n> kevin@sb.org\n> http://www.tildesoft.com\n>\n>\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65062","messageId":"alpine.LFD.1.00.0801111313220.3148@woody.linux-foundation.org","threadId":"11593","inReplyTo":"579DF776-4F4E-464C-88DB-B22C2EC291BD@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-01-11T21:18:10Z","receivedAt":"2008-01-11T21:18:10Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 11 Jan 2008, Kevin Ballard wrote:\n>\n> Anyway, here's the state of my directory:\n> \n> kevin@KBALLARD:~/Documents/School/C07> git ls-tree HEAD\n> 040000 tree b47c8103e2e01fcf145bdc237c0e56ffc61f1c47\tCS4536\n> 040000 tree dbf7fc51ef3effebdf9b4e9172e4c86cae52b163\tcs4536\n> 040000 tree 15834a7b6534a285bf6930be4e5404b37e1dc718\tece3601\n> 040000 tree 62d229b8c4a389b550df20a3752d666c48c767a4\tma2071\n\nHmm. You can do something like\n\n\tgit ls-files CS4536 | xargs git update-index --force-remove\n\nwhich will remove gits knowledge of that directory even though \"lstat()\" \nwill still claim that all the files still exists.\n\nCase-insensitive filesystems are a pain.\n\nI wish we had some way to handle it sanely, but I don't think a sane \nsolution to case-insensitivity exists. If you limit it to strictly just \n7bit ascii names (like in your example), then some of the problems do go \naway, but it would still be probably fairly major surgery to try to teach \ngit about the whole insanity of a case-independent working tree.\n\n\t\tLinus\n"},{"id":"65064","messageId":"EF244B67-35F5-412C-9668-8802CE7F328A@sb.org","threadId":"11593","inReplyTo":"CCDF6298-4F17-4F70-AF00-E63F2C2A2816@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-11T21:19:26Z","receivedAt":"2008-01-11T21:19:26Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"Err, make that git-rebase (for removing the commit).\n\n-Kevin Ballard\n\nOn Jan 11, 2008, at 4:09 PM, Kevin Ballard wrote:\n\n> Wow, it's even worse. I made a tmp branch and used git-filter-branch  \n> to remove the commit that introduced CS4536, leaving only the cs4536  \n> directory. But now if I try and run `git co master` it refuses, as  \n> it thinks it's going to overwrite the untracked file CS4536/ \n> introduction.txt. I believe it's actually seeing the tracked file  \n> cs4536/introduction.txt.\n>\n> -Kevin Ballard\n>\n> On Jan 11, 2008, at 3:19 PM, Kevin Ballard wrote:\n>\n>> Somehow I managed to change the case of a directory without git  \n>> realizing it. I thought I issued `git mv CS4536 cs4536` but since  \n>> that won't work in my efforts to reproduce the problem, I must have  \n>> simply issued the `mv` outside of git and then re-added it.\n>>\n>> Anyway, here's the state of my directory:\n>>\n>> kevin@KBALLARD:~/Documents/School/C07> git ls-tree HEAD\n>> 040000 tree b47c8103e2e01fcf145bdc237c0e56ffc61f1c47\tCS4536\n>> 040000 tree dbf7fc51ef3effebdf9b4e9172e4c86cae52b163\tcs4536\n>> 040000 tree 15834a7b6534a285bf6930be4e5404b37e1dc718\tece3601\n>> 040000 tree 62d229b8c4a389b550df20a3752d666c48c767a4\tma2071\n>>\n>> Note that I have both versions of the directory present.  \n>> Unfortunately, only one of them can be present on the filesystem.  \n>> If I run `mv cs4536 CS4536; git reset --hard` I end up with a  \n>> different working tree.\n>>\n>> Git should be able to detect this sort of conflict on a case- \n>> insensitive system. I didn't even realize what I'd done until I  \n>> pushed back to the master repo and ran `git reset --hard` there,  \n>> then wondered why the new file I added to cs4536/ was missing and  \n>> why my directory was still named CS4536.\n>>\n>> -Kevin Ballard\n>>\n>> -- \n>> Kevin Ballard\n>> http://kevin.sb.org\n>> kevin@sb.org\n>> http://www.tildesoft.com\n>>\n>>\n>\n> -- \n> Kevin Ballard\n> http://kevin.sb.org\n> kevin@sb.org\n> http://www.tildesoft.com\n>\n>\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65066","messageId":"alpine.LFD.1.00.0801111318570.3148@woody.linux-foundation.org","threadId":"11593","inReplyTo":"CCDF6298-4F17-4F70-AF00-E63F2C2A2816@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-01-11T21:25:37Z","receivedAt":"2008-01-11T21:25:37Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 11 Jan 2008, Kevin Ballard wrote:\n>\n> Wow, it's even worse. I made a tmp branch and used git-filter-branch to remove\n> the commit that introduced CS4536, leaving only the cs4536 directory. But now\n> if I try and run `git co master` it refuses, as it thinks it's going to\n> overwrite the untracked file CS4536/introduction.txt. I believe it's actually\n> seeing the tracked file cs4536/introduction.txt.\n\nIf you don't have any dirty state, I'd suggest removing your working tree \nbefore doing a \"git checkout\". That's needed anyway to make sure that your \nworking tree has the same case as your index and git trees, because \notherwise since the crazy filesystem thinks that CS4536/cs4536 ar ethe \nsame, you might end up having all the wrong names.\n\nCase differences are hard anyway, but you probably made them even harder \nby them using a rename that actually meant that the old name still \n*existed* in the filesystem (since the new name would always map to the \nold name thanks to your crazy filesystem).\n\nI'm sure you can get into even more trouble with case-independent \nfilesystems, but I think you did a pretty good job of hitting on one of \nthe craziest cases ;)\n\n\t\tLinus\n"},{"id":"65068","messageId":"alpine.LSU.1.00.0801112127040.31053@racer.site","threadId":"11593","inReplyTo":"579DF776-4F4E-464C-88DB-B22C2EC291BD@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-11T21:29:25Z","receivedAt":"2008-01-11T21:29:25Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 11 Jan 2008, Kevin Ballard wrote:\n\n> Somehow I managed to change the case of a directory without git \n> realizing it. I thought I issued `git mv CS4536 cs4536` but since that \n> won't work in my efforts to reproduce the problem, I must have simply \n> issued the `mv` outside of git and then re-added it.\n> \n> Anyway, here's the state of my directory:\n> \n> kevin@KBALLARD:~/Documents/School/C07> git ls-tree HEAD\n> 040000 tree b47c8103e2e01fcf145bdc237c0e56ffc61f1c47\tCS4536\n> 040000 tree dbf7fc51ef3effebdf9b4e9172e4c86cae52b163\tcs4536\n> 040000 tree 15834a7b6534a285bf6930be4e5404b37e1dc718\tece3601\n> 040000 tree 62d229b8c4a389b550df20a3752d666c48c767a4\tma2071\n> \n> Note that I have both versions of the directory present. Unfortunately, \n> only one of them can be present on the filesystem. If I run `mv cs4536 \n> CS4536; git reset --hard` I end up with a different working tree.\n> \n> Git should be able to detect this sort of conflict on a case-insensitive \n> system.\n\nDo not blame git for the shortcomings of your setup!\n\nHowever, as luck has it, I looked into this issue again, as somebody \nraised it with msysgit (for obvious reasons; file systems on Windows are \ncase challenged).  If you are serious about this problem, I can give you \ntons of pointers how to go solve it.  (Although I might be disconnected \nthis weekend, because of the lack of competence of the IT department \nhere.)\n\nCiao,\nDscho\n"},{"id":"65069","messageId":"34F43A68-6041-42BE-85BD-3EF971875C0F@sb.org","threadId":"11593","inReplyTo":"alpine.LSU.1.00.0801112127040.31053@racer.site","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-11T21:44:41Z","receivedAt":"2008-01-11T21:44:41Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"On Jan 11, 2008, at 4:29 PM, Johannes Schindelin wrote:\n\n>> Git should be able to detect this sort of conflict on a case- \n>> insensitive\n>> system.\n>\n> Do not blame git for the shortcomings of your setup!\n\nOh, I'm not surprised git doesn't handle this case, nor do I think  \ngit's required to. I merely think that, given the increasing relevance  \nof OS X and the fact that it uses a case-insensitive system by  \ndefault, this sort of problem is going to occur more and more  \nfrequently and it's quite a learning experience trying to fix it by  \nhand. It would be very helpful if git could detect these problems  \nitself.\n\n> However, as luck has it, I looked into this issue again, as somebody\n> raised it with msysgit (for obvious reasons; file systems on Windows  \n> are\n> case challenged).  If you are serious about this problem, I can give  \n> you\n> tons of pointers how to go solve it.  (Although I might be  \n> disconnected\n> this weekend, because of the lack of competence of the IT department\n> here.)\n\nI think I've got a handle on it. I've already expunged the mis-cased  \nfile using git-rebase to remove the offending commit, now I just need  \nto rewrite the second commit's message so it looks like the original  \ncommit (luckily I didn't do any work in the directory before I re- \ncased it). Thanks anyway.\n\n-Kevin Ballard\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65070","messageId":"200801112259.40968.robin.rosenberg.lists@dewire.com","threadId":"11593","inReplyTo":"CCDF6298-4F17-4F70-AF00-E63F2C2A2816@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg.lists@dewire.com","sentAt":"2008-01-11T21:59:40Z","receivedAt":"2008-01-11T21:59:40Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"fredagen den 11 januari 2008 skrev Kevin Ballard:\n> Wow, it's even worse. I made a tmp branch and used git-filter-branch  \n> to remove the commit that introduced CS4536, leaving only the cs4536  \n> directory. But now if I try and run `git co master` it refuses, as it  \n> thinks it's going to overwrite the untracked file CS4536/ \n> introduction.txt. I believe it's actually seeing the tracked file  \n> cs4536/introduction.txt.\n\nI think you should try an index filter. That should help you avoid file system problems.\n\n-- robin\n"},{"id":"65071","messageId":"alpine.LSU.1.00.0801112202180.31053@racer.site","threadId":"11593","inReplyTo":"34F43A68-6041-42BE-85BD-3EF971875C0F@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-11T22:05:18Z","receivedAt":"2008-01-11T22:05:18Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 11 Jan 2008, Kevin Ballard wrote:\n\n> On Jan 11, 2008, at 4:29 PM, Johannes Schindelin wrote:\n> \n> > If you are serious about this problem, I can give you tons of pointers \n> > how to go solve it.  (Although I might be disconnected this weekend, \n> > because of the lack of competence of the IT department here.)\n> \n> I think I've got a handle on it. I've already expunged the mis-cased \n> file using git-rebase to remove the offending commit, now I just need to \n> rewrite the second commit's message so it looks like the original commit \n> (luckily I didn't do any work in the directory before I re-cased it). \n> Thanks anyway.\n\nI was not talking about fixing it up in your repository.  If you really \nthink that git should help you, you gotta teach it to.  Because people who \ndo not experience the same as you will be less likely to feel the urge to \nteach git to help in that situation (because they did not experience that \nsituation yet).  For example, I am one of these people.  And I guess a lot \nof these people hang out on this list.\n\nCiao,\nDscho\n"},{"id":"65072","messageId":"alpine.LFD.1.00.0801111356000.3148@woody.linux-foundation.org","threadId":"11593","inReplyTo":"34F43A68-6041-42BE-85BD-3EF971875C0F@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-01-11T22:08:35Z","receivedAt":"2008-01-11T22:08:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 11 Jan 2008, Kevin Ballard wrote:\n> \n> Oh, I'm not surprised git doesn't handle this case, nor do I think git's\n> required to. I merely think that, given the increasing relevance of OS X and\n> the fact that it uses a case-insensitive system by default, this sort of\n> problem is going to occur more and more frequently and it's quite a learning\n> experience trying to fix it by hand. It would be very helpful if git could\n> detect these problems itself.\n\nI do agree that we could/should do something to help with case-insensitive \nfilesystems.\n\nI absolutely *detest* those things, and I think that people who design \nthem are total morons - with MS-DOS, you could understand it (people \ndidn't know better), but with OS X?\n\nBut considering that they exist, we should probably offer at least *some* \nhelp for people who didn't realize that you could make OS X behave better.\n\nHowever, it's not like there is even a simple solution. The right place to \ndo that check would probably be in \"add_index_entry()\", but doing a check \nwhether the same file already exists (in a different case) is simply \n*extremely* expensive for a very critical piece of code, unless we were to \nchange that index data structure a lot (ie add a separate hash for the \nfilenames).\n\nInside the Linux kernel, we have support for insane case-insensitive \nfilesystems, and it really does need a lot of effort to do an even \nhalf-way decent thing while not penalizing the sane case. So it's hard.\n\n(And that's totally ignoring the fact that case-insensitivity then also \nhas tons of i18n issues and can get *really* messy - in the above I'm \ntalking purely about the issues that would hit us even with 7-bit straight \nASCII).\n\nSo handling case-sensitivity (even when you restrict it to ASCII-only) is \nactually rather messy. The obvious thing to do is to sort everything using \na case-insensitive sort, but that in turn would break all the rules git \nhas for the sorting of trees.\n\nSo you can't just change the sort order: you'd literally have to have two \n*different* lookup keys for the index (the \"strict sort\" and the \"case- \ninsensitive sort\"), and keep them both around.\n\nAlmost all of the code that actually touches the index is in read-cache.c, \nand it's not like that is a very complex data structure (or a very big \nfile), so adding another key to the sorting probably wouldn't be too \nhorrid. But it's definitely a lot more than just a few lines of code!\n\n\t\tLinus\n"},{"id":"65078","messageId":"85abnc6jsg.fsf@lola.goethe.zz","threadId":"11593","inReplyTo":"alpine.LFD.1.00.0801111356000.3148@woody.linux-foundation.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2008-01-11T23:10:07Z","receivedAt":"2008-01-11T23:10:07Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Linus Torvalds <torvalds@linux-foundation.org> writes:\n\n> I do agree that we could/should do something to help with case-insensitive \n> filesystems.\n>\n> I absolutely *detest* those things, and I think that people who design \n> them are total morons - with MS-DOS, you could understand it (people \n> didn't know better),\n\nAh, those young whippersnappers who think they are so smart...  there is\na history to that, you know.  Early character sets (like those on punch\ncards) had just capital letters.  Even when lowercase letters were\nintroduced, those tended to use more space (12 instead of 6 bit) and be\nharder to print (and the line printers who churned out 40 lines per\nsecond did not bother with such finesse, anyway).  But capital letters\nare not designed for readability of long lines.  So when printing or\neven screen terminals came into use, one tended to prefer writing in\nlowercase letters.  Which actually had the uppercase code points\nusually.  Some early microcomputers (for which CP/M was designed)\nactually were hooked up with a \"standard\" 50 or 110 Baud teletype as I/O\ndevice, and those tended to have only lowercase letters, too, in their\nbasic incantations.  So CP/M, not knowing which kind of input device\nwould be used and whether it would prefer (or offer exclusively) upper\nor lower case, had case-insensitive commands, and consequently also\ncase-insensitive file names.  And QDOS (whence MSDOS) was basically\nintended to be a CP/M ripoff.\n\n> but with OS X?\n\nOS X has an Apple inheritance, Apple has the same inheritance as other\nmicrocomputers which includes a case-insensitive BASIC interpreter\n(BASIC again coming from old teletype times).  It is, again, a decision\nto drag along old history.\n\nBut actually, there is more to it nowadays: two file names containing ü,\nbut one with a single letter and one with combining accent, look exactly\nthe same.  If they don't act exactly the same, one opens up quite a hole\nfor spoofing attacks.  Well, probably hard to avoid (since things like\nuppercase Alpha and uppercase A look the same and need to be different\ncode points, too).  But one also opens a can of worms for confusion.  So\nthe problem of canonical file names does not go away just with case\nsensitivity.\n\n> But considering that they exist, we should probably offer at least\n> *some* help for people who didn't realize that you could make OS X\n> behave better.\n\nIt is not like Linux does not support some case-insensitive file system\ntypes, too.  So the same problems can be had there as well.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"65080","messageId":"FCC447DA-34D1-47EC-98C7-62F380F5C303@sb.org","threadId":"11593","inReplyTo":"85abnc6jsg.fsf@lola.goethe.zz","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-11T23:12:07Z","receivedAt":"2008-01-11T23:12:07Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"On Jan 11, 2008, at 6:10 PM, David Kastrup wrote:\n\n>> But considering that they exist, we should probably offer at least\n>> *some* help for people who didn't realize that you could make OS X\n>> behave better.\n>\n> It is not like Linux does not support some case-insensitive file  \n> system\n> types, too.  So the same problems can be had there as well.\n\n\nIn addition, while there is an option for HFS+ Case-Sensitive, using  \nthat can cause bad things to happen as Mac OS X programs are written  \nunder a case-insensitive assumption and may behave badly when  \npresented with a case-sensitive filesystem.\n\n-Kevin Ballard\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65084","messageId":"200801120026.01930.robin.rosenberg@dewire.com","threadId":"11593","inReplyTo":"alpine.LFD.1.00.0801111356000.3148@woody.linux-foundation.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2008-01-11T23:26:01Z","receivedAt":"2008-01-11T23:26:01Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"fredagen den 11 januari 2008 skrev Linus Torvalds:\n> I do agree that we could/should do something to help with case-insensitive \n> filesystems.\n> \n> I absolutely *detest* those things, and I think that people who design \n> them are total morons - with MS-DOS, you could understand it (people \n> didn't know better), but with OS X?\n\nCould it be some comfort that the other SCM's I know of make a mess of\nthese cases, regardless of the number of digits in the price tag.\n\n[...]\n\n> Almost all of the code that actually touches the index is in read-cache.c, \n> and it's not like that is a very complex data structure (or a very big \n> file), so adding another key to the sorting probably wouldn't be too \n> horrid. But it's definitely a lot more than just a few lines of code!\n\nCould we just have a lookup table index extension for identifying the \nduplicates (when checking is enabled using core configuration option #3324)? \nThat table would keep a mapping from a normalized form (maybe include \ncanonical encoding while we're at it) to the actual octet sequence(s) used.\n\nMany operations would translate any supplied form throug the table before\ndoing the lookup so if we have Foo.h and give FOO.h to git add, it would \nnotice and perform add (update index) on Foo.h instead as that is the form we \nalreay know (or refuse yielding an error message; pick your poison). And, \nwell you get the picture.\n\n-- robin\n"},{"id":"65086","messageId":"1973E1D5-C8CC-4979-A085-85A2C5A13E57@sb.org","threadId":"11593","inReplyTo":"200801120026.01930.robin.rosenberg@dewire.com","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-12T00:03:13Z","receivedAt":"2008-01-12T00:03:13Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"Speaking of normalizing composed sequences, could that be the cause  \nfor the following?\n\nkevin@KBALLARD:~/Dev/git> ls\nkevin@KBALLARD:~/Dev/git> ls -a\n./          ../         .git/       .gitignore\nkevin@KBALLARD:~/Dev/git> git reset --hard\nHEAD is now at 58beb2c... Trim leading / off of paths in git-svn  \nprop_walk\nkevin@KBALLARD:~/Dev/git> git st\n# On branch master\n# Untracked files:\n#   (use \"git add <file>...\" to include in what will be committed)\n#\n#\tgitweb/test/Märchen\nnothing added to commit but untracked files present (use \"git add\" to  \ntrack)\n\nSome further exploration seems to support my cause:\n\nkevin@KBALLARD:~/Dev/git> git ls-files gitweb/test\n\"gitweb/test/M\\303\\244rchen\"\ngitweb/test/file with spaces\ngitweb/test/file+plus+sign\nkevin@KBALLARD:~/Dev/git/gitweb/test> ls Märchen | xxd\n0000000: 4d61 cc88 7263 6865 6e0a                 Ma..rchen.\n\nAs you can see, git has the file tracked using M\\303\\244rchen, where  \n\\303\\244 (or 0xC3A4, or U+00E4) is Latin Small Letter A With  \nDiaeresis, but the filesystem reports it as \"Ma\\xCC\\x88rchen\" where  \n0xCC88 (or U+0308) is Combining Diaeresis.\n\nIn other words, the git repository itself exhibits a problem under OS  \nX. I'm not sure if I didn't notice this untracked file before, or if  \nthe filesystem (or the index) actually used the other form previously,  \nbut regardless there's a problem that I believe would be present even  \nif I was using Case-Sensitive HFS+.\n\n-Kevin Ballard\n\nOn Jan 11, 2008, at 6:26 PM, Robin Rosenberg wrote:\n\n> Could we just have a lookup table index extension for identifying the\n> duplicates (when checking is enabled using core configuration option  \n> #3324)?\n> That table would keep a mapping from a normalized form (maybe include\n> canonical encoding while we're at it) to the actual octet  \n> sequence(s) used.\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65089","messageId":"200801120115.41274.robin.rosenberg@dewire.com","threadId":"11593","inReplyTo":"1973E1D5-C8CC-4979-A085-85A2C5A13E57@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2008-01-12T00:15:40Z","receivedAt":"2008-01-12T00:15:40Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"lördagen den 12 januari 2008 skrev Kevin Ballard:\n> Speaking of normalizing composed sequences, could that be the cause  \n> for the following?\n[...]\n> kevin@KBALLARD:~/Dev/git/gitweb/test> ls Märchen | xxd\n> 0000000: 4d61 cc88 7263 6865 6e0a                 Ma..rchen.\n> \n> As you can see, git has the file tracked using M\\303\\244rchen, where  \n> \\303\\244 (or 0xC3A4, or U+00E4) is Latin Small Letter A With  \n> Diaeresis, but the filesystem reports it as \"Ma\\xCC\\x88rchen\" where  \n> 0xCC88 (or U+0308) is Combining Diaeresis.\n\nYes that is due to normalization. When adding a file by name git uses the user \nsupplied name, but when adding files indirectly it gets the names from the \nfile system without denormalizing them. Likewize status gets the names from\nthe file system without denormalizing and thus you get a mismatch.\n\n-- robin\n"},{"id":"65092","messageId":"191B60D7-FD89-48D8-8D48-C91645D4814D@sb.org","threadId":"11593","inReplyTo":"200801120115.41274.robin.rosenberg@dewire.com","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-12T00:25:26Z","receivedAt":"2008-01-12T00:25:26Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"On Jan 11, 2008, at 7:15 PM, Robin Rosenberg wrote:\n\n> lördagen den 12 januari 2008 skrev Kevin Ballard:\n>> Speaking of normalizing composed sequences, could that be the cause\n>> for the following?\n> [...]\n>> kevin@KBALLARD:~/Dev/git/gitweb/test> ls Märchen | xxd\n>> 0000000: 4d61 cc88 7263 6865 6e0a                 Ma..rchen.\n>>\n>> As you can see, git has the file tracked using M\\303\\244rchen, where\n>> \\303\\244 (or 0xC3A4, or U+00E4) is Latin Small Letter A With\n>> Diaeresis, but the filesystem reports it as \"Ma\\xCC\\x88rchen\" where\n>> 0xCC88 (or U+0308) is Combining Diaeresis.\n>\n> Yes that is due to normalization. When adding a file by name git  \n> uses the user\n> supplied name, but when adding files indirectly it gets the names  \n> from the\n> file system without denormalizing them. Likewize status gets the  \n> names from\n> the file system without denormalizing and thus you get a mismatch.\n\nIs there a reason for this? It seems like it would be trivial to end  \nup with misdiagnosed \"untracked\" files when using any language other  \nthan English given this behaviuor.\n\n-Kevin Ballard\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65093","messageId":"7v7iif28i2.fsf@gitster.siamese.dyndns.org","threadId":"11593","inReplyTo":"191B60D7-FD89-48D8-8D48-C91645D4814D@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-01-12T00:27:33Z","receivedAt":"2008-01-12T00:27:33Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kevin Ballard <kevin@sb.org> writes:\n\n> Is there a reason for this? It seems like it would be trivial to end\n> up with misdiagnosed \"untracked\" files when using any language other\n> than English given this behaviuor.\n\nNo.  The assumption of the code has always been that sane\nfilesystems would return from readdir() the names you gave from\ncreat().\n"},{"id":"65095","messageId":"7v3at3280v.fsf@gitster.siamese.dyndns.org","threadId":"11593","inReplyTo":"200801120026.01930.robin.rosenberg@dewire.com","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-01-12T00:37:52Z","receivedAt":"2008-01-12T00:37:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Robin Rosenberg <robin.rosenberg@dewire.com> writes:\n\n> Could we just have a lookup table index extension for identifying the \n> duplicates (when checking is enabled using core configuration option #3324)? \n> That table would keep a mapping from a normalized form (maybe include \n> canonical encoding while we're at it) to the actual octet sequence(s) used.\n\nI would agree that the index extension, if we ever are going to\ndo this, would be the right place to store this information, at\nthe single repository level.\n\nHowever, this opens up a can of worms.  What's the canonical key\nshould be?  If you want to protect yourself from a unicode\nnormalizing filesystem, you would use one canonicalization,\nwhile if you want to protect from a case losing filesystem you\nwould use another?  Or do we at the same time downcase and NFD\nnormalize at the same time and be done with it?\n\nAnd where should the configuration be stored?  If a project\nwants to be interoperable across Linux and vfat, for example,\nthat canonicalization needs to be enabled in repositories of all\nparticipants, be they on Linux or vfat, so that people on Linux\ncan be prevented from creating and register two files xt_mark.c\nand xt_MARK.c in the same directory, so that people who extract\nthe source on vfat won't have troubles.\n\nWhich means the information needs to be in-tree.  But that\nshould not be in .gitattributes (which by definition is for\nper-path things).\n"},{"id":"65097","messageId":"alpine.LSU.1.00.0801120139170.8333@wbgn129.biozentrum.uni-wuerzburg.de","threadId":"11593","inReplyTo":"7v7iif28i2.fsf@gitster.siamese.dyndns.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-12T00:40:08Z","receivedAt":"2008-01-12T00:40:08Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 11 Jan 2008, Junio C Hamano wrote:\n\n> Kevin Ballard <kevin@sb.org> writes:\n> \n> > Is there a reason for this? It seems like it would be trivial to end \n> > up with misdiagnosed \"untracked\" files when using any language other \n> > than English given this behaviuor.\n> \n> No.  The assumption of the code has always been that sane filesystems \n> would return from readdir() the names you gave from creat().\n\nWe do not really have to rehash that whole discussion for the Nth time, do \nwe?\n\nCiao,\nDscho\n"},{"id":"65100","messageId":"200801120157.47854.robin.rosenberg@dewire.com","threadId":"11593","inReplyTo":"7v3at3280v.fsf@gitster.siamese.dyndns.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2008-01-12T00:57:46Z","receivedAt":"2008-01-12T00:57:46Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"lördagen den 12 januari 2008 skrev Junio C Hamano:\n> Robin Rosenberg <robin.rosenberg@dewire.com> writes:\n> \n> > Could we just have a lookup table index extension for identifying the \n> > duplicates (when checking is enabled using core configuration option #3324)? \n> > That table would keep a mapping from a normalized form (maybe include \n> > canonical encoding while we're at it) to the actual octet sequence(s) used.\n> \n> I would agree that the index extension, if we ever are going to\n> do this, would be the right place to store this information, at\n> the single repository level.\n> \n> However, this opens up a can of worms.  What's the canonical key\n> should be?  If you want to protect yourself from a unicode\n> normalizing filesystem, you would use one canonicalization,\n> while if you want to protect from a case losing filesystem you\n> would use another?  Or do we at the same time downcase and NFD\n> normalize at the same time and be done with it?\n\nThe worms are out already. So the question is whether there\nis a way of keeping them in the can instead of having them crawl\nall around. I think we could to both unicode (UTF-8 or NFD) and\ndowncase at the same time.\n\n> And where should the configuration be stored?  If a project\n> wants to be interoperable across Linux and vfat, for example,\n\nIn the brand new \".gitconfig\". It could in principle contain any config option,\nbut that would not be safe so I guess one should only allow \"safe\" options\nthere.\n\n-- robin\n"},{"id":"65102","messageId":"B2968E86-D43E-4F5E-921F-DF8A15886C98@sb.org","threadId":"11593","inReplyTo":"alpine.LSU.1.00.0801120139170.8333@wbgn129.biozentrum.uni-wuerzburg.de","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-12T01:16:40Z","receivedAt":"2008-01-12T01:16:40Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"On Jan 11, 2008, at 7:40 PM, Johannes Schindelin wrote:\n\n> On Fri, 11 Jan 2008, Junio C Hamano wrote:\n>\n>> Kevin Ballard <kevin@sb.org> writes:\n>>\n>>> Is there a reason for this? It seems like it would be trivial to end\n>>> up with misdiagnosed \"untracked\" files when using any language other\n>>> than English given this behaviuor.\n>>\n>> No.  The assumption of the code has always been that sane filesystems\n>> would return from readdir() the names you gave from creat().\n>\n> We do not really have to rehash that whole discussion for the Nth  \n> time, do\n> we?\n\n\nApparently so. By Junio's definition, HFS+ is not a sane filesystem,  \nand as git grows more popular with OS X users, this issue is going to  \ncrop up more frequently.\n\nAccording to the HFS+ Volume Format technote[1], filenames in HFS+ are  \nstored in normalized, canonical order. To be more specific, they're  \nstored in a special apple variant of Unicode Normal Form D (the  \nspecial variant is for preserving round-trip with older encodings with  \ncertain codepoint ranges[2]).\n\nIn other words, if you hand an HFS+ filesystem a filename that  \ncontains unicode characters, what you get back later may be in a  \ndifferent format. And that's going to be a problem if git doesn't deal  \nwith this.\n\n[1]: http://developer.apple.com/technotes/tn/tn1150.html#UnicodeSubtleties\n[2]: http://developer.apple.com/qa/qa2001/qa1173.html\n\nNote: CC list stripped because this is a re-sent email, as the list  \nbounced the last one that contained a text/html alternate.\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65103","messageId":"7vprw7zv7s.fsf@gitster.siamese.dyndns.org","threadId":"11593","inReplyTo":"B2968E86-D43E-4F5E-921F-DF8A15886C98@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-01-12T01:30:31Z","receivedAt":"2008-01-12T01:30:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Kevin Ballard <kevin@sb.org> writes:\n\n> On Jan 11, 2008, at 7:40 PM, Johannes Schindelin wrote:\n>\n>> On Fri, 11 Jan 2008, Junio C Hamano wrote:\n>>\n>>> Kevin Ballard <kevin@sb.org> writes:\n>>>\n>>>> Is there a reason for this? It seems like it would be trivial to end\n>>>> up with misdiagnosed \"untracked\" files when using any language other\n>>>> than English given this behaviuor.\n>>>\n>>> No.  The assumption of the code has always been that sane filesystems\n>>> would return from readdir() the names you gave from creat().\n>>\n>> We do not really have to rehash that whole discussion for the Nth\n>> time, do\n>> we?\n>\n> Apparently so. By Junio's definition, HFS+ is not a sane filesystem,\n> and as git grows more popular with OS X users, this issue is going to\n> crop up more frequently.\n\nIt's not \"my\" definition, but you asked the reason and I gave\nthe answer.  We can close this issue of \"is HFS+ sane\" now.\nHFS+ is insane, period.  And as Linus said, you cannot forgive\nits insanity using the historical baggage argument, like MS-DOS.\n\nHOWEVER.\n\nIt is a totally different issue if we want to refuse supporting\ninsane filesystems.  And the answer is no.  It was not my\nintention to say that we do not intend to support them, when I\nexplained the reason why the things are as they are, which was\nthe original question by you.\n\nSee Robin's proposal to let us translate random names we get\nback from readdir() from the filesystem using an additional\nlook-up table in the index extension section that stores mapping\nfrom canonicalized form to the form that the user registered to\nthe index.  I think that is a sane approach to tackle this issue\non insane filesystems like HFS+.\n"},{"id":"65106","messageId":"08E42D03-7AAA-4ADD-9F5B-215AB30E34E1@sb.org","threadId":"11593","inReplyTo":"7vprw7zv7s.fsf@gitster.siamese.dyndns.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Kevin Ballard","fromEmail":"kevin@sb.org","sentAt":"2008-01-12T01:43:35Z","receivedAt":"2008-01-12T01:43:35Z","isPatch":false,"sender":{"key":"kevin@sb.org","avatar":"https://avatars.githubusercontent.com/u/714?v=4"},"body":"On Jan 11, 2008, at 8:30 PM, Junio C Hamano wrote:\n\n>> Apparently so. By Junio's definition, HFS+ is not a sane filesystem,\n>> and as git grows more popular with OS X users, this issue is going to\n>> crop up more frequently.\n>\n> It's not \"my\" definition, but you asked the reason and I gave\n> the answer.  We can close this issue of \"is HFS+ sane\" now.\n> HFS+ is insane, period.  And as Linus said, you cannot forgive\n> its insanity using the historical baggage argument, like MS-DOS.\n\nFair enough, though I believe OS X has a good reason, namely it's an  \nOS designed for regular users rather than servers or programmers. Case- \nsensitivity would confuse my mother.\n\n> HOWEVER.\n>\n> It is a totally different issue if we want to refuse supporting\n> insane filesystems.  And the answer is no.  It was not my\n> intention to say that we do not intend to support them, when I\n> explained the reason why the things are as they are, which was\n> the original question by you.\n\nOk. I wasn't implying anything with that phrase there, I was just  \ntrying to reiterate that HFS+ is case-insensitive and emphasize that  \nthis issue will become more relevant as time goes by.\n\n> See Robin's proposal to let us translate random names we get\n> back from readdir() from the filesystem using an additional\n> look-up table in the index extension section that stores mapping\n> from canonicalized form to the form that the user registered to\n> the index.  I think that is a sane approach to tackle this issue\n> on insane filesystems like HFS+.\n\nIf I knew what the index extension section was, perhaps I would think  \nthat's a good idea ;) I have yet to dive into the gory details of how  \nthis stuff works.\n\n-- \nKevin Ballard\nhttp://kevin.sb.org\nkevin@sb.org\nhttp://www.tildesoft.com\n\n\n"},{"id":"65169","messageId":"85ejcn5jsc.fsf@lola.goethe.zz","threadId":"11593","inReplyTo":"08E42D03-7AAA-4ADD-9F5B-215AB30E34E1@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"David Kastrup","fromEmail":"dak@gnu.org","sentAt":"2008-01-12T12:07:47Z","receivedAt":"2008-01-12T12:07:47Z","isPatch":false,"sender":{"key":"dak@gnu.org","avatar":"https://avatars.githubusercontent.com/u/52141349?v=4"},"body":"Kevin Ballard <kevin@sb.org> writes:\n\n> On Jan 11, 2008, at 8:30 PM, Junio C Hamano wrote:\n>\n>>> Apparently so. By Junio's definition, HFS+ is not a sane filesystem,\n>>> and as git grows more popular with OS X users, this issue is going to\n>>> crop up more frequently.\n>>\n>> It's not \"my\" definition, but you asked the reason and I gave\n>> the answer.  We can close this issue of \"is HFS+ sane\" now.\n>> HFS+ is insane, period.  And as Linus said, you cannot forgive\n>> its insanity using the historical baggage argument, like MS-DOS.\n>\n> Fair enough, though I believe OS X has a good reason, namely it's an\n> OS designed for regular users rather than servers or\n> programmers. Case-sensitivity would confuse my mother.\n\nIf case-sensitivity would be the primary cause of confusion in\nmother-computer interoperation, you have a remarkable mother.\n\n\"Type things the same way and they work the same\" is a simple enough\nrule.\n\n-- \nDavid Kastrup, Kriemhildstr. 15, 44793 Bochum\n"},{"id":"65179","messageId":"20080112144629.GE2963@dpotapov.dyndns.org","threadId":"11593","inReplyTo":"alpine.LFD.1.00.0801111356000.3148@woody.linux-foundation.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-01-12T14:46:29Z","receivedAt":"2008-01-12T14:46:29Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Fri, Jan 11, 2008 at 02:08:35PM -0800, Linus Torvalds wrote:\n> \n> However, it's not like there is even a simple solution. The right place to \n> do that check would probably be in \"add_index_entry()\", but doing a check \n> whether the same file already exists (in a different case) is simply \n> *extremely* expensive for a very critical piece of code, unless we were to \n> change that index data structure a lot (ie add a separate hash for the \n> filenames).\n\nAfter cursory look at the source code, I wonder if converting name1\nand name2 to upper case before memcmp in cache_name_compare() can\nhelp case-insensitive systems. This change will change the order of\nfile names in the index, but I suppose that it should not be a problem,\nbecause the index is host specific. Though, this fix is too simple, so\nI guess, I missed something.\n\n> (And that's totally ignoring the fact that case-insensitivity then also \n> has tons of i18n issues and can get *really* messy \n\nThe proper support of i18n is not simple even without case-insensitivity.\nFor instance, there are four different encodings widely used for Russian\nletters. On Windows alone, you have two simulteniously in the default\nsettings -- Windows-1251 for Windows applications and CP866 for Console\napplications... Actually, some console applications can change its default\nencoding, and it seems Cygwin programs do that. So, based on whether you\nuse gcc from Cygwin or Visual C to compile your console program, you can\nget a different encoding. On *nix in Russia, koi8-r and utf-8 are most\npopular... So, if you have a repository shared between different systems,\nyou cannot think about a file name just as a sequence of bytes anymore.\nOTOH, I doubt that many people are really interested in using non-ASCII\nfile names with Git right now.\n\nDmitry\n"},{"id":"65180","messageId":"20080112150324.GF2963@dpotapov.dyndns.org","threadId":"11593","inReplyTo":"08E42D03-7AAA-4ADD-9F5B-215AB30E34E1@sb.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-01-12T15:03:24Z","receivedAt":"2008-01-12T15:03:24Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Fri, Jan 11, 2008 at 08:43:35PM -0500, Kevin Ballard wrote:\n> \n> Fair enough, though I believe OS X has a good reason, namely it's an  \n> OS designed for regular users rather than servers or programmers. Case- \n> sensitivity would confuse my mother.\n\nMany of *nix servers are running web-services and samba servers, yet most\nusers are even not aware of whether they dealing with case-sensitive file\nsystem or not, let alone being confused by that. This is because most\nregular users will type the name only once when they create a new file\nand then just click on this name. So case-sensitive file systems can\nreally confuse only some badly written applications...\n\nDmitry\n"},{"id":"65187","messageId":"alpine.LSU.1.00.0801121732230.8333@wbgn129.biozentrum.uni-wuerzburg.de","threadId":"11593","inReplyTo":"200801120157.47854.robin.rosenberg@dewire.com","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-01-12T16:33:12Z","receivedAt":"2008-01-12T16:33:12Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 12 Jan 2008, Robin Rosenberg wrote:\n\n> lördagen den 12 januari 2008 skrev Junio C Hamano:\n>\n> > And where should the configuration be stored?  If a project wants to \n> > be interoperable across Linux and vfat, for example,\n> \n> In the brand new \".gitconfig\". It could in principle contain any config \n> option, but that would not be safe so I guess one should only allow \n> \"safe\" options there.\n\nFunny: I had the same idea (.gitconfig) for the crlf issues...\n\nCiao,\nDscho\n"},{"id":"65204","messageId":"alpine.LFD.1.00.0801121040010.2806@woody.linux-foundation.org","threadId":"11593","inReplyTo":"20080112144629.GE2963@dpotapov.dyndns.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-01-12T18:47:10Z","receivedAt":"2008-01-12T18:47:10Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 12 Jan 2008, Dmitry Potapov wrote:\n> \n> After cursory look at the source code, I wonder if converting name1\n> and name2 to upper case before memcmp in cache_name_compare() can\n> help case-insensitive systems. This change will change the order of\n> file names in the index, but I suppose that it should not be a problem,\n> because the index is host specific. Though, this fix is too simple, so\n> I guess, I missed something.\n\nNo, the index isn't host-specific, and we also have a deep knowledge of \nthe fact that the index order is the same as the unpacked tree order.\n\nSo no, we absolutely cannot just sort the index differently. We literally \nneed to have a separate key for a \"upper case lookup\".\n\n(That separate key can be just a hash table - it doesn't need to be \nsomething you can iterate over, so it can be pretty simple).\n\n> > (And that's totally ignoring the fact that case-insensitivity then also \n> > has tons of i18n issues and can get *really* messy \n> \n> The proper support of i18n is not simple even without case-insensitivity.\n> For instance, there are four different encodings widely used for Russian\n> letters.\n\n.. and git is very clear about this: filenames are *not* \"characters\" in \nthe i18n sense, they are series of bytes. There is absolutely no room for \nambiguity, and there is no locale for those things.\n\nAnd that isn't going to change. It's the only sane way to do \nlocale-independent names: people can *choose* to see the filenames as some \nUTF-8 sequence, or a series of Latin1, or anything, but that's not \nsomething git itself will care about.\n\nTrying to involve locale in name comparison simply isn't possible. Two \ndifferent repositories on two different filesystems would get two \ndifferent answers. And that is simply unacceptable in a distributed \nsystem.\n\nWhat we can do is to make the simple cases (ie the locale-*independent* \nones) warn about problems with case insensitivity.\n\n\t\t\tLinus\n"},{"id":"65212","messageId":"20080112192903.GJ2963@dpotapov.dyndns.org","threadId":"11593","inReplyTo":"alpine.LFD.1.00.0801121040010.2806@woody.linux-foundation.org","subject":"Re: Re-casing directories on case-insensitive systems","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-01-12T19:29:03Z","receivedAt":"2008-01-12T19:29:03Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Sat, Jan 12, 2008 at 10:47:10AM -0800, Linus Torvalds wrote:\n> \n> And that isn't going to change. It's the only sane way to do \n> locale-independent names: people can *choose* to see the filenames as some \n> UTF-8 sequence, or a series of Latin1, or anything, but that's not \n> something git itself will care about.\n\nUnfortunately, to agree on a single encoding for different systems is\neven more difficult than agreeing on a single end-of-line encoding.\nOTOH, it is not a real issue as long as anyone use ASCII names only.\n\n> \n> Trying to involve locale in name comparison simply isn't possible.\n\nAgreed. However, the proper solution would be that all filenames are\nstored in UTF-8, so conversation is done when a file is added to the\nindex. But that requires a lot of work, and as I said before, I doubt\nthat many people really want to store files with non-ASCII names, after\nall, Git is a developer tool. So, as far as I am concern, it does not\nworth efforts.\n\nDmitry\n"}]}