{"thread":{"id":"53912","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","startedAt":"2020-07-23T15:20:55Z","lastAt":"2020-07-24T21:09:49Z","messageCount":8,"participants":["Casey Meijer","brian m. carlson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"401929","messageId":"8BABB6F0-517F-4AA0-9FF9-92AF8C33CD0E@strongestfamilies.com","threadId":"53912","inReplyTo":"EEA65ED1-2BE0-41AD-84CC-780A9F4D9215@strongestfamilies.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Casey Meijer","fromEmail":"cmeijer@strongestfamilies.com","sentAt":"2020-07-23T15:20:50Z","receivedAt":"2020-07-23T15:20:55Z","isPatch":false,"sender":{"key":"cmeijer@strongestfamilies.com","avatar":null},"body":"This just bit me; it seems quite old, and I wanted to propose an alternative solution (maybe it doesn’t work for some reason I’m unaware of):\nhttps://marc.info/?l=git&m=154473525401677&w=2\n \nWhy not just preserve the existing semantics of the main worktree by checking the worktree refs first unconditionally and only fall back to the main refs when the ref doesn’t exist locally in the worktree?\n \nThis would have the added benefit of allowing power users to override refs in their worktrees and would, if I’m not mistaken, preserve the semantics of the main worktree in case-insensitive and case-sensitive filesystems. \n \nAnywho, just a thought.  I could work on a patch if this approach makes sense at least as an intermediary until there’s a pluggable storage backend for non-FS stores 😉 (I'd also be somewhat interested in implementing a postgres/sql storage backend if this project is moving forwards __ ).\n \n \nBest,\n \nCasey Meijer\n\n"},{"id":"401985","messageId":"20200724011944.GD1758454@crustytoothpaste.net","threadId":"53912","inReplyTo":"8BABB6F0-517F-4AA0-9FF9-92AF8C33CD0E@strongestfamilies.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-07-24T01:19:44Z","receivedAt":"2020-07-24T01:20:23Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-07-23 at 15:20:50, Casey Meijer wrote:\n> This just bit me; it seems quite old, and I wanted to propose an alternative solution (maybe it doesn’t work for some reason I’m unaware of):\n> https://marc.info/?l=git&m=154473525401677&w=2\n>  \n> Why not just preserve the existing semantics of the main worktree by checking the worktree refs first unconditionally and only fall back to the main refs when the ref doesn’t exist locally in the worktree?\n>  \n> This would have the added benefit of allowing power users to override refs in their worktrees and would, if I’m not mistaken, preserve the semantics of the main worktree in case-insensitive and case-sensitive filesystems.\n\nIt isn't clear to me exactly what you're suggesting. Are you suggesting\nthat we allow \"head\" instead of \"HEAD\" in worktrees, or that we allow\nrefs in general to be case insensitive, or something else?\n\n> Anywho, just a thought.  I could work on a patch if this approach makes sense at least as an intermediary until there’s a pluggable storage backend for non-FS stores 😉 (I'd also be somewhat interested in implementing a postgres/sql storage backend if this project is moving forwards __ ).\n\nThere is a proposal for a ref storage backend called \"reftable\" which\nwill not store the ref names in the file system, and work is being done\non it. There has been a suggestion for an SQLite store in the past, but\nthat causes problems for certain implementations, such as JGit, which do\nnot want to have C bindings.\n-- \nbrian m. carlson: Houston, Texas, US\n"},{"id":"401986","messageId":"xmqqft9hn1pd.fsf@gitster.c.googlers.com","threadId":"53912","inReplyTo":"20200724011944.GD1758454@crustytoothpaste.net","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-24T01:25:50Z","receivedAt":"2020-07-24T01:26:11Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> It isn't clear to me exactly what you're suggesting. Are you suggesting\n> that we allow \"head\" instead of \"HEAD\" in worktrees, or that we allow\n> refs in general to be case insensitive, or something else?\n\n> There is a proposal for a ref storage backend called \"reftable\" which\n> will not store the ref names in the file system, and work is being done\n> on it. There has been a suggestion for an SQLite store in the past, but\n> that causes problems for certain implementations, such as JGit, which do\n> not want to have C bindings.\n\nYes, another important thing to point out is that one shared goal of\nthese efforts is so that users, even those on case insensitive\nfilesystems, can name their refs foo and FOO and have the system\ntreat these as two distinct refs. IOW, wanting to enhance \"support\"\nfor case insensitive treatment of refs will not fly---asking for\n\"head\" and getting contents of \"HEAD\" on certain platforms is a bug,\ninduced by limited filesystem these platforms use, and it is being\nfixed.\n\nThanks.\n\n"},{"id":"402008","messageId":"5352321F-0122-49BA-B778-80A84DDABB71@strongestfamilies.com","threadId":"53912","inReplyTo":"xmqqft9hn1pd.fsf@gitster.c.googlers.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Casey Meijer","fromEmail":"cmeijer@strongestfamilies.com","sentAt":"2020-07-24T18:07:29Z","receivedAt":"2020-07-24T18:07:36Z","isPatch":false,"sender":{"key":"cmeijer@strongestfamilies.com","avatar":null},"body":"It's definitely a bug and it's kind of amazing that it's been floating about for 2 years. \nI'm not suggesting anything really change except the way you determine whether a ref is \"work-tree local\" or not. \nThis way on case sensitive filesystems only HEAD will be accepted, and on case insensitive filesystems both head and HEAD\nwill be valid (and will refer to the same file/ref), replicating the semantics of the primary worktree. \n\nNamely, instead of checking explicitly for \"HEAD\" (or going through some hoops to determine if the filesystem \n*is* case sensitive), just look in the worktree refs. If it's in there, then it's worktree local. If not, then not. \n\nLike I said, maybe there are some problems with this approach that I'm not aware of, but if so, I think it's worth thinking \nabout whether those problems are resolvable 😊 \n\nAs far as alternate storage engines, I'd be more interested in seeing core git builtout to support a plugable storage engine than any specific implementation.\nTake a look at PostgreSQL's work on Table Access Methods for an example in this vein. I think this idea plays well with my proposal above as well because it \ndelegates the responsibility of case sensitivity to the storage backend (in this case, the filesystem). \n\n\nBest,\n\nCasey\n\nOn 2020-07-23, 10:25 PM, \"Junio C Hamano\" wrote:\n\n \"brian m. carlson\" writes:\n\n > It isn't clear to me exactly what you're suggesting. Are you suggesting\n > that we allow \"head\" instead of \"HEAD\" in worktrees, or that we allow\n > refs in general to be case insensitive, or something else?\n\n > There is a proposal for a ref storage backend called \"reftable\" which\n > will not store the ref names in the file system, and work is being done\n > on it. There has been a suggestion for an SQLite store in the past, but\n > that causes problems for certain implementations, such as JGit, which do\n > not want to have C bindings.\n\n Yes, another important thing to point out is that one shared goal of\n these efforts is so that users, even those on case insensitive\n filesystems, can name their refs foo and FOO and have the system\n treat these as two distinct refs. IOW, wanting to enhance \"support\"\n for case insensitive treatment of refs will not fly---asking for\n \"head\" and getting contents of \"HEAD\" on certain platforms is a bug,\n induced by limited filesystem these platforms use, and it is being\n fixed.\n\n Thanks.\n\n\n"},{"id":"402009","messageId":"EE35569F-6029-4659-86B3-29FBAAD7C491@strongestfamilies.com","threadId":"53912","inReplyTo":"20200724011944.GD1758454@crustytoothpaste.net","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Casey Meijer","fromEmail":"cmeijer@strongestfamilies.com","sentAt":"2020-07-24T18:14:03Z","receivedAt":"2020-07-24T18:14:07Z","isPatch":false,"sender":{"key":"cmeijer@strongestfamilies.com","avatar":null},"body":"I think I misunderstood your claim actually Brian. What is a bug is asking for worktree A's head and getting the main worktree's head. A super dangerous bug. \n\nI certainly disagree with your assertion that asking for head and not getting HEAD (or HeaD or hEAd) on a case-insensitive storage engine isn't a bug and it certainly \nshouldn't be a bug once extensible storage engines are in place: the storage engine should have final say on how objects are stored and retrieved, not git-core. \n\nBest,\n\nCasey\n\nOn 2020-07-23, 10:19 PM, \"brian m. carlson\" wrote:\n\n On 2020-07-23 at 15:20:50, Casey Meijer wrote:\n > This just bit me; it seems quite old, and I wanted to propose an alternative solution (maybe it doesn’t work for some reason I’m unaware of):\n > https://marc.info/?l=git&m=154473525401677&w=2\n > \n > Why not just preserve the existing semantics of the main worktree by checking the worktree refs first unconditionally and only fall back to the main refs when the ref doesn’t exist locally in the worktree?\n > \n > This would have the added benefit of allowing power users to override refs in their worktrees and would, if I’m not mistaken, preserve the semantics of the main worktree in case-insensitive and case-sensitive filesystems.\n\n It isn't clear to me exactly what you're suggesting. Are you suggesting\n that we allow \"head\" instead of \"HEAD\" in worktrees, or that we allow\n refs in general to be case insensitive, or something else?\n\n > Anywho, just a thought. I could work on a patch if this approach makes sense at least as an intermediary until there’s a pluggable storage backend for non-FS stores 😉 (I'd also be somewhat interested in implementing a postgres/sql storage backend if this project is moving forwards __ ).\n\n There is a proposal for a ref storage backend called \"reftable\" which\n will not store the ref names in the file system, and work is being done\n on it. There has been a suggestion for an SQLite store in the past, but\n that causes problems for certain implementations, such as JGit, which do\n not want to have C bindings.\n -- \n brian m. carlson: Houston, Texas, US\n\n"},{"id":"402010","messageId":"02819EF1-CCA7-4A0F-9497-F6A52E6CBF30@strongestfamilies.com","threadId":"53912","inReplyTo":"xmqqft9hn1pd.fsf@gitster.c.googlers.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Casey Meijer","fromEmail":"cmeijer@strongestfamilies.com","sentAt":"2020-07-24T18:17:12Z","receivedAt":"2020-07-24T18:17:16Z","isPatch":false,"sender":{"key":"cmeijer@strongestfamilies.com","avatar":null},"body":"Sorry I got mixed up,, that last message should have been addressed to Junio. \n\nMy apologies. \n\nTo put it very simply, I'm asking that git respect the separation of concerns between itself \nand its storage engine (regardless of whether that's pluggable, or just the current filesystem, which I guess is technically pluggable, lol).\n\n\nBest,\n\nCasey\n\nOn 2020-07-23, 10:25 PM, \"Junio C Hamano\" wrote:\n\n \"brian m. carlson\" writes:\n\n > It isn't clear to me exactly what you're suggesting. Are you suggesting\n > that we allow \"head\" instead of \"HEAD\" in worktrees, or that we allow\n > refs in general to be case insensitive, or something else?\n\n > There is a proposal for a ref storage backend called \"reftable\" which\n > will not store the ref names in the file system, and work is being done\n > on it. There has been a suggestion for an SQLite store in the past, but\n > that causes problems for certain implementations, such as JGit, which do\n > not want to have C bindings.\n\n Yes, another important thing to point out is that one shared goal of\n these efforts is so that users, even those on case insensitive\n filesystems, can name their refs foo and FOO and have the system\n treat these as two distinct refs. IOW, wanting to enhance \"support\"\n for case insensitive treatment of refs will not fly---asking for\n \"head\" and getting contents of \"HEAD\" on certain platforms is a bug,\n induced by limited filesystem these platforms use, and it is being\n fixed.\n\n Thanks.\n\n\n"},{"id":"402011","messageId":"xmqq7dusn1su.fsf@gitster.c.googlers.com","threadId":"53912","inReplyTo":"02819EF1-CCA7-4A0F-9497-F6A52E6CBF30@strongestfamilies.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-07-24T19:36:01Z","receivedAt":"2020-07-24T19:36:09Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Casey Meijer writes:\n\n> Sorry I got mixed up,, that last message should have been\n> addressed to Junio.\n>\n> My apologies. \n>\n> To put it very simply, I'm asking that git respect the separation\n> of concerns between itself and its storage engine (regardless of\n> whether that's pluggable, or just the current filesystem, which I\n> guess is technically pluggable, lol).\n\nIf \"git\" is told to store ref 'foo' pointing at object X and then\nref 'Foo' pointing at object Y by the end user, after claiming to\nhave done these two operations, if it is then asked about the value\nof 'foo', it must say that 'foo' points at object X and not Y. If a\nref backend is based on case insensitive filesystem, there are only\ntwo options available. (1) ignore case and violate the expectation\nof end user. (2) come up with a way to \"defeat\" the limitation of\ncase insensitivity imposed by the filesystem (e.g. your ref backend\nimplementation _could_ URLencode/decode the ref before using it as a\nfilename on such a filesystem). \n\nDoing (2) would be transparent to the rest of Git (i.e. the rest of\nGit does not have to care that each ref is stored in a file, whose\nfilename is encoded version of the refname) and gives us a good\nseparation of concerns between it and the storage backend. Those\nwho ported Git to case insensitive filesystems didn't and chose (1).\n\nAs (1) violates end-user expectation, I would think it is fair to\ndeclare it a bug.\n"},{"id":"402027","messageId":"20200724210940.GE1758454@crustytoothpaste.net","threadId":"53912","inReplyTo":"EE35569F-6029-4659-86B3-29FBAAD7C491@strongestfamilies.com","subject":"Re: BUG FOLLOWUP: Case insensitivity in worktrees","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2020-07-24T21:09:40Z","receivedAt":"2020-07-24T21:09:49Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2020-07-24 at 18:14:03, Casey Meijer wrote:\n> I think I misunderstood your claim actually Brian. What is a bug is\n> asking for worktree A's head and getting the main worktree's head. A\n> super dangerous bug.\n> \n> I certainly disagree with your assertion that asking for head and not\n> getting HEAD (or HeaD or hEAd) on a case-insensitive storage engine\n> isn't a bug and it certainly shouldn't be a bug once extensible\n> storage engines are in place: the storage engine should have final say\n> on how objects are stored and retrieved, not git-core.\n\nIf you want to refer to HEAD, writing it \"head\" is always wrong. \"head\"\nis not a special ref to Git, and on a case-sensitive system, I am fully\nentitled to create a branch, tag, or other ref with that name that is\nindependent from HEAD.\n\nIt's wrong because regardless of operating system, you don't\nintrinsically know whether the repository is case sensitive. Windows 10\npermits case-sensitive directories and macOS has case-sensitive file\nsystems, so you cannot assume that \"head\" and \"HEAD\" are the same\nwithout knowing the setting of \"core.ignorecase\" and the properties of\nthe file system.\n\nSo when you write \"head\", you are not asking for HEAD in any worktree or\nrepository at all.\n\nWe are fully aware that Git cannot consistently store refs differing in\ncase on case-insensitive file systems, and we agree that's a bug.\nReftable will fix that, and as I mentioned, it is being worked on. It\nis not, however, a deficiency that refs are intrinsically case\nsensitive, and let me explain why.\n\nFirst, Git does not require that refs are in any particular encoding.\nSpecifically, they need not be in Unicode or UTF-8. It is valid to have\nmany characters in a ref name, including 0xff. That means any type of\ncase folding is not possible, since a ref need not correspond to actual\ntext.\n\nSecond, even if we did require them to be UTF-8, it is impossible to\nconsistently fold case in a way that works for all locales. Turkish and\nother Turkic languages have a dotted I and a dotless I[0]. The ASCII\nuppercase I would fold to a dotless lowercase I for Turkish and to the\nASCII (dotted) lowercase I for English. Similarly, the ASCII lowercase\nI is dotted, and folds to a dotted uppercase I in Turkish and an ASCII\n(dotless) uppercase I in English.\n\nIt is literally not possible to correctly perform case-folding in a\nlocale-independent way. Every attempt to do so will get at least this\ncase wrong (not to mention other cases that occur), and Turkic languages\nare spoken by 200 million people, so ignoring their needs is not only\nharmful, but also impacts a massive number of people. That major OS\ndesigners have made this mistake doesn't mean that we should as well.\n\nWe wouldn't perform ASCII-only case folding for all of the reasons\nmentioned above and because it's Anglocentric. As someone who speaks\nboth Spanish and French, I would find that unsuitable and the results\nbizarre.\n\nSo I understand that you may expect that on Windows or macOS that you\ncan write \"head\" and get HEAD and be surprised when that doesn't work in\nall cases. But that is not, and never has been, expected to work, nor\nis it a bug that it doesn't.\n\n[0] https://en.wikipedia.org/wiki/Dotted_and_dotless_I\n-- \nbrian m. carlson: Houston, Texas, US\n"}]}