git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Git clone and case sensitivity

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Jul 31, 2018, 19:39 UTC
Message-ID
<c174d972-fc5b-88e7-b437-89b9dd29e579@jeffhostetler.com>
In-Reply-To
<20180729092759.GA14484@sigill.intra.peff.net>
On 7/29/2018 5:28 AM, Jeff King wrote:
Show 51 quoted lines
> On Sun, Jul 29, 2018 at 07:26:41AM +0200, Duy Nguyen wrote:
> 
>>> strcasecmp() will only catch a subset of the cases. We really need to
>>> follow the same folding rules that the filesystem would.
>>
>> True. But that's how we handle case insensitivity internally. If a
>> filesytem has more sophisticated folding rules then git will not work
>> well on that one anyway.
> 
> Hrm. Yeah, I guess that's the best we can do for the actual in-memory
> checks. Everything else depends on doing an actual filesystem operation,
> and our icase stuff kicks in way before then. I was mostly thinking of
> HFS+ utf8 normalization weirdness, but I guess people are accustomed to
> that by now.
> 
>>> For the case of clone, I actually wonder if we could detect during the
>>> checkout step that a file already exists. Since we know that the
>>> directory we started with was empty, then if it does, either:
>>>
>>>    - there's some funny case-folding going on that means two paths in the
>>>      repository map to the same name in the filesystem; or
>>>
>>>    - somebody else is writing to the directory at the same time as us
>>
>> This is exactly what my first patch does (minus the sparse checkout
>> part).
> 
> Right, sorry, I should have read that one more carefully.
> 
>> But without knowing the exact folding rules, I don't think we can
>> locate this "somebody else" who wrote the first path. So if N paths
>> are treated the same by this filesystem, we could only report N-1 of
>> them.
>>
>> If we want to report just one path when this happens though, then this
>> works quite well.
> 
> Hmm. Since most such systems are case-preserving, would it be possible
> to report the name of the existing file? Doing it via opendir/readdir is
> hacky, and anyway puts the burden on us to find the matching name. Doing
> it via fstat() on the opened file doesn't work because at that the
> filesystem has resolved the name to an inode.
> 
> So yeah, perhaps strcasecmp() is the best we can do (I do agree that
> being able to mention all of the conflicting names is a benefit).
> 
> I guess we should be using fspathcmp(), though, in case it later learns
> to be smarter.
> 
> -Peff
> 

As has already been mentioned, this gets into weird territory really fast, between case folding, final space/dot on windows, utf8 NFC/NFD weirdness on the mac, utf8 invisible chars on the mac, long/short names on windows, and etc.

And that's just for filenames. Things really get weird if directory names have these ambiguities.

Perhaps just print the problematic paths (where the collision is detected) and let the user decide how to correct them.

Perhaps we could have a separate tool that could scan the index or commit for potential conflicts and warn them in advance (granted, it might not be perfect and may report a few false positives).

Forcing them into a sparse-checkout situation might be over their skill level.

Jeff
Previous: Carlo Arenas
Message 96 of 96 in “Git clone and case sensitivity”
  1. Paweł ParuzelJul 27, 2018
  2. brian m. carlsonJul 27, 2018
  3. Duy NguyenJul 28, 2018
  4. Duy NguyenJul 28, 2018
  5. Jeff KingJul 28, 2018
  6. Duy NguyenJul 28, 2018
  7. Simon RuderichJul 28, 2018
  8. Jeff KingJul 28, 2018
  9. brian m. carlsonJul 28, 2018
  10. Duy NguyenJul 29, 2018
  11. Jeff KingJul 29, 2018
  12. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Jul 30, 2018
  13. Torsten BögershausenJul 31, 2018
  14. Duy NguyenAug 1, 2018
  15. Elijah NewrenJul 31, 2018
  16. Junio C HamanoJul 31, 2018
  17. Jeff KingJul 31, 2018
  18. Junio C HamanoJul 31, 2018
  19. Jeff KingJul 31, 2018
  20. Junio C HamanoJul 31, 2018
  21. Junio C HamanoAug 1, 2018
  22. Duy NguyenAug 2, 2018
  23. Junio C HamanoAug 2, 2018
  24. Jeff KingAug 2, 2018
  25. Junio C HamanoAug 2, 2018
  26. Jeff KingAug 2, 2018
  27. Jeff HostetlerAug 3, 2018
  28. Junio C HamanoAug 3, 2018
  29. Jeff KingAug 3, 2018
  30. Jeff HostetlerAug 5, 2018
  31. Torsten BögershausenAug 3, 2018
  32. Duy NguyenAug 1, 2018
  33. Junio C HamanoJul 31, 2018
  34. Duy NguyenAug 1, 2018
  35. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 7, 2018
  36. Junio C HamanoAug 7, 2018
  37. Jeff HostetlerAug 8, 2018
  38. Jeff KingAug 8, 2018
  39. Junio C HamanoAug 9, 2018
  40. Jeff KingAug 9, 2018
  41. Jeff HostetlerAug 9, 2018
  42. Jeff KingAug 9, 2018
  43. Elijah NewrenAug 9, 2018
  44. Jeff KingAug 9, 2018
  45. Elijah NewrenAug 9, 2018
  46. Jeff KingAug 9, 2018
  47. Elijah NewrenAug 9, 2018
  48. Junio C HamanoAug 9, 2018
  49. 0/1 clone: warn on colidding entries on checkoutNguyễn Thái Ngọc Duy, Aug 10, 2018
  50. 1/1 clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 10, 2018
  51. Junio C HamanoAug 10, 2018
  52. SZEDER GáborAug 11, 2018
  53. Duy NguyenAug 11, 2018
  54. Junio C HamanoAug 13, 2018
  55. Duy NguyenAug 13, 2018
  56. Junio C HamanoAug 10, 2018
  57. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 12, 2018
  58. Jeff HostetlerAug 13, 2018
  59. Junio C HamanoAug 13, 2018
  60. Torsten BögershausenAug 15, 2018
  61. Duy NguyenAug 15, 2018
  62. config.txt: clarify core.checkStat = minimalNguyễn Thái Ngọc Duy, Aug 16, 2018
  63. Junio C HamanoAug 16, 2018
  64. Duy NguyenAug 16, 2018
  65. Junio C HamanoAug 16, 2018
  66. Junio C HamanoAug 17, 2018
  67. Duy NguyenAug 17, 2018
  68. Junio C HamanoAug 15, 2018
  69. Torsten BögershausenAug 16, 2018
  70. Duy NguyenAug 16, 2018
  71. Junio C HamanoAug 16, 2018
  72. clone: report duplicate entries on case-insensitive filesystemsNguyễn Thái Ngọc Duy, Aug 17, 2018
  73. Junio C HamanoAug 17, 2018
  74. Duy NguyenAug 17, 2018
  75. Torsten BögershausenAug 17, 2018
  76. clone: report duplicate entries on case-insensitive filesystemsCarlo Marcelo Arenas Belón, Nov 19, 2018
  77. Torsten BögershausenNov 19, 2018
  78. Carlo ArenasNov 19, 2018
  79. Duy NguyenNov 19, 2018
  80. Duy NguyenNov 19, 2018
  81. Duy NguyenNov 19, 2018
  82. Duy NguyenNov 19, 2018
  83. Ramsay JonesNov 19, 2018
  84. Ramsay JonesNov 19, 2018
  85. Carlo ArenasNov 20, 2018
  86. Junio C HamanoNov 20, 2018
  87. clone: fix colliding file detection on APFSNguyễn Thái Ngọc Duy, Nov 20, 2018
  88. Ramsay JonesNov 20, 2018
  89. Carlo ArenasNov 20, 2018
  90. Duy NguyenNov 20, 2018
  91. 1/1 t5601-99: Enable colliding file detection for MINGWtboegi@web.de, Nov 22, 2018
  92. 1/1 t5601-99: Enable colliding file detection for MINGWCarlo Marcelo Arenas Belón, Nov 22, 2018
  93. Johannes SchindelinNov 23, 2018
  94. Ramsay JonesNov 19, 2018
  95. Carlo ArenasNov 19, 2018
  96. Jeff HostetlerJul 31, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.