git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH] Windows: Assume all file names to be UTF-8 encoded.

From
Peter Krefting <peter@softwolves.pp.se>
Date
Mar 3, 2009, 11:54 UTC
Message-ID
<alpine.DEB.2.00.0903031248040.3702@perkele.intern.softwolves.pp.se>
In-Reply-To
<20090303075655.GB9875@lars.home.noschinski.de>
Lars Noschinski:
> Using no encoding for filenames was the obvious (and I would argue) 
> correct choice. Unix filenames are specified to be a sequence of bytes, 
> excluding '/' and '\0'.

I know the Unix way of thinking lends itself to such a design. This is one of the few cases where I personally think Unix has got it wrong, and Windows (NT) has got it right. But then again, Unix' design pre-dates the locale issue by quite some time, so it is not difficult to see where it comes from.

> Changing the filename (on checkout), so that the user sees an Ü regardless 
> of his or her locale (instead of an \0xDC, which only resolves to an Ü on 
> latin-1) would be an absolutely broken concept here.

Why would it? It is my view as a user on my files that define how file names are looked upon. If I have three machines, one Linux box using a iso8859-1 locale, an OS X box (where, I would believe, file APIs use UTF-8, someone please correct me if I'm wrong), and a Windows box (which uses UTF-16 on the file system layer, but does provide compatibility functions that use char pointers), and create a file on each of these called "Ü.txt" (which would be the sequence "DC 2E 74 78 74" on the Linux box, "C3 9C 2E 74 78 74" (or probably something else since I believe OS X decomposes the string) on the OS X box and "00DC 002E 0074 0078 0074" on the Windows box, I see these three file names as equal.

If I would create a Git repo on each of the three machines and put the file name in it, and then clone that on one of the other machines. *I* would assume that the file names were converted to fit the host operating system.

> IMHO having encoding specific open functions is begging for problems.

Indeed. That's why I like Windows' wchar_t APIs, and dislike Unix' and Linux' char APIs that, in some ways, depend on the user locale.

-- 
\\// Peter - http://www.softwolves.pp.se/
Previous: Lars NoschinskiNext: Lars Noschinski
Message 19 of 26 in “Windows: Assume all file names to be UTF-8 encoded.”
  1. Windows: Assume all file names to be UTF-8 encoded.Peter Krefting, Mar 2, 2009
  2. Johannes SixtMar 2, 2009
  3. Peter KreftingMar 2, 2009
  4. Johannes SchindelinMar 2, 2009
  5. Peter KreftingMar 2, 2009
  6. Johannes SixtMar 2, 2009
  7. Peter KreftingMar 2, 2009
  8. Robin RosenbergMar 2, 2009
  9. Peter KreftingMar 2, 2009
  10. Robin RosenbergMar 2, 2009
  11. Peter KreftingMar 3, 2009
  12. Dmitry PotapovMar 3, 2009
  13. Peter KreftingMar 3, 2009
  14. Robin RosenbergMar 7, 2009
  15. Peter KreftingMar 2, 2009
  16. Thomas RastMar 2, 2009
  17. Peter KreftingMar 2, 2009
  18. Lars NoschinskiMar 3, 2009
  19. Peter KreftingMar 3, 2009
  20. Lars NoschinskiMar 3, 2009
  21. Robin RosenbergMar 3, 2009
  22. Dmitry PotapovMar 3, 2009
  23. Peter KreftingMar 3, 2009
  24. Dmitry PotapovMar 3, 2009
  25. Peter KreftingMar 4, 2009
  26. Dmitry PotapovMar 4, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.