git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC 1/8] UTF helpers

From
Jay Soffian <jaysoffian@gmail.com>
Date
May 14, 2009, 13:57 UTC
Message-ID
<76718490905140657h4f4544fdrc5b9ce15ca36c8a5@mail.gmail.com>
In-Reply-To
<7viqk4z4cv.fsf@alter.siamese.dyndns.org>
On Thu, May 14, 2009 at 12:38 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 29 quoted lines
> It is rather sad; I suspect that the core of the series is buried in too
> much cruft deep enough to discourage many potential reviewers.  I think
> the entire series look incoherent because attacking two largely unrelated
> things at once.
>
>  (1) Normalizing pathnames internally to UTF-8 and possibly convert it
>     back to native upon use (e.g. creat(), lstat(), unlink()) and output.
>     As Linus analyzed, this shouldn't be done too early in the callchain
>     for performance reasons, but I think your patch would give us a good
>     set of starting points to follow where the result from readdir(),
>     user input and other things that are pathnames come from and go.
>
>     This part of the patch series was inspiring.  You have to worry about
>     gitignore, gitattributes and readlink() vs contents of a blob object
>     that records a symbolic link values, which I think either escaped
>     analysis people have done so far or being ignored as a small detail,
>     but they are important;
>
>  (2) Passing cat-file output through iconv to convert it.
>
>     I think this is unwarranted, even if the object given to cat-file
>     happens to be a commit or a tag object and you want to convert their
>     messages in native encoding.
>
>     I am not sure what should happen to "cat-file tree", "ls-files" and
>     "ls-tree".  The output from these plumbing does show pathnames, but I
>     tend to think it is Porcelain's job to turn them into whatever
>     encoding they want to use.  So are input to "update-index --stdin",
>     but I am still just thinking out loud.

I definitely do not have the time to work on unicode/utf-8/i18n support for git right now, but as an OS X user, it is something that interests me. When this topic periodically pops up, I squirrel away the useful messages into my "someday" folder. So even though it may seem that reviewing these patches is wasted effort, comments like the above are helpful. I say this because I expect someday someone will work on this topic, even if it is not me, and hopefully they can locate prior discussion in the mailing list archives and as such it will be of some use.

IOW, your comments are useful and appreciated, even if it doesn't lead to improved patches right away.

j.
Previous: Junio C HamanoNext: Robin Rosenberg
Message 17 of 20 in “[RFC 0/8] Antique UTF-8 filename support”
  1. Robin RosenbergMay 12, 2009
  2. 1/8 UTF helpersRobin Rosenberg, May 12, 2009
  3. 2/8 Messages in locale.Robin Rosenberg, May 12, 2009
  4. 3/8 Extend tests to cover locale wrt to commit messages.Robin Rosenberg, May 12, 2009
  5. 4/8 UTF file names.Robin Rosenberg, May 12, 2009
  6. Johannes SchindelinMay 13, 2009
  7. Robin RosenbergMay 13, 2009
  8. Esko LuontolaMay 13, 2009
  9. Andreas EricssonMay 13, 2009
  10. Esko LuontolaMay 13, 2009
  11. Alex RiesenMay 13, 2009
  12. Junio C HamanoMay 13, 2009
  13. Esko LuontolaMay 13, 2009
  14. Junio C HamanoMay 13, 2009
  15. Johannes SchindelinMay 13, 2009
  16. Junio C HamanoMay 14, 2009
  17. Jay SoffianMay 14, 2009
  18. 6/8 test of utf_locallinksRobin Rosenberg, May 12, 2009
  19. 7/8 Convert symlink dest in diffRobin Rosenberg, May 12, 2009
  20. 8/8 UTF-8 in non-SHA1-objectsRobin Rosenberg, May 12, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.