git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC 1/8] UTF helpers

From
Andreas Ericsson <ae@op5.se>
Date
May 13, 2009, 10:02 UTC
Message-ID
<4A0A9AA2.1000004@op5.se>
In-Reply-To
<4A0A91CE.3080905@gmail.com>
Esko Luontola wrote:
Show 22 quoted lines
> Robin Rosenberg wrote on 13.5.2009 8:24:
>> If the conclusion is that this is a way forward, then I
>> could start working on a completely new set of much cleaner patches.,
> 
> That would be great!
> 
> I see that in those early patches you took the approach of converting 
> the filenames from the local encoding to UTF-8 at the outer edges of 
> Git. That obviously was the easiest way to make the changes with minimal 
> changes to Git.
> 
> I've been thinking about a bit more extensive approach, which should 
> serve the interest of all stakeholders:
> 
> 
> Now the tree object contains the following information for each file: 
> filename, mode, sha1. To that would be added one more string: filename
> encoding. Unless the encoding is specified (such as in old commits 
> before the encoding information was added), the default encoding is 
> "binary", which is the same as how Git works now (it thinks filenames as 
> series of bytes, ignoring their encoding completely).
> 
[ long and incompatible plan removed ]
Show 6 quoted lines
> One big question is, that will this change require a change to the 
> repository format? Will it be possible to add the encoding field to the 
> tree object, without breaking compatibility with older Git clients? If 
> compatibility needs to be broken, how it can be done in a controlled 
> fashion?
> 

Generally when one wants to change one of the basic object types in git, some extraordinary benefit has to be shown that is not aimed at just a few people. Academic benefits (ie, "non-real-worldy") do not fall into that category. In fact, it's so rare for someone to provide such enormous benefit that the only time a core object format in git has been incompatibly changed is when Linus decided that trees should be able to have subtrees. The change reduced the repository size for the early git-tracked Linux kernel to about 4% of its original size, so there was a clear, undisputable and obvious benefit huge enough to warrant breaking the git repository format entirely just to get it in (I might have gotten those details entirely wrong, but it was something along those lines).

So unless you can change tree objects in a way that lets older git clients understand them while still adding this encoding cruft (it's cruft to me), I think your chances of getting such a change into the git core are about the size of the colour green.

If you're *really* serious about it though, here's how to go about it:

1. Make the changes so that newer git can always read and operate
on trees without the encoding information, regardless of what the
configuration says.
2. Modify 1.4.x branch to support this new format too, at least
for reading trees with the information in it. Otherwise some
package maintainers will just ignore such compatibility.
3. Modify 1.5.x branch similarly.
5. Make it configurable, but turned off by default and with a big
fat warning when its turned on.
6. 2 years later, remove the warning.
7. 2 years lter, turn it on by default.
8. 2 years later, remove the config option and make it a new
major release, but maintain the two codepaths forever.

1.[45].x branches are imaginary. They represent the branch that gets created when a new release in that series is necessary for some reason.

I haven't perused Robin's patches enough to know how they would interact with older git, and I'm not really interested in encoding issues. English being the lingua franca of internet and opensource development anyways, every project I've ever seen has only files named in a manner that would fit nicely into 7-bit ascii.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Register now for Nordic Meet on Nagios, June 3-4 in Stockholm
 http://nordicmeetonnagios.op5.org/

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.
Previous: Esko LuontolaNext: Esko Luontola
Message 9 of 20 in “[RFC 0/8] Antique UTF-8 filename support”
  1. Robin RosenbergMay 12, 2009
  2. 1/8 UTF helpersRobin Rosenberg, May 12, 2009
  3. 2/8 Messages in locale.Robin Rosenberg, May 12, 2009
  4. 3/8 Extend tests to cover locale wrt to commit messages.Robin Rosenberg, May 12, 2009
  5. 4/8 UTF file names.Robin Rosenberg, May 12, 2009
  6. Johannes SchindelinMay 13, 2009
  7. Robin RosenbergMay 13, 2009
  8. Esko LuontolaMay 13, 2009
  9. Andreas EricssonMay 13, 2009
  10. Esko LuontolaMay 13, 2009
  11. Alex RiesenMay 13, 2009
  12. Junio C HamanoMay 13, 2009
  13. Esko LuontolaMay 13, 2009
  14. Junio C HamanoMay 13, 2009
  15. Johannes SchindelinMay 13, 2009
  16. Junio C HamanoMay 14, 2009
  17. Jay SoffianMay 14, 2009
  18. 6/8 test of utf_locallinksRobin Rosenberg, May 12, 2009
  19. 7/8 Convert symlink dest in diffRobin Rosenberg, May 12, 2009
  20. 8/8 UTF-8 in non-SHA1-objectsRobin Rosenberg, May 12, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.