git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC] Print diffs of UTF-16 to console / patches to email as UTF-8...?

From
Drew Northup <drew.northup@maine.edu>
Date
Oct 22, 2010, 16:06 UTC
Message-ID
<1287763608.31218.63.camel@drew-northup.unet.maine.edu>

I am currently thinking about what the best way to preset readable (and safely email-able) patches to the user may be when the content is UTF-16. This is part of my ongoing work to treat UTF-16 as text (in other words, the crlf options will work and .gitattributes hacks won't be required to display diffs, etc). I was also concerned that the result be re-importable to valid UTF-16 in the end. This has led me to consider printing diffs as UTF-8 (no data loss, at least 16->8) when the source text is UTF-16. This should also be git-gui / gitk friendly (in theory). I would favorably consider this as a configurable option (export_unicode_diff_as_utf8 ?) leaving plain UTF-16 output as the standard output from "git diff" (once I convince it that UTF-16 is indeed text). Also, there is the issue of being able to recognize UTF-16 as UTF-16 in diffs/patches. Is there a precedent/standard I should be aware of with respect to BOMs and patches? I would think that adhering to the UTF-16 standard with respect to whole text files would make sense here (no BOM == Big Endian, BOM used to match LE/BE otherwise).

Comments welcome!
-- 
-Drew Northup N1XIM
   AKA RvnPhnx on OPN
________________________________________________
"As opposed to vegetable or mineral error?"
-John Pescatore, SANS NewsBites Vol. 12 Num. 59
Next: Jonathan Nieder
Message 1 of 15 in “[RFC] Print diffs of UTF-16 to console / patches to email as UTF-8...?”
  1. Drew NorthupOct 22, 2010
  2. Jonathan NiederOct 22, 2010
  3. Drew NorthupOct 22, 2010
  4. Jonathan NiederOct 22, 2010
  5. Drew NorthupOct 22, 2010
  6. Jonathan NiederOct 22, 2010
  7. Jakub NarebskiOct 22, 2010
  8. Jakub NarebskiOct 22, 2010
  9. Drew NorthupOct 22, 2010
  10. Jakub NarebskiOct 22, 2010
  11. Joshua JuranOct 22, 2010
  12. Jeff KingOct 22, 2010
  13. Jonathan NiederOct 22, 2010
  14. Git Attribute: File Text Encoding {WAS: Re: [RFC] Print diffs of UTF-16 to console / patches to email as UTF-8...?}Drew Northup, Oct 22, 2010
  15. Jakub NarebskiOct 22, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.