Re: [PATCH] Try URI quoting for embedded TAB and LF in pathnames

Linus Torvalds <torvalds@osdl.org> writes:

I'd like to protect against:

  1.  File names that cannot be handled correctly with the current
      formats.  Newline is the obvious problem here, along with
      (arguably) tab and space.

  2.  Common transliterations of patches.  Many programs (and mailers,
      alas) expand tabs to spaces, append CR to lines, prepend spaces
      to lines, break lines at spaces, etc.  'patch' already deals
      with this to some extent, but it'd be nice if the format
      resisted these transliterations better.

  3.  Humans misreading patches.  The patch format is intended to be
      human-readable, after all.

  4.  Reencoded patches.  Programs like Emacs can and will convert
      patches from UTF-8 to EUC-JP, for example.

You convinced me that (4) is not worth the hassle, but I'd still like to address (1)-(3) when it's easy.

Yes, but (2) and (3) can lose information about invalid UTF-8 if we don't suitably protect the encoding errors. I daresay that many mailers will mishandle invalid UTF-8, for example.

If I understand you aright, we could do that by modifying my previous proposal to escape all bytes in the UTF-8 representation of a control character. In Unicode, the characters 0080 through 009F are control characters, so that should suffice to quote the terminal escapes you mentioned. (Perhaps we should also escape unassigned Unicode characters too, on the theory that they might become control characters in the future.)

Yes, quite true. But we don't have to come up with something that's perfect in all cases, just something that's good enough to handle cases that we expect will be common in practice, in a world where UTF-8 is the preferred encoding for non-ASCII characters.