Re: [PATCH] Try URI quoting for embedded TAB and LF in pathnames

From: Paul Eggert <eggert@cs.ucla.edu>
Date: Oct 12, 2005, 06:51 UTC
Message-ID: <877jcjmdmq.fsf@penguin.cs.ucla.edu>
In-Reply-To: <Pine.LNX.4.64.0510111346220.14597@g5.osdl.org>

Linus Torvalds <torvalds@osdl.org> writes:

> you can read it as a UTF-8 stream, but then quote things at a byte
> level (ie if you quote one "character", you quote _all_ bytes in
> that character).

Yes, that's what I had in mind.

> And you quote if:
>
>  - the UTF-8 _character_ is in the 0x80-0x9f control range

Yes.  Or more generally, if it's any UTF-8 control character.

>  - any _raw_byte_ is in the 0x80-0x9f range (it might not be UTF-8)

Why quote the raw bytes? Is this for terminal escapes on older xterm (or xterm-like) implementations that don't understand UTF-8? If so, I'm not sure I'd bother, as it would introduce a lot of annoying quoting with perfectly reasonable UTF-8, and (if we assume the world is moving to UTF-8) it addresses a problem that is going away.

>  - any _raw_byte_ is 0xfe-0xff (illegal UTF-8 character)
>  - misformed UTF-8 (non-shortest sequence, or just generally invalid 
>    sequences with missing or wrong high bits)

Yes, that makes sense.

> quite frankly, that's a pretty painful thing to write.

It's not trivially short, yes.  But it shouldn't be that hard.

Also, I guess we don't have to write it, at least not at first. As long as we specify something like the C quoted-string format mentioned earlier, we can encode into that format using a naive algorithm (e.g., quote any non-ASCII byte or ASCII control character), and beautify the encoding method later.

> The upside is that it's easy to decode: you can _unquote_ it just as
> a byte stream.

Yes, that's the idea.

Also, the interchange format is the most important thing. We have to decode anything that is in the format, and we must encode into the format. Encoding prettily is nice, but not necessary.