Re: Rss produced by git is not valid xml?

From: H. Peter Anvin <hpa@zytor.com>
Date: Nov 18, 2005, 20:28 UTC
Message-ID: <437E3971.3060803@zytor.com>
In-Reply-To: <Pine.LNX.4.64.0511181220350.13959@g5.osdl.org>

Linus Torvalds wrote:

Show 10 quoted lines

> 
> It's easy to say "just do the right thing", and ignore reality.
> 
> git commit logs have always been "8-bit data". It's actually gitweb that 
> is buggy if it claims it is UTF-8 without checking or converting it to 
> such.
> 
> I agree that UTF-8 is a good idea, but that's a totally different 
> argument.
>

I think the point is: what do you do with the data? If it *looks* like valid UTF-8, you pretty much have to assume it is; if it's not (it contains invalid UTF-8 sequences), what do you do? There are only a small handful of alternatives, and none are really good:

	- Reject it (it's kind of too late, should have been done at
	  checkin)
	- Show them as SUBSTITUTE characters (U+FFFD).
	- Show them as Latin-1 or Windows-1252
	- Provide a complex configuration mechanism

I think Kay is going with the second option.

Note this problem always exists for the data contents anyway. We can't do anything about that.

What's probably more important is that tools that rely on email or other outside data sources (like CVS) do the necessary conversions, so one doesn't end up with an inadvertently incorrect repository.

	-hpa