Re: Encoding problem on OSX?

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Aug 9, 2010, 23:46 UTC
Message-ID: <20100809234620.GA6418@burratino>
In-Reply-To: <AANLkTiky2uUHfOsh8rNXZPEqTLcNSZowy=Qcm+4Hjn_n@mail.gmail.com>

İsmail Dönmez wrote:

Show 17 quoted lines

> git shortlog HEAD >log &&
> fuzz log >log.predictable &&
> test_cmp expect.template log.predictable
> 
> --- expect.template 2010-08-09 13:45:46.000000000 +0000
> +++ log.predictable 2010-08-09 13:45:46.000000000 +0000
> @@ -1,8 +1,8 @@
>  A U Thor (5):
>        SUBJECT
>        SUBJECT
> -      SUBJECT
> -      SUBJECT
> +      SUBJECT𝄞s 𝄞s a very, very long f𝄞rst l𝄞ne for the comm𝄞t
> message to see 𝄞f 𝄞t 𝄞s wrapped correctly
> +      SUBJECT????s ????s a very, very long f????rst l????ne for the
> comm????t message to see ????f ????t ????s wrapped correctly
>        SUBJECT

Very interesting; thanks for a report.

From the definition of fuzz(), it looks like

	sed "
			s/$_x40/OBJECT_NAME/g
			s/$_x05/OBJID/g
			s/^ \{6\}[CTa].*/      SUBJECT/g
			s/^ \{8\}[^ ].*/        CONTINUATION/g
		" <log >log.fuzzy

failed to completely match the fourth and five lines of the shortlog:

	A U Thor (5):
	      Test
	      This is a very, very long first[etc]
	      Th𝄞s 𝄞s a very, very long f𝄞rst[etc]
	      Th<malformed treble clef>s <malformed treble clef>s a...

Could you confirm this?  What does

	locale
	printf 'Th\360\235\204\236s\n' | sed 's/.*//g'

print?

Jonathan