Re: [PATCH] t: use user-specific utf-8 locale for testing

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jun 8, 2021, 10:49 UTC
Message-ID: <874ke62f41.fsf@evledraar.gmail.com>
In-Reply-To: <YLfiYXxQqXL7RyHC@nand.local>

On Wed, Jun 02 2021, Taylor Blau wrote:

Show 7 quoted lines

> On Wed, Jun 02, 2021 at 06:46:46PM +0700, Đoàn Trần Công Danh wrote:
>> Despite being required by POSIX, locale(1) is unavailable in some
>> systems, e.g. Linux with musl libc.  Some of those systems support
>> utf-8 locale out of the box.
>
> Hmmph. I would have imagined that locale was available everywhere, but
> unfortunately not.

Small and unsolicited history lesson from a person with funny characters in their name & language :)

Today it seems like *nix systems have always had UTF-8, but this was a relatively late development.

It's Plan9 that had UTF-8 from the start, on *nix systems it was US-ASCII, and anything else was tacked on top later on.

When I started using *nix systems I belive it was quite common to have default configurations with only ISO-8859-1 locales installed, and certainly that's what a lot of or most users who had the need for locales in European languages not covered by US-ASCII used by default.

This is from hazy memory, but I think it was even actively recommended against having or using UTF-8 locales on the system. If you e.g. connected to an IRC channel, or copy/pasted from your text editor into an E-Mail you could easily send the other end misencodedgibberish.

Later on things like IRC channels in these languages had a "switch day", it was a complete mess. Nowadays mostly nobody really notices or remembers anymore these encoding issues since we've mostly got UTF-8 everywhere as a result.

I mean, at least in the case of European languages, I understand e.g. Japanese and Chinese still have their own persistent encoding issues related to competing standards.

Even today you can't rely on UTF-8 even on Linux systems, and I think this has become even more true of late with minimal CI systems or other chroot-like test environments.