Re: [PATCH] t: use user-specific utf-8 locale for testing

From: Đoàn Trần Công Danh <congdanhqx@gmail.com>
Date: Jun 4, 2021, 05:20 UTC
Message-ID: <YLm4EXRqsXF/VenO@danh.dev>
In-Reply-To: <f8cbd947-3176-34bc-a0f0-dd816298e2c7@gmail.com>

On 2021-06-04 10:32:04+0700, Bagas Sanjaya <bagasdotme@gmail.com> wrote:

Show 10 quoted lines

> Hi Đoàn,
> 
> On 02/06/21 18.46, Đoàn Trần Công Danh wrote:
> > In some test-cases, utf-8 locale is required. To find such locale,
> > we're using the first available UTF-8 locale that returned by
> > "locale -a".
> > 
> 
> On most systems, the first available such locale is C.UTF-8, so why don't we
> just use it?

I don't know. I think in Linux with glibc, C.UTF-8 must be enable explicitly and run "localedef"? Worse, systems have a right to not have any UTF-8 locales. Someone may only enable C and en_US.ISO-8859-1

Show 12 quoted lines

> > Despite being required by POSIX, locale(1) is unavailable in some
> > systems, e.g. Linux with musl libc.  Some of those systems support
> > utf-8 locale out of the box.
> > 
> > However, without "locale -a", we can't guess provided UTF-8 locale.
> > 
> > Let's give users of those systems an option to have better test
> > coverage.
> > 
> 
> So can we assume to UTF-8 locale as default for testing on systems without
> locale(1)?

I don't know, either. POSIX only requires 2 locales, C and POSIX, All other locales are optional:

	Conforming implementations shall support one or more coded
	character sets.

I don't know if such systems exist or not.

C and POSIX treats any bytes higher than 0x80 as arbitrary characters [1].

	The POSIX locale shall contain 256 single-byte characters
	including the characters

Show 22 quoted lines

> > Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
> > ---
> >   Makefile                                 |  6 ++++++
> >   t/lib-git-svn.sh                         | 15 +++++++++------
> >   t/t9100-git-svn-basic.sh                 | 14 +++-----------
> >   t/t9115-git-svn-dcommit-funky-renames.sh |  6 +++---
> >   t/t9129-git-svn-i18n-commitencoding.sh   |  4 ++--
> >   5 files changed, 23 insertions(+), 22 deletions(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index c3565fc0f8..4b2c24e5ea 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -398,6 +398,9 @@ all::
> >   # with a different indexfile format version.  If it isn't set the index
> >   # file format used is index-v[23].
> >   #
> > +# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.
> > +# If it isn't set, use the first utf-8 locale returned by "locale -a".
> > +#
> 
> Better say "... to prefer utf-8 locale for testing".

I intended to say:

	... to *the* preferred ...

We need utf-8 locale to test anyway ;) So, we don't prefer utf-8 locale, we pick a utf-8 locale from a handful of utf-8 locales.

1: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02

-- 
Danh