{"thread":{"id":"55829","subject":"[PATCH] t: use user-specific utf-8 locale for testing","startedAt":"2021-06-02T11:47:16Z","lastAt":"2021-06-10T10:23:14Z","messageCount":19,"participants":["Đoàn Trần Công Danh","Taylor Blau","Jeff King","Bagas Sanjaya","Torsten Bögershausen","Junio C Hamano","Ævar Arnfjörð Bjarmason"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"426228","messageId":"20210602114646.17463-1-congdanhqx@gmail.com","threadId":"55829","inReplyTo":null,"subject":"[PATCH] t: use user-specific utf-8 locale for testing","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-02T11:46:46Z","receivedAt":"2021-06-02T11:47:16Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"In some test-cases, utf-8 locale is required. To find such locale,\nwe're using the first available UTF-8 locale that returned by\n\"locale -a\".\n\nDespite being required by POSIX, locale(1) is unavailable in some\nsystems, e.g. Linux with musl libc. Some of those systems support\nutf-8 locale out of the box.\n\nHowever, without \"locale -a\", we can't guess provided UTF-8 locale.\n\nLet's give users of those systems an option to have better test\ncoverage.\n\nSigned-off-by: Đoàn Trần Công Danh \n---\n Makefile | 6 ++++++\n t/lib-git-svn.sh | 15 +++++++++------\n t/t9100-git-svn-basic.sh | 14 +++-----------\n t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n 5 files changed, 23 insertions(+), 22 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex c3565fc0f8..4b2c24e5ea 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -398,6 +398,9 @@ all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n+# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n+# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n+#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n@@ -2801,6 +2804,9 @@ ifdef GIT_TEST_CMP\n endif\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n+endif\n+ifdef GIT_TEST_UTF8_LOCALE\n+\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n endif\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 547eb3c31a..df319593f7 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -121,12 +121,15 @@ start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n }\n \n-prepare_a_utf8_locale () {\n-\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n-\tp\n-\tq\n-}')\n-\tif test -n \"$a_utf8_locale\"\n+prepare_utf8_locale () {\n+\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n+\tthen\n+\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n+\t\tp\n+\t\tq\n+\t}')\n+\tfi\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n \t\ttest_set_prereq UTF8\n \telse\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 1d3fdcc997..d5563ec35f 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -4,21 +4,13 @@\n #\n \n test_description='git svn basic tests'\n-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n \n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./lib-git-svn.sh\n \n-case \"$GIT_SVN_LC_ALL\" in\n-*.UTF-8)\n-\ttest_set_prereq UTF8\n-\t;;\n-*)\n-\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n-\t;;\n-esac\n+prepare_utf8_locale\n \n test_expect_success 'git svn --version works anywhere' '\n \tnongit git svn --version\n@@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n \n-name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n-LC_ALL=\"$GIT_SVN_LC_ALL\"\n+name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n+LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n export LC_ALL\n # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 9b44a44bc1..743fbe1fe4 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n # > ... All of the above characters, except for the backslash, are converted\n # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n # > \"Private use area\") when creating or accessing files.\n-prepare_a_utf8_locale\n+prepare_utf8_locale\n test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tneq=$(printf \"\\201\\202\") &&\n \tgit config svn.pathnameencoding cp932 &&\n@@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n \n # See the comment on the above test for setting of LC_ALL.\n test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tinf=$(printf \"\\201\\207\") &&\n \tgit config svn.pathnameencoding cp932 &&\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 2c213ae654..01e1e8a8f7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -14,12 +14,12 @@ compare_git_head_with () {\n \ttest_cmp current \"$1\"\n }\n \n-prepare_a_utf8_locale\n+prepare_utf8_locale\n \n compare_svn_head_with () {\n \t# extract just the log message and strip out committer info.\n \t# don't use --limit here since svn 1.1.x doesn't have it,\n-\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n+\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n \t\tuse bytes;\n \t\t$/ = (\"-\"x72) . \"\\n\";\n \t\tmy @x = ;\n-- \n2.32.0.rc2.1.gd85b21334f\n\n"},{"id":"426253","messageId":"YLfiYXxQqXL7RyHC@nand.local","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"Re: [PATCH] t: use user-specific utf-8 locale for testing","fromName":"Taylor Blau","fromEmail":"ttaylorr@github.com","sentAt":"2021-06-02T19:56:38Z","receivedAt":"2021-06-02T19:57:54Z","isPatch":true,"sender":{"key":"ttaylorr@github.com","avatar":"https://gravatar.com/avatar/d5f3476f26b6f99cbb6b467e7ed7482f5762c8157bc73f569196e428bdcbea25?d=mp&s=160"},"body":"On Wed, Jun 02, 2021 at 06:46:46PM +0700, Đoàn Trần Công Danh wrote:\n> Despite being required by POSIX, locale(1) is unavailable in some\n> systems, e.g. Linux with musl libc. Some of those systems support\n> utf-8 locale out of the box.\n\nHmmph. I would have imagined that locale was available everywhere, but\nunfortunately not.\n\n> diff --git a/Makefile b/Makefile\n> index c3565fc0f8..4b2c24e5ea 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -398,6 +398,9 @@ all::\n> # with a different indexfile format version. If it isn't set the index\n> # file format used is index-v[23].\n> #\n> +# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n> +# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n\ns/prefered/preferred\n\n> +#\n> # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n> #\n> # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n> @@ -2801,6 +2804,9 @@ ifdef GIT_TEST_CMP\n> endif\n> ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n> \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n> +endif\n> +ifdef GIT_TEST_UTF8_LOCALE\n> +\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n> endif\n> \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n> ifdef GIT_PERF_REPEAT_COUNT\n> diff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\n> index 547eb3c31a..df319593f7 100644\n> --- a/t/lib-git-svn.sh\n> +++ b/t/lib-git-svn.sh\n> @@ -121,12 +121,15 @@ start_svnserve () {\n> \t\t --listen-host 127.0.0.1 &\n> }\n>\n> -prepare_a_utf8_locale () {\n> -\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n> -\tp\n> -\tq\n> -}')\n> -\tif test -n \"$a_utf8_locale\"\n> +prepare_utf8_locale () {\n> +\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n> +\tthen\n> +\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n> +\t\tp\n> +\t\tq\n> +\t}')\n> +\tfi\n\nOK, so we bind GIT_TEST_UTF8_LOCALE to the value of $a_utf8_locale in\nthe pre-image, unless the user said otherwise.\n\n> +\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n\n...Then we go on to handle things like before, except we read from\n\"$GIT_TEST_UTF8_LOCALE\" instead of \"$a_utf8_locale\". Makes sense to me.\n\n> \tthen\n> \t\ttest_set_prereq UTF8\n> \telse\n> diff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\n> index 1d3fdcc997..d5563ec35f 100755\n> --- a/t/t9100-git-svn-basic.sh\n> +++ b/t/t9100-git-svn-basic.sh\n> @@ -4,21 +4,13 @@\n> #\n>\n> test_description='git svn basic tests'\n> -GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n>\n> GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n>\n> . ./lib-git-svn.sh\n>\n> -case \"$GIT_SVN_LC_ALL\" in\n> -*.UTF-8)\n> -\ttest_set_prereq UTF8\n> -\t;;\n> -*)\n> -\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n> -\t;;\n> -esac\n> +prepare_utf8_locale\n\nThis change (and the omitted ones below in later hunks) look like it\nisn't changing any behavior (and just running the same code behind the\nprepare_utf8_locale function instead of inlining it).\n\nThey all look right to me, but it may be helpful to either point it out\nin the commit message and/or prepare the separately. I'd probably err on\nthe side of the former.\n\nThat said, this patch looks good to me with minor touch-ups (my only\nnits are the above and the spelling mistake in the Makefile).\n\nThanks,\nTaylor\n"},{"id":"426348","messageId":"YLktKeMXoZ/6wHqT@coredump.intra.peff.net","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"Re: [PATCH] t: use user-specific utf-8 locale for testing","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-06-03T19:27:37Z","receivedAt":"2021-06-03T19:27:41Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jun 02, 2021 at 06:46:46PM +0700, Đoàn Trần Công Danh wrote:\n\n> In some test-cases, utf-8 locale is required. To find such locale,\n> we're using the first available UTF-8 locale that returned by\n> \"locale -a\".\n> \n> Despite being required by POSIX, locale(1) is unavailable in some\n> systems, e.g. Linux with musl libc. Some of those systems support\n> utf-8 locale out of the box.\n> \n> However, without \"locale -a\", we can't guess provided UTF-8 locale.\n> \n> Let's give users of those systems an option to have better test\n> coverage.\n\nThat makes sense. At first I thought this was fixing a test breakage,\nand wondered if we should simply be skipping the tests. But after\nreading more carefully, we already are doing that, and this is just\nallowing those tests to run in more places (you even said so in the last\nsentence above, but I missed it, so I am calling it out).\n\nI kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\nthat people don't even have to set this extra Makefile knob. But I'm not\nsure if we have a good way of testing if that locale works (if we can't\nfind the \"locale\" binary). So this is probably a reasonable medium\nground.\n\n> ---\n> Makefile | 6 ++++++\n> t/lib-git-svn.sh | 15 +++++++++------\n> t/t9100-git-svn-basic.sh | 14 +++-----------\n> t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n> t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n> 5 files changed, 23 insertions(+), 22 deletions(-)\n\nI think we test musl in our ci builds; should we do something like:\n\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex d848c036c5..ec67398c18 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -229,6 +229,7 @@ linux-musl)\n \tCC=gcc\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n+\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=something.UTF-8?\"\n \t;;\n esac\n \n\nhere, too (I don't know what the correct \"something\" is here; maybe\n\"C\")?\n\n-Peff\n"},{"id":"426383","messageId":"f8cbd947-3176-34bc-a0f0-dd816298e2c7@gmail.com","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"Re: [PATCH] t: use user-specific utf-8 locale for testing","fromName":"Bagas Sanjaya","fromEmail":"bagasdotme@gmail.com","sentAt":"2021-06-04T03:32:04Z","receivedAt":"2021-06-04T03:32:09Z","isPatch":true,"sender":{"key":"bagasdotme@gmail.com","avatar":"https://avatars.githubusercontent.com/u/40219486?v=4"},"body":"Hi Đoàn,\n\nOn 02/06/21 18.46, Đoàn Trần Công Danh wrote:\n> In some test-cases, utf-8 locale is required. To find such locale,\n> we're using the first available UTF-8 locale that returned by\n> \"locale -a\".\n> \n\nOn most systems, the first available such locale is C.UTF-8, so why \ndon't we just use it?\n\n> Despite being required by POSIX, locale(1) is unavailable in some\n> systems, e.g. Linux with musl libc. Some of those systems support\n> utf-8 locale out of the box.\n> \n> However, without \"locale -a\", we can't guess provided UTF-8 locale.\n> \n> Let's give users of those systems an option to have better test\n> coverage.\n> \n\nSo can we assume to UTF-8 locale as default for testing on systems \nwithout locale(1)?\n\n> Signed-off-by: Đoàn Trần Công Danh \n> ---\n> Makefile | 6 ++++++\n> t/lib-git-svn.sh | 15 +++++++++------\n> t/t9100-git-svn-basic.sh | 14 +++-----------\n> t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n> t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n> 5 files changed, 23 insertions(+), 22 deletions(-)\n> \n> diff --git a/Makefile b/Makefile\n> index c3565fc0f8..4b2c24e5ea 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -398,6 +398,9 @@ all::\n> # with a different indexfile format version. If it isn't set the index\n> # file format used is index-v[23].\n> #\n> +# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n> +# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n> +#\n\nBetter say \"... to prefer utf-8 locale for testing\".\n\n-- \nAn old man doll... just what I always wanted! - Clara\n"},{"id":"426396","messageId":"YLm4EXRqsXF/VenO@danh.dev","threadId":"55829","inReplyTo":"f8cbd947-3176-34bc-a0f0-dd816298e2c7@gmail.com","subject":"Re: [PATCH] t: use user-specific utf-8 locale for testing","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-04T05:20:17Z","receivedAt":"2021-06-04T05:20:33Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2021-06-04 10:32:04+0700, Bagas Sanjaya wrote:\n> Hi Đoàn,\n> \n> On 02/06/21 18.46, Đoàn Trần Công Danh wrote:\n> > In some test-cases, utf-8 locale is required. To find such locale,\n> > we're using the first available UTF-8 locale that returned by\n> > \"locale -a\".\n> > \n> \n> On most systems, the first available such locale is C.UTF-8, so why don't we\n> just use it?\n\nI don't know. I think in Linux with glibc, C.UTF-8 must be enable\nexplicitly and run \"localedef\"?\nWorse, systems have a right to not have any UTF-8 locales.\nSomeone may only enable C and en_US.ISO-8859-1\n\n> > Despite being required by POSIX, locale(1) is unavailable in some\n> > systems, e.g. Linux with musl libc. Some of those systems support\n> > utf-8 locale out of the box.\n> > \n> > However, without \"locale -a\", we can't guess provided UTF-8 locale.\n> > \n> > Let's give users of those systems an option to have better test\n> > coverage.\n> > \n> \n> So can we assume to UTF-8 locale as default for testing on systems without\n> locale(1)?\n\nI don't know, either. POSIX only requires 2 locales, C and POSIX,\nAll other locales are optional:\n\n\tConforming implementations shall support one or more coded\n\tcharacter sets. \n\nI don't know if such systems exist or not.\n\nC and POSIX treats any bytes higher than 0x80 as arbitrary characters [1].\n\n\tThe POSIX locale shall contain 256 single-byte characters\n\tincluding the characters\n\n> > Signed-off-by: Đoàn Trần Công Danh \n> > ---\n> > Makefile | 6 ++++++\n> > t/lib-git-svn.sh | 15 +++++++++------\n> > t/t9100-git-svn-basic.sh | 14 +++-----------\n> > t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n> > t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n> > 5 files changed, 23 insertions(+), 22 deletions(-)\n> > \n> > diff --git a/Makefile b/Makefile\n> > index c3565fc0f8..4b2c24e5ea 100644\n> > --- a/Makefile\n> > +++ b/Makefile\n> > @@ -398,6 +398,9 @@ all::\n> > # with a different indexfile format version. If it isn't set the index\n> > # file format used is index-v[23].\n> > #\n> > +# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n> > +# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n> > +#\n> \n> Better say \"... to prefer utf-8 locale for testing\".\n\nI intended to say:\n\n\t... to *the* preferred ...\n\nWe need utf-8 locale to test anyway ;) So, we don't prefer utf-8 locale,\nwe pick a utf-8 locale from a handful of utf-8 locales.\n\n\n1: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap06.html#tag_06_02\n\n-- \nDanh\n"},{"id":"426537","messageId":"20210606163316.8630-1-congdanhqx@gmail.com","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"[PATCH v2] t: use user-specific utf-8 locale for testing","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-06T16:33:16Z","receivedAt":"2021-06-06T16:34:39Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"In some test-cases, utf-8 locale is required. To find such locale,\nwe're using the first available UTF-8 locale that returned by\n\"locale -a\".\n\nDespite being required by POSIX, locale(1) is unavailable in some\nsystems, e.g. Linux with musl libc. Some of those systems support\nutf-8 locale out of the box.\n\nHowever, without \"locale -a\", we can't guess provided UTF-8 locale.\n\nLet's give users of those systems an option to have better test\ncoverage.\n\nThis change also rename t/lib-git-svn.sh:prepare_a_utf8_locale to\nprepare_utf8_locale, since we no longer prepare the variable named\n\"a_utf8_locale\" but set up a fallback value for GIT_TEST_UTF8_LOCALE\ninstead. The fallback will be LC_ALL, LANG environment variable,\nor the first utf-8 locale from output of \"locale -a\", in that order.\n\nSigned-off-by: Đoàn Trần Công Danh \n---\nRange-diff against v1:\n1: d242ce64c4 ! 1: f299ae2239 t: use user-specific utf-8 locale for testing\n @@ Commit message\n Let's give users of those systems an option to have better test\n coverage.\n \n + This change also rename t/lib-git-svn.sh:prepare_a_utf8_locale to\n + prepare_utf8_locale, since we no longer prepare the variable named\n + \"a_utf8_locale\" but set up a fallback value for GIT_TEST_UTF8_LOCALE\n + instead. The fallback will be LC_ALL, LANG environment variable,\n + or the first utf-8 locale from output of \"locale -a\", in that order.\n +\n Signed-off-by: Đoàn Trần Công Danh \n \n ## Makefile ##\n @@ Makefile: all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n -+# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n -+# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n ++# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n ++# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n ++# locale returned by \"locale -a\".\n +#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n @@ Makefile: ifdef GIT_TEST_CMP\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\n \n + ## ci/lib.sh ##\n +@@ ci/lib.sh: linux-musl)\n + \tCC=gcc\n + \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n + \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n ++\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n + \t;;\n + esac\n + \n +\n ## t/lib-git-svn.sh ##\n @@ t/lib-git-svn.sh: start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n @@ t/lib-git-svn.sh: start_svnserve () {\n -}')\n -\tif test -n \"$a_utf8_locale\"\n +prepare_utf8_locale () {\n -+\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n ++\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n ++\tthen\n ++\t\t: test_set_prereq UTF8\n ++\telif test -n \"${LC_ALL:-$LANG}\"\n +\tthen\n ++\t\tcase \"${LC_ALL:-$LANG}\" in\n ++\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n ++\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n ++\t\t\t;;\n ++\t\tesac\n ++\telse\n +\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n +\t\tp\n +\t\tq\n\n Makefile | 7 +++++++\n ci/lib.sh | 1 +\n t/lib-git-svn.sh | 24 ++++++++++++++++++------\n t/t9100-git-svn-basic.sh | 14 +++-----------\n t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n 6 files changed, 34 insertions(+), 22 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex c3565fc0f8..502e0c9a81 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -398,6 +398,10 @@ all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n+# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n+# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n+# locale returned by \"locale -a\".\n+#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n@@ -2801,6 +2805,9 @@ ifdef GIT_TEST_CMP\n endif\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n+endif\n+ifdef GIT_TEST_UTF8_LOCALE\n+\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n endif\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex d848c036c5..476c3f369f 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -229,6 +229,7 @@ linux-musl)\n \tCC=gcc\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n+\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n \t;;\n esac\n \ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 547eb3c31a..83efc17661 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -121,12 +121,24 @@ start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n }\n \n-prepare_a_utf8_locale () {\n-\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n-\tp\n-\tq\n-}')\n-\tif test -n \"$a_utf8_locale\"\n+prepare_utf8_locale () {\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n+\tthen\n+\t\t: test_set_prereq UTF8\n+\telif test -n \"${LC_ALL:-$LANG}\"\n+\tthen\n+\t\tcase \"${LC_ALL:-$LANG}\" in\n+\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n+\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n+\t\t\t;;\n+\t\tesac\n+\telse\n+\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n+\t\tp\n+\t\tq\n+\t}')\n+\tfi\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n \t\ttest_set_prereq UTF8\n \telse\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 1d3fdcc997..d5563ec35f 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -4,21 +4,13 @@\n #\n \n test_description='git svn basic tests'\n-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n \n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./lib-git-svn.sh\n \n-case \"$GIT_SVN_LC_ALL\" in\n-*.UTF-8)\n-\ttest_set_prereq UTF8\n-\t;;\n-*)\n-\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n-\t;;\n-esac\n+prepare_utf8_locale\n \n test_expect_success 'git svn --version works anywhere' '\n \tnongit git svn --version\n@@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n \n-name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n-LC_ALL=\"$GIT_SVN_LC_ALL\"\n+name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n+LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n export LC_ALL\n # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 9b44a44bc1..743fbe1fe4 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n # > ... All of the above characters, except for the backslash, are converted\n # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n # > \"Private use area\") when creating or accessing files.\n-prepare_a_utf8_locale\n+prepare_utf8_locale\n test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tneq=$(printf \"\\201\\202\") &&\n \tgit config svn.pathnameencoding cp932 &&\n@@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n \n # See the comment on the above test for setting of LC_ALL.\n test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tinf=$(printf \"\\201\\207\") &&\n \tgit config svn.pathnameencoding cp932 &&\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 2c213ae654..01e1e8a8f7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -14,12 +14,12 @@ compare_git_head_with () {\n \ttest_cmp current \"$1\"\n }\n \n-prepare_a_utf8_locale\n+prepare_utf8_locale\n \n compare_svn_head_with () {\n \t# extract just the log message and strip out committer info.\n \t# don't use --limit here since svn 1.1.x doesn't have it,\n-\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n+\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n \t\tuse bytes;\n \t\t$/ = (\"-\"x72) . \"\\n\";\n \t\tmy @x = ;\n-- \n2.32.0.rc3.5.gf3d78db977\n\n"},{"id":"426547","messageId":"20210606200613.muanikoqgpjsgk66@tb-raspi4","threadId":"55829","inReplyTo":"20210606163316.8630-1-congdanhqx@gmail.com","subject":"Re: [PATCH v2] t: use user-specific utf-8 locale for testing","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2021-06-06T20:06:13Z","receivedAt":"2021-06-06T20:06:28Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"This all looks good.\nSome suggestions about the commit message are inline.\n\nOn Sun, Jun 06, 2021 at 11:33:16PM +0700, Đoàn Trần Công Danh wrote:\n> In some test-cases, utf-8 locale is required. To find such locale,\n> we're using the first available UTF-8 locale that returned by\n> \"locale -a\".\n\nGood explanation.\nI think that in generaral \"utf-8\" as a specification/specifier is better\nwritten as \"UTF-8\", with uppercase.\n\"utf-8\" or utf8 may be used inside the code, depending on the language.\n\n>\n> Despite being required by POSIX, locale(1) is unavailable in some\n> systems, e.g. Linux with musl libc. Some of those systems support\n> utf-8 locale out of the box.\nThis reads a little bit harsh (the first sentence) and it is not\nfully clear which systems do what (the second sentence).\nOr are 2 things mentioned - the locale(1) utility and the support\nof one UTF-8 locale \"out of the box\" ?\nDoes Linux with musl libs support an UTF-8 locale, but not\nthe locale(1) untility ?\nGit itself supports many systems, that are not POSIX compliant,\nstrictly speaking. But if avaliable, the functions defined in POSIX\nare used, whenever available.\n\nCould we write:\nHowever, the locale(1) utility is unavailable on some systems,\ne.g. Linux with musl libc.\n\n>\n> However, without \"locale -a\", we can't guess provided UTF-8 locale.\n>\n> Let's give users of those systems an option to have better test\n> coverage.\n\nAdd a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for linux-musl\n\n>\n> This change also rename t/lib-git-svn.sh:prepare_a_utf8_locale to\n> prepare_utf8_locale, since we no longer prepare the variable named\n> \"a_utf8_locale\" but set up a fallback value for GIT_TEST_UTF8_LOCALE\n> instead. The fallback will be LC_ALL, LANG environment variable,\n> or the first utf-8 locale from output of \"locale -a\", in that order.\n\nrename -> renames, may be drop \"This change\", like this ?\nRename t/lib-git-svn.sh:prepare_a_utf8_locale into prepare_utf8_locale,\nsince we no longer prepare the variable named \"a_utf8_locale\",\nbut set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\nThe fallback will be LC_ALL, LANG environment variable,\nor the first UTF-8 locale from output of \"locale -a\", in that order.\n\n\n>\n> Signed-off-by: Đoàn Trần Công Danh \n> ---\n> Range-diff against v1:\n> 1: d242ce64c4 ! 1: f299ae2239 t: use user-specific utf-8 locale for testing\n> @@ Commit message\n> Let's give users of those systems an option to have better test\n> coverage.\n>\n> + This change also rename t/lib-git-svn.sh:prepare_a_utf8_locale to\n> + prepare_utf8_locale, since we no longer prepare the variable named\n> + \"a_utf8_locale\" but set up a fallback value for GIT_TEST_UTF8_LOCALE\n> + instead. The fallback will be LC_ALL, LANG environment variable,\n> + or the first utf-8 locale from output of \"locale -a\", in that order.\n> +\n> Signed-off-by: Đoàn Trần Công Danh \n>\n> ## Makefile ##\n> @@ Makefile: all::\n> # with a different indexfile format version. If it isn't set the index\n> # file format used is index-v[23].\n> #\n> -+# Define GIT_TEST_UTF8_LOCALE to prefered utf-8 locale for testing.\n> -+# If it isn't set, use the first utf-8 locale returned by \"locale -a\".\n> ++# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n> ++# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n> ++# locale returned by \"locale -a\".\n> +#\n> # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n> #\n> @@ Makefile: ifdef GIT_TEST_CMP\n> \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n> ifdef GIT_PERF_REPEAT_COUNT\n>\n> + ## ci/lib.sh ##\n> +@@ ci/lib.sh: linux-musl)\n> + \tCC=gcc\n> + \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n> + \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n> ++\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n> + \t;;\n> + esac\n> +\n> +\n> ## t/lib-git-svn.sh ##\n> @@ t/lib-git-svn.sh: start_svnserve () {\n> \t\t --listen-host 127.0.0.1 &\n> @@ t/lib-git-svn.sh: start_svnserve () {\n> -}')\n> -\tif test -n \"$a_utf8_locale\"\n> +prepare_utf8_locale () {\n> -+\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n> ++\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n> ++\tthen\n> ++\t\t: test_set_prereq UTF8\n> ++\telif test -n \"${LC_ALL:-$LANG}\"\n> +\tthen\n> ++\t\tcase \"${LC_ALL:-$LANG}\" in\n> ++\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n> ++\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n> ++\t\t\t;;\n> ++\t\tesac\n> ++\telse\n> +\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n> +\t\tp\n> +\t\tq\n>\n> Makefile | 7 +++++++\n> ci/lib.sh | 1 +\n> t/lib-git-svn.sh | 24 ++++++++++++++++++------\n> t/t9100-git-svn-basic.sh | 14 +++-----------\n> t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n> t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n> 6 files changed, 34 insertions(+), 22 deletions(-)\n>\n> diff --git a/Makefile b/Makefile\n> index c3565fc0f8..502e0c9a81 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -398,6 +398,10 @@ all::\n> # with a different indexfile format version. If it isn't set the index\n> # file format used is index-v[23].\n> #\n> +# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n> +# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n> +# locale returned by \"locale -a\".\n> +#\n> # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n> #\n> # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n> @@ -2801,6 +2805,9 @@ ifdef GIT_TEST_CMP\n> endif\n> ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n> \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n> +endif\n> +ifdef GIT_TEST_UTF8_LOCALE\n> +\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n> endif\n> \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n> ifdef GIT_PERF_REPEAT_COUNT\n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index d848c036c5..476c3f369f 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -229,6 +229,7 @@ linux-musl)\n> \tCC=gcc\n> \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n> \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n> +\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n> \t;;\n> esac\n>\n> diff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\n> index 547eb3c31a..83efc17661 100644\n> --- a/t/lib-git-svn.sh\n> +++ b/t/lib-git-svn.sh\n> @@ -121,12 +121,24 @@ start_svnserve () {\n> \t\t --listen-host 127.0.0.1 &\n> }\n>\n> -prepare_a_utf8_locale () {\n> -\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n> -\tp\n> -\tq\n> -}')\n> -\tif test -n \"$a_utf8_locale\"\n> +prepare_utf8_locale () {\n> +\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n> +\tthen\n> +\t\t: test_set_prereq UTF8\n> +\telif test -n \"${LC_ALL:-$LANG}\"\n> +\tthen\n> +\t\tcase \"${LC_ALL:-$LANG}\" in\n> +\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n> +\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n> +\t\t\t;;\n> +\t\tesac\n> +\telse\n> +\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n> +\t\tp\n> +\t\tq\n> +\t}')\n> +\tfi\n> +\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n> \tthen\n> \t\ttest_set_prereq UTF8\n> \telse\n> diff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\n> index 1d3fdcc997..d5563ec35f 100755\n> --- a/t/t9100-git-svn-basic.sh\n> +++ b/t/t9100-git-svn-basic.sh\n> @@ -4,21 +4,13 @@\n> #\n>\n> test_description='git svn basic tests'\n> -GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n>\n> GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n> export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n>\n> . ./lib-git-svn.sh\n>\n> -case \"$GIT_SVN_LC_ALL\" in\n> -*.UTF-8)\n> -\ttest_set_prereq UTF8\n> -\t;;\n> -*)\n> -\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n> -\t;;\n> -esac\n> +prepare_utf8_locale\n>\n> test_expect_success 'git svn --version works anywhere' '\n> \tnongit git svn --version\n> @@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n> \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n> \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n>\n> -name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n> -LC_ALL=\"$GIT_SVN_LC_ALL\"\n> +name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n> +LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n> export LC_ALL\n> # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n> test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\n> diff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\n> index 9b44a44bc1..743fbe1fe4 100755\n> --- a/t/t9115-git-svn-dcommit-funky-renames.sh\n> +++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n> @@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n> # > ... All of the above characters, except for the backslash, are converted\n> # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n> # > \"Private use area\") when creating or accessing files.\n> -prepare_a_utf8_locale\n> +prepare_utf8_locale\n> test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n> -\tLC_ALL=$a_utf8_locale &&\n> +\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n> \texport LC_ALL &&\n> \tneq=$(printf \"\\201\\202\") &&\n> \tgit config svn.pathnameencoding cp932 &&\n> @@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n>\n> # See the comment on the above test for setting of LC_ALL.\n> test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n> -\tLC_ALL=$a_utf8_locale &&\n> +\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n> \texport LC_ALL &&\n> \tinf=$(printf \"\\201\\207\") &&\n> \tgit config svn.pathnameencoding cp932 &&\n> diff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\n> index 2c213ae654..01e1e8a8f7 100755\n> --- a/t/t9129-git-svn-i18n-commitencoding.sh\n> +++ b/t/t9129-git-svn-i18n-commitencoding.sh\n> @@ -14,12 +14,12 @@ compare_git_head_with () {\n> \ttest_cmp current \"$1\"\n> }\n>\n> -prepare_a_utf8_locale\n> +prepare_utf8_locale\n>\n> compare_svn_head_with () {\n> \t# extract just the log message and strip out committer info.\n> \t# don't use --limit here since svn 1.1.x doesn't have it,\n> -\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n> +\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n> \t\tuse bytes;\n> \t\t$/ = (\"-\"x72) . \"\\n\";\n> \t\tmy @x = ;\n> --\n> 2.32.0.rc3.5.gf3d78db977\n>\n"},{"id":"426548","messageId":"xmqqtumawmo5.fsf@gitster.g","threadId":"55829","inReplyTo":"20210606200613.muanikoqgpjsgk66@tb-raspi4","subject":"Re: [PATCH v2] t: use user-specific utf-8 locale for testing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-07T00:20:10Z","receivedAt":"2021-06-07T00:20:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen writes:\n\n> This all looks good.\n> Some suggestions about the commit message are inline.\n\nAll good comments.\n\nI have another one on the title.\n\n> Subject: Re: [PATCH v2] t: use user-specific utf-8 locale for testing\n\nIf we auto-detect to customize which locale to use per user, that is\nalready user-specific; letting the user to explicitly tell us which\nlocale to use, that is \"use user-specified UTF-8 locale\".\n\nThis affects only SVN tests and the change is in lib-git-svn, so\nwhen we test other parts of the system for UTF-8, this change will\nnot directly help, right? Perhaps we want to mention SVN somewhere\non the title? Perhaps \"for testing git-svn\"?\n\nThanks.\n\n\n"},{"id":"426550","messageId":"20210607004838.15235-1-congdanhqx@gmail.com","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"[PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-07T00:48:38Z","receivedAt":"2021-06-07T00:49:23Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"In some test-cases, UTF-8 locale is required. To find such locale,\nwe're using the first available UTF-8 locale that returned by\n\"locale -a\".\n\nHowever, the locale(1) utility is unavailable on some systems,\ne.g. Linux with musl libc.\n\nHowever, without \"locale -a\", we can't guess provided UTF-8 locale.\n\nAdd a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for\nlinux-musl in our CI system.\n\nRename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale,\nsince we no longer prepare the variable named \"a_utf8_locale\",\nbut set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\nThe fallback will be LC_ALL, LANG environment variable,\nor the first UTF-8 locale from output of \"locale -a\", in that order.\n\nSigned-off-by: Đoàn Trần Công Danh \n---\n\nJeff King wrote:\n\n> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n> that people don't even have to set this extra Makefile knob. But I'm not\n> sure if we have a good way of testing if that locale works (if we can't\n> find the \"locale\" binary).\n\nI also think we should fallback to \"C.UTF-8\" instead of not testing those\ntests. However, I don't know if there're any systems that not have \"C.UTF-8\"\nlocale.\n\n\nRange-diff against v2:\n1: f299ae2239 ! 1: 064cfc0fe3 t: use user-specific utf-8 locale for testing\n @@ Metadata\n Author: Đoàn Trần Công Danh \n \n ## Commit message ##\n - t: use user-specific utf-8 locale for testing\n + t: use pre-defined utf-8 locale for testing svn\n \n - In some test-cases, utf-8 locale is required. To find such locale,\n + In some test-cases, UTF-8 locale is required. To find such locale,\n we're using the first available UTF-8 locale that returned by\n \"locale -a\".\n \n - Despite being required by POSIX, locale(1) is unavailable in some\n - systems, e.g. Linux with musl libc. Some of those systems support\n - utf-8 locale out of the box.\n + However, the locale(1) utility is unavailable on some systems,\n + e.g. Linux with musl libc.\n \n However, without \"locale -a\", we can't guess provided UTF-8 locale.\n \n - Let's give users of those systems an option to have better test\n - coverage.\n + Add a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for\n + linux-musl in our CI system.\n \n - This change also rename t/lib-git-svn.sh:prepare_a_utf8_locale to\n - prepare_utf8_locale, since we no longer prepare the variable named\n - \"a_utf8_locale\" but set up a fallback value for GIT_TEST_UTF8_LOCALE\n - instead. The fallback will be LC_ALL, LANG environment variable,\n - or the first utf-8 locale from output of \"locale -a\", in that order.\n + Rename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale,\n + since we no longer prepare the variable named \"a_utf8_locale\",\n + but set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\n + The fallback will be LC_ALL, LANG environment variable,\n + or the first UTF-8 locale from output of \"locale -a\", in that order.\n \n Signed-off-by: Đoàn Trần Công Danh \n \n\n Makefile | 7 +++++++\n ci/lib.sh | 1 +\n t/lib-git-svn.sh | 24 ++++++++++++++++++------\n t/t9100-git-svn-basic.sh | 14 +++-----------\n t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n 6 files changed, 34 insertions(+), 22 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex c3565fc0f8..502e0c9a81 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -398,6 +398,10 @@ all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n+# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n+# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n+# locale returned by \"locale -a\".\n+#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n@@ -2801,6 +2805,9 @@ ifdef GIT_TEST_CMP\n endif\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n+endif\n+ifdef GIT_TEST_UTF8_LOCALE\n+\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n endif\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex d848c036c5..476c3f369f 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -229,6 +229,7 @@ linux-musl)\n \tCC=gcc\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n+\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n \t;;\n esac\n \ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 547eb3c31a..83efc17661 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -121,12 +121,24 @@ start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n }\n \n-prepare_a_utf8_locale () {\n-\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n-\tp\n-\tq\n-}')\n-\tif test -n \"$a_utf8_locale\"\n+prepare_utf8_locale () {\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n+\tthen\n+\t\t: test_set_prereq UTF8\n+\telif test -n \"${LC_ALL:-$LANG}\"\n+\tthen\n+\t\tcase \"${LC_ALL:-$LANG}\" in\n+\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n+\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n+\t\t\t;;\n+\t\tesac\n+\telse\n+\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n+\t\tp\n+\t\tq\n+\t}')\n+\tfi\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n \t\ttest_set_prereq UTF8\n \telse\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 1d3fdcc997..d5563ec35f 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -4,21 +4,13 @@\n #\n \n test_description='git svn basic tests'\n-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n \n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./lib-git-svn.sh\n \n-case \"$GIT_SVN_LC_ALL\" in\n-*.UTF-8)\n-\ttest_set_prereq UTF8\n-\t;;\n-*)\n-\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n-\t;;\n-esac\n+prepare_utf8_locale\n \n test_expect_success 'git svn --version works anywhere' '\n \tnongit git svn --version\n@@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n \n-name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n-LC_ALL=\"$GIT_SVN_LC_ALL\"\n+name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n+LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n export LC_ALL\n # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 9b44a44bc1..743fbe1fe4 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n # > ... All of the above characters, except for the backslash, are converted\n # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n # > \"Private use area\") when creating or accessing files.\n-prepare_a_utf8_locale\n+prepare_utf8_locale\n test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tneq=$(printf \"\\201\\202\") &&\n \tgit config svn.pathnameencoding cp932 &&\n@@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n \n # See the comment on the above test for setting of LC_ALL.\n test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tinf=$(printf \"\\201\\207\") &&\n \tgit config svn.pathnameencoding cp932 &&\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 2c213ae654..01e1e8a8f7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -14,12 +14,12 @@ compare_git_head_with () {\n \ttest_cmp current \"$1\"\n }\n \n-prepare_a_utf8_locale\n+prepare_utf8_locale\n \n compare_svn_head_with () {\n \t# extract just the log message and strip out committer info.\n \t# don't use --limit here since svn 1.1.x doesn't have it,\n-\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n+\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n \t\tuse bytes;\n \t\t$/ = (\"-\"x72) . \"\\n\";\n \t\tmy @x = ;\n-- \n2.32.0.rc3.5.gf3d78db977\n\n"},{"id":"426553","messageId":"xmqqk0n6wkrr.fsf@gitster.g","threadId":"55829","inReplyTo":"20210607004838.15235-1-congdanhqx@gmail.com","subject":"Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-06-07T01:01:12Z","receivedAt":"2021-06-07T01:01:15Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Đoàn Trần Công Danh writes:\n\n> Subject: Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn\n\nThat certainly is better than \"user-specific\", but a lot worse than\n\"user-specified\". \"pre-defined\" leaves it open to question \"who\ndefines it?\", and it is a reasonable interpretation that the locale\nmay be hardcoded in the makefile, but that is not what this patch\ndoes. Saying \"user-specified\" would not have such a problem.\n\n>> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n>> that people don't even have to set this extra Makefile knob. But I'm not\n>> sure if we have a good way of testing if that locale works (if we can't\n>> find the \"locale\" binary).\n>\n> I also think we should fallback to \"C.UTF-8\" instead of not testing those\n> tests. However, I don't know if there're any systems that not have \"C.UTF-8\"\n> locale.\n\nI do share the feeling, but have a hunch that systems lacking\n\"locale -a\" may be either superset of, or has at least large overlap\nwith, those lacking \"C.UTF-8\", and the new mechanism introduced here\nwill cover both of them, so I think it is OK to stop here, at least\nfor now.\n\nThanks.\n"},{"id":"426555","messageId":"20210607010844.24587-1-congdanhqx@gmail.com","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"[PATCH v4] t: use user-specified utf-8 locale for testing svn","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-07T01:08:44Z","receivedAt":"2021-06-07T01:09:08Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"In some test-cases, UTF-8 locale is required. To find such locale,\nwe're using the first available UTF-8 locale that returned by\n\"locale -a\".\n\nHowever, the locale(1) utility is unavailable on some systems,\ne.g. Linux with musl libc.\n\nHowever, without \"locale -a\", we can't guess provided UTF-8 locale.\n\nAdd a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for\nlinux-musl in our CI system.\n\nRename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale,\nsince we no longer prepare the variable named \"a_utf8_locale\",\nbut set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\nThe fallback will be LC_ALL, LANG environment variable,\nor the first UTF-8 locale from output of \"locale -a\", in that order.\n\nSigned-off-by: Đoàn Trần Công Danh \n---\n\nJunio C. Hamano wrote:\n\n> That certainly is better than \"user-specific\", but a lot worse than\n> \"user-specified\". \"pre-defined\" leaves it open to question \"who\n> defines it?\", and it is a reasonable interpretation that the locale\n> may be hardcoded in the makefile, but that is not what this patch\n> does. Saying \"user-specified\" would not have such a problem.\n\nAhh, I scanned your comment too fast. Sorry for the noise.\n\nThanks for the quick turn-around.\n\nRange-diff against v3:\n1: 064cfc0fe3 ! 1: 6036f670d9 t: use pre-defined utf-8 locale for testing svn\n @@ Metadata\n Author: Đoàn Trần Công Danh \n \n ## Commit message ##\n - t: use pre-defined utf-8 locale for testing svn\n + t: use user-specified utf-8 locale for testing svn\n \n In some test-cases, UTF-8 locale is required. To find such locale,\n we're using the first available UTF-8 locale that returned by\n\n Makefile | 7 +++++++\n ci/lib.sh | 1 +\n t/lib-git-svn.sh | 24 ++++++++++++++++++------\n t/t9100-git-svn-basic.sh | 14 +++-----------\n t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n 6 files changed, 34 insertions(+), 22 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex c3565fc0f8..502e0c9a81 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -398,6 +398,10 @@ all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n+# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n+# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n+# locale returned by \"locale -a\".\n+#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n@@ -2801,6 +2805,9 @@ ifdef GIT_TEST_CMP\n endif\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n+endif\n+ifdef GIT_TEST_UTF8_LOCALE\n+\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n endif\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex d848c036c5..476c3f369f 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -229,6 +229,7 @@ linux-musl)\n \tCC=gcc\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n+\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n \t;;\n esac\n \ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 547eb3c31a..83efc17661 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -121,12 +121,24 @@ start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n }\n \n-prepare_a_utf8_locale () {\n-\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n-\tp\n-\tq\n-}')\n-\tif test -n \"$a_utf8_locale\"\n+prepare_utf8_locale () {\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n+\tthen\n+\t\t: test_set_prereq UTF8\n+\telif test -n \"${LC_ALL:-$LANG}\"\n+\tthen\n+\t\tcase \"${LC_ALL:-$LANG}\" in\n+\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n+\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n+\t\t\t;;\n+\t\tesac\n+\telse\n+\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n+\t\tp\n+\t\tq\n+\t}')\n+\tfi\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n \t\ttest_set_prereq UTF8\n \telse\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 1d3fdcc997..d5563ec35f 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -4,21 +4,13 @@\n #\n \n test_description='git svn basic tests'\n-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n \n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./lib-git-svn.sh\n \n-case \"$GIT_SVN_LC_ALL\" in\n-*.UTF-8)\n-\ttest_set_prereq UTF8\n-\t;;\n-*)\n-\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n-\t;;\n-esac\n+prepare_utf8_locale\n \n test_expect_success 'git svn --version works anywhere' '\n \tnongit git svn --version\n@@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n \n-name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n-LC_ALL=\"$GIT_SVN_LC_ALL\"\n+name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n+LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n export LC_ALL\n # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 9b44a44bc1..743fbe1fe4 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n # > ... All of the above characters, except for the backslash, are converted\n # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n # > \"Private use area\") when creating or accessing files.\n-prepare_a_utf8_locale\n+prepare_utf8_locale\n test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tneq=$(printf \"\\201\\202\") &&\n \tgit config svn.pathnameencoding cp932 &&\n@@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n \n # See the comment on the above test for setting of LC_ALL.\n test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tinf=$(printf \"\\201\\207\") &&\n \tgit config svn.pathnameencoding cp932 &&\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 2c213ae654..01e1e8a8f7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -14,12 +14,12 @@ compare_git_head_with () {\n \ttest_cmp current \"$1\"\n }\n \n-prepare_a_utf8_locale\n+prepare_utf8_locale\n \n compare_svn_head_with () {\n \t# extract just the log message and strip out committer info.\n \t# don't use --limit here since svn 1.1.x doesn't have it,\n-\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n+\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n \t\tuse bytes;\n \t\t$/ = (\"-\"x72) . \"\\n\";\n \t\tmy @x = ;\n-- \n2.32.0.rc3.5.gf3d78db977\n\n"},{"id":"426609","messageId":"20210607143815.77djyvxf4xec2qht@tb-raspi4","threadId":"55829","inReplyTo":"xmqqk0n6wkrr.fsf@gitster.g","subject":"Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2021-06-07T14:38:16Z","receivedAt":"2021-06-07T14:38:32Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Mon, Jun 07, 2021 at 10:01:12AM +0900, Junio C Hamano wrote:\n> Đoàn Trần Công Danh writes:\n>\n> > Subject: Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn\n>\n> That certainly is better than \"user-specific\", but a lot worse than\n> \"user-specified\". \"pre-defined\" leaves it open to question \"who\n> defines it?\", and it is a reasonable interpretation that the locale\n> may be hardcoded in the makefile, but that is not what this patch\n> does. Saying \"user-specified\" would not have such a problem.\n>\n> >> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n> >> that people don't even have to set this extra Makefile knob. But I'm not\n> >> sure if we have a good way of testing if that locale works (if we can't\n> >> find the \"locale\" binary).\n> >\n> > I also think we should fallback to \"C.UTF-8\" instead of not testing those\n> > tests. However, I don't know if there're any systems that not have \"C.UTF-8\"\n> > locale.\n>\n> I do share the feeling, but have a hunch that systems lacking\n> \"locale -a\" may be either superset of, or has at least large overlap\n> with, those lacking \"C.UTF-8\", and the new mechanism introduced here\n> will cover both of them, so I think it is OK to stop here, at least\n> for now.\n>\n> Thanks.\n\nI did a little bit of digging, the first (?) usage of locale was probably\nintroduced in the i18n patch to use gettext and msgfmt and stuff:\n\ncommit 5e9637c629702e3d41ad01d95956d1835d7338e0\nAuthor: Ævar Arnfjörð Bjarmason \nDate: Fri Nov 18 00:14:42 2011 +0100\n\n i18n: add infrastructure for translating Git with gettext\n\n--------------------------\n\nI can't get rid of the feeling, that most systems have an UTF-8\nlocale enabled by default these days.\n\nWhat does the following give us under Linux musl ?\nset | grep UTF\n\n\nMacOs (my box here) has this:\n\nLANG=en_US.UTF-8\nLC_ALL=en_US.UTF-8\nLC_CTYPE=UTF-8\nXTERM_LOCALE=en_US.UTF-8\n\nIf, and only if, Linux musl has a similar setup,\nthen we may be able to skip the Makefile knob for the moment.\nJust skip locale -a, if locale is not available.\nThis is just a loose idea, I need to install the git-svn bindings and\ntest if the git-svn tests pass.\n\n\n"},{"id":"426619","messageId":"YL4+ao9L9R6LcN9W@danh.dev","threadId":"55829","inReplyTo":"20210607143815.77djyvxf4xec2qht@tb-raspi4","subject":"Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-07T15:42:34Z","receivedAt":"2021-06-07T15:42:42Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2021-06-07 16:38:16+0200, Torsten Bögershausen wrote:\n> On Mon, Jun 07, 2021 at 10:01:12AM +0900, Junio C Hamano wrote:\n> > Đoàn Trần Công Danh writes:\n> >\n> > > Subject: Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn\n> >\n> > That certainly is better than \"user-specific\", but a lot worse than\n> > \"user-specified\". \"pre-defined\" leaves it open to question \"who\n> > defines it?\", and it is a reasonable interpretation that the locale\n> > may be hardcoded in the makefile, but that is not what this patch\n> > does. Saying \"user-specified\" would not have such a problem.\n> >\n> > >> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n> > >> that people don't even have to set this extra Makefile knob. But I'm not\n> > >> sure if we have a good way of testing if that locale works (if we can't\n> > >> find the \"locale\" binary).\n> > >\n> > > I also think we should fallback to \"C.UTF-8\" instead of not testing those\n> > > tests. However, I don't know if there're any systems that not have \"C.UTF-8\"\n> > > locale.\n> >\n> > I do share the feeling, but have a hunch that systems lacking\n> > \"locale -a\" may be either superset of, or has at least large overlap\n> > with, those lacking \"C.UTF-8\", and the new mechanism introduced here\n> > will cover both of them, so I think it is OK to stop here, at least\n> > for now.\n> >\n> > Thanks.\n> \n> I did a little bit of digging, the first (?) usage of locale was probably\n> introduced in the i18n patch to use gettext and msgfmt and stuff:\n> \n> commit 5e9637c629702e3d41ad01d95956d1835d7338e0\n> Author: Ævar Arnfjörð Bjarmason \n> Date: Fri Nov 18 00:14:42 2011 +0100\n> \n> i18n: add infrastructure for translating Git with gettext\n> \n> --------------------------\n> \n> I can't get rid of the feeling, that most systems have an UTF-8\n> locale enabled by default these days.\n> \n> What does the following give us under Linux musl ?\n> set | grep UTF\n\nIt's empty.\n\nOK, I lied. Distributions include a file in /etc/profile.d to enable\nLC_ALL/LANG. Without that file, it's empty. However, such use-case\nis covered by our fallback to LC_ALL and LANG.\n\nTechnically, Linux with musl always knows those locales C, POSIX,\nC.UTF-8\n\nI worried more about other systems instead.\n\n> MacOs (my box here) has this:\n> \n> LANG=en_US.UTF-8\n> LC_ALL=en_US.UTF-8\n> LC_CTYPE=UTF-8\n> XTERM_LOCALE=en_US.UTF-8\n> \n> If, and only if, Linux musl has a similar setup,\n> then we may be able to skip the Makefile knob for the moment.\n\nYes, this patch allows to skip the Makefile knob. Because LC_ALL and\nLANG will be the first and second fallback, respectively. And we can\nskip a (potentially expensive) call to \"locale -a\".\n\n> Just skip locale -a, if locale is not available.\n> This is just a loose idea, I need to install the git-svn bindings and\n> test if the git-svn tests pass.\n\n-- \nDanh\n"},{"id":"426713","messageId":"YL8PsmLaU+MpWv58@coredump.intra.peff.net","threadId":"55829","inReplyTo":"xmqqk0n6wkrr.fsf@gitster.g","subject":"Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-06-08T06:35:30Z","receivedAt":"2021-06-08T06:35:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 07, 2021 at 10:01:12AM +0900, Junio C Hamano wrote:\n\n> >> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n> >> that people don't even have to set this extra Makefile knob. But I'm not\n> >> sure if we have a good way of testing if that locale works (if we can't\n> >> find the \"locale\" binary).\n> >\n> > I also think we should fallback to \"C.UTF-8\" instead of not testing those\n> > tests. However, I don't know if there're any systems that not have \"C.UTF-8\"\n> > locale.\n> \n> I do share the feeling, but have a hunch that systems lacking\n> \"locale -a\" may be either superset of, or has at least large overlap\n> with, those lacking \"C.UTF-8\", and the new mechanism introduced here\n> will cover both of them, so I think it is OK to stop here, at least\n> for now.\n\nYeah, I think it is OK to stop here, too. I'd worry that we'd make life\nunnecessarily complicated for people testing on platforms that lack\n\"locale\" and C.UTF-8, for little gain.\n\nThe v4 patch looks good to me.\n\n-Peff\n"},{"id":"426714","messageId":"YL8Qe31z1Q5/DkmF@coredump.intra.peff.net","threadId":"55829","inReplyTo":"20210607010844.24587-1-congdanhqx@gmail.com","subject":"Re: [PATCH v4] t: use user-specified utf-8 locale for testing svn","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-06-08T06:38:51Z","receivedAt":"2021-06-08T06:38:55Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 07, 2021 at 08:08:44AM +0700, Đoàn Trần Công Danh wrote:\n\n> In some test-cases, UTF-8 locale is required. To find such locale,\n> we're using the first available UTF-8 locale that returned by\n> \"locale -a\".\n> \n> However, the locale(1) utility is unavailable on some systems,\n> e.g. Linux with musl libc.\n> \n> However, without \"locale -a\", we can't guess provided UTF-8 locale.\n> \n> Add a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for\n> linux-musl in our CI system.\n> \n> Rename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale,\n> since we no longer prepare the variable named \"a_utf8_locale\",\n> but set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\n> The fallback will be LC_ALL, LANG environment variable,\n> or the first UTF-8 locale from output of \"locale -a\", in that order.\n\nThis patch looks good to me. I have one tiny nit below, but I'm OK if\nit's left as-is:\n\n> +prepare_utf8_locale () {\n> +\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n> +\tthen\n> +\t\t: test_set_prereq UTF8\n\nThis \":\" is really just a comment, but since it contains code we would\nlike to run (but which actually gets run later), it confused me.\nSomething like:\n\n : already set by the caller, nothing to do\n\nwould have made it easier to understand.\n\nAnd then the rest of the function is pretty self-explanatory.\n\n-Peff\n"},{"id":"426718","messageId":"YL8SEnS9QWu/qvnb@danh.dev","threadId":"55829","inReplyTo":"YL8PsmLaU+MpWv58@coredump.intra.peff.net","subject":"Re: [PATCH v3] t: use pre-defined utf-8 locale for testing svn","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-08T06:45:38Z","receivedAt":"2021-06-08T06:46:52Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"On 2021-06-08 02:35:30-0400, Jeff King wrote:\n> On Mon, Jun 07, 2021 at 10:01:12AM +0900, Junio C Hamano wrote:\n> \n> > >> I kind of wonder if trying \"C.UTF-8\" would be a reasonable fallback so\n> > >> that people don't even have to set this extra Makefile knob. But I'm not\n> > >> sure if we have a good way of testing if that locale works (if we can't\n> > >> find the \"locale\" binary).\n> > >\n> > > I also think we should fallback to \"C.UTF-8\" instead of not testing those\n> > > tests. However, I don't know if there're any systems that not have \"C.UTF-8\"\n> > > locale.\n> > \n> > I do share the feeling, but have a hunch that systems lacking\n> > \"locale -a\" may be either superset of, or has at least large overlap\n> > with, those lacking \"C.UTF-8\", and the new mechanism introduced here\n> > will cover both of them, so I think it is OK to stop here, at least\n> > for now.\n> \n> Yeah, I think it is OK to stop here, too. I'd worry that we'd make life\n> unnecessarily complicated for people testing on platforms that lack\n> \"locale\" and C.UTF-8, for little gain.\n\nArgh, reading this and my patch again, I figured out a regression with\nmy patch when either LC_ALL and LANG is set but to something not\nending with utf-8 will ignore \"locale -a\" completely. I'll send\na reroll later. (With this reroll, we'll ignore the dumb \":\", too.\n\n> \n> The v4 patch looks good to me.\n> \n> -Peff\n\n-- \nDanh\n"},{"id":"426721","messageId":"20210608065628.23767-1-congdanhqx@gmail.com","threadId":"55829","inReplyTo":"20210602114646.17463-1-congdanhqx@gmail.com","subject":"[PATCH v5] t: use user-specified utf-8 locale for testing svn","fromName":"Đoàn Trần Công Danh","fromEmail":"congdanhqx@gmail.com","sentAt":"2021-06-08T06:56:28Z","receivedAt":"2021-06-08T06:57:57Z","isPatch":true,"sender":{"key":"congdanhqx@gmail.com","avatar":"https://avatars.githubusercontent.com/u/42673067?v=4"},"body":"In some test-cases, UTF-8 locale is required. To find such locale,\nwe're using the first available UTF-8 locale that returned by\n\"locale -a\".\n\nHowever, the locale(1) utility is unavailable on some systems,\ne.g. Linux with musl libc.\n\nHowever, without \"locale -a\", we can't guess provided UTF-8 locale.\n\nAdd a Makefile knob GIT_TEST_UTF8_LOCALE and activate it for\nlinux-musl in our CI system.\n\nRename t/lib-git-svn.sh:prepare_a_utf8_locale to prepare_utf8_locale,\nsince we no longer prepare the variable named \"a_utf8_locale\",\nbut set up a fallback value for GIT_TEST_UTF8_LOCALE instead.\nThe fallback will be LC_ALL, LANG environment variable,\nor the first UTF-8 locale from output of \"locale -a\", in that order.\n\nSigned-off-by: Đoàn Trần Công Danh \n---\n\n New in v5: fix a regression when LC_ALL or LANG set to something that not\n ends with utf-8, e.g. \"C\", \"locale -a\" will be ignored completely.\n\n Thanks to Peff's comment, I didn't notice this regression earlier.\n\nRange-diff against v4:\n1: 6036f670d9 ! 1: 6675de6866 t: use user-specified utf-8 locale for testing svn\n @@ t/lib-git-svn.sh: start_svnserve () {\n -}')\n -\tif test -n \"$a_utf8_locale\"\n +prepare_utf8_locale () {\n -+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n -+\tthen\n -+\t\t: test_set_prereq UTF8\n -+\telif test -n \"${LC_ALL:-$LANG}\"\n ++\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n +\tthen\n +\t\tcase \"${LC_ALL:-$LANG}\" in\n +\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n +\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n +\t\t\t;;\n ++\t\t*)\n ++\t\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n ++\t\t\t\tp\n ++\t\t\t\tq\n ++\t\t\t}')\n ++\t\t\t;;\n +\t\tesac\n -+\telse\n -+\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n -+\t\tp\n -+\t\tq\n -+\t}')\n +\tfi\n +\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n\n Makefile | 7 +++++++\n ci/lib.sh | 1 +\n t/lib-git-svn.sh | 22 ++++++++++++++++------\n t/t9100-git-svn-basic.sh | 14 +++-----------\n t/t9115-git-svn-dcommit-funky-renames.sh | 6 +++---\n t/t9129-git-svn-i18n-commitencoding.sh | 4 ++--\n 6 files changed, 32 insertions(+), 22 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex c3565fc0f8..502e0c9a81 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -398,6 +398,10 @@ all::\n # with a different indexfile format version. If it isn't set the index\n # file format used is index-v[23].\n #\n+# Define GIT_TEST_UTF8_LOCALE to preferred utf-8 locale for testing.\n+# If it isn't set, fallback to $LC_ALL, $LANG or use the first utf-8\n+# locale returned by \"locale -a\".\n+#\n # Define HAVE_CLOCK_GETTIME if your platform has clock_gettime.\n #\n # Define HAVE_CLOCK_MONOTONIC if your platform has CLOCK_MONOTONIC.\n@@ -2801,6 +2805,9 @@ ifdef GIT_TEST_CMP\n endif\n ifdef GIT_TEST_CMP_USE_COPIED_CONTEXT\n \t@echo GIT_TEST_CMP_USE_COPIED_CONTEXT=YesPlease >>$@+\n+endif\n+ifdef GIT_TEST_UTF8_LOCALE\n+\t@echo GIT_TEST_UTF8_LOCALE=\\''$(subst ','\\'',$(subst ','\\'',$(GIT_TEST_UTF8_LOCALE)))'\\' >>$@+\n endif\n \t@echo NO_GETTEXT=\\''$(subst ','\\'',$(subst ','\\'',$(NO_GETTEXT)))'\\' >>$@+\n ifdef GIT_PERF_REPEAT_COUNT\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex d848c036c5..476c3f369f 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -229,6 +229,7 @@ linux-musl)\n \tCC=gcc\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes\"\n \tMAKEFLAGS=\"$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes\"\n+\tMAKEFLAGS=\"$MAKEFLAGS GIT_TEST_UTF8_LOCALE=C.UTF-8\"\n \t;;\n esac\n \ndiff --git a/t/lib-git-svn.sh b/t/lib-git-svn.sh\nindex 547eb3c31a..2fde2353fd 100644\n--- a/t/lib-git-svn.sh\n+++ b/t/lib-git-svn.sh\n@@ -121,12 +121,22 @@ start_svnserve () {\n \t\t --listen-host 127.0.0.1 &\n }\n \n-prepare_a_utf8_locale () {\n-\ta_utf8_locale=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n-\tp\n-\tq\n-}')\n-\tif test -n \"$a_utf8_locale\"\n+prepare_utf8_locale () {\n+\tif test -z \"$GIT_TEST_UTF8_LOCALE\"\n+\tthen\n+\t\tcase \"${LC_ALL:-$LANG}\" in\n+\t\t*.[Uu][Tt][Ff]8 | *.[Uu][Tt][Ff]-8)\n+\t\t\tGIT_TEST_UTF8_LOCALE=\"${LC_ALL:-$LANG}\"\n+\t\t\t;;\n+\t\t*)\n+\t\t\tGIT_TEST_UTF8_LOCALE=$(locale -a | sed -n '/\\.[uU][tT][fF]-*8$/{\n+\t\t\t\tp\n+\t\t\t\tq\n+\t\t\t}')\n+\t\t\t;;\n+\t\tesac\n+\tfi\n+\tif test -n \"$GIT_TEST_UTF8_LOCALE\"\n \tthen\n \t\ttest_set_prereq UTF8\n \telse\ndiff --git a/t/t9100-git-svn-basic.sh b/t/t9100-git-svn-basic.sh\nindex 1d3fdcc997..d5563ec35f 100755\n--- a/t/t9100-git-svn-basic.sh\n+++ b/t/t9100-git-svn-basic.sh\n@@ -4,21 +4,13 @@\n #\n \n test_description='git svn basic tests'\n-GIT_SVN_LC_ALL=${LC_ALL:-$LANG}\n \n GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n \n . ./lib-git-svn.sh\n \n-case \"$GIT_SVN_LC_ALL\" in\n-*.UTF-8)\n-\ttest_set_prereq UTF8\n-\t;;\n-*)\n-\tsay \"# UTF-8 locale not set, some tests skipped ($GIT_SVN_LC_ALL)\"\n-\t;;\n-esac\n+prepare_utf8_locale\n \n test_expect_success 'git svn --version works anywhere' '\n \tnongit git svn --version\n@@ -187,8 +179,8 @@ test_expect_success POSIXPERM,SYMLINKS \"$name\" '\n \ttest ! -h \"$SVN_TREE\"/exec-2.sh &&\n \ttest_cmp help \"$SVN_TREE\"/exec-2.sh'\n \n-name=\"commit with UTF-8 message: locale: $GIT_SVN_LC_ALL\"\n-LC_ALL=\"$GIT_SVN_LC_ALL\"\n+name=\"commit with UTF-8 message: locale: $GIT_TEST_UTF8_LOCALE\"\n+LC_ALL=\"$GIT_TEST_UTF8_LOCALE\"\n export LC_ALL\n # This test relies on the previous test, hence requires POSIXPERM,SYMLINKS\n test_expect_success UTF8,POSIXPERM,SYMLINKS \"$name\" \"\ndiff --git a/t/t9115-git-svn-dcommit-funky-renames.sh b/t/t9115-git-svn-dcommit-funky-renames.sh\nindex 9b44a44bc1..743fbe1fe4 100755\n--- a/t/t9115-git-svn-dcommit-funky-renames.sh\n+++ b/t/t9115-git-svn-dcommit-funky-renames.sh\n@@ -93,9 +93,9 @@ test_expect_success 'git svn rebase works inside a fresh-cloned repository' '\n # > ... All of the above characters, except for the backslash, are converted\n # > to special UNICODE characters in the range 0xf000 to 0xf0ff (the\n # > \"Private use area\") when creating or accessing files.\n-prepare_a_utf8_locale\n+prepare_utf8_locale\n test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new file on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tneq=$(printf \"\\201\\202\") &&\n \tgit config svn.pathnameencoding cp932 &&\n@@ -107,7 +107,7 @@ test_expect_success UTF8,!MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 new\n \n # See the comment on the above test for setting of LC_ALL.\n test_expect_success !MINGW,!UTF8_NFD_TO_NFC 'svn.pathnameencoding=cp932 rename on dcommit' '\n-\tLC_ALL=$a_utf8_locale &&\n+\tLC_ALL=$GIT_TEST_UTF8_LOCALE &&\n \texport LC_ALL &&\n \tinf=$(printf \"\\201\\207\") &&\n \tgit config svn.pathnameencoding cp932 &&\ndiff --git a/t/t9129-git-svn-i18n-commitencoding.sh b/t/t9129-git-svn-i18n-commitencoding.sh\nindex 2c213ae654..01e1e8a8f7 100755\n--- a/t/t9129-git-svn-i18n-commitencoding.sh\n+++ b/t/t9129-git-svn-i18n-commitencoding.sh\n@@ -14,12 +14,12 @@ compare_git_head_with () {\n \ttest_cmp current \"$1\"\n }\n \n-prepare_a_utf8_locale\n+prepare_utf8_locale\n \n compare_svn_head_with () {\n \t# extract just the log message and strip out committer info.\n \t# don't use --limit here since svn 1.1.x doesn't have it,\n-\tLC_ALL=\"$a_utf8_locale\" svn log $(git svn info --url) | perl -w -e '\n+\tLC_ALL=\"$GIT_TEST_UTF8_LOCALE\" svn log $(git svn info --url) | perl -w -e '\n \t\tuse bytes;\n \t\t$/ = (\"-\"x72) . \"\\n\";\n \t\tmy @x = ;\n-- \n2.32.0.278.gd42b80f139\n\n"},{"id":"426728","messageId":"YL8bk5f4Xu91xwIL@coredump.intra.peff.net","threadId":"55829","inReplyTo":"20210608065628.23767-1-congdanhqx@gmail.com","subject":"Re: [PATCH v5] t: use user-specified utf-8 locale for testing svn","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2021-06-08T07:26:11Z","receivedAt":"2021-06-08T07:26:14Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jun 08, 2021 at 01:56:28PM +0700, Đoàn Trần Công Danh wrote:\n\n> New in v5: fix a regression when LC_ALL or LANG set to something that not\n> ends with utf-8, e.g. \"C\", \"locale -a\" will be ignored completely.\n> \n> Thanks to Peff's comment, I didn't notice this regression earlier.\n\nDoh, I totally missed that, too. I'm glad my comment was helpful in that\nrespect, at least. :)\n\nYour fix here looks good.\n\n-Peff\n"},{"id":"426975","messageId":"874ke62f41.fsf@evledraar.gmail.com","threadId":"55829","inReplyTo":"YLfiYXxQqXL7RyHC@nand.local","subject":"Re: [PATCH] t: use user-specific utf-8 locale for testing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2021-06-08T10:49:21Z","receivedAt":"2021-06-10T10:23:14Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Jun 02 2021, Taylor Blau wrote:\n\n> On Wed, Jun 02, 2021 at 06:46:46PM +0700, Đoàn Trần Công Danh wrote:\n>> Despite being required by POSIX, locale(1) is unavailable in some\n>> systems, e.g. Linux with musl libc. Some of those systems support\n>> utf-8 locale out of the box.\n>\n> Hmmph. I would have imagined that locale was available everywhere, but\n> unfortunately not.\n\nSmall and unsolicited history lesson from a person with funny characters\nin their name & language :)\n\nToday it seems like *nix systems have always had UTF-8, but this was a\nrelatively late development. \n\nIt's Plan9 that had UTF-8 from the start, on *nix systems it was\nUS-ASCII, and anything else was tacked on top later on.\n\nWhen I started using *nix systems I belive it was quite common to have\ndefault configurations with only ISO-8859-1 locales installed, and\ncertainly that's what a lot of or most users who had the need for\nlocales in European languages not covered by US-ASCII used by default.\n\nThis is from hazy memory, but I think it was even actively recommended\nagainst having or using UTF-8 locales on the system. If you\ne.g. connected to an IRC channel, or copy/pasted from your text editor\ninto an E-Mail you could easily send the other end misencodedgibberish.\n\nLater on things like IRC channels in these languages had a \"switch day\",\nit was a complete mess. Nowadays mostly nobody really notices or\nremembers anymore these encoding issues since we've mostly got UTF-8\neverywhere as a result.\n\nI mean, at least in the case of European languages, I understand\ne.g. Japanese and Chinese still have their own persistent encoding\nissues related to competing standards.\n\nEven today you can't rely on UTF-8 even on Linux systems, and I think\nthis has become even more true of late with minimal CI systems or other\nchroot-like test environments.\n"}]}