Accept-language test fails on Mac OS

> Test #25 'git client sends Accept-Language based on LANGUAGE, LC_ALL,
> LC_MESSAGES and LANG' in t5550 fails consistently on my mac, and has
> since the test was introduced. Test 26 and 27 ('git client sends
> Accept-Language with many preferred languages' and 'git client does
> not send Accept-Language') seem fine.
>
> I'm building git with NO_GETTEXT=1, which may be an issue? But in that
> case the test should probably be gated on gettext?

> Michael Blume <blume.mike@gmail.com> writes:
>
>> Test #25 'git client sends Accept-Language based on LANGUAGE, LC_ALL,
>> LC_MESSAGES and LANG' in t5550 fails consistently on my mac, and has
>> since the test was introduced. Test 26 and 27 ('git client sends
>> Accept-Language with many preferred languages' and 'git client does
>> not send Accept-Language') seem fine.
>>
>> I'm building git with NO_GETTEXT=1, which may be an issue? But in that
>> case the test should probably be gated on gettext?
>
> I recall queuing a SQUASH??? on top of the posted patch; does these
> tests pass with it reverted?

> On Fri, Dec 5, 2014 at 2:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Michael Blume <blume.mike@gmail.com> writes:
>>
>>> Test #25 'git client sends Accept-Language based on LANGUAGE, LC_ALL,
>>> LC_MESSAGES and LANG' in t5550 fails consistently on my mac, and has
>>> since the test was introduced. Test 26 and 27 ('git client sends
>>> Accept-Language with many preferred languages' and 'git client does
>>> not send Accept-Language') seem fine.
>>>
>>> I'm building git with NO_GETTEXT=1, which may be an issue? But in that
>>> case the test should probably be gated on gettext?
>>
>> I recall queuing a SQUASH??? on top of the posted patch; does these
>> tests pass with it reverted?
> 
> The test fails both on pu and on 7567fad which is prior to the
> SQUASH??? commit, so the squash does not seem to change anything.

> On 2014-12-06 00.01, Michael Blume wrote:
>> On Fri, Dec 5, 2014 at 2:51 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>> Michael Blume <blume.mike@gmail.com> writes:
>>>
>>>> Test #25 'git client sends Accept-Language based on LANGUAGE, LC_ALL,
>>>> LC_MESSAGES and LANG' in t5550 fails consistently on my mac, and has
>>>> since the test was introduced. Test 26 and 27 ('git client sends
>>>> Accept-Language with many preferred languages' and 'git client does
>>>> not send Accept-Language') seem fine.
>>>>
>>>> I'm building git with NO_GETTEXT=1, which may be an issue? But in that
>>>> case the test should probably be gated on gettext?
>>>
>>> I recall queuing a SQUASH??? on top of the posted patch; does these
>>> tests pass with it reverted?
>>
>> The test fails both on pu and on 7567fad which is prior to the
>> SQUASH??? commit, so the squash does not seem to change anything.
> pu@c83418638e0c326f:
> t5550 fails here under Debian Linux,
> I haven't digged further
>
> not ok 25 - git client sends Accept-Language based on LANGUAGE, LC_ALL, LC_MESSAGES and LANG
> #
> #               check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
> #
>

> On 2014-12-06 20.44, Yi, EungJun wrote:
>> Could you tell me your locale information from executing 'locale'
>> command and the verbose message you can get by accessing any git
>> repository via HTTP protocol? (e.g. GIT_CURL_VERBOSE=1 git clone
>> http://github.com/foo/bar )
> locale -a
> C
> C.UTF-8
> en_US.utf8
> icelandic
> is_IS
> is_IS.iso88591
> is_IS.utf8
> POSIX

>
> -----------------------------
> After patching like this:
> diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
> index fc6e7fc..1e42b42 100755
> --- a/t/t5550-http-fetch-dumb.sh
> +++ b/t/t5550-http-fetch-dumb.sh
> @@ -205,8 +205,8 @@ check_language () {
>         LANG=$5 \
>         git ls-remote "$HTTPD_URL/dumb/repo.git" 2>&1 |
>         tr -d '\015' |
> -       sort -u >stderr &&
> -       grep -i ^Accept-Language: stderr >actual &&
> +       sort -u >stderr
> +       grep -i ^Accept-Language: stderr >actual
>         test_cmp expect actual
>  }
> -----------
> I get this:
>
>
> expecting success:
>         check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
>
> --- expect      2014-12-06 21:00:59.000000000 +0000
> +++ actual      2014-12-06 21:00:59.000000000 +0000
> @@ -1 +0,0 @@
> -Accept-Language: de-DE, *;q=0.1
> not ok 25 - git client sends Accept-Language based on LANGUAGE, LC_ALL, LC_MESSAGES and LANG
> #
> #               check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
> #               check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
> #
>
>

> I'm sorry for bothering you, but could you tell me the result of
> "locale" command without "-a" option? What I want to know is locale
> environment variables and its values, so I want to reproduce the test
> failures on my laptop.
> 

> I get this:
> 
> 
> expecting success: 
>         check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
>         check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
> 
> --- expect      2014-12-06 21:00:59.000000000 +0000
> +++ actual      2014-12-06 21:00:59.000000000 +0000
> @@ -1 +0,0 @@
> -Accept-Language: de-DE, *;q=0.1
> not ok 25 - git client sends Accept-Language based on LANGUAGE, LC_ALL, LC_MESSAGES and LANG

> On Sat, Dec 06, 2014 at 10:04:06PM +0100, Torsten Bögershausen wrote:
> 
>> I get this:
>>
>>
>> expecting success: 
>>         check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
>>
>> --- expect      2014-12-06 21:00:59.000000000 +0000
>> +++ actual      2014-12-06 21:00:59.000000000 +0000
>> @@ -1 +0,0 @@
>> -Accept-Language: de-DE, *;q=0.1
>> not ok 25 - git client sends Accept-Language based on LANGUAGE, LC_ALL, LC_MESSAGES and LANG
> 
> I can reproduce the same problem here (Debian unstable). I actually ran
> into three issues (aside from needing to use Junio's SQUASH commit, to
> avoid the "\r" bash-ism):
> 
>   1. I couldn't build without including locale.h, for the
>      definition of setlocale() and the LC_MESSAGES constant (both used
>      in get_preferred_languages).
> 
>      I'm not sure what portability issues there are with including it
>      unconditionally. Should this possibly be tied into gettext.c, which
>      already uses setlocale?
> 
>   2. The call to setlocale(LC_MESSAGES, NULL) in get_preferred_languages
>      always returns "C" for me. This seems related to building with
>      NO_GETTEXT (which I typically do), as we never init setlocale
>      if NO_GETTEXT is set. This program demonstrates it:
> 
> 	#include <stdio.h>
> 	#include <string.h>
> 	#include <locale.h>
> 	
> 	int main(int argc, char **argv)
> 	{
> 		if (argv[1] && !strcmp(argv[1], "init"))
> 			setlocale(LC_MESSAGES, "");
> 		printf("%s", setlocale(LC_MESSAGES, NULL));
> 		return 0;
> 	}
> 
>      If I run it as "LANG=en_US.UTF-8 ./a.out", it prints "C". If I run
>      it as "LANG=en_US.UTF-8 ./a.out init", it prints "en_US.UTF-8". I
>      think we either need to start unconditionally calling setlocale()
>      as we do in git_setup_gettext, or we need to tie your feature to
>      using gettext.
> 
>      This is what causes the failure of the de-DE test for me; building
>      without NO_GETTEXT makes it work. Note that this doesn't affect the
>      first test for ko-KR, because that test sets LANGUAGE, which we
>      read ourselves (so we never make a setlocale() call).
> 
>   3. Even building with NO_GETTEXT, setlocale() does not want to
>      report ja_JP.UTF-8 for me, making the third test fail.
> 
>      I think the issue is that I do not build the ja_JP locale on my
>      system. Running "dpkg-reconfigure locales" and asking it to build
>      ja_JP.UTF-8 makes the test pass. This is somewhat of a Debian-ism.
>      From "man locale-gen":
> 
>        By default, the locale package which provides the base support
>        for localisation of libc-based programs does not contain usable
>        localisation files for every supported language. This limitation
>        has became necessary because of the substantial size of such
>        files and the large number of languages supported by libc. As a
>        result, Debian uses a special mechanism where we prepare the
>        actual localisation files on the target host and distribute only
>        the templates for them.
> 
>      I suspect it is inherited by Debian derivatives like Ubuntu. But I
>      also don't know that we can count on other platforms having all of
>      the locales either (e.g., they may ship them as separate packages,
>      not all of which are installed).
> 
>      So I'm not sure of an easy way around this. You want 4 separate
>      locales to thoroughly test, but you cannot rely on any particular
>      locale being present on the user's system.
> 
>      Note that this is just a problem with the tests, probably not with
>      the feature itself. Presumably people setting LANG=ja_JP actually
>      have that locale on their system (though technically this feature
>      is about asking the _server_ to use that language, it seems like
>      you would do so because you were using that language locally, too).
> 
> -Peff
> --

> On Sat, Dec 06, 2014 at 10:04:06PM +0100, Torsten Bögershausen wrote:
>
>> I get this:
>>
>>
>> expecting success:
>>         check_language "ko-KR, *;q=0.1" ko_KR.UTF-8 de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "de-DE, *;q=0.1" ""          de_DE.UTF-8 ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "ja-JP, *;q=0.1" ""          ""          ja_JP.UTF-8 en_US.UTF-8 &&
>>         check_language "en-US, *;q=0.1" ""          ""          ""          en_US.UTF-8
>>
>> --- expect      2014-12-06 21:00:59.000000000 +0000
>> +++ actual      2014-12-06 21:00:59.000000000 +0000
>> @@ -1 +0,0 @@
>> -Accept-Language: de-DE, *;q=0.1
>> not ok 25 - git client sends Accept-Language based on LANGUAGE, LC_ALL, LC_MESSAGES and LANG
>
> I can reproduce the same problem here (Debian unstable). I actually ran
> into three issues (aside from needing to use Junio's SQUASH commit, to
> avoid the "\r" bash-ism):
>
>   1. I couldn't build without including locale.h, for the
>      definition of setlocale() and the LC_MESSAGES constant (both used
>      in get_preferred_languages).
>
>      I'm not sure what portability issues there are with including it
>      unconditionally. Should this possibly be tied into gettext.c, which
>      already uses setlocale?