threads / bug / 49348

[Bug report] Git incorrectly selects language in macos

Subject: [Bug report] Git incorrectly selects language in macos

## tl;dr

11 messages between Sep 14, 2018 and Jan 16, 2019.

replies: 10people: 5as markdown or json

Niko Dzhus· Sep 14, 2018, 19:08 UTC · lore

It doesn't use English when other language is available as a secondary language.

Reproducing:

1. Open "Language & Region" in macos settings
2. In "Preferred languages" box, set English as a primary language.
3. Add another language, that git is translated to, as a secondary
language, for example, French or German.
4. Run any git command - git will use the secondary language, instead
of English.

When the secondary language is removed, then git starts using English again.

I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

Ævar Arnfjörð Bjarmason· Sep 14, 2018, 21:57 UTC · re: Niko Dzhus · lore

Re: [Bug report] Git incorrectly selects language in macos

On Fri, Sep 14 2018, Niko Dzhus wrote:

Show 14 quoted lines

> It doesn't use English when other language is available as a secondary language.
>
> Reproducing:
>
> 1. Open "Language & Region" in macos settings
> 2. In "Preferred languages" box, set English as a primary language.
> 3. Add another language, that git is translated to, as a secondary
> language, for example, French or German.
> 4. Run any git command - git will use the secondary language, instead
> of English.
>
> When the secondary language is removed, then git starts using English again.
>
> I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

What's the output of these two commands for you:

 1. locale
 2. env | grep -e LC -e LANG

We don't do any such magic ourselves, so whatever this is is down to how i18n in general works on your system, do you have any other translated command-line program that works differently?

I suspect there's some DWYM logic here that always treats English as a secondary language.

Do you also e.g. get the same results if you select say Swedish as a primary language and German as a secondary? I.e. a Git in German, as opposed to Swedish?

Niko Dzhus· Sep 15, 2018, 00:51 UTC · re: Ævar Arnfjörð Bjarmason · lore

Re: [Bug report] Git incorrectly selects language in macos

Tried what you suggested - it seems, it only ignores English. In you example, with Swedish as primary and German as secondary, git uses Swedish.

With more that one secondary language, the one with a higher priority is being used, as expected. I also tried using non-generic English (English-UK and English-US), but they also get ignored.

Terminal commands return the following:

➜ ~ locale LANG= LC_COLLATE="C" LC_CTYPE="UTF-8" LC_MESSAGES="C" LC_MONETARY="C" LC_NUMERIC="C" LC_TIME="C" LC_ALL= ➜ ~ env | grep -e LC -e LANG LC_CTYPE=UTF-8 ➜ ~

It doesn't change with primary/secondary language switching. I don't have any manual overrides in my .zshrc and .zprofile for those neither.

On Sat, Sep 15, 2018 at 12:57 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

Show 34 quoted lines

>
>
> On Fri, Sep 14 2018, Niko Dzhus wrote:
>
> > It doesn't use English when other language is available as a secondary language.
> >
> > Reproducing:
> >
> > 1. Open "Language & Region" in macos settings
> > 2. In "Preferred languages" box, set English as a primary language.
> > 3. Add another language, that git is translated to, as a secondary
> > language, for example, French or German.
> > 4. Run any git command - git will use the secondary language, instead
> > of English.
> >
> > When the secondary language is removed, then git starts using English again.
> >
> > I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .
>
> What's the output of these two commands for you:
>
>  1. locale
>  2. env | grep -e LC -e LANG
>
> We don't do any such magic ourselves, so whatever this is is down to how
> i18n in general works on your system, do you have any other translated
> command-line program that works differently?
>
> I suspect there's some DWYM logic here that always treats English as a
> secondary language.
>
> Do you also e.g. get the same results if you select say Swedish as a
> primary language and German as a secondary? I.e. a Git in German, as
> opposed to Swedish?

Niko Dzhus· Sep 15, 2018, 02:20 UTC · re: Niko Dzhus · lore

Re: [Bug report] Git incorrectly selects language in macos

Looks like the issue appeared after updating git from brew.

I tried it on a different mac laptop, git 2.18 still used English, but after updating to 2.19 it started using secondary language.

A quick search revealed that brew changed how it builds git recently. I think, it just didn't include i18n by default before, so I never noticed this. Here's the history of formula changes: https://github.com/Homebrew/homebrew-core/commits/master/Formula/git.rb Also, I found this thread https://github.com/Homebrew/homebrew-core/issues/31980

Anybody here familiar enough with the build process and dependencies of git to pinpoint what exactly is causing this and how to fix it?...

On Fri, Sep 14, 2018 at 10:08 PM Niko Dzhus <primenico@gmail.com> wrote:

Show 15 quoted lines

>
> It doesn't use English when other language is available as a secondary language.
>
> Reproducing:
>
> 1. Open "Language & Region" in macos settings
> 2. In "Preferred languages" box, set English as a primary language.
> 3. Add another language, that git is translated to, as a secondary
> language, for example, French or German.
> 4. Run any git command - git will use the secondary language, instead
> of English.
>
> When the secondary language is removed, then git starts using English again.
>
> I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

Eric Sunshine· Sep 16, 2018, 07:15 UTC · re: Niko Dzhus · lore

Re: [Bug report] Git incorrectly selects language in macos

On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:

Show 8 quoted lines

> Looks like the issue appeared after updating git from brew.
>
> A quick search revealed that brew changed how it builds git recently.
> I think, it just didn't include i18n by default before, so I never
> noticed this.
>
> Anybody here familiar enough with the build process and dependencies
> of git to pinpoint what exactly is causing this and how to fix it?...

This problem is not specific to Git. Earlier in the thread, Ævar asked[1] if the problem also occurs with other command-line programs, and indeed it does. For instance, I tried with 'wget' installed via brew, and it exhibits the same odd behavior. Ævar suggested that there might be some magic special-casing English, which makes me wonder if brew builds such magic into gettext(?) or if the magic is part of MacOS itself.

[1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/

Nate Weaver· Jan 15, 2019, 15:35 UTC · re: Eric Sunshine · lore

Re: [Bug report] Git incorrectly selects language in macos

Show 21 quoted lines

> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>> Looks like the issue appeared after updating git from brew.
>> 
>> A quick search revealed that brew changed how it builds git recently.
>> I think, it just didn't include i18n by default before, so I never
>> noticed this.
>> 
>> Anybody here familiar enough with the build process and dependencies
>> of git to pinpoint what exactly is causing this and how to fix it?...
> 
> This problem is not specific to Git. Earlier in the thread, Ævar
> asked[1] if the problem also occurs with other command-line programs,
> and indeed it does. For instance, I tried with 'wget' installed via
> brew, and it exhibits the same odd behavior. Ævar suggested that there
> might be some magic special-casing English, which makes me wonder if
> brew builds such magic into gettext(?) or if the magic is part of
> MacOS itself.
> 
> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/

I discovered that moving/renaming /usr/local/share/locale made this issue go away for me. While 'de' and 'fr' have a git.mo file in their LC_MESSAGES subdirectory, there is no 'en' directory.

If I copy the 'fr' version to a newly-created 'en' directory, then all my git output is in French; my language order in System Preferences is English (US) -> German -> Japanese.

Not sure if this is gettext's or macOS's issue.

--Nate Weaver (Wevah)

Nate Weaver· Jan 15, 2019, 16:43 UTC · re: Nate Weaver · lore

Re: [Bug report] Git incorrectly selects language in macos

Show 35 quoted lines

> On Jan 15, 2019, at 09:35:52, Nate Weaver <wevah@derailer.org> wrote:
> 
>> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
>> 
>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>>> Looks like the issue appeared after updating git from brew.
>>> 
>>> A quick search revealed that brew changed how it builds git recently.
>>> I think, it just didn't include i18n by default before, so I never
>>> noticed this.
>>> 
>>> Anybody here familiar enough with the build process and dependencies
>>> of git to pinpoint what exactly is causing this and how to fix it?...
>> 
>> This problem is not specific to Git. Earlier in the thread, Ævar
>> asked[1] if the problem also occurs with other command-line programs,
>> and indeed it does. For instance, I tried with 'wget' installed via
>> brew, and it exhibits the same odd behavior. Ævar suggested that there
>> might be some magic special-casing English, which makes me wonder if
>> brew builds such magic into gettext(?) or if the magic is part of
>> MacOS itself.
>> 
>> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/
> 
> I discovered that moving/renaming /usr/local/share/locale made this
> issue go away for me. While 'de' and 'fr' have a git.mo file in their
> LC_MESSAGES subdirectory, there is no 'en' directory.
> 
> If I copy the 'fr' version to a newly-created 'en' directory, then all my
> git output is in French; my language order in System Preferences is
> English (US) -> German -> Japanese.
> 
> Not sure if this is gettext's or macOS's issue.
> 
> --Nate Weaver (Wevah)

Upon further digging, this is an issue in gettext's code on macOS: The function _nl_language_preferences_default (in langprefs.c) specifically breaks early when it sees the literal string "en" in the list (from the "AppleLanguages" defaults key), but not when it gets "en-US", etc.

I.e., it doesn't check for a prefix.

This can be fixed (though there might be a better way) simply by replacing both instances of

if (strcmp (buf, "en") == 0)

with

if (strncmp (buf, "en", 2) == 0)

in gettext's langprefs.c.

Nate Weaver· Jan 15, 2019, 19:06 UTC · re: Nate Weaver · lore

Re: [Bug report] Git incorrectly selects language in macos

--Nate Weaver (Wevah)

Show 55 quoted lines

> On Jan 15, 2019, at 10:43:48, Nate Weaver <wevah@derailer.org> wrote:
> 
>> On Jan 15, 2019, at 09:35:52, Nate Weaver <wevah@derailer.org> wrote:
>> 
>>> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>> 
>>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>>>> Looks like the issue appeared after updating git from brew.
>>>> 
>>>> A quick search revealed that brew changed how it builds git recently.
>>>> I think, it just didn't include i18n by default before, so I never
>>>> noticed this.
>>>> 
>>>> Anybody here familiar enough with the build process and dependencies
>>>> of git to pinpoint what exactly is causing this and how to fix it?...
>>> 
>>> This problem is not specific to Git. Earlier in the thread, Ævar
>>> asked[1] if the problem also occurs with other command-line programs,
>>> and indeed it does. For instance, I tried with 'wget' installed via
>>> brew, and it exhibits the same odd behavior. Ævar suggested that there
>>> might be some magic special-casing English, which makes me wonder if
>>> brew builds such magic into gettext(?) or if the magic is part of
>>> MacOS itself.
>>> 
>>> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/
>> 
>> I discovered that moving/renaming /usr/local/share/locale made this
>> issue go away for me. While 'de' and 'fr' have a git.mo file in their
>> LC_MESSAGES subdirectory, there is no 'en' directory.
>> 
>> If I copy the 'fr' version to a newly-created 'en' directory, then all my
>> git output is in French; my language order in System Preferences is
>> English (US) -> German -> Japanese.
>> 
>> Not sure if this is gettext's or macOS's issue.
>> 
>> --Nate Weaver (Wevah)
> 
> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.
> 
> I.e., it doesn't check for a prefix.
> 
> This can be fixed (though there might be a better way) simply by replacing
> both instances of
> 
> if (strcmp (buf, "en") == 0)
> 
> with
> 
> if (strncmp (buf, "en", 2) == 0)
> 
> in gettext's langprefs.c.

Nevermind: I've been informed that this has already been fixed on the git trunk
for gettext, and will be in the next release.

(See: https://savannah.gnu.org/bugs/?49560)

Eric Sunshine· Jan 15, 2019, 20:36 UTC · re: Nate Weaver · lore

Re: [Bug report] Git incorrectly selects language in macos

On Tue, Jan 15, 2019 at 2:06 PM Nate Weaver <wevah@derailer.org> wrote:

> Nevermind: I've been informed that this has already been fixed on the git trunk
> for gettext, and will be in the next release.
>
> (See: https://savannah.gnu.org/bugs/?49560)

Thanks for the follow-up report. It's good to know that this is fixed at the correct location upstream.

I tried updating the Homebrew bug report[1] regarding this problem with respect to Git in order to add the new information you provided, but the ticket has been "locked" so no additional comments can be made there, which is unfortunate.

[1]: https://github.com/Homebrew/homebrew-core/issues/31980

Jonathan Nieder· Jan 16, 2019, 02:16 UTC · re: Nate Weaver · lore

Re: [Bug report] Git incorrectly selects language in macos

Nate Weaver wrote:

>>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:

>>>> Looks like the issue appeared after updating git from brew.

[...]

>>>> A quick search revealed that brew changed how it builds git recently.
>>>> I think, it just didn't include i18n by default before, so I never
>>>> noticed this.

[...]

Show 17 quoted lines

> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.
>
> I.e., it doesn't check for a prefix.
>
> This can be fixed (though there might be a better way) simply by replacing
> both instances of
>
> if (strcmp (buf, "en") == 0)
>
> with
>
> if (strncmp (buf, "en", 2) == 0)
>
> in gettext's langprefs.c.

Nice sleuthing. Do you have a bug tracking this in homebrew's issue tracker, either for the git formula or the gettext formula?

Thanks, Jonathan

Jonathan Nieder· Jan 16, 2019, 02:28 UTC · re: Nate Weaver · lore

Re: [Bug report] Git incorrectly selects language in macos

Nate Weaver wrote:

> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.

Sorry I missed your other followup.

As you mentioned, the fix is at https://savannah.gnu.org/bugs/?49560. There isn't a gettext release yet with that fix, so perhaps https://github.com/Homebrew/homebrew-core/blob/master/Formula/gettext.rb could be updated to apply that fix as a patch (https://docs.brew.sh/Formula-Cookbook#patches).

Thanks again and sorry for the noise, Jonathan