Re: Git 2.26.2 and failed self tests on OS X

From: Torsten Bögershausen <tboegi@web.de>
Date: May 5, 2020, 04:10 UTC
Message-ID: <20200505041033.w2q7h5k7otetfrus@tb-raspi4>
In-Reply-To: <CAH8yC8nyg6pRStrQxkO6DzqRbLhhza0vx1U1cGUV4P4MQ3jZ7Q@mail.gmail.com>

On Mon, May 04, 2020 at 05:39:50PM -0400, Jeffrey Walton wrote:

Show 60 quoted lines

> On Mon, May 4, 2020 at 4:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
> >
> > On Mon, May 4, 2020 at 4:19 PM Torsten Bögershausen <tboegi@web.de> wrote:
> > > On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
> > > > ...
> > > > > Does
> > > > > ./t3902-quoted.sh   -v -d
> > > > > give anything useful ?
> > > >
> > > > Yes, it looks like the command reveals a couple of problems.
> > > >
> > > > I'll have to look into my iconv recipe. It has not changed in a while,
> > > > so something else may be responsible for the failure.
> > > > https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.
> > > >
> > > > I'm not sure what to do with the HEAD problems.
> > > >
> > > > $ ./t3902-quoted.sh   -v -d
> > > > Initialized empty Git repository in
> > > > /Users/jwalton/Build-Scripts/git-2.26.2/t/trash
> > > > directory.t3902-quoted/.git/
> > > > expecting success of 3902.1 'setup':
> > > >
> > > >         mkdir "$FN" &&
> > > >         for_each_name "echo initial >\"\$name\"" &&
> > > >         git add . &&
> > > >         git commit -q -m Initial &&
> > > >
> > > >         for_each_name "echo second >\"\$name\"" &&
> > > >         git commit -a -m Second &&
> > > >
> > > >         for_each_name "echo modified >\"\$name\""
> > > >
> > > >
> > > > fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
> > > >     precomposed unicode is not supported.
> > > >     If you want to use decomposed unicode, run
> > > >     "git config core.precomposeunicode false"
> > > >
> > >
> > > I am quite sure, that the trouble starts here.
> > > It seems as if you are building iconv yourself ?
> > > And that iconv does not include "UTF-8-MAC", which is a specialty
> > > for Mac.
> >
> > Oh, thanks. I was not aware.
> >
> > I'll open a bug report with the iconv folks letting them know
> > configure has some gaps on OS X.
>
> Thanks again Torsten.
>
> I am not seeing an option to enable the encoding in iConv. Would you
> happen to know what is needed?
>
> Thanks again.
>
> $ cd libiconv-1.16
> $ ./configure --help
> `configure' configures libiconv 1.16 to adapt to many kinds of systems.

Hej Jeffrey, I am not an iconv expert - and don't intend to become one. The UTF-8-MAC conversion does probably only exist in the code for iconv shipped on Mac OS - so it could happen that this code is not open source. There may be ways to "extract it" : Either that information is part of the unicode specification, and there may be tables that can be converted into c-code that fits into the iconv code base. I don't know. Or you can use the iconv on MacOs itself to generate such a table. That is all somewhat behind my horizon.

For Git (under MacOS) we need UTF-8-MAC to compensate for HPS, (or HPF+) so that all filenames with non-ASCII codepoints are stored in the repo in precomposed unicode. This is what Linux and Windoes and Unix is using, this commit may give more information about the what and why:

  commit 76759c7dff53e8c84e975b88cb8245587c14c7ba
  Author: Torsten Bögershausen <tboegi@web.de>
  Date:   Sun Jul 8 15:50:25 2012 +0200
    git on Mac OS and precomposed unicode

By the way, does ./t3910-mac-os-precompose.sh work on your system ?

So far I am not aware about any problems with iconv from MacOs being reported here on the list. Which problems/bugs did you found in the iconv shipped with MacOs, especially together with Git ?

[snip]