Re: Broken interactive rebase text after some UTF-8 characters

From: Michal Nowak <mnowak@startmail.com>
Date: Feb 1, 2019, 16:24 UTC
Message-ID: <eda7b1c4-5fd6-5440-8998-75eab9f73e47@startmail.com>
In-Reply-To: <nycvar.QRO.7.76.6.1902011530440.41@tvgsbejvaqbjf.bet>

On 02/01/19 03:33 PM, Johannes Schindelin wrote:

Show 54 quoted lines

> Hi Michal,
> 
> On Fri, 1 Feb 2019, Michal Nowak wrote:
> 
>> On Friday, February 1, 2019 at 8:38 AM, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>>>
>>> On Thu, 31 Jan 2019, Junio C Hamano wrote:
>>>
>>>> Phillip Wood <phillip.wood@talktalk.net> writes:
>>>>
>>>>>> Are we misusing C formats?
>>>>>
>>>>> The C standard and POSIX both say that the * refers to the maximum
>>>>> number of bytes to print but it looks like it is being treated as the
>>>>> maximum number of characters on OpenIndiana.
>>>>>
>>>>> Johannes - Perhaps we should change it to use fwrite() unless
>>>> printf()
>>>>> gets fixed and we're sure no other operating systems are affected?
>>>>
>>>> Avoid such a rewrite, as "%*.s" that takes (int, char *) are used in
>>>> many other places in our codebase, if you can.
>>>
>>> Yes, this would be painful in particular in cases like
>>>
>>> 	master:advice.c:101:           fprintf(stderr, _("%shint: %.*s%s\n"),
>>>
>>> where we want to write more than just a variable-length buffer.
>>>
>>> I am curious: is libintl (gettext) used on OpenIndiana? I ask because
>>> AFAIR fprintf() is overridden in that case, and the bug might be a lot
>>> easier to fix if it is in libintl rather than in libc.
>>
>> here you can see the full output of the OpenIndiana git build: https://hipster.openindiana.org/logs/oi-userland/latest/git.publish.log.
>>
>>  From what I see there, libintl was found.
>>
>> If you believe this is illumos libc bug, it would be cool if someone created an simple testcase, which I can forward to the illumos developers.
> 
> You already have that example. Just take the UTF-8 text in your original
> bug report, put it into something like
> 
> 	int main(int argc, char **argv)
> 	{
> 		char utf8[] = "... your text here...";
> 
> 		printf("%.*s", (int)(sizeof(utf8) - 1), utf8);
> 
> 		return 0;
> 	}
> 
> You should first verify, though, that this replicate the problem, and if
> it does not, use libintl (I think you have to `#include <gettext.h>` and
> `-lintl` or some such) and see whether that reproduces your problem.

Thank you, Johannes for the test case.

However, I don't see any problem with the output on OpenIndiana:

{global} newman@lenovo:~ $ cat printf.c
#include <stdio.h>
//#include <gettext.h>
int main(int argc, char **argv) {
   char utf8[] = "Gergő Mihály Doma\n";
   printf("%.*s", (int)(sizeof(utf8) - 1), utf8);
   return 0;
}

{global} newman@lenovo:~ $ gcc printf.c -o printf && ./printf Gergő Mihály Doma

Enabled the gettext header in the source file.

{global} newman@lenovo:~ $ gcc printf.c -o printf_intl -lintl -I/usr/share/gettext/ && ./printf_intl Gergő Mihály Doma

{global} newman@lenovo:~ $ ldd printf printf_intl
printf:
         libc.so.1 =>     /lib/libc.so.1
         libm.so.2 =>     /lib/libm.so.2
printf_intl:
         libintl.so.1 =>  /lib/libintl.so.1
         libc.so.1 =>     /lib/libc.so.1
         libm.so.2 =>     /lib/libm.so.2

{global} newman@lenovo:~ $ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_ALL=

I even tried more arcane characters from https://www.w3.org/2001/06/utf-8-test/UTF-8-demo.html but they are displayed correctly as well.

Michal

Show 17 quoted lines

> 
> Ciao,
> Johannes
> 
>>
>> Thanks,
>> Michal
>>
>>>
>>> Of course, it might *still* be a bug in libc by virtue of handing '%.*s'
>>> through to libc's implementation.
>>>
>>> Alban, can you test this with NO_GETTEXT?
>>>
>>> Thanks,
>>> Johannes
>>