From: Johannes Sixt <j6t@kdbg.org>
Date: Mon, 19 Dec 2016 19:34:30 GMT
Subject: Re: [PATCH] winansi_isatty(): fix when Git is used from CMD
Message-ID: <3f06ea33-b4de-48b4-593e-239eb6e87dd4@kdbg.org>
In-Reply-To: <d661dbf1-9852-965a-2ca9-67d763115b9e@kdbg.org>

Am 18.12.2016 um 16:37 schrieb Johannes Sixt:
> winansi.c is all about overriding MSVCRT's console handling. If we are
> connected to a console, then by the time isatty() is called (from
> outside the emulation layer), all handling of file descriptors 1 and 2
> is already outside MSVCRT's control. In particular, we have determined
> unambiguously whether a terminal is connected (see is_console()). I
> suggest to have the implementation below (on top of the patch I'm
> responding to).
>
> What do you think?

I thought a bit more about this approach, and I retract it. I think it 
does not work when Git is connected to an MSYS TTY, i.e., when the 
"console" is in reality the pipe that is detected in detect_msys_tty().

At the same time I wonder how your original winansi_isatty() could have 
worked: In this case, MSVCRT's isatty() would return 1 (because 
detect_msys_tty() has set things up that this happens), but then 
winansi_isatty() checks whether the handle underlying fd 0, 1 or 2 is a 
real Windows console. But it is not: it is a pipe. Am I missing something?

>
> diff --git a/compat/winansi.c b/compat/winansi.c
> index ba360be69b..1748d17777 100644
> --- a/compat/winansi.c
> +++ b/compat/winansi.c
> @@ -575,9 +575,8 @@ static void detect_msys_tty(int fd)
>
>  int winansi_isatty(int fd)
>  {
> -	int res = isatty(fd);
> -
> -	if (res) {
> +	switch (fd) {
> +	case 0:
>  		/*
>  		 * Make sure that /dev/null is not fooling Git into believing
>  		 * that we are connected to a terminal, as "_isatty() returns a
> @@ -586,21 +585,19 @@ int winansi_isatty(int fd)
>  		 *
>  		 * https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx
>  		 */
> -		HANDLE handle = winansi_get_osfhandle(fd);
> -		if (fd == STDIN_FILENO) {
> +		{
> +			HANDLE handle = (HANDLE)_get_osfhandle(fd);
>  			DWORD dummy;
>
> -			if (!GetConsoleMode(handle, &dummy))
> -				res = 0;
> -		} else if (fd == STDOUT_FILENO || fd == STDERR_FILENO) {
> -			CONSOLE_SCREEN_BUFFER_INFO dummy;
> -
> -			if (!GetConsoleScreenBufferInfo(handle, &dummy))
> -				res = 0;
> +			return !!GetConsoleMode(handle, &dummy);
>  		}
> +	case 1:
> +		return !!hconsole1;
> +	case 2:
> +		return !!hconsole2;
>  	}
>
> -	return res;
> +	return isatty(fd);
>  }
>
>  void winansi_init(void)
>