[PATCH 1/1] mingw: fix isatty() after dup2()

From: Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com>
Date: Oct 30, 2018, 18:44 UTC
Message-ID: <dbea6293c72d7f94fd6114a8de3b0b9a7fa09bba.1540925078.git.gitgitgadget@gmail.com>
In-Reply-To: <pull.61.git.gitgitgadget@gmail.com>

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Since a9b8a09c3c30 (mingw: replace isatty() hack, 2016-12-22), we handle isatty() by special-casing the stdin/stdout/stderr file descriptors, caching the return value. However, we missed the case where dup2() overrides the respective file descriptor.

That poses a problem e.g. where the `show` builtin asks for a pager very early, the `setup_pager()` function sets the pager depending on the return value of `isatty()` and then redirects stdout. Subsequently, `cmd_log_init_finish()` calls `setup_pager()` *again*. What should happen now is that `isatty()` reports that stdout is *not* a TTY and consequently stdout should be left alone.

Let's override dup2() to handle this appropriately.

This fixes https://github.com/git-for-windows/git/issues/1077

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 compat/mingw.h   |  3 +++
 compat/winansi.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/compat/mingw.h b/compat/mingw.h
index f31dcff2b..b04556ce0 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -390,6 +390,9 @@ int mingw_raise(int sig);
 int winansi_isatty(int fd);
 #define isatty winansi_isatty
 
+int winansi_dup2(int oldfd, int newfd);
+#define dup2 winansi_dup2
+
 void winansi_init(void);
 HANDLE winansi_get_osfhandle(int fd);
 
diff --git a/compat/winansi.c b/compat/winansi.c
index a11a0f16d..f4f08237f 100644
--- a/compat/winansi.c
+++ b/compat/winansi.c
@@ -474,6 +474,18 @@ static void die_lasterr(const char *fmt, ...)
 	va_end(params);
 }
 
+#undef dup2
+int winansi_dup2(int oldfd, int newfd)
+{
+	int ret = dup2(oldfd, newfd);
+
+	if (!ret && newfd >= 0 && newfd <= 2)
+		fd_is_interactive[newfd] = oldfd < 0 || oldfd > 2 ?
+			0 : fd_is_interactive[oldfd];
+
+	return ret;
+}
+
 static HANDLE duplicate_handle(HANDLE hnd)
 {
 	HANDLE hresult, hproc = GetCurrentProcess();

-- 
gitgitgadget