threads / rfc / 27930

RFC patchnotice error exit from pager

Subject: [PATCH/RFC] notice error exit from pager

## tl;dr

7 messages between Jul 26, 2011 and Aug 1, 2011. Diffs are folded; open one to read it.

replies: 6people: 4as markdown or json

Clemens Buchacher· Jul 26, 2011, 21:04 UTC · lore
If the pager fails to run, git produces no output, e.g.:
 $ GIT_PAGER=not-a-command git log
The error reporting fails for two reasons:
 (1) start_command: There is a mechanism that detects errors during
     execvp introduced in 2b541bf8 (start_command: detect execvp
     failures early). The child writes one byte to a pipe only if
     execvp fails.  The parent waits for either EOF, when the
     successful execvp automatically closes the pipe (see
     FD_CLOEXEC in fcntl(1)), or it reads a single byte, in which
     case it knows that the execvp failed. This mechanism is
     incompatible with the workaround introduced in 35ce8622
     (pager: Work around window resizing bug in 'less'), which
     waits for input from the parent before the exec. Since both
     the parent and the child are waiting for input from each
     other, that would result in a deadlock. In order to avoid
     that, the mechanism is disabled by closing the child_notifier
     file descriptor. That is my best guess, at least, from reading
     the code. Hannes, can you confirm if this analysis is correct?
 (2) finish_command: The parent correctly detects the 127 exit
     status from the child, but the error output goes nowhere,
     since by that time it is already being redirected to the
     child.
No simple solution for (1) comes to mind. I am not sure if this bug
is still relevant. I could not reproduce it here with less version
436. If this bug was fixed a long time ago, maybe we can remove the
workaround by now?

Number (2) can be solved by not sending error output to the pager. But I am not completely satisfied with that solution. Not redirecting error output to the pager can result in the pager overwriting error output with standard output. Right now, I can't think of any common situations where this would be the case. But, for example, a git log on a broken repo with some missing commits will abort at some point with "error: could not read <sha1> [...]". With this patch, if there is at least one page of standard output before that error, the error message is not visible at all and the output simply stops as if git log had reached the root commit.

The easiest solution is to produce the error output in the child. Below is a patch for that. The previous code seems to go through some pain to let the parent handle the error output. But I don't see the point. Is there more to this than meets my eye?

Clemens
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---
 run-command.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)
Show changes to run-command.c +6 −5
diff --git a/run-command.c b/run-command.c
index 70e8a24..944a882 100644
--- a/run-command.c
+++ b/run-command.c
@@ -127,9 +127,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
 		if (code == 127) {
 			code = -1;
 			failed_errno = ENOENT;
-			if (!silent_exec_failure)
-				error("cannot run %s: %s", argv0,
-					strerror(ENOENT));
 		}
 	} else {
 		error("waitpid is confused (%s)", argv0);
@@ -287,10 +284,14 @@ fail_pipe:
 		 * Do not check for cmd->silent_exec_failure; the parent
 		 * process will check it when it sees this exit code.
 		 */
-		if (errno == ENOENT)
+		if (errno == ENOENT) {
+			if (!cmd->silent_exec_failure)
+				error("cannot run %s: %s", cmd->argv[0],
+					strerror(ENOENT));
 			exit(127);
-		else
+		} else {
 			die_errno("cannot exec '%s'", cmd->argv[0]);
+		}
 	}
 	if (cmd->pid < 0)
 		error("cannot fork() for %s: %s", cmd->argv[0],

-- 
1.7.3.1.105.g84915
Linus Torvalds· Jul 26, 2011, 21:35 UTC · re: Clemens Buchacher · lore

Re: [PATCH/RFC] notice error exit from pager

On Tue, Jul 26, 2011 at 2:04 PM, Clemens Buchacher <drizzd@aon.at> wrote:
Show 5 quoted lines
>
> No simple solution for (1) comes to mind. I am not sure if this bug
> is still relevant. I could not reproduce it here with less version
> 436. If this bug was fixed a long time ago, maybe we can remove the
> workaround by now?

Hmm. I can confirm that it doesn't happen for me either any more, although I don't know if that is an actual less bugfix, or it was timing-dependent.

That said, the workaround has other nice properties too, like delaying the screen cleaning that "less" often does. So I don't know if we really want to get rid of it.

> Number (2) can be solved by not sending error output to the pager.
I think your patch looks reasonable.
                  Linus
Johannes Sixt· Jul 27, 2011, 19:36 UTC · re: Clemens Buchacher · lore

Re: [PATCH/RFC] notice error exit from pager

Am 26.07.2011 23:04, schrieb Clemens Buchacher:
Show 11 quoted lines
> If the pager fails to run, git produces no output, e.g.:
> 
>  $ GIT_PAGER=not-a-command git log
> 
> The error reporting fails for two reasons:
> 
>  (1) start_command: There is a mechanism that detects errors during
>      execvp introduced in 2b541bf8 (start_command: detect execvp
>      failures early). ... This mechanism is
>      incompatible with the workaround introduced in 35ce8622
>      (pager: Work around window resizing bug in 'less')

You analysis is correct. I think the bug in less was fixed shortly after this workaround was introduced. I would like to remove this workaround for a different reason, but it seems there are people who are quite fond of it ;)

Show 23 quoted lines
> diff --git a/run-command.c b/run-command.c
> index 70e8a24..944a882 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -127,9 +127,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
>  		if (code == 127) {
>  			code = -1;
>  			failed_errno = ENOENT;
> -			if (!silent_exec_failure)
> -				error("cannot run %s: %s", argv0,
> -					strerror(ENOENT));
>  		}
>  	} else {
>  		error("waitpid is confused (%s)", argv0);
> @@ -287,10 +284,14 @@ fail_pipe:
>  		 * Do not check for cmd->silent_exec_failure; the parent
>  		 * process will check it when it sees this exit code.
>  		 */
> -		if (errno == ENOENT)
> +		if (errno == ENOENT) {
> +			if (!cmd->silent_exec_failure)
> +				error("cannot run %s: %s", cmd->argv[0],
> +					strerror(ENOENT));

This change is not good enough: There is no guarantee that this message goes to the right channel (stderr in the child can be redirected). Look carefully: We set a special die routine in the forked child that ensures that the die() messages are written to the right channel; we do not have a similar facility for error() messages (or do we?).

Show 8 quoted lines
>  			exit(127);
> -		else
> +		} else {
>  			die_errno("cannot exec '%s'", cmd->argv[0]);
> +		}
>  	}
>  	if (cmd->pid < 0)
>  		error("cannot fork() for %s: %s", cmd->argv[0],
-- Hannes
Clemens Buchacher· Jul 27, 2011, 21:32 UTC · re: Johannes Sixt · lore

[PATCH] error_routine: use parent's stderr if exec fails

The new process's error output may be redirected elsewhere, but if the exec fails, output should still go to the parent's stderr. This has already been done for the die_routine. Do the same for error_routine.

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---
On Wed, Jul 27, 2011 at 09:36:52PM +0200, Johannes Sixt wrote:
> 
> This change is not good enough: There is no guarantee that this message
> goes to the right channel (stderr in the child can be redirected).

Ah, good point. I skipped over that part thinking it was not relevant. This should take care of it.

Clemens
 git-compat-util.h |    2 ++
 run-command.c     |   15 +++++++--------
 usage.c           |   18 ++++++++++++++++++
 3 files changed, 27 insertions(+), 8 deletions(-)
Show changes to 3 files +27 −8

git-compat-util.h, run-command.c, usage.c

diff --git a/git-compat-util.h b/git-compat-util.h
index a75530d..07cbfe9 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -240,6 +240,7 @@ extern char *gitbasename(char *);
 
 /* General helper functions */
 extern void vreportf(const char *prefix, const char *err, va_list params);
+extern void vwritef(int fd, const char *prefix, const char *err, va_list params);
 extern NORETURN void usage(const char *err);
 extern NORETURN void usagef(const char *err, ...) __attribute__((format (printf, 1, 2)));
 extern NORETURN void die(const char *err, ...) __attribute__((format (printf, 1, 2)));
@@ -248,6 +249,7 @@ extern int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 extern void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 
 extern void set_die_routine(NORETURN_PTR void (*routine)(const char *err, va_list params));
+extern void set_error_routine(void (*routine)(const char *err, va_list params));
 
 extern int prefixcmp(const char *str, const char *prefix);
 extern int suffixcmp(const char *str, const char *suffix);
diff --git a/run-command.c b/run-command.c
index c752174..a2796c4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -77,16 +77,14 @@ static void notify_parent(void)
 
 static NORETURN void die_child(const char *err, va_list params)
 {
-	char msg[4096];
-	int len = vsnprintf(msg, sizeof(msg), err, params);
-	if (len > sizeof(msg))
-		len = sizeof(msg);
-
-	write_in_full(child_err, "fatal: ", 7);
-	write_in_full(child_err, msg, len);
-	write_in_full(child_err, "\n", 1);
+	vwritef(child_err, "fatal: ", err, params);
 	exit(128);
 }
+
+static void error_child(const char *err, va_list params)
+{
+	vwritef(child_err, "error: ", err, params);
+}
 #endif
 
 static inline void set_cloexec(int fd)
@@ -214,6 +212,7 @@ fail_pipe:
 			set_cloexec(child_err);
 		}
 		set_die_routine(die_child);
+		set_error_routine(error_child);
 
 		close(notify_pipe[0]);
 		set_cloexec(notify_pipe[1]);
diff --git a/usage.c b/usage.c
index b5e67e3..a2a6678 100644
--- a/usage.c
+++ b/usage.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "git-compat-util.h"
+#include "cache.h"
 
 void vreportf(const char *prefix, const char *err, va_list params)
 {
@@ -12,6 +13,18 @@ void vreportf(const char *prefix, const char *err, va_list params)
 	fprintf(stderr, "%s%s\n", prefix, msg);
 }
 
+void vwritef(int fd, const char *prefix, const char *err, va_list params)
+{
+	char msg[4096];
+	int len = vsnprintf(msg, sizeof(msg), err, params);
+	if (len > sizeof(msg))
+		len = sizeof(msg);
+
+	write_in_full(fd, prefix, strlen(prefix));
+	write_in_full(fd, msg, len);
+	write_in_full(fd, "\n", 1);
+}
+
 static NORETURN void usage_builtin(const char *err, va_list params)
 {
 	vreportf("usage: ", err, params);
@@ -46,6 +59,11 @@ void set_die_routine(NORETURN_PTR void (*routine)(const char *err, va_list param
 	die_routine = routine;
 }
 
+void set_error_routine(void (*routine)(const char *err, va_list params))
+{
+	error_routine = routine;
+}
+
 void NORETURN usagef(const char *err, ...)
 {
 	va_list params;

-- 
1.7.3.1.105.g84915
Clemens Buchacher· Aug 1, 2011, 17:59 UTC · re: Clemens Buchacher · lore

[PATCH] notice error exit from pager

If the pager fails to run, git produces no output, e.g.:
 $ GIT_PAGER=not-a-command git log
The error reporting fails for two reasons:
 (1) start_command: There is a mechanism that detects errors during
     execvp introduced in 2b541bf8 (start_command: detect execvp
     failures early). The child writes one byte to a pipe only if
     execvp fails.  The parent waits for either EOF, when the
     successful execvp automatically closes the pipe (see
     FD_CLOEXEC in fcntl(1)), or it reads a single byte, in which
     case it knows that the execvp failed. This mechanism is
     incompatible with the workaround introduced in 35ce8622
     (pager: Work around window resizing bug in 'less'), which
     waits for input from the parent before the exec. Since both
     the parent and the child are waiting for input from each
     other, that would result in a deadlock. In order to avoid
     that, the mechanism is disabled by closing the child_notifier
     file descriptor.
 (2) finish_command: The parent correctly detects the 127 exit
     status from the child, but the error output goes nowhere,
     since by that time it is already being redirected to the
     child.
No simple solution for (1) comes to mind.

Number (2) can be solved by not sending error output to the pager. Not redirecting error output to the pager can result in the pager overwriting error output with standard output, however.

Since there is no reliable way to handle error reporting in the parent, produce the output in the child instead.

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---

Nothing new compared to the RFC, just a slightly trimmed commit message.

 run-command.c |   15 ++++++---------
 1 files changed, 6 insertions(+), 9 deletions(-)
Show changes to run-command.c +6 −9
diff --git a/run-command.c b/run-command.c
index 5c91f37..a2796c4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -125,9 +125,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
 		if (code == 127) {
 			code = -1;
 			failed_errno = ENOENT;
-			if (!silent_exec_failure)
-				error("cannot run %s: %s", argv0,
-					strerror(ENOENT));
 		}
 	} else {
 		error("waitpid is confused (%s)", argv0);
@@ -282,14 +279,14 @@ fail_pipe:
 		} else {
 			execvp(cmd->argv[0], (char *const*) cmd->argv);
 		}
-		/*
-		 * Do not check for cmd->silent_exec_failure; the parent
-		 * process will check it when it sees this exit code.
-		 */
-		if (errno == ENOENT)
+		if (errno == ENOENT) {
+			if (!cmd->silent_exec_failure)
+				error("cannot run %s: %s", cmd->argv[0],
+					strerror(ENOENT));
 			exit(127);
-		else
+		} else {
 			die_errno("cannot exec '%s'", cmd->argv[0]);
+		}
 	}
 	if (cmd->pid < 0)
 		error("cannot fork() for %s: %s", cmd->argv[0],

-- 
1.7.3.1.105.g84915
Junio C Hamano· Aug 1, 2011, 20:17 UTC · re: Clemens Buchacher · lore

Re: [PATCH] notice error exit from pager

Clemens Buchacher <drizzd@aon.at> writes:
> Since there is no reliable way to handle error reporting in the
> parent, produce the output in the child instead.

Hmm, how does this interact with your earlier "error_routine: use parent's stderr if exec fails" patch?

Show 35 quoted lines
> diff --git a/run-command.c b/run-command.c
> index 5c91f37..a2796c4 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -125,9 +125,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
>  		if (code == 127) {
>  			code = -1;
>  			failed_errno = ENOENT;
> -			if (!silent_exec_failure)
> -				error("cannot run %s: %s", argv0,
> -					strerror(ENOENT));
>  		}
>  	} else {
>  		error("waitpid is confused (%s)", argv0);
> @@ -282,14 +279,14 @@ fail_pipe:
>  		} else {
>  			execvp(cmd->argv[0], (char *const*) cmd->argv);
>  		}
> -		/*
> -		 * Do not check for cmd->silent_exec_failure; the parent
> -		 * process will check it when it sees this exit code.
> -		 */
> -		if (errno == ENOENT)
> +		if (errno == ENOENT) {
> +			if (!cmd->silent_exec_failure)
> +				error("cannot run %s: %s", cmd->argv[0],
> +					strerror(ENOENT));
>  			exit(127);
> -		else
> +		} else {
>  			die_errno("cannot exec '%s'", cmd->argv[0]);
> +		}
>  	}
>  	if (cmd->pid < 0)
>  		error("cannot fork() for %s: %s", cmd->argv[0],
Clemens Buchacher· Aug 1, 2011, 20:35 UTC · re: Junio C Hamano · lore

Re: [PATCH] notice error exit from pager

On Mon, Aug 01, 2011 at 01:17:28PM -0700, Junio C Hamano wrote:
Show 7 quoted lines
> Clemens Buchacher <drizzd@aon.at> writes:
> 
> > Since there is no reliable way to handle error reporting in the
> > parent, produce the output in the child instead.
> 
> Hmm, how does this interact with your earlier "error_routine: use parent's
> stderr if exec fails" patch?

The use case addressed in this patch (i.e., the pager) does not strictly dependent on the previous patch. But Johannes noted that in general, error output of the child could be redirected, hiding the execve failure again. The error_routine patch avoids that by writing to the parent's stderr stream.

Clemens

← back to recent threads