threads / patch / 37588

v2unblock and unignore SIGPIPE

Subject: [PATCH v2] unblock and unignore SIGPIPE

## tl;dr

6 messages between Sep 18, 2014 and Sep 22, 2014. Diffs are folded; open one to read it.

replies: 5people: 4as markdown or json

Patrick Reynolds· Sep 18, 2014, 16:57 UTC · lore

Blocked and ignored signals -- but not caught signals -- are inherited across exec. Some callers with sloppy signal-handling behavior can call git with SIGPIPE blocked or ignored, even non-deterministically. When SIGPIPE is blocked or ignored, several git commands can run indefinitely, ignoring EPIPE returns from write() calls, even when the process that called them has gone away. Our specific case involved a pipe of git diff-tree output to a script that reads a limited amount of diff data.

In an ideal world, git would never be called with SIGPIPE blocked or ignored. But in the real world, several real potential callers, including Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE ignored. It is easier and more productive to harden git against this mistake than to clean it up in every potential parent process.

Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
1. Merged Junio's work from pu: moved restore_sigpipe_to_default into
git.c and restyled the new tests.
2. Moved the new tests into t0005.  This meant switching back to `git
diff` as our data generator, as the sample repo in t0005 doesn't have any
files for `git ls-files` to output.
3. Squashed.
 git.c              | 22 ++++++++++++++++++++++
 t/t0005-signals.sh | 22 ++++++++++++++++++++++
 2 files changed, 44 insertions(+)
Show changes to 2 files +44 −0

git.c, t/t0005-signals.sh

diff --git a/git.c b/git.c
index 210f1ae..0f03d56 100644
--- a/git.c
+++ b/git.c
@@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
 	return done_alias;
 }
 
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of the pipe to die with SIGPIPE and the downstream process
+ * even knows to check and handle EPIPE correctly.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
 
 int main(int argc, char **av)
 {
@@ -612,6 +632,8 @@ int main(int argc, char **av)
 	 */
 	sanitize_stdfds();
 
+	restore_sigpipe_to_default();
+
 	git_setup_gettext();
 
 	trace_command_performance(argv);
diff --git a/t/t0005-signals.sh b/t/t0005-signals.sh
index 981437b..638a355 100755
--- a/t/t0005-signals.sh
+++ b/t/t0005-signals.sh
@@ -27,4 +27,26 @@ test_expect_success !MINGW 'signals are propagated using shell convention' '
 	test_expect_code 143 git sigterm
 '
 
+large_git () {
+	for i in $(test_seq 1 100)
+	do
+		git diff --cached --binary || return
+	done
+}
+
+test_expect_success 'create blob' '
+	test-genrandom foo 16384 >file &&
+	git add file
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE' '
+	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
 test_done
-- 
2.0.1
James Nylen· Sep 18, 2014, 18:39 UTC · re: Junio C Hamano · lore

Re: [PATCH v2] unblock and unignore SIGPIPE

Today I learned that software can be constipated.
On Thu, Sep 18, 2014 at 12:35 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 5 quoted lines
> Thanks!
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Johannes Sixt· Sep 20, 2014, 08:42 UTC · re: Patrick Reynolds · lore

Re: [PATCH v2] unblock and unignore SIGPIPE

Am 18.09.2014 um 18:57 schrieb Patrick Reynolds:
Show 56 quoted lines
> Blocked and ignored signals -- but not caught signals -- are inherited
> across exec.  Some callers with sloppy signal-handling behavior can call
> git with SIGPIPE blocked or ignored, even non-deterministically.  When
> SIGPIPE is blocked or ignored, several git commands can run indefinitely,
> ignoring EPIPE returns from write() calls, even when the process that
> called them has gone away.  Our specific case involved a pipe of git
> diff-tree output to a script that reads a limited amount of diff data.
> 
> In an ideal world, git would never be called with SIGPIPE blocked or
> ignored.  But in the real world, several real potential callers, including
> Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE
> ignored.  It is easier and more productive to harden git against this
> mistake than to clean it up in every potential parent process.
> 
> Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> 1. Merged Junio's work from pu: moved restore_sigpipe_to_default into
> git.c and restyled the new tests.
> 2. Moved the new tests into t0005.  This meant switching back to `git
> diff` as our data generator, as the sample repo in t0005 doesn't have any
> files for `git ls-files` to output.
> 3. Squashed.
> 
>  git.c              | 22 ++++++++++++++++++++++
>  t/t0005-signals.sh | 22 ++++++++++++++++++++++
>  2 files changed, 44 insertions(+)
> 
> diff --git a/git.c b/git.c
> index 210f1ae..0f03d56 100644
> --- a/git.c
> +++ b/git.c
> @@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
>  	return done_alias;
>  }
>  
> +/*
> + * Many parts of Git have subprograms communicate via pipe, expect the
> + * upstream of the pipe to die with SIGPIPE and the downstream process
> + * even knows to check and handle EPIPE correctly.  Some third-party
> + * programs that ignore or block SIGPIPE for their own reason forget
> + * to restore SIGPIPE handling to the default before spawning Git and
> + * break this carefully orchestrated machinery.
> + *
> + * Restore the way SIGPIPE is handled to default, which is what we
> + * expect.
> + */
> +static void restore_sigpipe_to_default(void)
> +{
> +	sigset_t unblock;
> +
> +	sigemptyset(&unblock);
> +	sigaddset(&unblock, SIGPIPE);
> +	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
> +	signal(SIGPIPE, SIG_DFL);
> +}

This does not build on MinGW due to missing sigaddset() and sigprocmask(). I've a patch that adds dummies for them (but I ran out of time to complete it for submission). But then the test cases ...

Show 9 quoted lines
> +test_expect_success 'a constipated git dies with SIGPIPE' '
> +	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
> +	test "$OUT" -eq 141
> +'
> +
> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
> +	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
> +	test "$OUT" -eq 141
> +'

... fail always because we neither get SIGPIPE (we don't have it on Windows) nor do we see a write error (e.g. EPIPE) when writing to the pipe. Should I protect these tests with !MINGW or would it be an option to drop these tests alltogether?

-- Hannes
Junio C Hamano· Sep 22, 2014, 17:30 UTC · re: Johannes Sixt · lore

Re: [PATCH v2] unblock and unignore SIGPIPE

Johannes Sixt <j6t@kdbg.org> writes:
Show 28 quoted lines
>> +static void restore_sigpipe_to_default(void)
>> +{
>> +	sigset_t unblock;
>> +
>> +	sigemptyset(&unblock);
>> +	sigaddset(&unblock, SIGPIPE);
>> +	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
>> +	signal(SIGPIPE, SIG_DFL);
>> +}
>
> This does not build on MinGW due to missing sigaddset() and
> sigprocmask(). I've a patch that adds dummies for them (but I ran out of
> time to complete it for submission). But then the test cases ...
>
>> +test_expect_success 'a constipated git dies with SIGPIPE' '
>> +	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
>> +	test "$OUT" -eq 141
>> +'
>> +
>> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
>> +	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
>> +	test "$OUT" -eq 141
>> +'
>
> ... fail always because we neither get SIGPIPE (we don't have it on
> Windows) nor do we see a write error (e.g. EPIPE) when writing to the
> pipe. Should I protect these tests with !MINGW or would it be an option
> to drop these tests alltogether?

Let's do !MINGW for now, unless somebody can think of a reason why this change and tests are a bad idea (e.g. "we are not in the business of preventing users from shooting themselves; have the users bug those who wrote the software that spawns us with SIGPIPE ignored", to which I am sympathetic to some degree but not very much because I am also a practical person).

Thanks.
Johannes Sixt· Sep 22, 2014, 18:24 UTC · re: Junio C Hamano · lore

[PATCH] mingw.h: add dummy functions for sigset_t operations

Windows does not have POSIX-like signals, and so we ignore all operations on the non-existent signal mask machinery.

Do not turn sigemptyset into a function, but leave it a macro that erases the code in the argument because it is used to set sa_mask of a struct sigaction, but our dummy in mingw.h does not have that member.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 compat/mingw.h     | 7 ++++++-
 t/t0005-signals.sh | 4 ++--
 2 files changed, 8 insertions(+), 3 deletions(-)
Show changes to 2 files +8 −3

compat/mingw.h, t/t0005-signals.sh

diff --git a/compat/mingw.h b/compat/mingw.h
index 0b5f2fe..0e42653 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -69,7 +69,6 @@ struct sigaction {
 	sig_handler_t sa_handler;
 	unsigned sa_flags;
 };
-#define sigemptyset(x) (void)0
 #define SA_RESTART 0
 
 struct itimerval {
@@ -116,6 +115,12 @@ static inline int fcntl(int fd, int cmd, ...)
 }
 /* bash cannot reliably detect negative return codes as failure */
 #define exit(code) exit((code) & 0xff)
+#define sigemptyset(x) (void)0
+static inline int sigaddset(sigset_t *set, int signum)
+{ return 0; }
+#define SIG_UNBLOCK 0
+static inline int sigprocmask(int how, const sigset_t *set, sigset_t *oldset)
+{ return 0; }
 
 /*
  * simple adaptors
diff --git a/t/t0005-signals.sh b/t/t0005-signals.sh
index 638a355..aeea50c 100755
--- a/t/t0005-signals.sh
+++ b/t/t0005-signals.sh
@@ -39,12 +39,12 @@ test_expect_success 'create blob' '
 	git add file
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE' '
 	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE even if parent ignores it' '
 	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
-- 
2.0.0.12.gbcf935e

← back to recent threads