{"thread":{"id":"37588","subject":"[PATCH v2] unblock and unignore SIGPIPE","startedAt":"2014-09-18T16:57:09Z","lastAt":"2014-09-22T18:24:34Z","messageCount":6,"participants":["Patrick Reynolds","Junio C Hamano","James Nylen","Johannes Sixt"],"isPatch":true,"patchVersion":2,"patchTotal":null},"messages":[{"id":"249639","messageId":"1411059429-23868-1-git-send-email-patrick.reynolds@github.com","threadId":"37588","inReplyTo":null,"subject":"[PATCH v2] unblock and unignore SIGPIPE","fromName":"Patrick Reynolds","fromEmail":"patrick.reynolds@github.com","sentAt":"2014-09-18T16:57:09Z","receivedAt":"2014-09-18T16:57:09Z","isPatch":true,"sender":{"key":"patrick.reynolds@github.com","avatar":"https://gravatar.com/avatar/883ab411a1f4a4d96fd05b139be699507f11dafe679b551194bd269370bffc61?d=mp&s=160"},"body":"Blocked and ignored signals -- but not caught signals -- are inherited\nacross exec.  Some callers with sloppy signal-handling behavior can call\ngit with SIGPIPE blocked or ignored, even non-deterministically.  When\nSIGPIPE is blocked or ignored, several git commands can run indefinitely,\nignoring EPIPE returns from write() calls, even when the process that\ncalled them has gone away.  Our specific case involved a pipe of git\ndiff-tree output to a script that reads a limited amount of diff data.\n\nIn an ideal world, git would never be called with SIGPIPE blocked or\nignored.  But in the real world, several real potential callers, including\nPerl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE\nignored.  It is easier and more productive to harden git against this\nmistake than to clean it up in every potential parent process.\n\nSigned-off-by: Patrick Reynolds <patrick.reynolds@github.com>\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n1. Merged Junio's work from pu: moved restore_sigpipe_to_default into\ngit.c and restyled the new tests.\n2. Moved the new tests into t0005.  This meant switching back to `git\ndiff` as our data generator, as the sample repo in t0005 doesn't have any\nfiles for `git ls-files` to output.\n3. Squashed.\n\n git.c              | 22 ++++++++++++++++++++++\n t/t0005-signals.sh | 22 ++++++++++++++++++++++\n 2 files changed, 44 insertions(+)\n\ndiff --git a/git.c b/git.c\nindex 210f1ae..0f03d56 100644\n--- a/git.c\n+++ b/git.c\n@@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)\n \treturn done_alias;\n }\n \n+/*\n+ * Many parts of Git have subprograms communicate via pipe, expect the\n+ * upstream of the pipe to die with SIGPIPE and the downstream process\n+ * even knows to check and handle EPIPE correctly.  Some third-party\n+ * programs that ignore or block SIGPIPE for their own reason forget\n+ * to restore SIGPIPE handling to the default before spawning Git and\n+ * break this carefully orchestrated machinery.\n+ *\n+ * Restore the way SIGPIPE is handled to default, which is what we\n+ * expect.\n+ */\n+static void restore_sigpipe_to_default(void)\n+{\n+\tsigset_t unblock;\n+\n+\tsigemptyset(&unblock);\n+\tsigaddset(&unblock, SIGPIPE);\n+\tsigprocmask(SIG_UNBLOCK, &unblock, NULL);\n+\tsignal(SIGPIPE, SIG_DFL);\n+}\n \n int main(int argc, char **av)\n {\n@@ -612,6 +632,8 @@ int main(int argc, char **av)\n \t */\n \tsanitize_stdfds();\n \n+\trestore_sigpipe_to_default();\n+\n \tgit_setup_gettext();\n \n \ttrace_command_performance(argv);\ndiff --git a/t/t0005-signals.sh b/t/t0005-signals.sh\nindex 981437b..638a355 100755\n--- a/t/t0005-signals.sh\n+++ b/t/t0005-signals.sh\n@@ -27,4 +27,26 @@ test_expect_success !MINGW 'signals are propagated using shell convention' '\n \ttest_expect_code 143 git sigterm\n '\n \n+large_git () {\n+\tfor i in $(test_seq 1 100)\n+\tdo\n+\t\tgit diff --cached --binary || return\n+\tdone\n+}\n+\n+test_expect_success 'create blob' '\n+\ttest-genrandom foo 16384 >file &&\n+\tgit add file\n+'\n+\n+test_expect_success 'a constipated git dies with SIGPIPE' '\n+\tOUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )\n+\ttest \"$OUT\" -eq 141\n+'\n+\n+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '\n+\tOUT=$( ((trap \"\" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )\n+\ttest \"$OUT\" -eq 141\n+'\n+\n test_done\n-- \n2.0.1\n"},{"id":"249644","messageId":"xmqqvbokomqx.fsf@gitster.dls.corp.google.com","threadId":"37588","inReplyTo":"1411059429-23868-1-git-send-email-patrick.reynolds@github.com","subject":"Re: [PATCH v2] unblock and unignore SIGPIPE","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-09-18T17:35:18Z","receivedAt":"2014-09-18T17:35:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thanks!\n"},{"id":"249646","messageId":"CABVa4NieHi8yuqydU10wghgsfig4XG39kS4Hfa+L5-v1gELpEQ@mail.gmail.com","threadId":"37588","inReplyTo":"xmqqvbokomqx.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH v2] unblock and unignore SIGPIPE","fromName":"James Nylen","fromEmail":"jnylen@gmail.com","sentAt":"2014-09-18T18:39:13Z","receivedAt":"2014-09-18T18:39:13Z","isPatch":true,"sender":{"key":"jnylen@gmail.com","avatar":"https://gravatar.com/avatar/96804ac655933f5b6380e992610d6ff9029c6d04db1042d4bec381312ff7ff1b?d=mp&s=160"},"body":"Today I learned that software can be constipated.\n\nOn Thu, Sep 18, 2014 at 12:35 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Thanks!\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n"},{"id":"249683","messageId":"541D3E0C.4030400@kdbg.org","threadId":"37588","inReplyTo":"1411059429-23868-1-git-send-email-patrick.reynolds@github.com","subject":"Re: [PATCH v2] unblock and unignore SIGPIPE","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2014-09-20T08:42:52Z","receivedAt":"2014-09-20T08:42:52Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 18.09.2014 um 18:57 schrieb Patrick Reynolds:\n> Blocked and ignored signals -- but not caught signals -- are inherited\n> across exec.  Some callers with sloppy signal-handling behavior can call\n> git with SIGPIPE blocked or ignored, even non-deterministically.  When\n> SIGPIPE is blocked or ignored, several git commands can run indefinitely,\n> ignoring EPIPE returns from write() calls, even when the process that\n> called them has gone away.  Our specific case involved a pipe of git\n> diff-tree output to a script that reads a limited amount of diff data.\n> \n> In an ideal world, git would never be called with SIGPIPE blocked or\n> ignored.  But in the real world, several real potential callers, including\n> Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE\n> ignored.  It is easier and more productive to harden git against this\n> mistake than to clean it up in every potential parent process.\n> \n> Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>\n> Signed-off-by: Junio C Hamano <gitster@pobox.com>\n> ---\n> 1. Merged Junio's work from pu: moved restore_sigpipe_to_default into\n> git.c and restyled the new tests.\n> 2. Moved the new tests into t0005.  This meant switching back to `git\n> diff` as our data generator, as the sample repo in t0005 doesn't have any\n> files for `git ls-files` to output.\n> 3. Squashed.\n> \n>  git.c              | 22 ++++++++++++++++++++++\n>  t/t0005-signals.sh | 22 ++++++++++++++++++++++\n>  2 files changed, 44 insertions(+)\n> \n> diff --git a/git.c b/git.c\n> index 210f1ae..0f03d56 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)\n>  \treturn done_alias;\n>  }\n>  \n> +/*\n> + * Many parts of Git have subprograms communicate via pipe, expect the\n> + * upstream of the pipe to die with SIGPIPE and the downstream process\n> + * even knows to check and handle EPIPE correctly.  Some third-party\n> + * programs that ignore or block SIGPIPE for their own reason forget\n> + * to restore SIGPIPE handling to the default before spawning Git and\n> + * break this carefully orchestrated machinery.\n> + *\n> + * Restore the way SIGPIPE is handled to default, which is what we\n> + * expect.\n> + */\n> +static void restore_sigpipe_to_default(void)\n> +{\n> +\tsigset_t unblock;\n> +\n> +\tsigemptyset(&unblock);\n> +\tsigaddset(&unblock, SIGPIPE);\n> +\tsigprocmask(SIG_UNBLOCK, &unblock, NULL);\n> +\tsignal(SIGPIPE, SIG_DFL);\n> +}\n\nThis does not build on MinGW due to missing sigaddset() and\nsigprocmask(). I've a patch that adds dummies for them (but I ran out of\ntime to complete it for submission). But then the test cases ...\n\n> +test_expect_success 'a constipated git dies with SIGPIPE' '\n> +\tOUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )\n> +\ttest \"$OUT\" -eq 141\n> +'\n> +\n> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '\n> +\tOUT=$( ((trap \"\" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )\n> +\ttest \"$OUT\" -eq 141\n> +'\n\n... fail always because we neither get SIGPIPE (we don't have it on\nWindows) nor do we see a write error (e.g. EPIPE) when writing to the\npipe. Should I protect these tests with !MINGW or would it be an option\nto drop these tests alltogether?\n\n-- Hannes\n"},{"id":"249739","messageId":"xmqqwq8vlg01.fsf@gitster.dls.corp.google.com","threadId":"37588","inReplyTo":"541D3E0C.4030400@kdbg.org","subject":"Re: [PATCH v2] unblock and unignore SIGPIPE","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-09-22T17:30:38Z","receivedAt":"2014-09-22T17:30:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt <j6t@kdbg.org> writes:\n\n>> +static void restore_sigpipe_to_default(void)\n>> +{\n>> +\tsigset_t unblock;\n>> +\n>> +\tsigemptyset(&unblock);\n>> +\tsigaddset(&unblock, SIGPIPE);\n>> +\tsigprocmask(SIG_UNBLOCK, &unblock, NULL);\n>> +\tsignal(SIGPIPE, SIG_DFL);\n>> +}\n>\n> This does not build on MinGW due to missing sigaddset() and\n> sigprocmask(). I've a patch that adds dummies for them (but I ran out of\n> time to complete it for submission). But then the test cases ...\n>\n>> +test_expect_success 'a constipated git dies with SIGPIPE' '\n>> +\tOUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )\n>> +\ttest \"$OUT\" -eq 141\n>> +'\n>> +\n>> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '\n>> +\tOUT=$( ((trap \"\" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )\n>> +\ttest \"$OUT\" -eq 141\n>> +'\n>\n> ... fail always because we neither get SIGPIPE (we don't have it on\n> Windows) nor do we see a write error (e.g. EPIPE) when writing to the\n> pipe. Should I protect these tests with !MINGW or would it be an option\n> to drop these tests alltogether?\n\nLet's do !MINGW for now, unless somebody can think of a reason why\nthis change and tests are a bad idea (e.g. \"we are not in the\nbusiness of preventing users from shooting themselves; have the\nusers bug those who wrote the software that spawns us with SIGPIPE\nignored\", to which I am sympathetic to some degree but not very much\nbecause I am also a practical person).\n\nThanks.\n"},{"id":"249746","messageId":"54206962.9070005@kdbg.org","threadId":"37588","inReplyTo":"xmqqwq8vlg01.fsf@gitster.dls.corp.google.com","subject":"[PATCH] mingw.h: add dummy functions for sigset_t operations","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2014-09-22T18:24:34Z","receivedAt":"2014-09-22T18:24:34Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Windows does not have POSIX-like signals, and so we ignore all\noperations on the non-existent signal mask machinery.\n\nDo not turn sigemptyset into a function, but leave it a macro that\nerases the code in the argument because it is used to set sa_mask\nof a struct sigaction, but our dummy in mingw.h does not have that\nmember.\n\nSigned-off-by: Johannes Sixt <j6t@kdbg.org>\n---\n compat/mingw.h     | 7 ++++++-\n t/t0005-signals.sh | 4 ++--\n 2 files changed, 8 insertions(+), 3 deletions(-)\n\ndiff --git a/compat/mingw.h b/compat/mingw.h\nindex 0b5f2fe..0e42653 100644\n--- a/compat/mingw.h\n+++ b/compat/mingw.h\n@@ -69,7 +69,6 @@ struct sigaction {\n \tsig_handler_t sa_handler;\n \tunsigned sa_flags;\n };\n-#define sigemptyset(x) (void)0\n #define SA_RESTART 0\n \n struct itimerval {\n@@ -116,6 +115,12 @@ static inline int fcntl(int fd, int cmd, ...)\n }\n /* bash cannot reliably detect negative return codes as failure */\n #define exit(code) exit((code) & 0xff)\n+#define sigemptyset(x) (void)0\n+static inline int sigaddset(sigset_t *set, int signum)\n+{ return 0; }\n+#define SIG_UNBLOCK 0\n+static inline int sigprocmask(int how, const sigset_t *set, sigset_t *oldset)\n+{ return 0; }\n \n /*\n  * simple adaptors\ndiff --git a/t/t0005-signals.sh b/t/t0005-signals.sh\nindex 638a355..aeea50c 100755\n--- a/t/t0005-signals.sh\n+++ b/t/t0005-signals.sh\n@@ -39,12 +39,12 @@ test_expect_success 'create blob' '\n \tgit add file\n '\n \n-test_expect_success 'a constipated git dies with SIGPIPE' '\n+test_expect_success !MINGW 'a constipated git dies with SIGPIPE' '\n \tOUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )\n \ttest \"$OUT\" -eq 141\n '\n \n-test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '\n+test_expect_success !MINGW 'a constipated git dies with SIGPIPE even if parent ignores it' '\n \tOUT=$( ((trap \"\" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )\n \ttest \"$OUT\" -eq 141\n '\n-- \n2.0.0.12.gbcf935e\n"}]}