How to get bash to shut up about SIGPIPE?

Right now my major gripe with cogito is "cg-log" (which is actually the only command I use right now, everything else I just do by hand with the raw git archive) is that bash is being an ass about SIGPIPE, and when I only look at the top part of the log, ie I do something like:

	torvalds@ppc970:~/src/cogito> git log | head

it spews out the ten first lines of the log, and then it spews about a million lines (well, 20, but anyway) of crap:

	/home/torvalds/bin/cg-log: line 87:  6338 Done                    echo -n $color$key $rest
	      6339 Broken pipe             | sed "s/>.*/> ${pdate/+0000/$tz}/"
	/home/torvalds/bin/cg-log: line 87:  6328 Done                    cat-file commit $commit
	      6329 Broken pipe             | while read key rest; do
	    case "$key" in 
	        "author" | "committer")
	            if [ "$key" = "author" ]; then
	                color="$colauthor";
	            else
	                color="$colcommitter";
	            fi; date=(${rest#*> }); sec=${date[0]}; tz=${date[1]}; dtz=${tz/+/}; lsec=$(expr $dtz / 100 \* 3600 + $dtz % 100 \* 60 + $sec); pdate="$(date -Rud "1970-01-01 UTC + $lsec sec" 2>/dev/null)"; if [ "$pdate" ]; then
	                echo -n $color$key $rest | sed "s/>.*/> ${pdate/+0000/$tz}/"; echo $coldefault;
	            else
	                echo $color$key $rest $coldefault;
	            fi
	        ;;
	        "")
	            echo; sed -re '
	                                        / 
	*Signed-off-by:.*/Is//'$colsignoff'&'$coldefault'/
	                                        s/./    &/
	                                '
	        ;;
	        *)
	            echo $colheader$key $rest $coldefault
	        ;;
	    esac;
	done

which is just incredibly annoying, since it makes the ten lines I actually _wanted_ to get scroll off the screen..

Damn bash. What's the magic incantation that says SHUT UP!?

		Linus

Linus Torvalds schrieb:

I think you misspelled "cg-log". :-D

Doing "cg-log | head" gives me 10 lines of log and nothing else. Maybe the problem has been fixed between 0.7 and the current version I'm using (commit ID 49612c471eebd26efe926a71752e254c1cdc382d)?

Rene

On Thu, 28 Apr 2005, Rene Scharfe wrote:

My fingers have a hard time learning new patterns, so I've got:

	torvalds@ppc970:~/git> cat ~/bin/git   
	#!/bin/sh
	cmd="cg-$1"
	shift
	$cmd "$@"

until my fingers learn the new thing.

no, this is current as of an hour ago, same head you have.

It's a bash-specific thing, and depends on how you compiled bash.

Defining "DONT_REPORT_SIGPIPE" in config-top.h when building bash gets rid of it, but I really don't want to rebuild bash just because of this ;)

		Linus

Linus Torvalds schrieb:

As a workaround, add this line after the shift to suppress the ugly messages:

	[ "$cmd" = cg-log ] && exec 2>/dev/null

Yes, I just compiled it and in the version from gnu.org this defaults to not being defined. Man, am I glad my distro uncommented that line. :-)

Looking briefly at the source there doesn't seem to be a way to turn it off besides this compile time setting. I really wonder why..

Rene

On Thu, Apr 28, 2005 at 12:21:26PM -0700, Linus Torvalds wrote:

Debian's bash seems to have that set, so it's a bit hard for me to test (I don't really have any non-Debian machines around these days for some reason.)

Try adding "set -e" to the beginning of cg-log.

That may cause some *other* problems, but I think it will fix this problem.

On Thu, 28 Apr 2005, Ryan Anderson wrote:

Nope. I actually downloaded bash-3.0, and it literally seem sto be impossible to get bash to not do it.

It has some noise there about signal trapping, but that doesn't actually seem to work.

		Linus

Ryan Anderson <ryan@michonline.com> wrote:

This issue has been around for years. The discussion that led to Debian setting this option may be helpful in understanding it:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=10494

A brief time line:

11 Jun 1997 The issue was reported to Debian.
20 Nov 1999 Chet Ramey remarks that bash's default will not change.
 4 Sep 2004 Debian sets DONT_REPORT_SIGPIPE.

Chet Ramey claims that setting DONT_REPORT_SIGPIPE by default would make bash incompatible with every other shell out there. Interestingly, all the non-bash shells that I've tried disagree with him.

Cheers,

Herbert Xu wrote:

Thanks for the pointer. That discussion points to some alternative fixes that may be more useful (instead of installing a repaired shell).

One approach is to install a trap for SIGPIPE in
non-terminating command in a pipeline where the
later items might not process all the data, e.g.:
   (trap {} SIGPIPE; find .) | head -1

<rant> THIS IS A REALLY, REALLY BAD DECISION BY THE BASH TEAM. Why should the default be "create annoying spurious error report?". This should at least be a run-time settable option, with the OPPOSITE default. </rant>

Which is sad, bash is generally reasonably good. I guess now I have to say "bash, once properly configured using DONT_REPORT_SIGPIPE, is reasonably good."

--- David A. Wheeler

David wrote:

Both the versions of bash that I looked at (2.05 and 3.0) _still_ complain even if SIGPIPE is trapped - they just complain with a more terse message, unless DONT_REPORT_SIGPIPE is not defined.

Linus's version apparently isn't even more terse with this trap.

What bash do you have that this trap silences?

agreed

I reported:

Paul Jackson wrote:

...

Actually, I have a working bash, so I can't test any of these work-arounds. I was merely foolish enough to quote the Debian discussion, where someone reported that as a work-around. I had hopes that it would help. That'll teach me to pay attention to documentation :-).

I wonder, does a top-level trap work? E.g.:
  trap "" SIGPIPE
  ...

Anyone, good luck to those with broken bashes...

--- David A. Wheeler

On Tue, 3 May 2005, David A. Wheeler wrote:

No.

But putting the traps _inside_ the loops seems to help. So something like the appended at least makes it somewhat useful

And yes, you need them at both levels, it appears. Or maybe that just changes the timing enough. Whatever.

			Linus
----

Linus Torvalds <torvalds@osdl.org> wrote:

It's not the loop that's important, but the subshell where the signal is caught. Every time you enter a subshell all traps are reset. Since anything that comes (before or) after a pipe symbol goes into a subshell...

You need it in both places because the signal may be received in either place, depending on whether it's the echo or something inside the loop that dies while writing output.

Cheers,

Linus Torvalds wrote:

Try this:

+	{ set -e; trap 'exit 1' SIGPIPE
	$revls | $revsort | while read time commit parents; do
		[ "$revfmt" = "rev-list" ] && commit="$time"
		...
-	done | ${PAGER:-less} ${PAGER_FLAGS:--R}
+	done; } | ${PAGER:-less} ${PAGER_FLAGS:--R}

Ciao, ET.

On Thu, 28 Apr 2005, Edgar Toernig wrote:

Nope. No difference what-so-ever.

Side note: it's timing-dependent, so apparently especially on SMP, sometimes it doesn't complain, and then I think soemthing worked. Then I try again, and it's always back.

I've made a bug-report against bash.

		Linus

On 28 April 2005 14:28, you wrote:

If cg-log is all you use, then you could get away with using yagf:
  rsync://node1.wunjo.org/yagf.git

Features of cg-log missing in yagf log:
  * colors
  * sigpipe gripes ;)
  * #!/bin/bash (or /usr/bin/env bash)
  * ability to log between two commits with rev-tree (it's a planned feature
    in the near future.)

Cheers, Josh

You have a multiprocessor, don't you.

The following silly little shell script will provoke the bash SIGPIPE complaint reliably on a multiprocessor. It writes a big file, twice, from a for-loop in a separate bash subshell through a pipe to a command that exits after seeing one line.

Code Sample 1:

    #!/bin/bash
    for x in 1 2
    do
        cat /etc/termcap	# a big text file
    done | sed 1q

Adding a right trap _inside_ the shell loop that is _before_ the pipe will reduce the verbosity of the complaint substantially (not show the line number and text for each command inside the loop that is killed by the SIGPIPE; rather just show the simple "Broken pipe" error message):

Code Sample 2:

    #!/bin/bash
    for x in 1 2
    do
	trap continue PIPE	# reduce broken pipe screeching
	cat /etc/termcap	# a big text file
    done | sed 1q

Then wrapping the entire pipeline (now that the bogus output is a constant "Broken pipe" string) in the following manner will filter out just that noise, leaving whatever else was on stdout and/or stderr unscathed:

Code Sample 3:

    #!/bin/bash
    ( ( (
	for x in 1 2
	do
		trap continue PIPE      # reduce broken pipe screeching
		cat /etc/termcap        # a big text file
	done | sed 1q
    ) 1>&3 ) 2>&1 | grep -vxF 'Broken pipe' 1>&2 ) 3>&1

The following patch to bash jobs.c will enable "Code Sample 2" to do the right thing, without depending (so much) on the DONT_REPORT_SIGPIPE compile time flag. With this patch, you don't have to go all the way to the baroque code in "Code Sample 3" to shut bash up. Just a well placed trap is sufficient.

Whether or not this is actually worth persuing (or was even worth reading ;) I don't know.

--- jobs.c.orig 2001-03-26 10:08:24.000000000 -0800 +++ jobs.c 2005-04-29 17:09:44.294763496 -0700

On Fri, 29 Apr 2005, Paul Jackson wrote:

Yes. 

Yup:

	t: line 5:  3853 Broken pipe             cat /etc/termcap
	t: line 5:  3855 Broken pipe             cat /etc/termcap

Didn't change anything for me. Same thing.

It will also grep out any occurrence of "Broken pipe", so if we're talking about a kernel changelog, where we fix a pipe bug...

I don't know why the bash people have that stupid pipe reporting in the first place, considering that no other shell seems to do it, and everybody just wants to shut it up..

		Linus

Linus replied to pj:

I don't believe you did what I did.

The source code for bash, both 2.x and 3.x versions, clearly displays a simpler error message (no line number or redisplay of your script commands) in the case that you set a trap. And I tested both shells on a multiprocessor, to verify that they behaved as I expected, running these silly little scripts.

To labour the point, just now on a multiprocessor near me, the following six line script:

    ======================== begin ========================
    #!/usr/people/pj/etc/bash/bash-3.0/bash
    for x in 1 2
    do
	    trap continue PIPE      # reduce broken pipe screeching
	    cat /etc/termcap        # a big text file
    done | sed 1q
    ========================= end =========================

produced the following three lines of output:

    ======================== begin ========================
    ######## TERMINAL TYPE DESCRIPTIONS SOURCE FILE
    Broken pipe
    Broken pipe
    ========================= end =========================

whereas the following five line script (no trap):

    ======================== begin ========================
    #!/usr/people/pj/etc/bash/bash-3.0/bash
    for x in 1 2
    do
	    cat /etc/termcap        # a big text file
    done | sed 1q
    ========================= end =========================

produced the following three __noisier__ lines of output:

    ======================== begin ========================
    ######## TERMINAL TYPE DESCRIPTIONS SOURCE FILE
    foo: line 2: 11663 Broken pipe             cat /etc/termcap
    foo: line 2: 11665 Broken pipe             cat /etc/termcap
    ========================= end =========================

No no no. You didn't read the code or the comment ;). That or my code and comment were both unclear ... far more likely.

The following line noise is not plagarized from last years Obfuscated Perl Contest:

    ( ( (
	...
    ) 1>&3 ) 2>&1 | grep -vxF 'Broken pipe' 1>&2 ) 3>&1

It's the magic shell incantation to run grep only on the stderr stream, while passing through the stdout stream untouched. Even on the stderr stream, it only zaps lines that are exactly the eleven characters 'Broken pipe' (plus newline).

Now there we agree. I might speculate that they were trying to get an early lead in the Linus Git of the Year contest. But this is relatively mild compared to some of the crap I've seen on other projects I won't name here. So I too am at a loss to know why.

On Fri, Apr 29, 2005 at 11:29:22PM -0700, Paul Jackson wrote:

I don't have a multiprocessor and I see the same. Are you sure it's SMP dependant?

Your solution (trapping _inside_ the job, too) works for me, btw. Here's a patch for cg-log that reduces the clutter to two "Broken pipe" lines (pun not intended).

Rene

--- cg-log~ 2005-04-29 23:43:09.000000000 +0200 +++ cg-log 2005-04-30 12:15:40.000000000 +0200

Rene wrote:

No - I'm not sure. It just happened to be that way on the couple of systems I looked at (and I figured that in any case, it was a good bet that Linus had multiple processors ;).

Looks plausible to me.

Dear diary, on Sat, Apr 30, 2005 at 01:04:10PM CEST, I got a letter where Rene Scharfe <rene.scharfe@lsrfire.ath.cx> told me that...

Could you elaborate on how exactly is it supposed to help? I see identical behaviour with the traps and without them (UP, bash-2.05b).

The key code is in bash/jobs.c.

If you have a bash while or for loop feeding a pipe that shuts down while the loop is still running commands that try to write the pipe (perhaps you were pipe'ing to "head -1", and it exit'd, having read its one line), then the next command to attempt to write that pipe will die, and the bash instance that is handling that loop (and forked that command that just died) will notice the command died with a SIGPIPE signal.

At this point, one of three possible things happens:

 1) If your bash is compiled with DONT_REPORT_SIGPIPE defined, then
    that bash instance quietly leaves.  The concensus around here
    is that is "good(tm)."

 2) If not so compiled, then:

	2a) If you set a trap on SIGPIPE in that shell, it prints:

		fprintf (stderr, "%s", j_strsignal (termsig));

	2b) Else if you did not trap SIGPIPE, it prints:

		fprintf (stderr, "%s: line %d: ", get_name_for_error (),
				(line_number == 0) ? 1 : line_number);
		pretty_print_job (job, JLIST_NONINTERACTIVE, stderr);

The pretty_print_job() in (2b) can be a multi-line confusion.

Sample output for (2a):

	Broken pipe

Sample output in simple one line case for (2b):

	foo: line 2: 11663 Broken pipe             cat /etc/termcap

A couple of others are reporting different behaviour than what I report above - including Linus.

So it is almost certain that I don't understand all I know about this.

What behaviour do you see?