Re: [PATCH 02/10] t7422: fix flaky test caused by buffered stdout

From: Jeff King <peff@peff.net>
Date: Jan 7, 2025, 02:48 UTC
Message-ID: <20250107024829.GC2363@coredump.intra.peff.net>
In-Reply-To: <Z3u6lj_bpM7N93Fd@pks.im>

On Mon, Jan 06, 2025 at 12:12:22PM +0100, Patrick Steinhardt wrote:

Show 11 quoted lines

> >   - I tried using test_seq to avoid the inline perl, but it doesn't
> >     work! The problem is that it's implemented as a shell function. So
> >     when it gets SIGPIPE, the whole subshell is killed, and we never
> >     even run git-submodule at all. So it has to be a separate process
> >     (though I guess it could be test_seq in a subshell).
> 
> And that one should also work if we retain the grep. I wonder though
> whether we shouldn't prefer to use Perl regardless as it's likely to be
> faster when generating all that gibberish. Perl is basically a hard
> prerequisite for our tests anyway, so it doesn't really hurt to call it
> here.

I don't think we should pursue this direction any more because we need to get the SIGPIPE mid-way through the git-submodule command (see the other message I just sent).

But because it is a basic technique for establishing a reliable SIGPIPE, and we might end up using it elsewhere, I thought I'd post a slightly improved version.

The two things I didn't like about what I posted earlier were:

  - the guess at the pipe buffer size. 128k is probably enough in
    practice, but it's not guaranteed.

  - piping to "head" actually made our buffer size guess worse. We know
    that "head" is going to read the first line and then exit. But how
    much more data might it read? It might easily buffer 4k or even 8k,
    leaving the buffer not-quite full.

So I think a simpler and more robust version is just this:

  {
	{ yes || true; } &&
	command_expecting_sigpipe; echo $? >status
  } | true

We'll keep producing data in "yes" until the pipe is closed. So it will closed before command_expecting_sigpipe even starts, and there is no race there. And because we're using "true" on the right-hand side of the pipe, nothing is read at all from the pipe. So there's no guessing about how much might have been read.

And it works no matter how slow the right-hand side of the pipe is (e.g., you can add a "sleep 1" there and it still works).

Like I said, this won't help our current situation, but after having spent a little time on it (before realizing that) I figured it was worth documenting.

-Peff