git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: O_NONBLOCK: should I blame git or ssh?

From
Jeff King <peff@peff.net>
Date
Sep 9, 2019, 17:04 UTC
Message-ID
<20190909170403.GB30399@sigill.intra.peff.net>
In-Reply-To
<BN8PR15MB302520248BB1AD49B28C3AB3F0B40@BN8PR15MB3025.namprd15.prod.outlook.com>
On Sun, Sep 08, 2019 at 02:18:15PM +0000, Douglas Graham wrote:
Show 7 quoted lines
> When I collected that strace output, I had stdout redirected to a pipe to my
> workaround program, but I did not redirect stderr.  So ssh made stdout non-blocking,
> but since stderr was still connected to my terminal, it didn't touch that. But when
> this build is run under Jenkins, both stdout and stderr are connected to a pipe that
> Jenkins creates to collect output from the build. I assume that when git runs ssh, it
> does not redirect ssh's stderr to its own pipe, it only redirects stdout. So I think
> ssh will be messing with both pipes when this build is run under Jenkins.
OK, that makes sense.
Show 13 quoted lines
> Now that I have a fairly good understanding of what's happening, I think I can work
> around this occasional error by redirecting git's stderr to a file or something like
> that, but it's taken us a long time to figure this out, so I wonder if a more permanent
> fix shouldn't be implement, so others don't run into the same problem.  A google for
> "make: write error" indicates that we're not the first to have this problem with
> parallel builds, although in the other cases I've found, a specific version of the
> Linux kernel was being blamed.  Maybe that was a different problem.
> 
> I guess git could workaround this by redirecting stderr, but the real problem is probably
> with ssh, although it's not clear to me what it should do differently. It does some
> somehow backwards to me that that it only makes a descriptor non-blocking if it doesn't
> refer to a TTY, but it does the same thing in at least three different places so I guess
> that's  not a mistake.

Where would git redirect the stderr to? We definitely want to make sure it goes to our original stderr, since it can have useful content for the user to see. We could make a new pipe and then pump the output back to our original stderr. But besides being complex, that fools the downstream programs about whether stderr is a tty (I don't know whether ssh cares, but certainly git itself uses that to decide on some elements of the output, mostly progress meters).

So I think it would make more sense to talk to ssh folks about why this momentary O_NONBLOCK setting happens, and if it can be avoided.

-Peff
Previous: Douglas GrahamNext: Douglas Graham
Message 4 of 5 in “O_NONBLOCK: should I blame git or ssh?”
  1. Douglas GrahamSep 7, 2019
  2. Jeff KingSep 8, 2019
  3. Douglas GrahamSep 8, 2019
  4. Jeff KingSep 9, 2019
  5. Douglas GrahamSep 9, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.