git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH] pipe_command(): mark stdin descriptor as non-blocking

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 2, 2022, 15:04 UTC
Message-ID
<xmqqbkt2k7p4.fsf@gitster.g>
In-Reply-To
<YuikU//9OrdpKQcE@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
Show 16 quoted lines
> Our pipe_command() helper lets you both write to and read from a child
> process on its stdin/stdout. It's supposed to work without deadlocks
> because we use poll() to check when descriptors are ready for reading or
> writing. But there's a bug: if both the data to be written and the data
> to be read back exceed the pipe buffer, we'll deadlock.
> ...
> If you set add.interactive.useBuiltin to false, the problem goes away,
> because now we're not using pipe_command() anymore (instead, that part
> happens in perl). But this isn't a bug in the interactive code at all.
> It's the underlying pipe_command() code which is broken, and has been
> all along.
> ...
> The obvious fix is to put the descriptor into non-blocking mode, and
> indeed, that makes the problem go away. Callers shouldn't need to
> care, because they never see the descriptor (they hand us a buffer to
> feed into it).
Thanks for a very well reasoned and explained patch.
>   - more importantly, I'm not sure of the portability implications of
>     the fix. This is our first use of O_NONBLOCK outside of the
>     compat/simple-ipc unix-socket code. Do we need to abstract this
>     behind a compat/ layer for Windows?
Yup.  A very good question to ask for the platform maintainer.
Show 38 quoted lines
>  run-command.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
>
> diff --git a/run-command.c b/run-command.c
> index 14f17830f5..45bffb4b11 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -1418,6 +1418,14 @@ static int pump_io(struct io_pump *slots, int nr)
>  	return 0;
>  }
>  
> +static int make_nonblock(int fd)
> +{
> +	int flags = fcntl(fd, F_GETFL);
> +	if (flags < 0)
> +		return -1;
> +	flags |= O_NONBLOCK;
> +	return fcntl(fd, F_SETFL, flags);
> +}
>  
>  int pipe_command(struct child_process *cmd,
>  		 const char *in, size_t in_len,
> @@ -1438,6 +1446,15 @@ int pipe_command(struct child_process *cmd,
>  		return -1;
>  
>  	if (in) {
> +		if (make_nonblock(cmd->in) < 0) {
> +			error_errno("unable to make pipe non-blocking");
> +			close(cmd->in);
> +			if (out)
> +				close(cmd->out);
> +			if (err)
> +				close(cmd->err);
> +			return -1;
> +		}
>  		io[nr].fd = cmd->in;
>  		io[nr].type = POLLOUT;
>  		io[nr].u.out.buf = in;
Previous: Jeff KingNext: Jeff King
Message 2 of 44 in “pipe_command(): mark stdin descriptor as non-blocking”
  1. pipe_command(): mark stdin descriptor as non-blockingJeff King, Aug 2, 2022
  2. Junio C HamanoAug 2, 2022
  3. Jeff KingAug 2, 2022
  4. Junio C HamanoAug 2, 2022
  5. pipe_command(): mark stdin descriptor as non-blockingJeff King, Aug 3, 2022
  6. René ScharfeAug 3, 2022
  7. Jeff KingAug 3, 2022
  8. René ScharfeAug 3, 2022
  9. Jeff KingAug 5, 2022
  10. René ScharfeAug 5, 2022
  11. René ScharfeAug 7, 2022
  12. Jeff KingAug 7, 2022
  13. mingw: handle writes to non-blocking pipeRené Scharfe, Aug 10, 2022
  14. Johannes SchindelinAug 10, 2022
  15. Jeff KingAug 10, 2022
  16. René ScharfeAug 10, 2022
  17. Jeff KingAug 11, 2022
  18. René ScharfeAug 11, 2022
  19. Jeff KingAug 11, 2022
  20. René ScharfeAug 14, 2022
  21. Jeff KingAug 17, 2022
  22. 0/6 fix pipe_command() deadlockJeff King, Aug 17, 2022
  23. 1/6 compat: add function to enable nonblocking pipesJeff King, Aug 17, 2022
  24. Junio C HamanoAug 17, 2022
  25. Jeff KingAug 18, 2022
  26. 2/6 nonblock: support WindowsJeff King, Aug 17, 2022
  27. 3/6 git-compat-util: make MAX_IO_SIZE define globally availableJeff King, Aug 17, 2022
  28. 4/6 pipe_command(): avoid xwrite() for writing to pipeJeff King, Aug 17, 2022
  29. 5/6 pipe_command(): handle ENOSPC when writing to a pipeJeff King, Aug 17, 2022
  30. Junio C HamanoAug 17, 2022
  31. Jeff KingAug 18, 2022
  32. 6/6 pipe_command(): mark stdin descriptor as non-blockingJeff King, Aug 17, 2022
  33. Jeff KingAug 17, 2022
  34. René ScharfeAug 19, 2022
  35. Jeff KingAug 20, 2022
  36. René ScharfeAug 10, 2022
  37. Jeff KingAug 10, 2022
  38. René ScharfeAug 10, 2022
  39. Jeff KingAug 11, 2022
  40. René ScharfeAug 7, 2022
  41. Johannes SchindelinAug 8, 2022
  42. Johannes SchindelinAug 8, 2022
  43. Jeff KingAug 9, 2022
  44. Johannes SchindelinAug 9, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.