git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv3 02/11] run-command: report failure for degraded output just once

From
Jeff King <peff@peff.net>
Date
Nov 4, 2015, 22:56 UTC
Message-ID
<20151104225618.GA18805@sigill.intra.peff.net>
In-Reply-To
<xmqq4mh1a37i.fsf@gitster.mtv.corp.google.com>
On Wed, Nov 04, 2015 at 01:01:53PM -0800, Junio C Hamano wrote:
Show 7 quoted lines
> But the symptom does not have to be as severe as a total deadlock to
> be problematic.  If we block B (and other tasks) by not reading from
> them quickly because we are blocked on reading from A, which may
> take forever (in timescale of B and other tasks) to feed us enough
> to satisfy strbuf_read_once(), we are wasting resource by spawning B
> (and other tasks) early when we are not prepared to service them
> well, on both our end and on the other side of the connection.

I'm not sure I understand this line of reasoning. It is entirely possible that I have not been paying close enough attention and am missing something subtle, so please feel free to hit me with the clue stick.

But why would we ever block reading from A? If poll() reported to us that "A" is ready to read, and we call strbuf_read_once(), we will make a _single_ read call (which was, after all, the point of adding strbuf_read_once in the first place).

So even if descriptor "A" isn't non-blocking, why would we block? Only if the OS told us we are ready to read via poll(), but we are somehow not (which, AFAIK, would be a bug in the OS).

So I'm not sure I see why we need to be non-blocking at all here, if we are correctly hitting poll() and doing a single read on anybody who claims to be ready (rather than trying to soak up all of their available data), then we should never block, and we should never starve one process (even without blocking, we could be in a busy loop slurping from A and starve B, but by hitting the descriptors in round-robin for each poll(), we make sure they all progress).

What am I missing?
-Peff
Previous: Junio C HamanoNext: Junio C Hamano
Message 8 of 34 in “[PATCHv3 00/11] Expose the submodule parallelism to the user”
  1. Stefan BellerNov 4, 2015
  2. 01/11 run_processes_parallel: delimit intermixed task outputStefan Beller, Nov 4, 2015
  3. 02/11 run-command: report failure for degraded output just onceStefan Beller, Nov 4, 2015
  4. Junio C HamanoNov 4, 2015
  5. Stefan BellerNov 4, 2015
  6. Johannes SixtNov 4, 2015
  7. Junio C HamanoNov 4, 2015
  8. Jeff KingNov 4, 2015
  9. Junio C HamanoNov 5, 2015
  10. Jeff KingNov 5, 2015
  11. Junio C HamanoNov 5, 2015
  12. Stefan BellerNov 5, 2015
  13. Junio C HamanoNov 4, 2015
  14. Stefan BellerNov 4, 2015
  15. Junio C HamanoNov 4, 2015
  16. Stefan BellerNov 4, 2015
  17. 03/11 run-command: omit setting file descriptors to non blocking in WindowsStefan Beller, Nov 4, 2015
  18. 04/11 submodule-config: keep update strategy aroundStefan Beller, Nov 4, 2015
  19. 05/11 submodule-config: drop check against NULLStefan Beller, Nov 4, 2015
  20. 06/11 submodule-config: remove name_and_item_from_varStefan Beller, Nov 4, 2015
  21. 07/11 submodule-config: introduce parse_generic_submodule_configStefan Beller, Nov 4, 2015
  22. 08/11 fetching submodules: respect `submodule.jobs` config optionStefan Beller, Nov 4, 2015
  23. Jens LehmannNov 10, 2015
  24. Stefan BellerNov 10, 2015
  25. Jens LehmannNov 11, 2015
  26. Stefan BellerNov 11, 2015
  27. Jens LehmannNov 13, 2015
  28. Stefan BellerNov 13, 2015
  29. 09/11 git submodule update: have a dedicated helper for cloningStefan Beller, Nov 4, 2015
  30. 10/11 submodule update: expose parallelism to the userStefan Beller, Nov 4, 2015
  31. 11/11 clone: allow an explicit argument for parallel submodule clonesStefan Beller, Nov 4, 2015
  32. Junio C HamanoNov 4, 2015
  33. Stefan BellerNov 4, 2015
  34. Junio C HamanoNov 4, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.