git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv3 06/13] run-command: add an asynchronous parallel child processor

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 22, 2015, 21:41 UTC
Message-ID
<xmqqh9mm1690.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<CAGZ79kYwMrpU-gW6FsbH1W4TRPisZH9GJoyZ6hoimXdxz4HCYQ@mail.gmail.com>
Stefan Beller <sbeller@google.com> writes:
Show 30 quoted lines
> On Tue, Sep 22, 2015 at 12:53 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Stefan Beller <sbeller@google.com> writes:
>>
>>> So how would you find out when we are done?
>>
>>         while (1) {
>>                 if (we have available slot)
>>                         ask to start a new one;
>>                 if (nobody is running anymore)
>>                         break;
>>                 collect the output from running ones;
>>                 drain the output from the foreground one;
>>                 cull the finished process;
>>         }
>>
>
> Personally I do not like the break; in the middle of
> the loop, but that's personal preference, I'd guess.
> I'll also move the condition for (we have available slot)
> back inside the called function.
>
> So I am thinking about using this in the reroll instead:
>
>     run_processes_parallel_start_as_needed(&pp);
>     while (pp.nr_processes > 0) {
>         run_processes_parallel_buffer_stderr(&pp);
>         run_processes_parallel_output(&pp);
>         run_processes_parallel_collect_finished(&pp);
>         run_processes_parallel_start_as_needed(&pp);
>     }

If you truly think the latter is easier to follow its logic (with the duplicated call to the same function only to avoid break that makes it clear why we are quitting the loop, and without the explicit "start only if we can afford to"), then I have to say that our design tastes are fundamentally incompatible. I have nothing more to add.

Previous: Stefan BellerNext: Stefan Beller
Message 36 of 48 in “[PATCHv3 00/13] fetch submodules in parallel and a preview on parallel "submodule update"”
  1. Stefan BellerSep 21, 2015
  2. 01/13 Sending "Fetching submodule <foo>" output to stderrStefan Beller, Sep 21, 2015
  3. Junio C HamanoSep 21, 2015
  4. 02/13 xread: poll on non blocking fdsStefan Beller, Sep 21, 2015
  5. Junio C HamanoSep 21, 2015
  6. Torsten BögershausenSep 22, 2015
  7. Jacob KellerSep 22, 2015
  8. Torsten BögershausenSep 22, 2015
  9. Junio C HamanoSep 22, 2015
  10. Jeff KingSep 22, 2015
  11. Junio C HamanoSep 22, 2015
  12. Stefan BellerSep 23, 2015
  13. Junio C HamanoSep 23, 2015
  14. Jeff KingSep 23, 2015
  15. Eric SunshineSep 21, 2015
  16. Junio C HamanoSep 22, 2015
  17. Stefan BellerSep 22, 2015
  18. Junio C HamanoSep 22, 2015
  19. Stefan BellerSep 22, 2015
  20. 03/13 xread_nonblock: add functionality to read from fds nonblockinglyStefan Beller, Sep 21, 2015
  21. Junio C HamanoSep 22, 2015
  22. Junio C HamanoSep 22, 2015
  23. Jacob KellerSep 22, 2015
  24. Jacob KellerSep 22, 2015
  25. Junio C HamanoSep 22, 2015
  26. 04/13 strbuf: add strbuf_read_once to read without blockingStefan Beller, Sep 21, 2015
  27. Junio C HamanoSep 22, 2015
  28. Jacob KellerSep 22, 2015
  29. 05/13 run-command: factor out return value computationStefan Beller, Sep 21, 2015
  30. Junio C HamanoSep 22, 2015
  31. 06/13 run-command: add an asynchronous parallel child processorStefan Beller, Sep 21, 2015
  32. Junio C HamanoSep 22, 2015
  33. Stefan BellerSep 22, 2015
  34. Junio C HamanoSep 22, 2015
  35. Stefan BellerSep 22, 2015
  36. Junio C HamanoSep 22, 2015
  37. Stefan BellerSep 22, 2015
  38. Junio C HamanoSep 22, 2015
  39. 07/13 fetch_populated_submodules: use new parallel job processingStefan Beller, Sep 21, 2015
  40. Junio C HamanoSep 22, 2015
  41. 08/13 submodules: allow parallel fetching, add tests and documentationStefan Beller, Sep 21, 2015
  42. 09/13 submodule config: keep update strategy aroundStefan Beller, Sep 21, 2015
  43. Eric SunshineSep 22, 2015
  44. Stefan BellerSep 22, 2015
  45. 10/13 git submodule update: cmd_update_recursiveStefan Beller, Sep 21, 2015
  46. 11/13 git submodule update: cmd_update_cloneStefan Beller, Sep 21, 2015
  47. 12/13 git submodule update: cmd_update_fetchStefan Beller, Sep 21, 2015
  48. 13/13 Rewrite submodule update in CStefan Beller, Sep 21, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.