git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv3 06/13] run-command: add an asynchronous parallel child processor

From
Stefan Beller <sbeller@google.com>
Date
Sep 22, 2015, 21:31 UTC
Message-ID
<CAGZ79kYwMrpU-gW6FsbH1W4TRPisZH9GJoyZ6hoimXdxz4HCYQ@mail.gmail.com>
In-Reply-To
<xmqqtwqm2puk.fsf@gitster.mtv.corp.google.com>
On Tue, Sep 22, 2015 at 12:53 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 14 quoted lines
> Stefan Beller <sbeller@google.com> writes:
>
>> So how would you find out when we are done?
>
>         while (1) {
>                 if (we have available slot)
>                         ask to start a new one;
>                 if (nobody is running anymore)
>                         break;
>                 collect the output from running ones;
>                 drain the output from the foreground one;
>                 cull the finished process;
>         }
>

Personally I do not like the break; in the middle of the loop, but that's personal preference, I'd guess. I'll also move the condition for (we have available slot) back inside the called function.

So I am thinking about using this in the reroll instead:
    run_processes_parallel_start_as_needed(&pp);
    while (pp.nr_processes > 0) {
        run_processes_parallel_buffer_stderr(&pp);
        run_processes_parallel_output(&pp);
        run_processes_parallel_collect_finished(&pp);
        run_processes_parallel_start_as_needed(&pp);
    }

This eliminates the need for the flag and also exits the loop just after the possible startup of new processes.

Previous: Junio C HamanoNext: Junio C Hamano
Message 35 of 48 in “[PATCHv3 00/13] fetch submodules in parallel and a preview on parallel "submodule update"”
  1. Stefan BellerSep 21, 2015
  2. 01/13 Sending "Fetching submodule <foo>" output to stderrStefan Beller, Sep 21, 2015
  3. Junio C HamanoSep 21, 2015
  4. 02/13 xread: poll on non blocking fdsStefan Beller, Sep 21, 2015
  5. Junio C HamanoSep 21, 2015
  6. Torsten BögershausenSep 22, 2015
  7. Jacob KellerSep 22, 2015
  8. Torsten BögershausenSep 22, 2015
  9. Junio C HamanoSep 22, 2015
  10. Jeff KingSep 22, 2015
  11. Junio C HamanoSep 22, 2015
  12. Stefan BellerSep 23, 2015
  13. Junio C HamanoSep 23, 2015
  14. Jeff KingSep 23, 2015
  15. Eric SunshineSep 21, 2015
  16. Junio C HamanoSep 22, 2015
  17. Stefan BellerSep 22, 2015
  18. Junio C HamanoSep 22, 2015
  19. Stefan BellerSep 22, 2015
  20. 03/13 xread_nonblock: add functionality to read from fds nonblockinglyStefan Beller, Sep 21, 2015
  21. Junio C HamanoSep 22, 2015
  22. Junio C HamanoSep 22, 2015
  23. Jacob KellerSep 22, 2015
  24. Jacob KellerSep 22, 2015
  25. Junio C HamanoSep 22, 2015
  26. 04/13 strbuf: add strbuf_read_once to read without blockingStefan Beller, Sep 21, 2015
  27. Junio C HamanoSep 22, 2015
  28. Jacob KellerSep 22, 2015
  29. 05/13 run-command: factor out return value computationStefan Beller, Sep 21, 2015
  30. Junio C HamanoSep 22, 2015
  31. 06/13 run-command: add an asynchronous parallel child processorStefan Beller, Sep 21, 2015
  32. Junio C HamanoSep 22, 2015
  33. Stefan BellerSep 22, 2015
  34. Junio C HamanoSep 22, 2015
  35. Stefan BellerSep 22, 2015
  36. Junio C HamanoSep 22, 2015
  37. Stefan BellerSep 22, 2015
  38. Junio C HamanoSep 22, 2015
  39. 07/13 fetch_populated_submodules: use new parallel job processingStefan Beller, Sep 21, 2015
  40. Junio C HamanoSep 22, 2015
  41. 08/13 submodules: allow parallel fetching, add tests and documentationStefan Beller, Sep 21, 2015
  42. 09/13 submodule config: keep update strategy aroundStefan Beller, Sep 21, 2015
  43. Eric SunshineSep 22, 2015
  44. Stefan BellerSep 22, 2015
  45. 10/13 git submodule update: cmd_update_recursiveStefan Beller, Sep 21, 2015
  46. 11/13 git submodule update: cmd_update_cloneStefan Beller, Sep 21, 2015
  47. 12/13 git submodule update: cmd_update_fetchStefan Beller, Sep 21, 2015
  48. 13/13 Rewrite submodule update in CStefan Beller, Sep 21, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.