git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv4 06/14] run-command: add an asynchronous parallel child processor

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 23, 2015, 18:04 UTC
Message-ID
<xmqqlhbxxbai.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<CAGZ79kYaqFRPfRORbknTyez5u0d6_BD0d5wBtAnjSkb4sUBUBA@mail.gmail.com>
Stefan Beller <sbeller@google.com> writes:
Show 8 quoted lines
> I modified the test-run-command test function to start up to 400 processes.
> (Most people will use less than 400 processes in the next 5 years), and run
> just as in t0061:
>
>     ./test-run-command run-command-parallel-400 sh -c "printf
> \"%s\n%s\n\" Hello World"
>
> The output felt immediate (not slowed down or anything).

I doubt that such an experiment has any value. You are not driving printf in real life.

You are running "git fetch" that are is a lot more heavy-weight. Because once each of them started fully they will be network bound, it is likely that you would want to run more processes than you have core.

Previous: Stefan BellerNext: Junio C Hamano
Message 10 of 39 in “[PATCHv4 00/14] fetch submodules in parallel and a preview on parallel "submodule update"”
  1. Stefan BellerSep 23, 2015
  2. 01/14 submodule: Send "Fetching submodule <foo>" to standard errorStefan Beller, Sep 23, 2015
  3. 02/14 xread: poll on non blocking fdsStefan Beller, Sep 23, 2015
  4. 03/14 xread_nonblock: add functionality to read from fds without blockingStefan Beller, Sep 23, 2015
  5. 04/14 strbuf: add strbuf_read_once to read without blockingStefan Beller, Sep 23, 2015
  6. 05/14 run-command: factor out return value computationStefan Beller, Sep 23, 2015
  7. 06/14 run-command: add an asynchronous parallel child processorStefan Beller, Sep 23, 2015
  8. Junio C HamanoSep 23, 2015
  9. Stefan BellerSep 23, 2015
  10. Junio C HamanoSep 23, 2015
  11. Junio C HamanoSep 23, 2015
  12. Stefan BellerSep 23, 2015
  13. Junio C HamanoSep 23, 2015
  14. Junio C HamanoSep 23, 2015
  15. Junio C HamanoSep 23, 2015
  16. Junio C HamanoSep 23, 2015
  17. [PATCHv5] Another squash on run-command: add an asynchronous parallel child processorStefan Beller, Sep 23, 2015
  18. Junio C HamanoSep 24, 2015
  19. 0/2 Another squash on run-command: add an asynchronous parallel child processorStefan Beller, Sep 24, 2015
  20. 2/2 SQUASH for "fetch_populated_submodules: use new parallel job processing"Stefan Beller, Sep 24, 2015
  21. 1/2 SQUASH???Stefan Beller, Sep 24, 2015
  22. Junio C HamanoSep 25, 2015
  23. Junio C HamanoSep 25, 2015
  24. Stefan BellerSep 25, 2015
  25. Junio C HamanoSep 25, 2015
  26. Junio C HamanoSep 25, 2015
  27. Stefan BellerSep 25, 2015
  28. Junio C HamanoSep 25, 2015
  29. Stefan BellerSep 25, 2015
  30. Junio C HamanoSep 25, 2015
  31. 07/14 fetch_populated_submodules: use new parallel job processingStefan Beller, Sep 23, 2015
  32. 08/14 submodules: allow parallel fetching, add tests and documentationStefan Beller, Sep 23, 2015
  33. 09/14 submodule-config: Untangle logic in parse_configStefan Beller, Sep 23, 2015
  34. 10/14 submodule config: keep update strategy aroundStefan Beller, Sep 23, 2015
  35. 11/14 git submodule update: cmd_update_recursiveStefan Beller, Sep 23, 2015
  36. 12/14 git submodule update: cmd_update_cloneStefan Beller, Sep 23, 2015
  37. Junio C HamanoSep 23, 2015
  38. 13/14 git submodule update: cmd_update_fetchStefan Beller, Sep 23, 2015
  39. 14/14 Rewrite submodule update in CStefan Beller, Sep 23, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.