git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] submodule: implement `module_name` as a builtin helper

From
Stefan Beller <sbeller@google.com>
Date
Aug 7, 2015, 23:19 UTC
Message-ID
<CAGZ79kaGe59=z4Eeuno8DRuWWRD12OyLETi7NA+utfoLsJ4+=A@mail.gmail.com>
In-Reply-To
<xmqqpp2ywwj1.fsf@gitster.dls.corp.google.com>
On Fri, Aug 7, 2015 at 3:42 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 8 quoted lines
> Stefan Beller <sbeller@google.com> writes:
>
>> That's why I want to be a bit more generic and have this thread pool API
>> done in C, such that "any for loop" in git can be easily replaced by using
>> the thread pool. I think of "git fetch --all" specially.
>
> One more thing, as I didn't notice that you kept repeating "thread"
> pool API.

Yeah I intended to use both threads and processes for the heavy submodule operations.

Each thread in the thread pool would setup the right argument lists and environment and then spawn a process for heavy weight operations, wait for the process, maybe process its output.

Maybe I should omit the whole thread pool and only use select from a single threaded main program.

Show 12 quoted lines
>
> While I doubt that you would gain much by using threads in place of
> processes to perform parallel "submodule update", "submodule clone",
> "fetch all", etc., all of which are fairly well isolated and heavy
> weight operations themselves, and I suspect that the implementation
> simplicity of using separate processes would probably be huge plus
> compared to any possible upside you may gain from using threads, if
> you really want to go the "thread" route, the first thing to try
> would be to see if a few places we already use threads for
> parallelism (namely, "grep", "pack-objects", "preload-index" and
> "index-pack") can be factored out and model your new API around the
> commonality among them.
Previous: Junio C HamanoNext: Junio C Hamano
Message 22 of 27 in “submodule: implement `module_list` as a builtin helper”
  1. 1/4 submodule: implement `module_list` as a builtin helperStefan Beller, Aug 5, 2015
  2. 2/4 submodule: implement `module_name` as a builtin helperStefan Beller, Aug 5, 2015
  3. Stefan BellerAug 5, 2015
  4. Eric SunshineAug 5, 2015
  5. Stefan BellerAug 5, 2015
  6. Jens LehmannAug 5, 2015
  7. Stefan BellerAug 5, 2015
  8. submodule: implement `module_name` as a builtin helperStefan Beller, Aug 5, 2015
  9. Jens LehmannAug 6, 2015
  10. submodule: implement `module_name` as a builtin helperStefan Beller, Aug 6, 2015
  11. Junio C HamanoAug 7, 2015
  12. Stefan BellerAug 7, 2015
  13. Junio C HamanoAug 7, 2015
  14. Stefan BellerAug 7, 2015
  15. Junio C HamanoAug 7, 2015
  16. Stefan BellerAug 7, 2015
  17. Junio C HamanoAug 7, 2015
  18. Stefan BellerAug 7, 2015
  19. Junio C HamanoAug 7, 2015
  20. Stefan BellerAug 7, 2015
  21. Junio C HamanoAug 7, 2015
  22. Stefan BellerAug 7, 2015
  23. Junio C HamanoAug 8, 2015
  24. Junio C HamanoAug 8, 2015
  25. Stefan BellerAug 10, 2015
  26. Jens LehmannAug 5, 2015
  27. Junio C HamanoAug 7, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.