Re: [PATCH] submodule: implement `module_name` as a builtin helper

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 8, 2015, 00:21 UTC
Message-ID: <xmqqlhdmwrxx.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CAGZ79kaGe59=z4Eeuno8DRuWWRD12OyLETi7NA+utfoLsJ4+=A@mail.gmail.com>

Stefan Beller <sbeller@google.com> writes:

Show 5 quoted lines

>> One more thing, as I didn't notice that you kept repeating "thread"
>> pool API.
>
> Yeah I intended to use both threads and processes for the heavy submodule
> operations.

OK. I somehow had an impression that it might be more tricky than it is worth to spawn/run_command out of a thread/run_async, but if it makes it easier and more generic to correctly arrange the thread-pool API to allow the per-thread functions to run_command(), I wouldn't object to that approach at all.

Then for-each-parallel would truly become a trivial application of that API. Your per-thread function happens to be a one that prepares appropriate "struct child_process" and calls run_command() to interact with that single child, receiving its output and culling it when it is done.

Show 6 quoted lines

>> ... if
>> you really want to go the "thread" route, the first thing to try
>> would be to see if a few places we already use threads for
>> parallelism (namely, "grep", "pack-objects", "preload-index" and
>> "index-pack") can be factored out and model your new API around the
>> commonality among them.

And obviously, doing your pool API around threads will allow you to throw future per-thread function that do not involve run_command() at all at your API, and it will make it easy to adapt the current threaded parts of the system to the API.

Perfect.

;-)