From: Junio C Hamano <gitster@pobox.com>
Date: Sat, 08 Aug 2015 00:21:30 GMT
Subject: Re: [PATCH] submodule: implement `module_name` as a builtin helper
Message-ID: <xmqqlhdmwrxx.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CAGZ79kaGe59=z4Eeuno8DRuWWRD12OyLETi7NA+utfoLsJ4+=A@mail.gmail.com>

Stefan Beller <sbeller@google.com> writes:

>> One more thing, as I didn't notice that you kept repeating "thread"
>> pool API.
>
> Yeah I intended to use both threads and processes for the heavy submodule
> operations.

OK.  I somehow had an impression that it might be more tricky than
it is worth to spawn/run_command out of a thread/run_async, but if
it makes it easier and more generic to correctly arrange the
thread-pool API to allow the per-thread functions to run_command(),
I wouldn't object to that approach at all.

Then for-each-parallel would truly become a trivial application of
that API.  Your per-thread function happens to be a one that
prepares appropriate "struct child_process" and calls run_command()
to interact with that single child, receiving its output and culling
it when it is done.

>> ... if
>> you really want to go the "thread" route, the first thing to try
>> would be to see if a few places we already use threads for
>> parallelism (namely, "grep", "pack-objects", "preload-index" and
>> "index-pack") can be factored out and model your new API around the
>> commonality among them.

And obviously, doing your pool API around threads will allow you to
throw future per-thread function that do not involve run_command()
at all at your API, and it will make it easy to adapt the current
threaded parts of the system to the API.

Perfect.

;-)