git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 2/3] run-commands: add an async queue processor

From
Stefan Beller <sbeller@google.com>
Date
Aug 21, 2015, 20:56 UTC
Message-ID
<CAGZ79kaj0=2yw+KBLKQCkMzZpmcdFk93yDd7m+3x6Jn7gGap8w@mail.gmail.com>
In-Reply-To
<xmqqy4h44bdp.fsf@gitster.dls.corp.google.com>
On Fri, Aug 21, 2015 at 1:47 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 26 quoted lines
>
> I do not think we are on the same wavelength.  What I meant was to
> do this:
>
>         aq = xmalloc(...);
>         set up _everything_ in aq and make it a consistent state;
>         /* aq->first and aq->last are part of _everything_ in aq */
>         for (many times)
>                 pthread_create(...);
>
>         /* No aq->first = aq->last = NULL assignment here */
>
> instead of
>
>         aq = xmalloc(...);
>         set up part of aq;
>         for (many times)
>                 pthread_create(...);
>         belatedly initialize aq->first and aq->last and finally
>         aq becomes a consistent state.
>
> which is what we see above.  The latter works _only_ because the
> threads created are blocked waiting on aq->workingcount which is
> initialized to block before threads are created to run dispatch,
> and one of the early things dispatch does is to try acquiring that
> semaphore to block before accessing aq->first and aq->last.

I see your point and it makes sense to me as it makes the mental memory model cleaner

I viewed the methods as atomic units (you would not call add_task before the constructor has finished, so no harm there. And a dispatcher in the pthread would just block on the `workingcount` semaphore as the semaphores are the main inter thread communication. I viewed the queue as just a secondary thing to distribute the work load.

Previous: Junio C HamanoNext: Stefan Beller
Message 13 of 16 in “submodule: implement `module_clone` as a builtin helper”
  1. 1/3 submodule: implement `module_clone` as a builtin helperStefan Beller, Aug 21, 2015
  2. 2/3 run-commands: add an async queue processorStefan Beller, Aug 21, 2015
  3. Junio C HamanoAug 21, 2015
  4. Jeff KingAug 21, 2015
  5. Stefan BellerAug 21, 2015
  6. Jeff KingAug 21, 2015
  7. Stefan BellerAug 21, 2015
  8. Junio C HamanoAug 21, 2015
  9. Stefan BellerAug 21, 2015
  10. Junio C HamanoAug 24, 2015
  11. Stefan BellerAug 21, 2015
  12. Junio C HamanoAug 21, 2015
  13. Stefan BellerAug 21, 2015
  14. 3/3 submodule: helper to run foreach in parallelStefan Beller, Aug 21, 2015
  15. Junio C HamanoAug 21, 2015
  16. Stefan BellerAug 21, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.