git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/5] index-pack: Use the new worker pool

From
Stefan Beller <sbeller@google.com>
Date
Aug 25, 2015, 19:23 UTC
Message-ID
<CAGZ79kaAT1C1XscOmkN8UzeJ7cmwOhxiTu1HsoTwN+HR5H2A5Q@mail.gmail.com>
In-Reply-To
<20150825190329.GA11271@sigill.intra.peff.net>
On Tue, Aug 25, 2015 at 12:03 PM, Jeff King <peff@peff.net> wrote:
Show 12 quoted lines
> On Tue, Aug 25, 2015 at 10:28:25AM -0700, Stefan Beller wrote:
>
>> By treating each object as its own task the workflow is easier to follow
>> as the function used in the worker threads doesn't need any control logic
>> any more.
>
> Have you tried running t/perf/p5302 on this?
>
> I seem to get a pretty consistent 2%-ish slowdown, both against git.git
> and linux.git. That's not a lot, but I'm wondering if there is some
> low-hanging fruit in the locking, or in the pattern of work being
> dispatched. Or it may just be noise, but it seems fairly consistent.
I did not run any perf tests, just the standard tests.

Maybe the progress display can be moved out to another thread, such that it doesn't block the threads doing actual work. Also the progress display can be done a bit more sloppy, we are allowed to drop old events, if we have newer information. [Though this reasoning is not looking at the threading pool code, because ..um.. my code is perfect]

>
> -Peff
Previous: Jeff KingNext: Junio C Hamano
Message 13 of 19 in “Demonstrate new parallel threading API”
  1. 0/5 Demonstrate new parallel threading APIStefan Beller, Aug 25, 2015
  2. 1/5 FIXUP submodule: implement `module_clone` as a builtin helperStefan Beller, Aug 25, 2015
  3. 2/5 thread-utils: add a threaded task queueStefan Beller, Aug 25, 2015
  4. 3/5 submodule: helper to run foreach in parallelStefan Beller, Aug 25, 2015
  5. Junio C HamanoAug 25, 2015
  6. Stefan BellerAug 25, 2015
  7. Junio C HamanoAug 25, 2015
  8. Junio C HamanoAug 25, 2015
  9. Jeff KingAug 26, 2015
  10. Stefan BellerAug 26, 2015
  11. 4/5 index-pack: Use the new worker poolStefan Beller, Aug 25, 2015
  12. Jeff KingAug 25, 2015
  13. Stefan BellerAug 25, 2015
  14. Junio C HamanoAug 25, 2015
  15. Stefan BellerAug 25, 2015
  16. Junio C HamanoAug 25, 2015
  17. Stefan BellerAug 25, 2015
  18. Junio C HamanoAug 25, 2015
  19. 5/5 pack-objects: Use new worker poolStefan Beller, Aug 25, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.