git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/2] Add support for multi threaded checkout

From
JPJames Pickens <jepicken@gmail.com>
Date
Dec 18, 2008, 23:35 UTC
Message-ID
<885649360812181535h36d24b0gb31acddded452a0@mail.gmail.com>
In-Reply-To
<alpine.LFD.2.00.0812181333150.14014@localhost.localdomain>
On Thu, Dec 18, 2008, Linus Torvalds <torvalds@linux-foundation.org> wrote:
Show 5 quoted lines
> So instead of doing all the unpacking etc in parallel (with locking around
> it to serialize it), I'd suggest doing ll the unpacking serially since
> that isn't the problem anyway (and since you have to protect it with a
> lock anyway), and just have a "write out and free the buffer" phase that
> is done in the threads.

That's certainly a more elegant way to do it, but unless I'm missing something, it requires rewriting a good bit of code. The main reason I went with the locking was to keep the patch as simple and non-intrusive as possible.

Show 5 quoted lines
> The alternative would be to actually do what your patch suggests, but
> actually try to make the code git SHA1 object handling be thread-safe. At
> that point, the ugly locking in write_entry() would go away, and you might
> actually improve performance on SMP thanks to doing the CPU part in
> parallel.

I started down that path at one point, and quickly got in over my head. Making all that code thread safe looks like a big task to me. From my perspective, I get a ~350% speedup from this easy patch, and I might get an additional 25% (blind guess) from a much more difficult patch. It didn't seem worth the effort.

James
Previous: Linus TorvaldsNext: Linus Torvalds
Message 5 of 11 in “Add support for multi threaded checkout”
  1. 0/2 Add support for multi threaded checkoutPickens, James E, Dec 18, 2008
  2. 1/2 Add support for multi threaded checkoutJames Pickens, Dec 18, 2008
  3. 2/2 Add core.threadedcheckout config optionJames Pickens, Dec 18, 2008
  4. Linus TorvaldsDec 18, 2008
  5. James PickensDec 18, 2008
  6. Linus TorvaldsDec 19, 2008
  7. Nicolas PitreDec 18, 2008
  8. James PickensDec 18, 2008
  9. Nicolas Morey-ChaisemartinDec 18, 2008
  10. James PickensDec 18, 2008
  11. James PickensDec 19, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.