git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 0/2] Add support for multi threaded checkout

From
PEPickens, James E <james.e.pickens@intel.com>
Date
Dec 18, 2008, 20:51 UTC
Message-ID
<3BA20DF9B35F384F8B7395B001EC3FB3265B2A01@azsmsx507.amr.corp.intel.com>

There was some discussion a while back about improving git performance on NFS (http://article.gmane.org/gmane.comp.version-control.git/100950). This led to Linus adding the 'preload_index' function, which improves performance of several commands using multi threading. He also briefly described how to do the same for 'git checkout'. Well, I finally found some time to work on it, and will post patches shortly. This is my first patch; apologies if I screwed something up.

Patch 1 adds the functionality, and 2 adds a config option to enable/disable it.

Much of the patch is literally copy/paste from preload-index.c into unpack-trees.c. Many of the functions called during checkout are not thread safe, so I added a mutex in entry.c to serialize basically everything except writing the files to disk. I also added a mutex in unpack-trees.c for the progress meter.

It passes the test suite, and seems fairly safe to my naïve eyes.

Here are some benchmarks, cloning a linux kernel repo I had on an NFS drive:

                   NFS->NFS    NFS->Local
master (53682f0c)    2:46.1          13.3
with threads           36.6          18.2

So it improved performance on NFS significantly. Unfortunately it also degraded performance on the local disk significantly. I'm hoping someone will suggest a way to mitigate that... I think it would be reasonable to disable the threading except when the work dir is on NFS, but I don't know how to detect that. Even in that case it will have *some* impact from locking/unlocking the mutex, but I think it would be in the noise.

James
Next: James Pickens
Message 1 of 11 in “Add support for multi threaded checkout”
  1. 0/2 Add support for multi threaded checkoutPickens, James E, Dec 18, 2008
  2. 1/2 Add support for multi threaded checkoutJames Pickens, Dec 18, 2008
  3. 2/2 Add core.threadedcheckout config optionJames Pickens, Dec 18, 2008
  4. Linus TorvaldsDec 18, 2008
  5. James PickensDec 18, 2008
  6. Linus TorvaldsDec 19, 2008
  7. Nicolas PitreDec 18, 2008
  8. James PickensDec 18, 2008
  9. Nicolas Morey-ChaisemartinDec 18, 2008
  10. James PickensDec 18, 2008
  11. James PickensDec 19, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.