git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Improving performance with pthreads in refresh_index().

From
SKSimon Klinkert <klinkert@webgods.de>
Date
Oct 12, 2011, 07:21 UTC
Message-ID
<470EDA14-707D-42C7-A57A-CFE7208E1D03@webgods.de>
In-Reply-To
<4E942D76.7030908@drmicha.warpmail.net>
Okay. It seems like my idea was already implemented by Linus. I wasn't aware of that fact.
Anyway, I've learned a bit more about git's bowels. Thanks a lot.
Simon
On 11.10.2011, at 13:50, Michael J Gruber wrote:
Show 46 quoted lines
> klinkert@webgods.de venit, vidit, dixit 11.10.2011 11:32:
>> Git performs for every file in a repository at least one (with a cold cache)
>> lstat(). In larger repositories operations like git status take a
>> long time. In case your local repository is located on a remote server
>> (e. g. mounted via nfs) it ends up in an *incredible* slow git.
>> 
>> With this patch you're able to determine a number of threads (maxthreads)
>> in your config file to run these tons of lstats in threads. There
>> won't be created any pthreads if you haven't set maxthreads. In my
>> test cases a git status with this patch performs enormously faster (over
>> two minutes before and approximately 25 seconds now). Of course, it
>> has a positive impact on other git commands, too.
> 
> Can you specify under which circumstances one should get a speedup? Our
> NFS isn't slow enough... but on a dead slow sshfs work tree I get the
> following for "git status -s":
> 
> maxthreads: 0, preloadindex: false, time: 14.73
> maxthreads: 1, preloadindex: false, time: 14.25
> maxthreads: 2, preloadindex: false, time: 13.32
> maxthreads: 3, preloadindex: false, time: 12.40
> maxthreads: 4, preloadindex: false, time: 12.65
> maxthreads: 5, preloadindex: false, time: 12.16
> maxthreads: 8, preloadindex: false, time: 12.32
> maxthreads: 10, preloadindex: false, time: 11.98
> maxthreads: 15, preloadindex: false, time: 12.31
> maxthreads: 20, preloadindex: false, time: 12.00
> maxthreads: 0, preloadindex: true, time: 12.17
> maxthreads: 1, preloadindex: true, time: 11.98
> maxthreads: 2, preloadindex: true, time: 12.21
> maxthreads: 3, preloadindex: true, time: 11.99
> maxthreads: 4, preloadindex: true, time: 12.14
> maxthreads: 5, preloadindex: true, time: 12.21
> maxthreads: 8, preloadindex: true, time: 12.14
> maxthreads: 10, preloadindex: true, time: 12.08
> maxthreads: 15, preloadindex: true, time: 12.16
> maxthreads: 20, preloadindex: true, time: 11.96
> 
> So it seams it gives me what preloadindex does, which is not much.
> 
> Note: I'm not saying the patch is bad. I'm just wondering whether that
> is expected.
> 
> Michael
> P.S.: It's actually sshfs with ssh to an NFSv3 client (server restricts
> exports) :(
Previous: Michael J Gruber
Message 5 of 5 in “Improving performance with pthreads in refresh_index().”
  1. Improving performance with pthreads in refresh_index().klinkert@webgods.de, Oct 11, 2011
  2. Johannes SixtOct 11, 2011
  3. Nguyen Thai Ngoc DuyOct 11, 2011
  4. Michael J GruberOct 11, 2011
  5. Simon KlinkertOct 12, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.