git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Improving performance with pthreads in refresh_index().

From
Michael J Gruber <git@drmicha.warpmail.net>
Date
Oct 11, 2011, 11:50 UTC
Message-ID
<4E942D76.7030908@drmicha.warpmail.net>
In-Reply-To
<1318325521-23262-1-git-send-email-klinkert@webgods.de>
klinkert@webgods.de venit, vidit, dixit 11.10.2011 11:32:
Show 11 quoted lines
> Git performs for every file in a repository at least one (with a cold cache)
> lstat(). In larger repositories operations like git status take a
> long time. In case your local repository is located on a remote server
> (e. g. mounted via nfs) it ends up in an *incredible* slow git.
> 
> With this patch you're able to determine a number of threads (maxthreads)
> in your config file to run these tons of lstats in threads. There
> won't be created any pthreads if you haven't set maxthreads. In my
> test cases a git status with this patch performs enormously faster (over
> two minutes before and approximately 25 seconds now). Of course, it
> has a positive impact on other git commands, too.

Can you specify under which circumstances one should get a speedup? Our NFS isn't slow enough... but on a dead slow sshfs work tree I get the following for "git status -s":

maxthreads: 0, preloadindex: false, time: 14.73 maxthreads: 1, preloadindex: false, time: 14.25 maxthreads: 2, preloadindex: false, time: 13.32 maxthreads: 3, preloadindex: false, time: 12.40 maxthreads: 4, preloadindex: false, time: 12.65 maxthreads: 5, preloadindex: false, time: 12.16 maxthreads: 8, preloadindex: false, time: 12.32 maxthreads: 10, preloadindex: false, time: 11.98 maxthreads: 15, preloadindex: false, time: 12.31 maxthreads: 20, preloadindex: false, time: 12.00 maxthreads: 0, preloadindex: true, time: 12.17 maxthreads: 1, preloadindex: true, time: 11.98 maxthreads: 2, preloadindex: true, time: 12.21 maxthreads: 3, preloadindex: true, time: 11.99 maxthreads: 4, preloadindex: true, time: 12.14 maxthreads: 5, preloadindex: true, time: 12.21 maxthreads: 8, preloadindex: true, time: 12.14 maxthreads: 10, preloadindex: true, time: 12.08 maxthreads: 15, preloadindex: true, time: 12.16 maxthreads: 20, preloadindex: true, time: 11.96

So it seams it gives me what preloadindex does, which is not much.
Note: I'm not saying the patch is bad. I'm just wondering whether that
is expected.

Michael P.S.: It's actually sshfs with ssh to an NFSv3 client (server restricts exports) :(

Previous: Nguyen Thai Ngoc DuyNext: Simon Klinkert
Message 4 of 5 in “Improving performance with pthreads in refresh_index().”
  1. Improving performance with pthreads in refresh_index().klinkert@webgods.de, Oct 11, 2011
  2. Johannes SixtOct 11, 2011
  3. Nguyen Thai Ngoc DuyOct 11, 2011
  4. Michael J GruberOct 11, 2011
  5. Simon KlinkertOct 12, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.