git/list[1] front-page[2] threads[3] people[4] search[5] about
 

RE: [PATCH v4] Add git-grep threads param

From
VLVictor Leschuk <vleschuk@accesssoftek.com>
Date
Nov 9, 2015, 18:32 UTC
Message-ID
<6AE1604EE3EC5F4296C096518C6B77EE5D0FDABA19@mail.accesssoftek.com>
In-Reply-To
<CA+55aFzHic5AN05QkbERFszRC=i3aDDGy9yhXEjgzZjwzFVBLQ@mail.gmail.com>

On Mon, Nov 9, 2015 at 9:28 AM, Victor Leschuk <vleschuk@accesssoftek.com> wrote:

Show 7 quoted lines
>
> Maybe use the simplest version (and keep num_numbers == 0 also as flag for all other checks in code like if(num_flags) .... ):
>
> if (list.nr || cached )
>   num_threads = 0; // do not use threads
> else if (num_threads == 0)
>   num_threads = online_cpus() <= 1 ? 0 : GREP_NUM_THREADS_DEFAULT;
> I will say this AGAIN.
> The number of threads is *not* about the number of CPU's. Stop this
craziness. It's wrong.
Actually I have never said the nCPUs played main role in it. The patch is intended to provide user ability to change this threads number according to their needs and to touch as small amount of other code as possible.
Show 6 quoted lines
> The number of threads is about parallelism. Yes, CPU's is a small part
> of it. But as mentioned earlier, the *big* wins are for slow
> filesystems, NFS in particular. On NFS, even if you have things
> cached, the cache revalidation is going to cause network traffic
> almost every time. Being able to have several of those outstanding is
> a big deal.
> So stop with the "online_cpus()" stuff. And don't base your benchmarks
> purely on the CPU-bound case. Because the CPU-bound case is the case
> that is already generally so good that few people will care all *that*
> deeply.
I have performed a cold-cached FS test (in previous thread to minimize the CPU part in the results) and it showed high correlation between speed and thread_num. Isn't it what you said? Even on systems with small number of cores we can gain profit of multithreading. That's I why I suggest this param to be customizable and not HARDCODED.
We need to create a clear text for the documentation that this number should not based on number of CPU-s only. Currently do not mention anything on it.

-- Victor

Previous: Jeff KingNext: Linus Torvalds
Message 15 of 19 in “Add git-grep threads param”
  1. Add git-grep threads paramVictor Leschuk, Oct 27, 2015
  2. Victor LeschukNov 2, 2015
  3. Junio C HamanoNov 2, 2015
  4. Victor LeschukNov 3, 2015
  5. Junio C HamanoNov 3, 2015
  6. Jeff KingNov 4, 2015
  7. Victor LeschukNov 9, 2015
  8. Jeff KingNov 9, 2015
  9. Victor LeschukNov 9, 2015
  10. Jeff KingNov 9, 2015
  11. Victor LeschukNov 9, 2015
  12. Jeff KingNov 9, 2015
  13. Linus TorvaldsNov 9, 2015
  14. Jeff KingNov 9, 2015
  15. Victor LeschukNov 9, 2015
  16. Linus TorvaldsNov 9, 2015
  17. Victor LeschukNov 9, 2015
  18. Stefan BellerNov 9, 2015
  19. Victor LeschukNov 9, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.