git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 0/2] grep: make threading smarter

From
Carlo Marcelo Arenas Belón <carenas@gmail.com>
Date
Aug 4, 2019, 21:15 UTC
Message-ID
<20190804211509.39229-1-carenas@gmail.com>

833e3df171 (pack-objects: Add runtime detection of online CPU's, 2008-02-22) added the capability to check the number of online CPUs at runtime to do better threading, so use that as well with grep.

Testing with a large (more than 4) number of cores and no grep.threads configuration in real hardware encouraged, to confirm that no other bottleneck is preventing the additional threads to improve performance.

If platform specific testing shows degradation (specially with HP-UX), make sure that the right number of CPUs is reported by :

  $ ./t/helper/test-tool online-cpus
  4

There are additional cleanups possible in the grep code but had left it out of this RFC to avoid confusion and make the change in patch 2 as straight forward as possible.

There is also a chance that the online_cpus() function will be updated as it predates POSIX and might be associated with one known performance issue in HP-UX[1]

Lastly the performance numbers point to deficiencies in kwset and the compat/regex code that will need to be addressed independently.

Carlo Marcelo Arenas Belón (2):
  p7810: add more grep performance relevant cases
  grep: make default number of threads reflect runtime
 Documentation/git-grep.txt |  2 +-
 builtin/grep.c             |  3 +--
 t/perf/p7810-grep.sh       | 25 +++++++++++++++++++++++--
 3 files changed, 25 insertions(+), 5 deletions(-)
[1] https://public-inbox.org/git/TU4PR8401MB121664A8A588D799803F1E84E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM/
-- 
2.23.0.rc1
Next: Carlo Marcelo Arenas Belón
Message 1 of 4 in “grep: make threading smarter”
  1. 0/2 grep: make threading smarterCarlo Marcelo Arenas Belón, Aug 4, 2019
  2. 1/2 p7810: add more grep performance relevant casesCarlo Marcelo Arenas Belón, Aug 4, 2019
  3. 2/2 grep: make default number of threads reflect runtimeCarlo Marcelo Arenas Belón, Aug 4, 2019
  4. Junio C HamanoAug 5, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.