git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git grep --threads 12 --textconv is effectively single-threaded

From
Zach Riggle <zachriggle@gmail.com>
Date
Jul 7, 2020, 21:25 UTC
Message-ID
<CAMP9c5nUteg_HouuYJZtq7g4MrSE638mns=HeKhNpNTYgQB4=w@mail.gmail.com>

It looks like the bit of code that is responsible for performing textconv conversions is single-threaded, even if git-grep is provided a number of threads to use.

This greatly limits the usability of textconv filters for grep expressions over a large number of files.

Note the difference in total CPU usage in the following expressions:

$ git grep --threads 12 -e foobar --and -e fizzbuzz &> /dev/null 0.24s user 0.28s system 710% cpu 0.073 total

$ git grep --threads 12 -e foobar --and -e fizzbuzz --textconv &> /dev/null 0.90s user 1.75s system 110% cpu 2.390 total

$ git version git version 2.27.0

Zach Riggle
Next: Jeff King
Message 1 of 7 in “git grep --threads 12 --textconv is effectively single-threaded”
  1. Zach RiggleJul 7, 2020
  2. Jeff KingJul 7, 2020
  3. Zach RiggleJul 8, 2020
  4. Jeff KingJul 8, 2020
  5. Junio C HamanoJul 8, 2020
  6. Jeff KingJul 9, 2020
  7. Junio C HamanoJul 10, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.