git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: "git grep" parallelism question

From
Junio C Hamano <gitster@pobox.com>
Date
Apr 26, 2013, 18:47 UTC
Message-ID
<7vr4hxw2mp.fsf@alter.siamese.dyndns.org>
In-Reply-To
<CA+55aFxY2PJ+L=vCfvQ39UGBr7E6m5q76hO=z3Mqm6vTQmmMbw@mail.gmail.com>
Linus Torvalds <torvalds@linux-foundation.org> writes:
> Wouldn't it be lovely if it was slightly smarter (something more akin
> to the index preloading that takes number of files into account) or at
> least allowed people to set the parallelism explicitly with a command
> line switch?

Yeah, a reasonable starting point for auto-tuning may be to use the same kind of parameters and heuristics (i.e. max parallel of 20 threads, assume a cost to use an extra thread is the same as running 500 greps), and then tweak them (for example, thread cost of 500 may be reasonable for lstat() but it would be way too big for grep()).

The real issue may be that we do not have a good estimate of how many paths are involved in the request before starting these threads, though.

Previous: Linus TorvaldsNext: Linus Torvalds
Message 2 of 16 in “"git grep" parallelism question”
  1. Linus TorvaldsApr 26, 2013
  2. Junio C HamanoApr 26, 2013
  3. Linus TorvaldsApr 26, 2013
  4. Junio C HamanoApr 26, 2013
  5. Linus TorvaldsApr 26, 2013
  6. Thomas RastApr 27, 2013
  7. Ramkumar RamachandraApr 29, 2013
  8. John KeepingApr 29, 2013
  9. Thomas RastApr 29, 2013
  10. John KeepingApr 29, 2013
  11. Junio C HamanoApr 29, 2013
  12. John KeepingApr 30, 2013
  13. Jeff KingApr 30, 2013
  14. John KeepingApr 30, 2013
  15. Jeff KingApr 30, 2013
  16. Pete WyckoffMay 5, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.