git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 11/12] grep: move driver pre-load out of critical section

From
Matheus Tavares <matheus.bernardino@usp.br>
Date
Jan 16, 2020, 02:39 UTC
Message-ID
<2f72f3034118432381f3c9378e70a65d27e3dfbb.1579141989.git.matheus.bernardino@usp.br>
In-Reply-To
<cover.1579141989.git.matheus.bernardino@usp.br>

In builtin/grep.c:add_work() we pre-load the userdiff drivers before adding the grep_source in the todo list. This operation is currently being performed after acquiring the grep_mutex, but as it's already thread-safe, we don't need to protect it here. So let's move it out of the critical section which should avoid thread contention and improve performance.

Running[1] `git grep --threads=8 abcd[02] HEAD` on chromium's repository[2], I got the following mean times for 30 executions after 2 warmups:

        Original         |  6.2886s
-------------------------|-----------
 Out of critical section |  5.7852s
[1]: Tests performed on an i7-7700HQ with 16GB of RAM and SSD, running
     Manjaro Linux.
[2]: chromium’s repo at commit 03ae96f (“Add filters testing at DSF=2”,
         04-06-2019), after a 'git gc' execution.
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/grep.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 6aaa8d4406..a85b710b48 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -92,8 +92,11 @@ static pthread_cond_t cond_result;
 
 static int skip_first_line;
 
-static void add_work(struct grep_opt *opt, const struct grep_source *gs)
+static void add_work(struct grep_opt *opt, struct grep_source *gs)
 {
+	if (opt->binary != GREP_BINARY_TEXT)
+		grep_source_load_driver(gs, opt->repo->index);
+
 	grep_lock();
 
 	while ((todo_end+1) % ARRAY_SIZE(todo) == todo_done) {
@@ -101,9 +104,6 @@ static void add_work(struct grep_opt *opt, const struct grep_source *gs)
 	}
 
 	todo[todo_end].source = *gs;
-	if (opt->binary != GREP_BINARY_TEXT)
-		grep_source_load_driver(&todo[todo_end].source,
-					opt->repo->index);
 	todo[todo_end].done = 0;
 	strbuf_reset(&todo[todo_end].out);
 	todo_end = (todo_end + 1) % ARRAY_SIZE(todo);

-- 
2.24.1
Previous: Matheus TavaresNext: Matheus Tavares
Message 44 of 47 in “grep: re-enable threads when cached, w/ parallel inflation”
  1. Matheus TavaresAug 10, 2019
  2. [GSoC][PATCH 1/4] object-store: add lock to read_object_file_extended()Matheus Tavares, Aug 10, 2019
  3. [GSoC][PATCH 2/4] grep: allow locks to be enabled individuallyMatheus Tavares, Aug 10, 2019
  4. [GSoC][PATCH 3/4] grep: disable grep_read_mutex when possibleMatheus Tavares, Aug 10, 2019
  5. [GSoC][PATCH 4/4] grep: re-enable threads in some non-worktree casesMatheus Tavares, Aug 10, 2019
  6. 00/11 grep: improve threading and fix race conditionsMatheus Tavares, Sep 30, 2019
  7. 01/11 grep: fix race conditions on userdiff callsMatheus Tavares, Sep 30, 2019
  8. 02/11 grep: fix race conditions at grep_submodule()Matheus Tavares, Sep 30, 2019
  9. 03/11 grep: fix racy calls in grep_objects()Matheus Tavares, Sep 30, 2019
  10. 04/11 replace-object: make replace operations thread-safeMatheus Tavares, Sep 30, 2019
  11. 05/11 object-store: allow threaded access to object readingMatheus Tavares, Sep 30, 2019
  12. Jonathan TanNov 12, 2019
  13. Jeff KingNov 13, 2019
  14. Matheus Tavares BernardinoNov 14, 2019
  15. Jeff KingNov 14, 2019
  16. Jonathan TanNov 14, 2019
  17. Jeff KingNov 15, 2019
  18. Matheus Tavares BernardinoDec 19, 2019
  19. Matheus Tavares BernardinoJan 9, 2020
  20. Christian CouderJan 10, 2020
  21. 06/11 grep: replace grep_read_mutex by internal obj read lockMatheus Tavares, Sep 30, 2019
  22. squash! grep: replace grep_read_mutex by internal obj read lockMatheus Tavares, Oct 1, 2019
  23. 07/11 submodule-config: add skip_if_read option to repo_read_gitmodules()Matheus Tavares, Sep 30, 2019
  24. 08/11 grep: allow submodule functions to run in parallelMatheus Tavares, Sep 30, 2019
  25. 09/11 grep: protect packed_git [re-]initializationMatheus Tavares, Sep 30, 2019
  26. 10/11 grep: re-enable threads in non-worktree caseMatheus Tavares, Sep 30, 2019
  27. 11/11 grep: move driver pre-load out of critical sectionMatheus Tavares, Sep 30, 2019
  28. 00/12 grep: improve threading and fix race conditionsMatheus Tavares, Jan 16, 2020
  29. 01/12 grep: fix race conditions on userdiff callsMatheus Tavares, Jan 16, 2020
  30. 02/12 grep: fix race conditions at grep_submodule()Matheus Tavares, Jan 16, 2020
  31. 03/12 grep: fix racy calls in grep_objects()Matheus Tavares, Jan 16, 2020
  32. 04/12 replace-object: make replace operations thread-safeMatheus Tavares, Jan 16, 2020
  33. 05/12 object-store: allow threaded access to object readingMatheus Tavares, Jan 16, 2020
  34. 06/12 grep: replace grep_read_mutex by internal obj read lockMatheus Tavares, Jan 16, 2020
  35. 07/12 submodule-config: add skip_if_read option to repo_read_gitmodules()Matheus Tavares, Jan 16, 2020
  36. 08/12 grep: allow submodule functions to run in parallelMatheus Tavares, Jan 16, 2020
  37. SZEDER GáborJan 29, 2020
  38. Junio C HamanoJan 29, 2020
  39. Junio C HamanoJan 29, 2020
  40. Matheus Tavares BernardinoJan 29, 2020
  41. Philippe BlainJan 30, 2020
  42. 09/12 grep: protect packed_git [re-]initializationMatheus Tavares, Jan 16, 2020
  43. 10/12 grep: re-enable threads in non-worktree caseMatheus Tavares, Jan 16, 2020
  44. 11/12 grep: move driver pre-load out of critical sectionMatheus Tavares, Jan 16, 2020
  45. 12/12 grep: use no. of cores as the default no. of threadsMatheus Tavares, Jan 16, 2020
  46. Victor LeschukJan 16, 2020
  47. Matheus TavaresJan 16, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.