git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 08/14] grep: clean up num_threads handling

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Nov 3, 2018, 08:48 UTC
Message-ID
<20181103084850.9584-9-pclouds@gmail.com>
In-Reply-To
<20181103084850.9584-1-pclouds@gmail.com>

When NO_PTHREADS is still used in this file, we have two separate code paths for thread and no thread support. The latter will always have num_threads remain zero while the former uses num_threads zero as "default number of threads".

With recent changes blur the line between thread and no-thread support, this num_threads handling becomes a bit strange so let's redefine it like this:

- num_threads == 0 means default number of threads and should become
  positive after all configuration and option parsing is done if
  multithread is supported.
- num_threads <= 1 runs no threads. It does not matter if the platform
  supports threading or not.
- num_threads > 1 will run multiple threads and is invalid if
  HAVE_THREADS is false. pthread API is only used in this case.

PS. a new warning is also added when num_threads is forced back to one because a thread-incompatible option is specified.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/grep.c | 58 +++++++++++++++++++++++---------------------------
 1 file changed, 27 insertions(+), 31 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 6dd15dbaa2..de3f568cee 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -69,13 +69,11 @@ static pthread_mutex_t grep_mutex;
 
 static inline void grep_lock(void)
 {
-	assert(num_threads);
 	pthread_mutex_lock(&grep_mutex);
 }
 
 static inline void grep_unlock(void)
 {
-	assert(num_threads);
 	pthread_mutex_unlock(&grep_mutex);
 }
 
@@ -234,7 +232,7 @@ static int wait_all(void)
 	int i;
 
 	if (!HAVE_THREADS)
-		return 0;
+		BUG("Never call this function unless you have started threads");
 
 	grep_lock();
 	all_work_added = 1;
@@ -279,14 +277,14 @@ static int grep_cmd_config(const char *var, const char *value, void *cb)
 		if (num_threads < 0)
 			die(_("invalid number of threads specified (%d) for %s"),
 			    num_threads, var);
-		else if (!HAVE_THREADS && num_threads && num_threads != 1) {
+		else if (!HAVE_THREADS && num_threads > 1) {
 			/*
 			 * TRANSLATORS: %s is the configuration
 			 * variable for tweaking threads, currently
 			 * grep.threads
 			 */
 			warning(_("no threads support, ignoring %s"), var);
-			num_threads = 0;
+			num_threads = 1;
 		}
 	}
 
@@ -323,7 +321,7 @@ static int grep_oid(struct grep_opt *opt, const struct object_id *oid,
 	grep_source_init(&gs, GREP_SOURCE_OID, pathbuf.buf, path, oid);
 	strbuf_release(&pathbuf);
 
-	if (HAVE_THREADS && num_threads) {
+	if (num_threads > 1) {
 		/*
 		 * add_work() copies gs and thus assumes ownership of
 		 * its fields, so do not call grep_source_clear()
@@ -353,7 +351,7 @@ static int grep_file(struct grep_opt *opt, const char *filename)
 	grep_source_init(&gs, GREP_SOURCE_FILE, buf.buf, filename, filename);
 	strbuf_release(&buf);
 
-	if (HAVE_THREADS && num_threads) {
+	if (num_threads > 1) {
 		/*
 		 * add_work() copies gs and thus assumes ownership of
 		 * its fields, so do not call grep_source_clear()
@@ -1025,36 +1023,34 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	pathspec.recursive = 1;
 	pathspec.recurse_submodules = !!recurse_submodules;
 
-	if (HAVE_THREADS) {
-		if (list.nr || cached || show_in_pager)
-			num_threads = 0;
-		else if (num_threads == 0)
-			num_threads = GREP_NUM_THREADS_DEFAULT;
-		else if (num_threads < 0)
-			die(_("invalid number of threads specified (%d)"), num_threads);
-		if (num_threads == 1)
-			num_threads = 0;
+	if (list.nr || cached || show_in_pager) {
+		if (num_threads > 1)
+			warning(_("invalid option combination, ignoring --threads"));
+		num_threads = 1;
+	} else if (!HAVE_THREADS && num_threads > 1) {
+		warning(_("no threads support, ignoring --threads"));
+		num_threads = 1;
+	} else if (num_threads < 0)
+		die(_("invalid number of threads specified (%d)"), num_threads);
+	else if (num_threads == 0)
+		num_threads = HAVE_THREADS ? GREP_NUM_THREADS_DEFAULT : 1;
+
+	if (num_threads > 1) {
+		if (!HAVE_THREADS)
+			BUG("Somebody got num_threads calculation wrong!");
+		if (!(opt.name_only || opt.unmatch_name_only || opt.count)
+		    && (opt.pre_context || opt.post_context ||
+			opt.file_break || opt.funcbody))
+			skip_first_line = 1;
+		start_threads(&opt);
 	} else {
-		if (num_threads)
-			warning(_("no threads support, ignoring --threads"));
-		num_threads = 0;
-	}
-
-	if (!num_threads)
 		/*
 		 * The compiled patterns on the main path are only
 		 * used when not using threading. Otherwise
-		 * start_threads() below calls compile_grep_patterns()
+		 * start_threads() above calls compile_grep_patterns()
 		 * for each thread.
 		 */
 		compile_grep_patterns(&opt);
-
-	if (HAVE_THREADS && num_threads) {
-		if (!(opt.name_only || opt.unmatch_name_only || opt.count)
-		    && (opt.pre_context || opt.post_context ||
-			opt.file_break || opt.funcbody))
-			skip_first_line = 1;
-		start_threads(&opt);
 	}
 
 	if (show_in_pager && (cached || list.nr))
@@ -1106,7 +1102,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		hit = grep_objects(&opt, &pathspec, &list);
 	}
 
-	if (num_threads)
+	if (num_threads > 1)
 		hit |= wait_all();
 	if (hit && show_in_pager)
 		run_pager(&opt, prefix);
-- 
2.19.1.1005.gac84295441
Previous: Nguyễn Thái Ngọc DuyNext: Nguyễn Thái Ngọc Duy
Message 60 of 66 in “Reduce #ifdef NO_PTHREADS”
  1. 00/10 Reduce #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  2. 01/10 thread-utils: macros to unconditionally compile pthreads APINguyễn Thái Ngọc Duy, Oct 27, 2018
  3. Jeff KingOct 27, 2018
  4. Duy NguyenOct 27, 2018
  5. Jeff KingOct 27, 2018
  6. Duy NguyenOct 27, 2018
  7. 03/10 name-hash.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  8. 04/10 attr.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  9. 05/10 send-pack.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  10. Jeff KingOct 27, 2018
  11. 06/10 grep: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  12. Jeff KingOct 27, 2018
  13. Junio C HamanoOct 29, 2018
  14. Jeff KingOct 29, 2018
  15. Duy NguyenOct 29, 2018
  16. Jeff KingOct 29, 2018
  17. Junio C HamanoOct 30, 2018
  18. 02/10 index-pack: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  19. Jeff KingOct 27, 2018
  20. 07/10 preload-index.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  21. Ben PeartOct 29, 2018
  22. 08/10 pack-objects: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  23. 09/10 read-cache.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  24. Ben PeartOct 29, 2018
  25. Duy NguyenOct 29, 2018
  26. Ben PeartOct 29, 2018
  27. Junio C HamanoOct 30, 2018
  28. 10/10 Clean up pthread_create() error handlingNguyễn Thái Ngọc Duy, Oct 27, 2018
  29. Jeff KingOct 27, 2018
  30. Jeff KingOct 27, 2018
  31. Duy NguyenOct 27, 2018
  32. 00/10 Reduce #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  33. 01/10 thread-utils: macros to unconditionally compile pthreads APINguyễn Thái Ngọc Duy, Oct 27, 2018
  34. 02/10 index-pack: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  35. 04/10 attr.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  36. 03/10 name-hash.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  37. 05/10 grep: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  38. 06/10 preload-index.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  39. Ben PeartOct 29, 2018
  40. Duy NguyenOct 29, 2018
  41. Ben PeartOct 29, 2018
  42. Jeff KingOct 29, 2018
  43. 07/10 pack-objects: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  44. 08/10 read-cache.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Oct 27, 2018
  45. Jeff KingOct 29, 2018
  46. Ben PeartOct 29, 2018
  47. Ben PeartOct 29, 2018
  48. 09/10 Clean up pthread_create() error handlingNguyễn Thái Ngọc Duy, Oct 27, 2018
  49. 10/10 read-cache.c: initialize copy_len to shut up gcc 8Nguyễn Thái Ngọc Duy, Oct 27, 2018
  50. Jeff KingOct 29, 2018
  51. 00/14 Reduce #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  52. 01/14 thread-utils: macros to unconditionally compile pthreads APINguyễn Thái Ngọc Duy, Nov 3, 2018
  53. 02/14 run-command.h: include thread-utils.h instead of pthread.hNguyễn Thái Ngọc Duy, Nov 3, 2018
  54. 03/14 send-pack.c: move async's #ifdef NO_PTHREADS back to run-command.cNguyễn Thái Ngọc Duy, Nov 3, 2018
  55. 04/14 index-pack: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  56. 05/14 name-hash.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  57. 06/14 attr.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  58. 07/14 grep: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  59. 09/14 preload-index.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  60. 08/14 grep: clean up num_threads handlingNguyễn Thái Ngọc Duy, Nov 3, 2018
  61. 10/14 pack-objects: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  62. 11/14 read-cache.c: remove #ifdef NO_PTHREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  63. 12/14 read-cache.c: reduce branching based on HAVE_THREADSNguyễn Thái Ngọc Duy, Nov 3, 2018
  64. 13/14 read-cache.c: initialize copy_len to shut up gcc 8Nguyễn Thái Ngọc Duy, Nov 3, 2018
  65. 14/14 Clean up pthread_create() error handlingNguyễn Thái Ngọc Duy, Nov 3, 2018
  66. Junio C HamanoNov 6, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.