git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: racy leak sanitizer builds, was Re: [PATCH 0/9] commit-reach: -Wsign-compare follow-ups

From
Jeff King <peff@peff.net>
Date
Dec 28, 2024, 19:31 UTC
Message-ID
<20241228193151.GA1515174@coredump.intra.peff.net>
In-Reply-To
<20241228192307.GC815586@coredump.intra.peff.net>
On Sat, Dec 28, 2024 at 02:23:07PM -0500, Jeff King wrote:
> It would probably be pretty easy to do something similar for index-pack,
> though really this is something I guess we'd need to do manually
> anywhere we spawn worker threads.

Here's what that would look like. Confirmed that this fixes the issue running t5309 with --stress. Dropping the extra locks in resolve_deltas() is just backing out that earlier insufficient attempt to synchronize things.

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index d773809c4c..de1f5d8628 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -185,6 +185,8 @@ static pthread_mutex_t deepest_delta_mutex;
 
 static pthread_key_t key;
 
+static pthread_barrier_t start_barrier;
+
 static inline void lock_mutex(pthread_mutex_t *mutex)
 {
 	if (threads_active)
@@ -209,6 +211,7 @@ static void init_thread(void)
 	if (show_stat)
 		pthread_mutex_init(&deepest_delta_mutex, NULL);
 	pthread_key_create(&key, NULL);
+	pthread_barrier_init(&start_barrier, NULL, nr_threads);
 	CALLOC_ARRAY(thread_data, nr_threads);
 	for (i = 0; i < nr_threads; i++) {
 		thread_data[i].pack_fd = xopen(curr_pack, O_RDONLY);
@@ -231,6 +234,7 @@ static void cleanup_thread(void)
 	for (i = 0; i < nr_threads; i++)
 		close(thread_data[i].pack_fd);
 	pthread_key_delete(key);
+	pthread_barrier_destroy(&start_barrier);
 	free(thread_data);
 }
 
@@ -1100,6 +1104,8 @@ static int compare_ref_delta_entry(const void *a, const void *b)
 
 static void *threaded_second_pass(void *data)
 {
+	if (threads_active)
+		pthread_barrier_wait(&start_barrier);
 	if (data)
 		set_thread_data(data);
 	for (;;) {
@@ -1336,15 +1342,13 @@ static void resolve_deltas(struct pack_idx_option *opts)
 	base_cache_limit = opts->delta_base_cache_limit * nr_threads;
 	if (nr_threads > 1 || getenv("GIT_FORCE_THREADS")) {
 		init_thread();
-		work_lock();
 		for (i = 0; i < nr_threads; i++) {
 			int ret = pthread_create(&thread_data[i].thread, NULL,
 						 threaded_second_pass, thread_data + i);
 			if (ret)
 				die(_("unable to create thread: %s"),
 				    strerror(ret));
 		}
-		work_unlock();
 		for (i = 0; i < nr_threads; i++)
 			pthread_join(thread_data[i].thread, NULL);
 		cleanup_thread();
Previous: Jeff KingNext: René Scharfe
Message 30 of 34 in “commit-reach: -Wsign-compare follow-ups”
  1. 0/9 commit-reach: -Wsign-compare follow-upsPatrick Steinhardt, Dec 27, 2024
  2. 1/9 prio-queue: fix type of `insertion_ctr`Patrick Steinhardt, Dec 27, 2024
  3. Jeff KingDec 27, 2024
  4. 2/9 commit-reach: fix index used to loop through unsigned integerPatrick Steinhardt, Dec 27, 2024
  5. Jeff KingDec 27, 2024
  6. 3/9 commit-reach: fix type of `min_commit_date`Patrick Steinhardt, Dec 27, 2024
  7. 4/9 commit-reach: use `size_t` to track indices in `remove_redundant()`Patrick Steinhardt, Dec 27, 2024
  8. Justin ToblerJan 3, 2025
  9. 5/9 commit-reach: use `size_t` to track indices in `get_reachable_subset()`Patrick Steinhardt, Dec 27, 2024
  10. 6/9 builtin/log: use `size_t` to track indicesPatrick Steinhardt, Dec 27, 2024
  11. Justin ToblerJan 3, 2025
  12. Patrick SteinhardtJan 3, 2025
  13. 7/9 builtin/log: fix remaining -Wsign-compare warningsPatrick Steinhardt, Dec 27, 2024
  14. shejialuoDec 27, 2024
  15. Patrick SteinhardtDec 27, 2024
  16. shejialuoDec 27, 2024
  17. 8/9 shallow: fix -Wsign-compare warningsPatrick Steinhardt, Dec 27, 2024
  18. 9/9 commit-reach: use `size_t` to track indices when computing merge basesPatrick Steinhardt, Dec 27, 2024
  19. Justin ToblerJan 3, 2025
  20. Patrick SteinhardtJan 3, 2025
  21. Junio C HamanoDec 27, 2024
  22. Junio C HamanoDec 27, 2024
  23. Jeff KingDec 27, 2024
  24. Patrick SteinhardtDec 28, 2024
  25. Junio C HamanoDec 28, 2024
  26. Patrick SteinhardtDec 28, 2024
  27. Junio C HamanoDec 28, 2024
  28. racy leak sanitizer builds, was Re: [PATCH 0/9] commit-reach: -Wsign-compare follow-upsJeff King, Dec 28, 2024
  29. Jeff KingDec 28, 2024
  30. Jeff KingDec 28, 2024
  31. René ScharfeDec 29, 2024
  32. Jeff KingDec 29, 2024
  33. Jeff KingDec 30, 2024
  34. Junio C HamanoDec 30, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.