git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/5] pack-objects: take lock before accessing `remaining`

From
Johannes Sixt <j6t@kdbg.org>
Date
Aug 15, 2017, 19:50 UTC
Message-ID
<88ddf942-d7e4-ef1c-3f9c-9816d0571502@kdbg.org>
In-Reply-To
<5815ea4f27226b604751961c8b70355a8925f0c5.1502780344.git.martin.agren@gmail.com>
Am 15.08.2017 um 14:53 schrieb Martin Ågren:
Show 51 quoted lines
> When checking the conditional of "while (me->remaining)", we did not
> hold the lock. Calling find_deltas would still be safe, since it checks
> "remaining" (after taking the lock) and is able to handle all values. In
> fact, this could (currently) not trigger any bug: a bug could happen if
> `remaining` transitioning from zero to non-zero races with the evaluation
> of the while-condition, but these are always separated by the
> data_ready-mechanism.
> 
> Make sure we have the lock when we read `remaining`. This does mean we
> release it just so that find_deltas can take it immediately again. We
> could tweak the contract so that the lock should be taken before calling
> find_deltas, but let's defer that until someone can actually show that
> "unlock+lock" has a measurable negative impact.
> 
> Signed-off-by: Martin Ågren <martin.agren@gmail.com>
> ---
> I don't think this corrects any real error. The benefits of this patch
> would be "future-proofs things slightly" and "silences tsan, so that
> other errors don't drown in noise". Feel free to tell me those benefits
> are negligible and that this change actually hurts.
> 
>   builtin/pack-objects.c | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index c753e9237..bd391e97a 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -2170,7 +2170,10 @@ static void *threaded_find_deltas(void *arg)
>   {
>   	struct thread_params *me = arg;
>   
> +	progress_lock();
>   	while (me->remaining) {
> +		progress_unlock();
> +
>   		find_deltas(me->list, &me->remaining,
>   			    me->window, me->depth, me->processed);
>   
> @@ -2192,7 +2195,10 @@ static void *threaded_find_deltas(void *arg)
>   			pthread_cond_wait(&me->cond, &me->mutex);
>   		me->data_ready = 0;
>   		pthread_mutex_unlock(&me->mutex);
> +
> +		progress_lock();
>   	}
> +	progress_unlock();
>   	/* leave ->working 1 so that this doesn't get more work assigned */
>   	return NULL;
>   }
> 

It is correct that this access of me->remaining requires a lock. It could be solved in the way you did. I tried to do it differently, but all cleaner solutions that I can think of are overkill...

-- Hannes
Previous: Martin ÅgrenNext: Martin Ågren
Message 7 of 64 in “Some ThreadSanitizer-results”
  1. 0/5 Some ThreadSanitizer-resultsMartin Ågren, Aug 15, 2017
  2. 1/5 convert: initialize attr_action in convert_attrsMartin Ågren, Aug 15, 2017
  3. Torsten BögershausenAug 15, 2017
  4. Torsten BögershausenAug 15, 2017
  5. Martin ÅgrenAug 15, 2017
  6. 2/5 pack-objects: take lock before accessing `remaining`Martin Ågren, Aug 15, 2017
  7. Johannes SixtAug 15, 2017
  8. 5/5 ThreadSanitizer: add suppressionsMartin Ågren, Aug 15, 2017
  9. tsan: t3008: hashmap_add touches size from multiple threadsMartin Ågren, Aug 15, 2017
  10. Jeff HostetlerAug 15, 2017
  11. Stefan BellerAug 15, 2017
  12. Martin ÅgrenAug 15, 2017
  13. Stefan BellerAug 15, 2017
  14. Martin ÅgrenAug 15, 2017
  15. Jeff HostetlerAug 15, 2017
  16. hashmap: address ThreadSanitizer concernsJeff Hostetler, Aug 30, 2017
  17. hashmap: add API to disable item counting when threadedJeff Hostetler, Aug 30, 2017
  18. Johannes SchindelinSep 1, 2017
  19. Jonathan NiederSep 1, 2017
  20. Jeff HostetlerSep 5, 2017
  21. Martin ÅgrenSep 5, 2017
  22. Jeff KingSep 2, 2017
  23. Johannes SchindelinSep 4, 2017
  24. Jeff HostetlerSep 5, 2017
  25. Junio C HamanoSep 6, 2017
  26. Jeff HostetlerSep 5, 2017
  27. Jeff KingSep 2, 2017
  28. Jeff HostetlerSep 5, 2017
  29. Simon RuderichSep 2, 2017
  30. Junio C HamanoSep 6, 2017
  31. Jeff HostetlerSep 6, 2017
  32. hashmap: address ThreadSanitizer concernsJeff Hostetler, Sep 6, 2017
  33. hashmap: add API to disable item counting when threadedJeff Hostetler, Sep 6, 2017
  34. tsan: t5400: set_try_to_free_routineMartin Ågren, Aug 15, 2017
  35. Stefan BellerAug 15, 2017
  36. Martin ÅgrenAug 15, 2017
  37. Jeff KingAug 17, 2017
  38. 4/5 strbuf_reset: don't write to slopbuf with ThreadSanitizerMartin Ågren, Aug 15, 2017
  39. Junio C HamanoAug 15, 2017
  40. Martin ÅgrenAug 15, 2017
  41. Junio C HamanoAug 15, 2017
  42. 3/5 Makefile: define GIT_THREAD_SANITIZERMartin Ågren, Aug 15, 2017
  43. Jeff KingAug 20, 2017
  44. Martin ÅgrenAug 20, 2017
  45. 0/4 Some ThreadSanitizer-resultsMartin Ågren, Aug 21, 2017
  46. 1/4 convert: always initialize attr_action in convert_attrsMartin Ågren, Aug 21, 2017
  47. 2/4 pack-objects: take lock before accessing `remaining`Martin Ågren, Aug 21, 2017
  48. 3/4 strbuf_setlen: don't write to strbuf_slopbufMartin Ågren, Aug 21, 2017
  49. Junio C HamanoAug 23, 2017
  50. Martin ÅgrenAug 23, 2017
  51. Junio C HamanoAug 23, 2017
  52. Brandon CaseyAug 23, 2017
  53. Junio C HamanoAug 23, 2017
  54. Brandon CaseyAug 23, 2017
  55. Brandon CaseyAug 23, 2017
  56. Brandon CaseyAug 23, 2017
  57. Junio C HamanoAug 24, 2017
  58. Brandon CaseyAug 24, 2017
  59. Martin ÅgrenAug 24, 2017
  60. Junio C HamanoAug 23, 2017
  61. Brandon CaseyAug 23, 2017
  62. 4/4 ThreadSanitizer: add suppressionsMartin Ågren, Aug 21, 2017
  63. Jeff KingAug 25, 2017
  64. Jeff HostetlerAug 28, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.