git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Make xmalloc and xrealloc thread-safe

From
Fredrik Kuivinen <frekui@gmail.com>
Date
Mar 27, 2010, 13:26 UTC
Message-ID
<4c8ef71003270626y45685e69j28ccb8a8738b9083@mail.gmail.com>
In-Reply-To
<alpine.LFD.2.00.1003241613020.694@xanadu.home>
On Wed, Mar 24, 2010 at 21:22, Nicolas Pitre <nico@fluxnic.net> wrote:
Show 42 quoted lines
> By providing a hook for the routine responsible for trying to free some
> memory on malloc failure, we can ensure that the  called routine is
> protected by the appropriate locks when threads are in play.
>
> The obvious offender here was pack-objects which was calling xmalloc()
> within threads while release_pack_memory() is not thread safe.
>
> Signed-off-by: Nicolas Pitre <nico@fluxnic.net>
> ---
>
> On Wed, 24 Mar 2010, Shawn Pearce wrote:
>
>> On Wed, Mar 24, 2010 at 11:54 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
>> > Another solution could be for xmalloc() to use a function pointer for
>> > the method to use on malloc error path, which would default to a
>> > function calling release_pack_memory(size, -1).  Then pack-objects.c
>> > would override the default with its own to acquire the read_mutex around
>> > the call to release_pack_memory().  That is probably the easiest
>> > solution for now.
>>
>> Yea, that sounds like the most reasonable solution right now.
>
> So here it is.
>
> Note: there was a dubious usage of fd when calling release_pack_memory()
> in xmmap() which is now removed.
>
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index 9780258..65f797f 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -1522,6 +1522,13 @@ static void find_deltas(struct object_entry **list, unsigned *list_size,
>
>  #ifndef NO_PTHREADS
>
> +static void try_to_free_from_threads(size_t size)
> +{
> +       read_lock();
> +       release_pack_memory(size, -1);
> +       read_unlock();
> +}
> +

Will this really work in all cases? In the find_deltas -> try_delta -> read_sha1_file -> ... -> xmalloc call path, the mutex is already locked when we get to xmalloc. As the mutex is of the default type (NULL is passed as the mutex attribute argument to pthread_mutex_init), it is undefined behaviour to lock the mutex again (see http://www.opengroup.org/onlinepubs/007908799/xsh/pthread_mutexattr_gettype.html )

I just realised that builtin/grep.c also needs a fix for its use of xmalloc.
- Fredrik
Previous: Junio C HamanoNext: Nicolas Pitre
Message 16 of 39 in “Make xmalloc and xrealloc thread-safe”
  1. 1/2 Make xmalloc and xrealloc thread-safeFredrik Kuivinen, Mar 23, 2010
  2. Shawn O. PearceMar 23, 2010
  3. Fredrik KuivinenMar 23, 2010
  4. Nicolas PitreMar 23, 2010
  5. Fredrik KuivinenMar 24, 2010
  6. Nicolas PitreMar 24, 2010
  7. Shawn PearceMar 24, 2010
  8. Junio C HamanoMar 24, 2010
  9. Nicolas PitreMar 24, 2010
  10. Shawn PearceMar 24, 2010
  11. Make xmalloc and xrealloc thread-safeNicolas Pitre, Mar 24, 2010
  12. Shawn O. PearceMar 24, 2010
  13. Nicolas PitreMar 24, 2010
  14. Junio C HamanoMar 24, 2010
  15. Junio C HamanoMar 24, 2010
  16. Fredrik KuivinenMar 27, 2010
  17. Nicolas PitreMar 27, 2010
  18. Fredrik KuivinenMar 31, 2010
  19. Make xmalloc and xrealloc thread-safeNicolas Pitre, Apr 7, 2010
  20. Shawn O. PearceApr 7, 2010
  21. Nicolas PitreApr 7, 2010
  22. Shawn PearceApr 7, 2010
  23. Nicolas PitreApr 7, 2010
  24. Shawn PearceApr 7, 2010
  25. Nicolas PitreApr 7, 2010
  26. Fredrik KuivinenApr 7, 2010
  27. Nicolas PitreApr 7, 2010
  28. Fredrik KuivinenApr 7, 2010
  29. Erik Faye-LundApr 7, 2010
  30. Nicolas PitreApr 7, 2010
  31. Sverre RabbelierApr 7, 2010
  32. Fredrik KuivinenApr 7, 2010
  33. Junio C HamanoApr 7, 2010
  34. Johannes SixtApr 7, 2010
  35. Thread-safe xmalloc and xrealloc needs a recursive mutexJohannes Sixt, Apr 8, 2010
  36. Fredrik KuivinenApr 8, 2010
  37. Junio C HamanoApr 7, 2010
  38. 2/2 Make sha1_to_hex thread-safeFredrik Kuivinen, Mar 23, 2010
  39. Johannes SixtMar 23, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.