Re: [PATCH 1/2] Make xmalloc and xrealloc thread-safe

From: Fredrik Kuivinen <frekui@gmail.com>
Date: Mar 24, 2010, 15:23 UTC
Message-ID: <4c8ef71003240823o7cd733bn5f19699305c94cba@mail.gmail.com>
In-Reply-To: <alpine.LFD.2.00.1003231945480.31128@xanadu.home>

On Wed, Mar 24, 2010 at 00:50, Nicolas Pitre <nico@fluxnic.net> wrote:

Show 13 quoted lines

> On Tue, 23 Mar 2010, Fredrik Kuivinen wrote:
>
>> On Tue, Mar 23, 2010 at 19:43, Shawn O. Pearce <spearce@spearce.org> wrote:
>> > If that is what we are doing, disabling the release of pack windows
>> > when malloc fails, why can't we do that all of the time?
>>
>> The idea was that most git programs are single threaded, so they can
>> still benefit from releasing the pack windows when they are low on
>> memory.
>
> This is bobus. The Git program using the most memory is probably
> pack-objects and it is threaded.  Most single-threaded programs don't
> use close to as much memory.

Ok, you are right. But xmalloc/xrealloc cannot be used in multiple threads simultaneously without some serialization.

For example, I think there are some potential race conditions in the pack-objects code. In the threaded code we have the following call chains leading to xcalloc, xmalloc, and xrealloc:

find_deltas -> xcalloc find_deltas -> do_compress -> xmalloc find_deltas -> try_delta -> xrealloc find_deltas -> try_delta -> read_sha1_file -> ... -> xmalloc (called with read_lock held, but it can still race with the other calls)

As far as I can see there is no serialization between these calls.

- Fredrik