Re: [PATCH] Work around performance bug in subprocess.Popen.communicate()

From: Catalin Marinas <catalin.marinas@gmail.com>
Date: Aug 13, 2009, 22:18 UTC
Message-ID: <b0943d9e0908131518i3ac18331leb4c0c76313b0780@mail.gmail.com>
In-Reply-To: <20090804085125.GB27452@lux.e.vtech>

2009/8/4 Karl Wiberg <kha@virtutech.com>:

Show 12 quoted lines

> On 2009-07-31 12:27:53 +0100, Catalin Marinas wrote:
>
>> But can this not lead to a deadlock if the __indata is big? The
>> stdout of the created process is only processed once the whole
>> __indata is written. I thought communicate() was created to avoid
>> this issue.
>
> I don't think there's a problem. write() isn't supposed to have a
> limit on the amount of data it will accept in one call, as far as I'm
> aware. Plus, it works just fine with Erik's test case---which in my
> case was about 7 MB. If it can handle 7 MB, I doubt there's a limit
> we'll hit anytime soon.

write() itself doesn't have a limit, it's mainly what the application receiving the data can handle. In the Git case, I think it takes all the input as it isn't a filtering tool (things may be different with tools like sed etc.).

> Oh, and we still call communicate()---we just don't pass it any
> additional bytes to write to stdin.

Yes, but if write() is blocked, communicate() won't be called.

Since we are only using Git, I'll merge this patch (and maybe add a comment).

-- 
Catalin