git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Work around performance bug in subprocess.Popen.communicate()

From
KWKarl Wiberg <kha@virtutech.com>
Date
Jul 31, 2009, 09:36 UTC
Message-ID
<20090731093632.7018.24435.stgit@october.e.vtech>
From: Karl Wiberg <kha@treskal.com>

In Python 2.4 (specifically, I tested with 2.4.6 on Ubuntu 9.04), subprocess.Popen.communicate() seems to take time proportional to the square of the size of the indata, which makes it ridiculously expensive to write stack log entries when the diffs are large. Work around the bug by calling subprocess.Popen.stdin.write() manually instead of letting communicate() handle the indata.

The performance bug has been fixed in Python 2.6 (I tested with 2.6.2), so with that version this workaround doesn't affect the run time. I haven't tested with Python 2.5.

This fixes bug 13319.
Signed-off-by: Karl Wiberg <kha@treskal.com>
---
 stgit/run.py |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/stgit/run.py b/stgit/run.py
index 7493ed3..311d12f 100644
--- a/stgit/run.py
+++ b/stgit/run.py
@@ -110,7 +110,9 @@ class Run:
                                  stdin = subprocess.PIPE,
                                  stdout = subprocess.PIPE,
                                  stderr = subprocess.PIPE)
-            outdata, errdata = p.communicate(self.__indata)
+            if self.__indata:
+                p.stdin.write(self.__indata)
+            outdata, errdata = p.communicate()
             self.exitcode = p.returncode
         except OSError, e:
             raise self.exc('%s failed: %s' % (self.__cmd[0], e))
Next: Catalin Marinas
Message 1 of 7 in “Work around performance bug in subprocess.Popen.communicate()”
  1. Work around performance bug in subprocess.Popen.communicate()Karl Wiberg, Jul 31, 2009
  2. Catalin MarinasJul 31, 2009
  3. Karl WibergAug 4, 2009
  4. Catalin MarinasAug 13, 2009
  5. Karl WibergAug 14, 2009
  6. Erik SandbergAug 14, 2009
  7. Karl WibergAug 14, 2009

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.