git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] status: read submodule process output before calling wait()

From
Jeff King <peff@peff.net>
Date
Mar 22, 2015, 07:44 UTC
Message-ID
<20150322074455.GA1303@peff.net>
In-Reply-To
<CAPx-p4sKD0Nut3E1jnWfPPx4=--ZOxBgiVdt3RRb5tktw31qDg@mail.gmail.com>
On Sat, Mar 21, 2015 at 10:56:54PM -0700, Wincent Colaiuta wrote:
Show 7 quoted lines
> I've never seen this hang before despite frequent use of submodules.
> Oddly, I was able to work around the hang by moving the submodule in
> two hops (one from Ansible v1.6.10 to v1.7.0, then from v1.7.0 to
> v1.8.4). I am not sure if this is specific to the Ansible repo, or
> whether the length of the summary is crossing some threshold that
> triggers the bug to manifest. If I run the forked commands manually
> from an interactive shell, they complete just fine.
It's the length of the summary. The fix is below.

-- >8 -- The status code tries to read the output of "git submodule summary" over a pipe by waiting for the program to finish and then reading its output, like this:

  run_command(&sm_summary);
  len = strbuf_read(&cmd_stdout, sm_summary.out, 1024);

Besides being a violation of the run-command API (which makes no promises about the state of the struct after run_command returns), this can easily lead to deadlock. The "submodule status" process may fill up the pipe buffer and block on write(). Meanwhile, the reading side in the parent process is blocked in wait(), waiting for the child to finish.

Instead, we should start the process, read everything it produces, and only then call wait() to finish it off.

Signed-off-by: Jeff King <peff@peff.net>
---
I notice that we also don't detect when the sub-command fails. I don't
know what we would do in that case (abort the status? print a message?)
and it's orthogonal to this issue, so I left it for somebody more
clueful in the area to think about.
 wt-status.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/wt-status.c b/wt-status.c
index 7036fa2..96f0033 100644
--- a/wt-status.c
+++ b/wt-status.c
@@ -748,9 +748,9 @@ static void wt_status_print_submodule_summary(struct wt_status *s, int uncommitt
 	fflush(s->fp);
 	sm_summary.out = -1;
 
-	run_command(&sm_summary);
-
+	start_command(&sm_summary);
 	len = strbuf_read(&cmd_stdout, sm_summary.out, 1024);
+	finish_command(&sm_summary);
 
 	/* prepend header, only if there's an actual output */
 	if (len) {
-- 
2.3.3.618.ga041503
Previous: Wincent ColaiutaNext: Jeff King
Message 2 of 26 in “status hangs trying to get submodule summary”
  1. Wincent ColaiutaMar 22, 2015
  2. status: read submodule process output before calling wait()Jeff King, Mar 22, 2015
  3. Jeff KingMar 22, 2015
  4. 0/7 introduce strbuf_read_cmd to avoid deadlocksJeff King, Mar 22, 2015
  5. 1/7 wt-status: don't flush before running "submodule status"Jeff King, Mar 22, 2015
  6. 2/7 wt_status: fix signedness mismatch in strbuf_read callJeff King, Mar 22, 2015
  7. 3/7 strbuf: introduce strbuf_read_cmd helperJeff King, Mar 22, 2015
  8. Eric SunshineMar 22, 2015
  9. Junio C HamanoMar 22, 2015
  10. Junio C HamanoMar 22, 2015
  11. 0/7 introduce capture_command to avoid deadlocksJeff King, Mar 23, 2015
  12. 1/7 wt-status: don't flush before running "submodule status"Jeff King, Mar 23, 2015
  13. 2/7 wt_status: fix signedness mismatch in strbuf_read callJeff King, Mar 23, 2015
  14. 3/7 run-command: introduce capture_command helperJeff King, Mar 23, 2015
  15. 4/7 wt-status: use capture_commandJeff King, Mar 23, 2015
  16. 5/7 submodule: use capture_commandJeff King, Mar 23, 2015
  17. 6/7 trailer: use capture_commandJeff King, Mar 23, 2015
  18. 7/7 run-command: forbid using run_command with piped outputJeff King, Mar 23, 2015
  19. Junio C HamanoMar 23, 2015
  20. Jeff KingMar 22, 2015
  21. Junio C HamanoMar 22, 2015
  22. Jeff KingMar 22, 2015
  23. 4/7 wt-status: use strbuf_read_cmdJeff King, Mar 22, 2015
  24. 5/7 submodule: use strbuf_read_cmdJeff King, Mar 22, 2015
  25. 6/7 trailer: use strbuf_read_cmdJeff King, Mar 22, 2015
  26. 7/7 run-command: forbid using run_command with piped outputJeff King, Mar 22, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.