git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 6/8] run-command: Fix missing output from late callbacks

From
Stefan Beller <sbeller@google.com>
Date
Oct 20, 2015, 22:43 UTC
Message-ID
<1445381030-23912-7-git-send-email-sbeller@google.com>
In-Reply-To
<1445381030-23912-1-git-send-email-sbeller@google.com>

The callbacks in the parallel processing API were given the contract, that they are not allowed to print anything to stdout/err, but the API will take care of their output needs instead.

In case a child process is started, the callback can first add its messages to the buffer and then the child process output is buffered just in the same buffer, and so the output will be taken care of eventually once the child process is done.

When no child process is run, we also need to fulfill our promise to output the buffer eventually. So when no child process is started, we need to amend the contents of the buffer passed to the child to our buffer for finished processes. We cannot output directly at that point in time as another process may be in the middle of its output.

The buffer for finished child processes then also needs to be flushed in the cleanup phase as it may contain data.

Signed-off-by: Stefan Beller <sbeller@google.com>
---
 run-command.c          | 11 ++++++++++-
 t/t0061-run-command.sh |  9 +++++++++
 test-run-command.c     | 13 +++++++++++++
 3 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/run-command.c b/run-command.c
index d354c01..fa73dae 100644
--- a/run-command.c
+++ b/run-command.c
@@ -988,6 +988,12 @@ static void pp_cleanup(struct parallel_processes *pp)
 
 	free(pp->children);
 	free(pp->pfd);
+
+	/*
+	 * When get_next_task added messages to the buffer in its last
+	 * iteration, the buffered output is non empty.
+	 */
+	fputs(pp->buffered_output.buf, stderr);
 	strbuf_release(&pp->buffered_output);
 
 	sigchain_pop_common();
@@ -1023,8 +1029,11 @@ static int pp_start_one(struct parallel_processes *pp)
 	if (!pp->get_next_task(&pp->children[i].data,
 			       &pp->children[i].process,
 			       &pp->children[i].err,
-			       pp->data))
+			       pp->data)) {
+		strbuf_addbuf(&pp->buffered_output, &pp->children[i].err);
+		strbuf_reset(&pp->children[i].err);
 		return 1;
+	}
 
 	if (start_command(&pp->children[i].process)) {
 		int code = pp->start_failure(&pp->children[i].process,
diff --git a/t/t0061-run-command.sh b/t/t0061-run-command.sh
index f27ada7..12228b4 100755
--- a/t/t0061-run-command.sh
+++ b/t/t0061-run-command.sh
@@ -91,4 +91,13 @@ test_expect_success 'run_command is asked to abort gracefully' '
 	test_cmp expect actual
 '
 
+cat >expect <<-EOF
+no further jobs available
+EOF
+
+test_expect_success 'run_command outputs ' '
+	test-run-command run-command-no-jobs 3 sh -c "printf \"%s\n%s\n\" Hello World" 2>actual &&
+	test_cmp expect actual
+'
+
 test_done
diff --git a/test-run-command.c b/test-run-command.c
index c8770c2..13e5d44 100644
--- a/test-run-command.c
+++ b/test-run-command.c
@@ -34,6 +34,15 @@ static int parallel_next(void** task_cb,
 	return 1;
 }
 
+static int no_job(void** task_cb,
+		  struct child_process *cp,
+		  struct strbuf *err,
+		  void *cb)
+{
+	strbuf_addf(err, "no further jobs available\n");
+	return 0;
+}
+
 static int task_finished(int result,
 			 struct child_process *cp,
 			 struct strbuf *err,
@@ -73,6 +82,10 @@ int main(int argc, char **argv)
 		exit(run_processes_parallel(jobs, parallel_next,
 					    NULL, task_finished, &proc));
 
+	if (!strcmp(argv[1], "run-command-no-jobs"))
+		exit(run_processes_parallel(jobs, no_job,
+					    NULL, task_finished, &proc));
+
 	fprintf(stderr, "check usage\n");
 	return 1;
 }
-- 
2.5.0.275.gbfc1651.dirty
Previous: Stefan BellerNext: Stefan Beller
Message 11 of 17 in “Fixes for the parallel processing engine and git submodule update”
  1. 0/8 Fixes for the parallel processing engine and git submodule updateStefan Beller, Oct 20, 2015
  2. 1/8 run-command: Fix early shutdownStefan Beller, Oct 20, 2015
  3. 2/8 run-command: Call get_next_task with a clean child process.Stefan Beller, Oct 20, 2015
  4. Junio C HamanoOct 20, 2015
  5. Junio C HamanoOct 20, 2015
  6. Junio C HamanoOct 21, 2015
  7. Stefan BellerOct 21, 2015
  8. 3/8 run-command: Initialize the shutdown flagStefan Beller, Oct 20, 2015
  9. 4/8 test-run-command: Test for gracefully abortingStefan Beller, Oct 20, 2015
  10. 5/8 test-run-command: Increase test coverageStefan Beller, Oct 20, 2015
  11. 6/8 run-command: Fix missing output from late callbacksStefan Beller, Oct 20, 2015
  12. 7/8 submodule config: Keep update strategy aroundStefan Beller, Oct 20, 2015
  13. 8/8 git submodule update: Have a dedicated helper for cloningStefan Beller, Oct 20, 2015
  14. Junio C HamanoOct 21, 2015
  15. Stefan BellerOct 21, 2015
  16. Junio C HamanoOct 21, 2015
  17. Stefan BellerOct 21, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.