git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 2/6] run-command: add hide_output to run_processes_parallel_opts

From
Calvin Wan <calvinwan@google.com>
Date
Oct 24, 2022, 19:24 UTC
Message-ID
<CAFySSZCFrfhdKuOT=kxqPPBGBD0T2FtD4vJHfa9M3cMAPCSBtA@mail.gmail.com>
In-Reply-To
<221021.86lep9g9ja.gmgdl@evledraar.gmail.com>
Show 33 quoted lines
> I may just be missing something, but doesn't "struct child_process"
> already have e.g. "no_stderr", "no_stdout" etc. that we can use?
> I.e. isn't this thing equivalent to running:
>
>         your-command >/dev/null 2>/dev/null
>
> Which is what the non-parallel API already supports.
>
> Now, IIRC if you just set that in the "get_next_task" callback it won't
> work in the parallel API, or you'll block waiting for I/O that'll never
> come or whatever.
>
> But that'll be because the parallel interface currently only suppors a
> subset of the full "child_process" combination of options, and maybe it
> doesn't grok this.
>
> But if that's the case we should just extend the API to support
> "no_stdout", "no_stderr" etc., no?
>
> I.e. hypothetically the parallel one could support 100% of the "struct
> child_process" combination of options, we just haven't bothered yet.
>
> But I don't see why the parallel API should grow options that we already
> have in "struct child_process", instead we should set them there, and it
> should gradually learn to deal with them.
>
> I think it's also fine to have some basic sanity checks there, e.g. I
> could see how for something like this we don't want to support piping
> only some children to /dev/null but not others, and that it should be
> all or nothing (maybe it makes state management when we loop over them
> easier).
>
> Or again, maybe I'm missing something...
Shouldn't the options that are set in "child_process" be abstracted away
from "parallel_processes"? Setting "no_stdout", "no_stderr", etc. in a
"child_process" shouldn't imply that we still pass the stdout and stderr to
 "parallel_processes" and then we send the output to "/dev/null".

That being said, I can understand the aversion to adding an option like this that doesn't also add support for stdout and stderr. I can remove this patch and instead reset the buffer inside of pipe_output and task_finished in a later patch

Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 11 of 19 in “submodule: parallelize diff”
  1. 0/6 submodule: parallelize diffCalvin Wan, Oct 20, 2022
  2. 1/6 run-command: add pipe_output_fn to run_processes_parallel_optsCalvin Wan, Oct 20, 2022
  3. Ævar Arnfjörð BjarmasonOct 21, 2022
  4. Calvin WanOct 24, 2022
  5. Junio C HamanoOct 21, 2022
  6. Calvin WanOct 24, 2022
  7. Junio C HamanoOct 24, 2022
  8. Calvin WanOct 25, 2022
  9. 2/6 run-command: add hide_output to run_processes_parallel_optsCalvin Wan, Oct 20, 2022
  10. Ævar Arnfjörð BjarmasonOct 21, 2022
  11. Calvin WanOct 24, 2022
  12. Ævar Arnfjörð BjarmasonOct 25, 2022
  13. Calvin WanOct 25, 2022
  14. 3/6 submodule: strbuf variable renameCalvin Wan, Oct 20, 2022
  15. 4/6 submodule: move status parsing into functionCalvin Wan, Oct 20, 2022
  16. 5/6 diff-lib: refactor match_stat_with_submoduleCalvin Wan, Oct 20, 2022
  17. 6/6 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Oct 20, 2022
  18. Ævar Arnfjörð BjarmasonOct 21, 2022
  19. Calvin WanNov 3, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.