git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/6] hook API: fix v2.36.0 regression: hooks should be connected to a TTY

From
Emily Shaffer <emilyshaffer@google.com>
Date
Apr 28, 2022, 23:31 UTC
Message-ID
<Ymsj5Lv89xqLqZxD@google.com>
In-Reply-To
<patch-6.6-de3664f6d2b-20220421T122108Z-avarab@gmail.com>
On Thu, Apr 21, 2022 at 02:25:31PM +0200, Ævar Arnfjörð Bjarmason wrote:
Show 78 quoted lines
> 
> Fix a regression reported[1] in f443246b9f2 (commit: convert
> {pre-commit,prepare-commit-msg} hook to hook.h, 2021-12-22): Due to
> using the run_process_parallel() API in the earlier 96e7225b310 (hook:
> add 'run' subcommand, 2021-12-22) we'd capture the hook's stderr and
> stdout, and thus lose the connection to the TTY in the case of
> e.g. the "pre-commit" hook.
> 
> As a preceding commit notes GNU parallel's similar --ungroup option
> also has it emit output faster. While we're unlikely to have hooks
> that emit truly massive amounts of output (or where the performance
> thereof matters) it's still informative to measure the overhead. In a
> similar "seq" test we're now ~30% faster:
> 
> 	$ cat .git/hooks/seq-hook; git hyperfine -L rev origin/master,HEAD~0 -s 'make CFLAGS=-O3' './git hook run seq-hook'
> 	#!/bin/sh
> 
> 	seq 100000000
> 	Benchmark 1: ./git hook run seq-hook' in 'origin/master
> 	  Time (mean ± σ):     787.1 ms ±  13.6 ms    [User: 701.6 ms, System: 534.4 ms]
> 	  Range (min … max):   773.2 ms … 806.3 ms    10 runs
> 
> 	Benchmark 2: ./git hook run seq-hook' in 'HEAD~0
> 	  Time (mean ± σ):     603.4 ms ±   1.6 ms    [User: 573.1 ms, System: 30.3 ms]
> 	  Range (min … max):   601.0 ms … 606.2 ms    10 runs
> 
> 	Summary
> 	  './git hook run seq-hook' in 'HEAD~0' ran
> 	    1.30 ± 0.02 times faster than './git hook run seq-hook' in 'origin/master'
> 
> In the preceding commit we removed the "no_stdin=1" and
> "stdout_to_stderr=1" assignments. This change brings them back as with
> ".ungroup=1" the run_process_parallel() function doesn't provide them
> for us implicitly.
> 
> As an aside omitting the stdout_to_stderr=1 here would have all tests
> pass, except those that test "git hook run" itself in
> t1800-hook.sh. But our tests passing is the result of another test
> blind spot, as was the case with the regression being fixed here. The
> "stdout_to_stderr=1" for hooks is long-standing behavior, see
> e.g. 1d9e8b56fe3 (Split back out update_hook handling in receive-pack,
> 2007-03-10) and other follow-up commits (running "git log" with
> "--reverse -p -Gstdout_to_stderr" is a good start).
> 
> 1. https://lore.kernel.org/git/CA+dzEBn108QoMA28f0nC8K21XT+Afua0V2Qv8XkR8rAeqUCCZw@mail.gmail.com/
> 
> Reported-by: Anthony Sottile <asottile@umich.edu>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  hook.c          |  8 +++++++-
>  t/t1800-hook.sh | 37 +++++++++++++++++++++++++++++++++++++
>  2 files changed, 44 insertions(+), 1 deletion(-)
> 
> diff --git a/hook.c b/hook.c
> index 68ee4030551..f5eef1d561b 100644
> --- a/hook.c
> +++ b/hook.c
> @@ -53,7 +53,9 @@ static int pick_next_hook(struct child_process *cp,
>  	if (!hook_path)
>  		return 0;
>  
> +	cp->no_stdin = 1;
>  	strvec_pushv(&cp->env_array, hook_cb->options->env.v);
> +	cp->stdout_to_stderr = 1;
>  	cp->trace2_hook_name = hook_cb->hook_name;
>  	cp->dir = hook_cb->options->dir;
>  
> @@ -119,16 +121,20 @@ int run_hooks_opt(const char *hook_name, struct run_hooks_opt *options)
>  		.options = options,
>  	};
>  	const char *const hook_path = find_hook(hook_name);
> -	int jobs = 1;
> +	const int jobs = 1;
>  	int ret = 0;
>  	struct run_process_parallel_opts run_opts = {
>  		.tr2_category = "hook",
>  		.tr2_label = hook_name,
> +		.ungroup = jobs == 1,

So here we do set .ungroup based only on the job count - but we do that only for hooks, which means someone else could conceivably come across similar bug in their later use of run_processes_parallel. Is the reason for doing this in the context of the hook library instead of in the context of run_processes_parallel library just because we are not sure if it will break other parallel callers? Or some other reason?

 - Emily
Show 67 quoted lines
>  	};
>  
>  	if (!options)
>  		BUG("a struct run_hooks_opt must be provided to run_hooks");
>  
> +	if (jobs != 1 || !run_opts.ungroup)
> +		BUG("TODO: think about & document order & interleaving of parallel hook output");
> +
>  	if (options->invoked_hook)
>  		*options->invoked_hook = 0;
>  
> diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh
> index 1e4adc3d53e..f22754deccc 100755
> --- a/t/t1800-hook.sh
> +++ b/t/t1800-hook.sh
> @@ -4,6 +4,7 @@ test_description='git-hook command'
>  
>  TEST_PASSES_SANITIZE_LEAK=true
>  . ./test-lib.sh
> +. "$TEST_DIRECTORY"/lib-terminal.sh
>  
>  test_expect_success 'git hook usage' '
>  	test_expect_code 129 git hook &&
> @@ -120,4 +121,40 @@ test_expect_success 'git -c core.hooksPath=<PATH> hook run' '
>  	test_cmp expect actual
>  '
>  
> +test_hook_tty() {
> +	local fd="$1" &&
> +
> +	cat >expect &&
> +
> +	test_when_finished "rm -rf repo" &&
> +	git init repo &&
> +
> +	test_hook -C repo pre-commit <<-EOF &&
> +	{
> +		test -t 1 && echo >&$fd STDOUT TTY || echo >&$fd STDOUT NO TTY &&
> +		test -t 2 && echo >&$fd STDERR TTY || echo >&$fd STDERR NO TTY
> +	} $fd>actual
> +	EOF
> +
> +	test_commit -C repo A &&
> +	test_commit -C repo B &&
> +	git -C repo reset --soft HEAD^ &&
> +	test_terminal git -C repo commit -m"B.new" &&
> +	test_cmp expect repo/actual
> +}
> +
> +test_expect_success TTY 'git hook run: stdout and stderr are connected to a TTY: STDOUT redirect' '
> +	test_hook_tty 1 <<-\EOF
> +	STDOUT NO TTY
> +	STDERR TTY
> +	EOF
> +'
> +
> +test_expect_success TTY 'git hook run: stdout and stderr are connected to a TTY: STDERR redirect' '
> +	test_hook_tty 2 <<-\EOF
> +	STDOUT TTY
> +	STDERR NO TTY
> +	EOF
> +'
> +
>  test_done
> -- 
> 2.36.0.893.g80a51c675f6
> 
Previous: Ævar Arnfjörð BjarmasonNext: Junio C Hamano
Message 23 of 85 in “git 2.36.0 regression: pre-commit hooks no longer have stdout/stderr as tty”
  1. Anthony SottileApr 19, 2022
  2. Emily ShafferApr 19, 2022
  3. Anthony SottileApr 19, 2022
  4. Phillip WoodApr 20, 2022
  5. Ævar Arnfjörð BjarmasonApr 20, 2022
  6. Emily ShafferApr 20, 2022
  7. Junio C HamanoApr 20, 2022
  8. Emily ShafferApr 20, 2022
  9. Junio C HamanoApr 20, 2022
  10. Emily ShafferApr 20, 2022
  11. Ævar Arnfjörð BjarmasonApr 21, 2022
  12. Junio C HamanoApr 21, 2022
  13. Junio C HamanoApr 21, 2022
  14. Junio C HamanoApr 20, 2022
  15. 0/6 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, Apr 21, 2022
  16. 1/6 run-command API: replace run_processes_parallel_tr2() with opts structÆvar Arnfjörð Bjarmason, Apr 21, 2022
  17. Junio C HamanoApr 23, 2022
  18. Emily ShafferApr 28, 2022
  19. Junio C HamanoApr 29, 2022
  20. 2/6 run-command tests: test stdout of run_command_parallel()Ævar Arnfjörð Bjarmason, Apr 21, 2022
  21. Junio C HamanoApr 23, 2022
  22. 6/6 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, Apr 21, 2022
  23. Emily ShafferApr 28, 2022
  24. Junio C HamanoApr 29, 2022
  25. 3/6 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, Apr 21, 2022
  26. Junio C HamanoApr 23, 2022
  27. Emily ShafferApr 28, 2022
  28. 4/6 hook tests: fix redirection logic error in 96e7225b310Ævar Arnfjörð Bjarmason, Apr 21, 2022
  29. Junio C HamanoApr 23, 2022
  30. 5/6 hook API: don't redundantly re-set "no_stdin" and "stdout_to_stderr"Ævar Arnfjörð Bjarmason, Apr 21, 2022
  31. Junio C HamanoApr 29, 2022
  32. Junio C HamanoApr 21, 2022
  33. Ævar Arnfjörð BjarmasonApr 21, 2022
  34. 0/8 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, May 18, 2022
  35. 1/8 run-command tests: change if/if/... to if/else if/elseÆvar Arnfjörð Bjarmason, May 18, 2022
  36. 2/8 run-command API: use "opts" struct for run_processes_parallel{,_tr2}()Ævar Arnfjörð Bjarmason, May 18, 2022
  37. Junio C HamanoMay 18, 2022
  38. Emily ShafferMay 25, 2022
  39. 3/8 run-command tests: test stdout of run_command_parallel()Ævar Arnfjörð Bjarmason, May 18, 2022
  40. 4/8 run-command.c: add an initializer for "struct parallel_processes"Ævar Arnfjörð Bjarmason, May 18, 2022
  41. 7/8 hook API: don't redundantly re-set "no_stdin" and "stdout_to_stderr"Ævar Arnfjörð Bjarmason, May 18, 2022
  42. 6/8 hook tests: fix redirection logic error in 96e7225b310Ævar Arnfjörð Bjarmason, May 18, 2022
  43. 8/8 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, May 18, 2022
  44. Junio C HamanoMay 18, 2022
  45. Emily ShafferMay 26, 2022
  46. Ævar Arnfjörð BjarmasonMay 26, 2022
  47. Emily ShafferMay 26, 2022
  48. 5/8 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, May 18, 2022
  49. Junio C HamanoMay 18, 2022
  50. Emily ShafferMay 26, 2022
  51. Junio C HamanoMay 27, 2022
  52. Johannes SchindelinMay 25, 2022
  53. Ævar Arnfjörð BjarmasonMay 25, 2022
  54. Junio C HamanoMay 25, 2022
  55. Junio C HamanoMay 26, 2022
  56. Ævar Arnfjörð BjarmasonMay 26, 2022
  57. Junio C HamanoMay 26, 2022
  58. Ævar Arnfjörð BjarmasonMay 26, 2022
  59. Junio C HamanoMay 26, 2022
  60. 0/2 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, May 27, 2022
  61. 1/2 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, May 27, 2022
  62. Junio C HamanoMay 27, 2022
  63. 2/2 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, May 27, 2022
  64. Junio C HamanoMay 27, 2022
  65. 0/2 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, May 31, 2022
  66. 1/2 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, May 31, 2022
  67. Johannes SchindelinJun 1, 2022
  68. Junio C HamanoJun 1, 2022
  69. 2/2 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, May 31, 2022
  70. Johannes SchindelinJun 1, 2022
  71. Johannes SchindelinJun 1, 2022
  72. 0/2 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, Jun 2, 2022
  73. 1/2 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, Jun 2, 2022
  74. 2/2 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, Jun 2, 2022
  75. Junio C HamanoJun 2, 2022
  76. Phillip WoodJun 3, 2022
  77. Ævar Arnfjörð BjarmasonJun 3, 2022
  78. Phillip WoodJun 3, 2022
  79. Ævar Arnfjörð BjarmasonJun 7, 2022
  80. 0/2 hook API: connect hooks to the TTY again, fixes a v2.36.0 regressionÆvar Arnfjörð Bjarmason, Jun 7, 2022
  81. 2/2 hook API: fix v2.36.0 regression: hooks should be connected to a TTYÆvar Arnfjörð Bjarmason, Jun 7, 2022
  82. Junio C HamanoJun 7, 2022
  83. 1/2 run-command: add an "ungroup" option to run_process_parallel()Ævar Arnfjörð Bjarmason, Jun 7, 2022
  84. Emily ShafferJun 17, 2022
  85. Junio C HamanoJun 7, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.