Re: [PATCH v3 1/2] ci: annotate leaks and stop a leak-sanitizer script at its first failure

From: Phillip Wood <phillip.wood123@gmail.com>
Date: Sep 30, 2026, 14:56 UTC
Message-ID: <37628f91-eb36-426b-9f6b-b2083f917919@gmail.com>
In-Reply-To: <b6a36820ae3c50e36d71f751b7ff25b7f3275cea.1790748583.git.gitgitgadget@gmail.com>

Hi Harald

On 30/09/2026 07:09, Harald Nordgren via GitGitGadget wrote:

Show 12 quoted lines

> From: Harald Nordgren <haraldnordgren@gmail.com>
> 
> A leak is only discovered once, at the end of a whole script, well
> after every test has already reported ok, and it gets no annotation at
> all, so a leak-sanitizer job's only visible failure is:
> 
>      Process completed with exit code 1.
> 
> Give a leak its own annotation. Point it at the test script, the exact
> line isn't known, only which script the leak turned up in, and put the
> full sanitizer report in a log group next to it, so it stays visible
> and isn't capped to a handful of lines.

It is good that it reports the full LSAN but is it really necessary to emphasize that - why would anyone reading this think it might be abbreviated?

What does adding the file and line number to the annotation buy us? I was hoping there'd be a link in the output to the test source but I can't see anything like that. The output is displayed as

     Error: failed: t7603.3 pull c2, c3, c4, c5 into c1
     ▶failure: t7603.3 pull c2, c3, c4, c5 into c1
     Error: memory leak logged in t7603
     ▶leak: t7603.3

and clicking on the '▶' lines expands the output, but there is nothing about the source file as far as I can see. Ideally, if the test is failing due to a leak, it would be nice to report that as

     Error: leak detected in: t7603.3 pull c2, c3, c4, c5 into c1
     ▶failure: t7603.3 pull c2, c3, c4, c5 into c1

and display the test output and LSAN output together when the "▶failure:" line is clicked, rather than having separate sections for the test output and leak output. Having said that what you've already implemented is a clear improvement so I'd be happy to take that if you don't feel like devoting any more time to it.

Show 5 quoted lines

> Once a script has one leak, it keeps running: the sanitizer log
> directory is never cleared between tests, so every later test in the
> same script sees the same leftover log entries and also reports "not
> ok", burying the one real failure in copies of itself. Stop a
> leak-sanitizer script at its first failure with --immediate instead.

I think that is probably a welcome improvement though if two different tests have different leaks it would be nice to be able to show both.

Thanks for working on this, it makes the LSAN output much more accessible.

Phillip

Show 94 quoted lines

> A failing test already gets its own annotation once its script
> finishes, but --immediate exits as soon as that test fails, before
> reaching the code that writes it. Write the annotation first, so
> turning on --immediate here does not silently drop it.
> 
> Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
> ---
>   ci/lib.sh                            |  1 +
>   t/test-lib-github-workflow-markup.sh | 16 ++++++++++++++++
>   t/test-lib.sh                        |  6 +++++-
>   3 files changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/ci/lib.sh b/ci/lib.sh
> index c6ccbf8c17..a89f480a78 100755
> --- a/ci/lib.sh
> +++ b/ci/lib.sh
> @@ -382,6 +382,7 @@ linux-leaks|linux-reftable-leaks)
>   	export NO_CVS_TESTS=LetsSaveSomeTime
>   	export NO_SVN_TESTS=LetsSaveSomeTime
>   	export NO_P4_TESTS=LetsSaveSomeTime
> +	GIT_TEST_OPTS="$GIT_TEST_OPTS --immediate"
>   	;;
>   linux-asan-ubsan)
>   	export SANITIZE=address,undefined
> diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
> index fa29a62aa3..0d54496358 100644
> --- a/t/test-lib-github-workflow-markup.sh
> +++ b/t/test-lib-github-workflow-markup.sh
> @@ -28,6 +28,11 @@ start_test_output () {
>   	github_markup_output="${GIT_TEST_TEE_OUTPUT_FILE%.out}.markup"
>   	>$github_markup_output
>   	GIT_TEST_TEE_OFFSET=0
> +	github_markup_script_name=${0##*/}
> +}
> +
> +github_annotation_ () {
> +	echo >>$github_markup_output "::$1 file=$2,line=$3::$4"
>   }
>   
>   # No need to override start_test_case_output
> @@ -53,4 +58,15 @@ finalize_test_case_output () {
>   	echo >>$github_markup_output "::endgroup::"
>   }
>   
> +finalize_test_leak_output () {
> +	# The exact line the leak turned up on isn't known, only the script,
> +	# so point at line 1.
> +	github_annotation_ error "t/$github_markup_script_name" 1 \
> +		"memory leak logged in $this_test"
> +
> +	echo >>$github_markup_output "::group::leak: $this_test.$test_count"
> +	cat "$TEST_RESULTS_SAN_FILE".* >>$github_markup_output
> +	echo >>$github_markup_output "::endgroup::"
> +}
> +
>   # No need to override finalize_test_output
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 1f0505e412..e74a12f1dd 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -199,6 +199,7 @@ mark_option_requires_arg () {
>   start_test_output () { :; }
>   start_test_case_output () { :; }
>   finalize_test_case_output () { :; }
> +finalize_test_leak_output () { :; }
>   finalize_test_output () { :; }
>   
>   parse_option () {
> @@ -822,6 +823,9 @@ test_failure_ () {
>   	say_color error "not ok $test_count - ${pfx:+$pfx }$1"
>   	shift
>   	printf '%s\n' "$*" | sed -e 's/^/#	/'
> +	# Write the annotation before either --immediate exit path below,
> +	# both of which call exit and would otherwise skip it.
> +	finalize_test_case_output failure "$failure_label" "$@"
>   	if test -n "$immediate"
>   	then
>   		say_color error "1..$test_count"
> @@ -835,7 +839,6 @@ test_failure_ () {
>   		check_test_results_san_file_ "$test_failure"
>   		_error_exit
>   	fi
> -	finalize_test_case_output failure "$failure_label" "$@"
>   }
>   
>   test_known_broken_ok_ () {
> @@ -1218,6 +1221,7 @@ check_test_results_san_file_ () {
>   		return
>   	fi &&
>   	say_color >&4 error "$(cat "$TEST_RESULTS_SAN_FILE".*)" &&
> +	finalize_test_leak_output &&
>   
>   	if test "$test_failure" = 0
>   	then