[PATCH v4 2/2] ci: point test failures and fixed known breakages at their file and line

From: Harald Nordgren via GitGitGadget <gitgitgadget@gmail.com>
Date: Oct 1, 2026, 18:44 UTC
Message-ID: <8ec2b53d8265e1219b5f1279cadda2ac44c96ae0.1790880255.git.gitgitgadget@gmail.com>
In-Reply-To: <pull.2419.v4.git.git.1790880255.gitgitgadget@gmail.com>

From: Harald Nordgren <haraldnordgren@gmail.com>

A test failure or a fixed known breakage gets an annotation that names the test but says nothing about where it's defined, so a reviewer has to search the script by hand to find it.

Find the line a test is defined on by searching the script for its description as a fixed string, using the first match. Fall back to line 1 when the description is not found verbatim, which happens when a test builds its description at runtime instead of writing it out literally.

A GitHub annotation is a single line, so a `%` in a test description has to be percent-encoded as `%25`, or GitHub misreads it as its own escape sequence. for-each-ref's format atoms use plenty of them, e.g. `%(raw)`.

Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
---
 t/test-lib-github-workflow-markup.sh | 38 +++++++++++++++++++++++-----
 1 file changed, 32 insertions(+), 6 deletions(-)

diff --git a/t/test-lib-github-workflow-markup.sh b/t/test-lib-github-workflow-markup.sh
index 0d54496358..6fee4dfb22 100644
--- a/t/test-lib-github-workflow-markup.sh
+++ b/t/test-lib-github-workflow-markup.sh
@@ -31,6 +31,22 @@ start_test_output () {
 	github_markup_script_name=${0##*/}
 }
 
+github_escape_message_ () {
+	# % has to be escaped or GitHub misreads it as the start of its own
+	# percent-encoding (e.g. a literal %(raw) in a for-each-ref test
+	# description).
+	sed -e 's/%/%25/g'
+}
+
+find_test_case_line_ () {
+	# A description can contain characters like [ or * that would
+	# corrupt a regex search, so match it literally and take the first
+	# hit. The -- keeps a description starting with "-" from being read
+	# as an option.
+	grep -n -F -- "$1" "$TEST_DIRECTORY/$github_markup_script_name" |
+	head -n 1 | cut -d: -f1
+}
+
 github_annotation_ () {
 	echo >>$github_markup_output "::$1 file=$2,line=$3::$4"
 }
@@ -40,18 +56,28 @@ github_annotation_ () {
 finalize_test_case_output () {
 	test_case_result=$1
 	shift
+
+	case "$test_case_result" in
+	ok|broken)
+		# Exit without printing the "ok" or "broken" tests
+		return
+		;;
+	esac
+
+	test_case_line=$(find_test_case_line_ "$1")
+	test_case_description=$(printf '%s' "$1" | github_escape_message_)
+
 	case "$test_case_result" in
 	failure)
-		echo >>$github_markup_output "::error::failed: $this_test.$test_count $1"
+		github_annotation_ error "t/$github_markup_script_name" "${test_case_line:-1}" \
+			"failed: $this_test.$test_count $test_case_description"
 		;;
 	fixed)
-		echo >>$github_markup_output "::notice::fixed: $this_test.$test_count $1"
-		;;
-	ok|broken)
-		# Exit without printing the "ok" or ""broken" tests
-		return
+		github_annotation_ notice "t/$github_markup_script_name" "${test_case_line:-1}" \
+			"fixed: $this_test.$test_count $test_case_description"
 		;;
 	esac
+
 	echo >>$github_markup_output "::group::$test_case_result: $this_test.$test_count $*"
 	test-tool >>$github_markup_output path-utils skip-n-bytes \
 		"$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET

-- 
gitgitgadget