[PATCH v4 0/2] ci: link failure and leak annotations to the test script
- From
Harald Nordgren via GitGitGadget <gitgitgadget@gmail.com>
- Date
- Oct 1, 2026, 18:44 UTC
- Message-ID
- <pull.2419.v4.git.git.1790880255.gitgitgadget@gmail.com>
- In-Reply-To
- <pull.2419.git.git.1790362443893.gitgitgadget@gmail.com>
Link failure and leak annotations in CI to the test script, so both can be found from the job summary.
V4 CI Job where failures and leaks are reported: https://github.com/git/git/actions/runs/36760014277/job/110039964565?pr=2426
Changes in v4:
* Clarify commit messages and simplify escaping logic.
Changes in v3:
* Fixed bug in the --immediate exit ordering: the --immediate && --invert-exit-code path called exit 0 before the test's annotation was written, now a single unconditional call covers both exit paths. * github_escape_message_ no longer relies on \r being a portable sed escape sequence (not POSIX-guaranteed and BSD sed implementations can differ), it splices in the literal carriage-return byte via printf instead. * Reverted unrelated test-tool line back to its original form.
Changes in v2:
* Split into two commits, each explaining its own reasoning. * Leak output is no longer capped or embedded in the message, it's now an uncapped fold, so multiple leaks in the same test both show in full. A second leak in a different test still won't show in the same run, --immediate stops the script at the first failure, but it no longer gets buried under every later test falsely reporting "not ok" either. * Drops the giant unfolded message that annotations used to carry, which is what probably caused the scrolling behavior.
Harald Nordgren (2):
ci: annotate leaks and stop a leak-sanitizer script at its first
failure
ci: point test failures and fixed known breakages at their file and
lineci/lib.sh | 1 + t/test-lib-github-workflow-markup.sh | 54 ++++++++++++++++++++++++---- t/test-lib.sh | 6 +++- 3 files changed, 54 insertions(+), 7 deletions(-)
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2419%2FHaraldNordgren%2Fci-annotation-file-line-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2419/HaraldNordgren/ci-annotation-file-line-v4 Pull-Request: https://github.com/git/git/pull/2419
Range-diff vs v3:
1: b6a36820ae ! 1: 138394b48b ci: annotate leaks and stop a leak-sanitizer script at its first failure
@@ Commit message
Give a leak its own annotation. Point it at the test script, the exact
line isn't known, only which script the leak turned up in, and put the
- full sanitizer report in a log group next to it, so it stays visible
- and isn't capped to a handful of lines.
+ sanitizer report in a log group next to it, so it stays visible.
Once a script has one leak, it keeps running: the sanitizer log
directory is never cleared between tests, so every later test in the
2: 750c360512 ! 2: 8ec2b53d82 ci: point test failures and fixed known breakages at their file and line
@@ Commit message
ci: point test failures and fixed known breakages at their file and line
A test failure or a fixed known breakage gets an annotation that names
- the test but carries no file or line, so there is nothing to click
- through to from the GitHub UI.
+ the test but says nothing about where it's defined, so a reviewer has
+ to search the script by hand to find it.
Find the line a test is defined on by searching the script for its
- description as a fixed string, using the first match. A description
- can contain characters like `[` or `*` that a regex search would
- misread, so match it literally. Fall back to line 1 when the
- description is not found verbatim, which happens when a test builds
- its description at runtime instead of writing it out literally.
+ description as a fixed string, using the first match. Fall back to
+ line 1 when the description is not found verbatim, which happens when
+ a test builds its description at runtime instead of writing it out
+ literally.
- A GitHub annotation is a single line, and a test description is always
- one line too, so only a `%` or a stray carriage return in it needs
- percent-encoding to keep the annotation intact. Escape `%` first, or a
- carriage return's own encoding would be mangled by a `%` substitution
- that ran after it.
+ A GitHub annotation is a single line, so a `%` in a test description
+ has to be percent-encoded as `%25`, or GitHub misreads it as its own
+ escape sequence. for-each-ref's format atoms use plenty of them, e.g.
+ `%(raw)`.
Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
@@ t/test-lib-github-workflow-markup.sh: start_test_output () {
}
+github_escape_message_ () {
-+ # A test description is always one line, so only % and CR need
-+ # escaping here. Escape % first, or CR's own %-encoding gets mangled.
-+ # \r is not a portable sed escape, so splice in the actual byte.
-+ sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"
++ # % has to be escaped or GitHub misreads it as the start of its own
++ # percent-encoding (e.g. a literal %(raw) in a for-each-ref test
++ # description).
++ sed -e 's/%/%25/g'
+}
+
+find_test_case_line_ () {
+ # A description can contain characters like [ or * that would
+ # corrupt a regex search, so match it literally and take the first
-+ # hit; -- keeps a description starting with "-" from being read as
-+ # an option.
++ # hit. The -- keeps a description starting with "-" from being read
++ # as an option.
+ grep -n -F -- "$1" "$TEST_DIRECTORY/$github_markup_script_name" |
+ head -n 1 | cut -d: -f1
+}-- gitgitgadget