git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 0/2] ci: link failure and leak annotations to the test script

From
Harald Nordgren via GitGitGadget <gitgitgadget@gmail.com>
Date
Oct 1, 2026, 18:44 UTC
Message-ID
<pull.2419.v4.git.git.1790880255.gitgitgadget@gmail.com>
In-Reply-To
<pull.2419.git.git.1790362443893.gitgitgadget@gmail.com>

Link failure and leak annotations in CI to the test script, so both can be found from the job summary.

V4 CI Job where failures and leaks are reported: https://github.com/git/git/actions/runs/36760014277/job/110039964565?pr=2426

Changes in v4:
 * Clarify commit messages and simplify escaping logic.
Changes in v3:
 * Fixed bug in the --immediate exit ordering: the --immediate &&
   --invert-exit-code path called exit 0 before the test's annotation was
   written, now a single unconditional call covers both exit paths.
 * github_escape_message_ no longer relies on \r being a portable sed escape
   sequence (not POSIX-guaranteed and BSD sed implementations can differ),
   it splices in the literal carriage-return byte via printf instead.
 * Reverted unrelated test-tool line back to its original form.
Changes in v2:
 * Split into two commits, each explaining its own reasoning.
 * Leak output is no longer capped or embedded in the message, it's now an
   uncapped fold, so multiple leaks in the same test both show in full. A
   second leak in a different test still won't show in the same run,
   --immediate stops the script at the first failure, but it no longer gets
   buried under every later test falsely reporting "not ok" either.
 * Drops the giant unfolded message that annotations used to carry, which is
   what probably caused the scrolling behavior.
Harald Nordgren (2):
  ci: annotate leaks and stop a leak-sanitizer script at its first
    failure
  ci: point test failures and fixed known breakages at their file and
    line
 ci/lib.sh                            |  1 +
 t/test-lib-github-workflow-markup.sh | 54 ++++++++++++++++++++++++----
 t/test-lib.sh                        |  6 +++-
 3 files changed, 54 insertions(+), 7 deletions(-)
base-commit: a018953688f1b10bddf91bff8747068f5f4746a4
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2419%2FHaraldNordgren%2Fci-annotation-file-line-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2419/HaraldNordgren/ci-annotation-file-line-v4
Pull-Request: https://github.com/git/git/pull/2419
Range-diff vs v3:
 1:  b6a36820ae ! 1:  138394b48b ci: annotate leaks and stop a leak-sanitizer script at its first failure
     @@ Commit message
      
          Give a leak its own annotation. Point it at the test script, the exact
          line isn't known, only which script the leak turned up in, and put the
     -    full sanitizer report in a log group next to it, so it stays visible
     -    and isn't capped to a handful of lines.
     +    sanitizer report in a log group next to it, so it stays visible.
      
          Once a script has one leak, it keeps running: the sanitizer log
          directory is never cleared between tests, so every later test in the
 2:  750c360512 ! 2:  8ec2b53d82 ci: point test failures and fixed known breakages at their file and line
     @@ Commit message
          ci: point test failures and fixed known breakages at their file and line
      
          A test failure or a fixed known breakage gets an annotation that names
     -    the test but carries no file or line, so there is nothing to click
     -    through to from the GitHub UI.
     +    the test but says nothing about where it's defined, so a reviewer has
     +    to search the script by hand to find it.
      
          Find the line a test is defined on by searching the script for its
     -    description as a fixed string, using the first match. A description
     -    can contain characters like `[` or `*` that a regex search would
     -    misread, so match it literally. Fall back to line 1 when the
     -    description is not found verbatim, which happens when a test builds
     -    its description at runtime instead of writing it out literally.
     +    description as a fixed string, using the first match. Fall back to
     +    line 1 when the description is not found verbatim, which happens when
     +    a test builds its description at runtime instead of writing it out
     +    literally.
      
     -    A GitHub annotation is a single line, and a test description is always
     -    one line too, so only a `%` or a stray carriage return in it needs
     -    percent-encoding to keep the annotation intact. Escape `%` first, or a
     -    carriage return's own encoding would be mangled by a `%` substitution
     -    that ran after it.
     +    A GitHub annotation is a single line, so a `%` in a test description
     +    has to be percent-encoded as `%25`, or GitHub misreads it as its own
     +    escape sequence. for-each-ref's format atoms use plenty of them, e.g.
     +    `%(raw)`.
      
          Signed-off-by: Harald Nordgren <haraldnordgren@gmail.com>
      
     @@ t/test-lib-github-workflow-markup.sh: start_test_output () {
       }
       
      +github_escape_message_ () {
     -+	# A test description is always one line, so only % and CR need
     -+	# escaping here. Escape % first, or CR's own %-encoding gets mangled.
     -+	# \r is not a portable sed escape, so splice in the actual byte.
     -+	sed -e 's/%/%25/g' -e "s/$(printf '\r')/%0D/g"
     ++	# % has to be escaped or GitHub misreads it as the start of its own
     ++	# percent-encoding (e.g. a literal %(raw) in a for-each-ref test
     ++	# description).
     ++	sed -e 's/%/%25/g'
      +}
      +
      +find_test_case_line_ () {
      +	# A description can contain characters like [ or * that would
      +	# corrupt a regex search, so match it literally and take the first
     -+	# hit; -- keeps a description starting with "-" from being read as
     -+	# an option.
     ++	# hit. The -- keeps a description starting with "-" from being read
     ++	# as an option.
      +	grep -n -F -- "$1" "$TEST_DIRECTORY/$github_markup_script_name" |
      +	head -n 1 | cut -d: -f1
      +}
-- 
gitgitgadget
Previous: Junio C HamanoNext: Harald Nordgren via GitGitGadget
Message 21 of 40 in “ci: point leak-sanitizer failures at the actual test and error”
  1. ci: point leak-sanitizer failures at the actual test and errorHarald Nordgren via GitGitGadget, Sep 25, 2026
  2. Ben KnobleSep 25, 2026
  3. Harald NordgrenSep 25, 2026
  4. Phillip WoodSep 27, 2026
  5. Harald NordgrenSep 27, 2026
  6. 0/2 ci: link failure and leak annotations to the test scriptHarald Nordgren via GitGitGadget, Sep 28, 2026
  7. 2/2 ci: point test failures and fixed known breakages at their file and lineHarald Nordgren via GitGitGadget, Sep 28, 2026
  8. Junio C HamanoSep 28, 2026
  9. 1/2 ci: annotate leaks and stop a leak-sanitizer script at its first failureHarald Nordgren via GitGitGadget, Sep 28, 2026
  10. Junio C HamanoSep 28, 2026
  11. Harald NordgrenSep 29, 2026
  12. 0/2 ci: link failure and leak annotations to the test scriptHarald Nordgren via GitGitGadget, Sep 30, 2026
  13. 1/2 ci: annotate leaks and stop a leak-sanitizer script at its first failureHarald Nordgren via GitGitGadget, Sep 30, 2026
  14. Phillip WoodSep 30, 2026
  15. 2/2 ci: point test failures and fixed known breakages at their file and lineHarald Nordgren via GitGitGadget, Sep 30, 2026
  16. Phillip WoodSep 30, 2026
  17. Harald NordgrenSep 30, 2026
  18. Junio C HamanoSep 30, 2026
  19. Phillip WoodSep 30, 2026
  20. Junio C HamanoSep 30, 2026
  21. 0/2 ci: link failure and leak annotations to the test scriptHarald Nordgren via GitGitGadget, Oct 1, 2026
  22. 1/2 ci: annotate leaks and stop a leak-sanitizer script at its first failureHarald Nordgren via GitGitGadget, Oct 1, 2026
  23. 2/2 ci: point test failures and fixed known breakages at their file and lineHarald Nordgren via GitGitGadget, Oct 1, 2026
  24. Phillip WoodOct 1, 2026
  25. Junio C HamanoOct 1, 2026
  26. Harald NordgrenOct 2, 2026
  27. 0/2 ci: link failure and leak annotations to the test scriptHarald Nordgren via GitGitGadget, Oct 3, 2026
  28. 1/2 ci: annotate leaks and stop a leak-sanitizer script at its first failureHarald Nordgren via GitGitGadget, Oct 3, 2026
  29. 2/2 ci: point test failures and fixed known breakages at their file and lineHarald Nordgren via GitGitGadget, Oct 3, 2026
  30. Phillip WoodOct 3, 2026
  31. Phillip WoodOct 3, 2026
  32. Harald NordgrenOct 4, 2026
  33. Phillip WoodOct 5, 2026
  34. Harald NordgrenOct 5, 2026
  35. Phillip WoodOct 5, 2026
  36. Harald NordgrenOct 4, 2026
  37. 0/2 ci: link failure and leak annotations to the test scriptHarald Nordgren via GitGitGadget, Oct 6, 2026
  38. 1/2 ci: annotate leaks and stop a leak-sanitizer script at its first failureHarald Nordgren via GitGitGadget, Oct 6, 2026
  39. 2/2 ci: point test failures and fixed known breakages at their file and lineHarald Nordgren via GitGitGadget, Oct 6, 2026
  40. Phillip WoodOct 6, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.