Re: [PATCH v2 0/2] ci(GitHub): mark up compile errors, too

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jun 13, 2022, 22:41 UTC
Message-ID: <220614.86tu8oyxu9.gmgdl@evledraar.gmail.com>
In-Reply-To: <pull.1253.v2.git.1655125988.gitgitgadget@gmail.com>

On Mon, Jun 13 2022, Johannes Schindelin via GitGitGadget wrote:

Show 23 quoted lines

> Just like we mark up test failures, it makes sense to mark up compile
> errors, too.
>
> In a sense, it makes even more sense with compile errors than with test
> failures because we can link directly to the corresponding source code in
> the former case (if said code has been touched by the Pull Request, that
> is). The only downside is that this link currently is kind of misleading if
> the Pull Request did not even touch the offending source code (such as was
> the case when a GCC upgrade in Git for Windows' SDK all of a sudden pointed
> out problems in the source code that had existed for a long time already).
> We will see how the GitHub Actions engineers will develop this feature
> further.
>
> This patch series is based on js/ci-github-workflow-markup. Which also
> serves as an example how this looks like if the offending source code was
> not touched by the Pull Request:
> https://github.com/dscho/git/actions/runs/2477526645 because it still
> triggers the above-referenced GCC build failure.
>
> Changes since v1:
>
>  * Using a comma in the workflow command now, as described in the official
>    documentation ;-) (Thank you, Ævar)

You're welcome!

Show 7 quoted lines

>  * The curly bracket construct was replaced by a proper subshell, to avoid
>    jumbled output and a race where the exit.status file could be read before
>    it was written.
>
> Johannes Schindelin (2):
>   ci(github): use grouping also in the `win-build` job
>   ci(github): also mark up compile errors

It's still genuinely unclear to me what exactly the expected before/after result is, and I wish the 2/2 commit would discuss it.

So, in v1 we had this: https://github.com/gitgitgadget/git/actions/runs/2461737185

Where the *summary* for the CI said e.g. "syslog.c line=53#L1", so that was the "needs a comma" bug, now it says syslog.c#L53 instead: https://github.com/dscho/git/actions/runs/2477526645 (your link above). So that's good.

But re my earlier comment where I asked/wondered if fixing that would link to the source file at line 53 it still seems to just link to the diff.

Is that a bug? The desired result? If the commit was modifying syslog.c would the link work?

Clearly an end result where we link to the source file/lines at the rev we're testing is much more useful.

I found this discussion: https://github.community/t/are-github-actions-notice-warning-error-annotations-broken/225674

Which has a link to an example run at: https://github.com/IronTooch-ColdStorage/Github-AnnotationTest/actions/runs/1782265048

So isn't this for creating "annotations" for just the regions that would be involved in your diff? I.e. it shows a notice for the line(s) involved in the diff itself, but presumably nothing else?

If that's the case I think it would be much more useful to just e.g. wrap $(CC) in some "tee"-like command to spew its output somewhere, and then have a "step" where we extract the warnings/errors emitted, and emit URLs you could click on, unless there's some way to make the GitHub UX emit the same information.

I.e. it'll be quite hit & miss whether the annotation will show up in the diff, the compiler will often warn about a line some distance away from the change made, e.g. if a variable is made unused.

Unless the intent is only to aggregate them on the summary page, but then why do we need to link to the "line" at all, which will at best work unreliably, and at worst be actively misleading.

In any case, needing to do less reading of the tea leaves would be nice, i.e. if the commit message explain what the desired change is exactly, and how it should be handling these cases.

Thanks.