Re: [PATCH 5/9] tests: refactor --write-junit-xml code

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jan 26, 2022, 00:10 UTC
Message-ID: <220126.86o83zfj5m.gmgdl@evledraar.gmail.com>
In-Reply-To: <94dcbe1bc437dc9738cfc635e9085d743eb4ec98.1643050574.git.gitgitgadget@gmail.com>

On Mon, Jan 24 2022, Johannes Schindelin via GitGitGadget wrote:

Show 15 quoted lines

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The code writing JUnit XML is interspersed directly with all the code in
> `t/test-lib.sh`, and it is therefore not only ill-separated, but
> introducing yet another output format would make the situation even
> worse.
>
> Let's introduce an abstraction layer by hiding the JUnit XML code behind
> four new functions that are supposed to be called before and after each
> test and test case.
>
> This is not just an academic exercise, refactoring for refactoring's
> sake. We _actually_ want to introduce such a new output format, to
> make it substantially easier to diagnose test failures in our GitHub
> workflow, therefore we do need this refactoring.

I'm a bit confused about the need to patch this JUnit code & to generalize test-lib.sh to emit things in three machine-readable formats (TAP, JUnit, and now this Markdown format).

In
https://lore.kernel.org/git/nycvar.QRO.7.76.6.2112201834050.347@tvgsbejvaqbjf.bet/
you replied to my patch to remove this dead code with wanting to keep
it, because:
    
    The reason is that there are still some things that Azure Pipelines can do
    that GitHub workflows cannot, for example:
    
    - present the logs of failed tests in an intuitive manner,
    
    - re-run _only_ failed jobs.

Which is fair enough, but in this series we're further patching it, but it's still not used anywhere in-tree at the end of it, or am I missing something?

This series is seeking to address 1/2 of the points you mentioned, and presumably the latter is a question of us juggling around our GitHub CI job definitions.

Then in 6/9 you note:

    This seems not to faze Azure Pipelines' XML parser, but it still is
    incorrect, so let's fix it.

So this is running on Azure somehow, but not via the previously in-tree azure-pipeline.yml removed in your 6081d3898fe (ci: retire the Azure Pipelines definition, 2020-04-11)?