Re: win+VS environment has "cut" but not "paste"?

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Mar 10, 2022, 15:23 UTC
Message-ID: <220310.86o82dj02q.gmgdl@evledraar.gmail.com>
In-Reply-To: <nycvar.QRO.7.76.6.2203091320140.357@tvgsbejvaqbjf.bet>

On Wed, Mar 09 2022, Johannes Schindelin wrote:

Show 18 quoted lines

> Hi Junio,
>
> On Mon, 7 Mar 2022, Junio C Hamano wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>>
>> > I said that the current output is only useful to veterans. The output that
>> > hides the detailed log, under a separate job that is marked as
>> > non-failing.
>> >
>> > That's still as true as when I said it. :-)
>>
>> I think getting rid of the separate "print failures" CI step and
>> making it more discoverable how to reveal the details of failing
>> test step is a usability improvement.
>
> I'm so glad you're saying that! I was starting to doubt whether my caring
> about getting rid of that `print failures` step was maybe misguided.

I don't think anyone's been maintaining that getting rid of it wouldn't be ideal. I for one have just been commenting on issues in the proposed implementation.

I think we might still want to retain some such steps in the future, i.e. if we have a failure have subsequent steps that on failure() bump varying levels of verbosity / raw logs etc., or even try re-running the test in different ways (e.g. narrow it down with --run).

But the failure step you see when something fails should ideally have the failure plus the relevant error, just as we do with compile errors.

Show 9 quoted lines

>> I am not Ævar, but I think what was questioned was the improvement
>> justifies multi dozens of seconds extra waiting time, which is a
>> usability dis-improvement.  I do not have a good answer to that
>> question.
>
> It is definitely worrisome that we have to pay such a price. And if there
> was a good answer how to improve that (without sacrificing the
> discoverability of the command trace corresponding to the test failure), I
> would be more than just eager to hear it.

Isn't the answer to that what I suggested in [1]; I.e. the performance problem being that we include N number of lines of the output that *didn't fail*, and that's what slows down showing the relevant output that *did* fail.

I.e. if say t3070-wildmatch.sh fails in a couple of tests we'd show a *lot* of lines between the relevant failing tests, let's just elide the non-failing ones and show the output for the failing ones specifically.

*Sometimes* (but very rarely) it's relevant to still look at the full output, since the failure might be due to an earlier silent failure in a previous test (or the state it left behind), but I think that's rare enough that the right thing to do is just to stick that in a subsequent "verbose dump" step or whatever.

Show 11 quoted lines

>> But new "non-veteran" users may not share that.  That is something a
>> bit worrying about the new UI.
>
> Indeed. My goal, after all, is to improve the experience of contributors,
> not to make it harder.
>
> Still, given that you currently have to click quite a few times until you
> get to where you need to be, I have my doubts that what this patch series
> does is actually making things slower, measured in terms of the total time
> from seeing a failed build to being able to diagnose the cause by
> inspecting the command trace.

Yes, but wouldn't the "Test Summary Report" in [1] be the best of both worlds[1] (with some minor changes to adapt it to the GitHub "grouping" output, perhaps)?

Then you'd always see the specific of the failing test at the end, if you had N number of failures you might have a lot of output above that, but even that we could always tweak with some smart heuristic. I.e. show verbose "not ok" output if failures <10, if 10..100 elide some for the raw log, if >100 just print "this is completely screwed" or whatever :)

1. https://lore.kernel.org/git/220302.86mti87cj2.gmgdl@evledraar.gmail.com/