Re: [PATCH 2/2] diff tests: rewrite flakyness-causing test "aid"
- From
SZEDER Gábor <szeder.dev@gmail.com>
- Date
- Apr 15, 2021, 21:26 UTC
- Message-ID
- <20210415212600.GN2947267@szeder.dev>
- In-Reply-To
- <CAHd-oW6D7Tmp2TmrmGvXDt-NnT_wxhfr_bn_P_7=ot=m_2eyyA@mail.gmail.com>
On Tue, Apr 13, 2021 at 04:55:23PM -0300, Matheus Tavares Bernardino wrote:
Show 32 quoted lines
> On Tue, Apr 13, 2021 at 4:01 PM Ævar Arnfjörð Bjarmason
> > >> diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
> > >> index 67f6411aff9..228ff100c61 100755
> > >> --- a/t/t4013-diff-various.sh
> > >> +++ b/t/t4013-diff-various.sh
> > >> @@ -200,10 +200,12 @@ do
> > >> esac
> > >> test=$(echo "$label" | sed -e 's|[/ ][/ ]*|_|g')
> > >> pfx=$(printf "%04d" $test_count)
> > >> - expect="$TEST_DIRECTORY/t4013/diff.$test"
> > >> + expect_relative="t4013/diff.$test"
> > >> + expect="$TEST_DIRECTORY/$expect_relative"
> > >> actual="$pfx-diff.$test"
> > >>
> > >> test_expect_$status "git $cmd # magic is ${magic:-(not used)}" '
> > >> + test_when_finished "rm $actual" &&
> > >
> > > Nit: before these two patches, "$actual" was only removed when the
> > > test succeeded. So, in case of failure, the failed output files would
> > > still be there for debugging. It might be interesting to keep this
> > > behavior and only remove "$actual" at the end of the test.
> >
> > Either I'm missing something or you are, that's how test_when_finished
> > works.
> >
> > It's skipped under e.g. "--immediate --debug". See b586744a864 (test:
> > skip clean-up when running under --immediate mode, 2011-06-27)
>
> I was mostly thinking about the `artifacts` zip we get from our CI
> when a test fails. I find the final trash dir quite useful for some
> post-mortem analysis, especially to debug WIP tests that only fail
> occasionally or test failures on OSes I don't have quick access to.On Travis CI we run tests with '--immediate' for exactly this reason; I don't know why it's done differently on other CI systems, and, unfortunately, 'git log --grep=immediate ci/' didn't turn up any insights.