yz/p4-py3, was Re: What's cooking in git.git (Jan 2020, #04; Wed, 22)

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Jan 23, 2020, 12:11 UTC
Message-ID: <nycvar.QRO.7.76.6.2001231306351.46@tvgsbejvaqbjf.bet>
In-Reply-To: <xmqqftg6671g.fsf@gitster-ct.c.googlers.com>

Hi Junio,

On Wed, 22 Jan 2020, Junio C Hamano wrote:

Show 26 quoted lines

> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
> > On Wed, Jan 22, 2020 at 02:18:05PM -0800, Junio C Hamano wrote:
> >> * yz/p4-py3 (2020-01-15) 14 commits
> >>  - ci: also run linux-gcc pipeline with python3.5 environment
> >
> > I still think that this last patch needs to be reworked before this
> > series is merged any further.
> >
> > The only Python script we have is 'git p4', so the Python version is
> > only relevant for 'git p4' tests ('t98*'), while the rest of Git and
> > the test suite couldn't care less [1].  This patch, however, not only
> > builds Git and runs the full test suite for each of the two Python
> > versions, but, worse, runs the full test suite _twice_ for each, first
> > as a "regular" test run and then again with all the GIT_TEST_* knobs
> > enabled.  Consequently, it adds ~50mins to every build's runtime.
> >
> > That's just too wasteful.
>
> Thanks for a reminder.  Yes, I do recall you raised the above point
> and I agree with the assessment.
>
> What's the ideal endgame wrt the tests?  Build with Py$N and run
> full test suite once, and run full test suite again with the unusual
> knobs enabled, which is what is done without this series, plus build
> with Py(5-$N) and run and run only t98?? tests?

Should we declare `t98xx` to be the namespace for the Python-based scripts, or alternatively declare that we won't ever include another Python script but `git-p4`?

But yes, I think that we should probably "tack on" the Python 3.x tests to the `linux-gcc` job.

Or maybe finally split this job into three: one job that does what `linux-gcc` suggests, a `linux-gcc-knobs` one that sets all those `GIT_*` variables, and a python3x one that only runs t98*.sh.

The reason to split it off is this: on rare occasion, I have to restart the `linux-gcc` job because _one_ of those `git-p4` tests failed due to some reason or other, probably timing-related, I did not have time to investigate this. Having to re-run the entire test suite, twice, just to work around those flaky tests is rather wasteful.

That would actually be my prereference. If people agree, I will revive https://github.com/gitgitgadget/git/pull/266.

Ciao, Dscho