Re: [PATCH 1/1] ci: split the `linux-gcc` job into two jobs

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Jun 13, 2019, 16:51 UTC
Message-ID: <nycvar.QRO.7.76.6.1906131844440.44@tvgsbejvaqbjf.bet>
In-Reply-To: <xmqqv9x9wkba.fsf@gitster-ct.c.googlers.com>

Hi,

On Thu, 13 Jun 2019, Junio C Hamano wrote:

Show 21 quoted lines

> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
> > On Thu, Jun 13, 2019 at 05:53:51AM -0700, Johannes Schindelin via GitGitGadget wrote:
> >> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> >>
> >> This job was abused to not only run the test suite in a regular way but
> >> also with all kinds of `GIT_TEST_*` options set to non-default values.
> >>
> >> Let's split this into two
> >
> > Why...?
> >
> >> with the `linux-gcc` job running the default
> >> test suite, and the newly-introduced `linux-gcc-extra` job running the
> >> test suite in the "special" ways.
> >>
> >> Technically, we would have to build Git only once, but it would not be
> >> obvious how to teach Travis to transport build artifacts, so we keep it
> >> simple and just build Git in both jobs.
>
> I had the same reaction.

So basically you are saying that the cover letter was the wrong location for this:

	For people like me, who often look at our CI builds, it is hard to
	tell whether test suite failures in the linux-gcc job stem from
	the first make test run, or from the second one, after setting all
	kinds of GIT_TEST_* variables to non-default values.

	Let's make it easier on people like me.

	This also helps the problem where the CI builds often finish the other
	jobs waaaay before linux-gcc finally finishes, too: linux-gcc and
	linux-gcc-extra can be run in parallel, on different agents.

Of course, I would rephrase that a little, because it is kinda okay for a cover letter, but not for a commit message that needs to be understandable out of context.

If you agree that this would make the change easier to swallow, I will make it so.

Show 7 quoted lines

> If it said something like:
>
>     There is no logical reason why these extras need to be tied to
>     the linux-gcc platform.  instead of tying the extra
>     configuration tests only to linux-gcc, split it so that they are
>     also run on all other combinations, and this is merely a first
>     step of doing so

No, that would double the time taken by the CI build. It already taxes the patience so much that most contributors already don't bother with it, and you can see how many times I find issues and have to report them (which flies in the face of the idea of using automated builds to catch bugs early): we are seeing the detrimental effects of a regression test suite that takes too long.

Show 5 quoted lines

> it might at least have been possible to judge if the motivation is
> sane, but the proposed log message, while it is quite clear what is
> being done and what its shortcomings are, is silent on why we would
> want to do this in the first place, and that makes it even harder to
> swallow the shortcomings.

Okay, two things:

	- it makes it easier to identify *which* settings to use when a
	  test case fails in `linux-gcc` or `linux-gcc-extra`.

	- the overall CI build runtime is no longer dictated by the
	  runtime of this single build, which easily was the slowest of
	  the bunch.

Does that clear up the picture? Dscho