Re: [PATCH 4/5] travis-ci: don't run the test suite as root in the 32 bit Linux build

From: Jeff King <peff@peff.net>
Date: Jan 24, 2018, 18:01 UTC
Message-ID: <20180124180142.GA26290@sigill.intra.peff.net>
In-Reply-To: <20180124155622.GB15803@sigill.intra.peff.net>

On Wed, Jan 24, 2018 at 10:56:22AM -0500, Jeff King wrote:

Show 12 quoted lines

> > > (As an aside, I'm not sure the prove cache is doing much. Running in
> > > slow-to-fast order helps if you are trying to run massively in parallel,
> > > but we only use -j3 for our Travis builds).
> > 
> > It saves about a minute / 10% of runtime; it's mentioned in 7e72cfcee
> > (travis-ci: save prove state for the 32 bit Linux build, 2017-12-27).
> 
> I'm surprised we get that much benefit out of a 3-way parallel run, but
> I'll believe you if you measured it. I guess it's because a lot of the
> really long tests are right at the end, numerically (especially if svn
> tests are enabled). I wonder if "--shuffle" would yield similar
> benefits.

Just for fun, I tried running:

  cd t
  best-of-five make GIT_PROVE_OPTS='-j3'
  best-of-five make GIT_PROVE_OPTS='-j3 --state=slow,save'
  best-of-five make GIT_PROVE_OPTS='-j3 --shuffle'

and got:

  stock
  Attempt 1: 137.057
  Attempt 2: 137.635
  Attempt 3: 138.925
  Attempt 4: 134.693
  Attempt 5: 139.581

  slow,save
  Attempt 1: 133.157
  Attempt 2: 135.602
  Attempt 3: 133.225
  Attempt 4: 136.278
  Attempt 5: 133.382

  shuffle
  Attempt 1: 136.717
  Attempt 2: 138.805
  Attempt 3: 145.734
  Attempt 4: 145.226
  Attempt 5: 145.889

I had expected the shuffle to be sometimes-fast and sometimes-slow, but it seems like it is just-slow. So that's probably not a big win. It also doesn't look like state-saving gets us much.

Those runs don't have cvs/svn installed. I repeated with those installed. The whole run is much slower then (about 230s), but the relative timings are the same.

I wonder what is different between my setup and Travis. I guess one is that I use a tmpfs for the test-root. I wonder if that could throw off the relative timings, or the importance of parallelization.

-Peff