threads / patch / 41260

v2, 2 partstravis-ci: run previously failed tests first, then slowest to fastest

Subject: [PATCH v2 0/2] travis-ci: run previously failed tests first, then slowest to fastest

## tl;dr

4 messages between Jan 26, 2016 and Jan 26, 2016. Diffs are folded; open one to read it.

replies: 3people: 2as markdown or json

larsxschneider@gmail.com· Jan 26, 2016, 09:53 UTC · lore
From: Lars Schneider <larsxschneider@gmail.com>
diff to v1:
    - improve the commit message ($gmane/284465)
    - rename the Travis cache directory from $HOME/.prove-cache to
      $HOME/travis-cache for clarity
    - explicitly use container-based infrastructure ($gmane/284432)
Thanks Mike, Jeff, and Junio for the review!

Cheers, Lars

Lars Schneider (2):
  travis-ci: run previously failed tests first, then slowest to fastest
  travis-ci: explicity use container-based infrastructure
 .travis.yml | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

-- 2.5.1

larsxschneider@gmail.com· Jan 26, 2016, 09:53 UTC · re: larsxschneider@gmail.com · lore

[PATCH v2 1/2] travis-ci: run previously failed tests first, then slowest to fastest

From: Lars Schneider <larsxschneider@gmail.com>

The Travis-CI machines are in a clean state in the beginning of every run (transient by default). Use the Travis-CI cache feature to make the prove state persistent across consecutive Travis-CI runs on the same branch. This allows to run previously failed tests first and run remaining tests in slowest to fastest order. As a result it is less likely that Travis-CI needs to wait for a single test at the end which speeds up the test suite execution by ~2 min.

Travis-CI can only cache entire directories. Prove stores the .prove file always in the t/ directory but we don't want to cache the entire t/ directory. Therefore we create a symlink from $HOME/travis-cache/.prove to t/.prove and cache the $HOME/travis-cache directory.

Unfortunately the cache feature is only available (for free) on the Travis-CI Linux environment.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
Show changes to .travis.yml +7 −1
diff --git a/.travis.yml b/.travis.yml
index c3bf9c6..610881e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,9 @@
 language: c
 
+cache:
+  directories:
+    - $HOME/travis-cache
+
 os:
   - linux
   - osx
@@ -18,7 +22,7 @@ env:
     - P4_VERSION="15.2"
     - GIT_LFS_VERSION="1.1.0"
     - DEFAULT_TEST_TARGET=prove
-    - GIT_PROVE_OPTS="--timer --jobs 3"
+    - GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
     - GIT_TEST_OPTS="--verbose --tee"
     - CFLAGS="-g -O2 -Wall -Werror"
     - GIT_TEST_CLONE_2GB=YesPlease
@@ -67,6 +71,8 @@ before_install:
     p4 -V | grep Rev.;
     echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)";
     git-lfs version;
+    mkdir -p $HOME/travis-cache;
+    ln -s $HOME/travis-cache/.prove t/.prove;
 
 before_script: make --jobs=2
 
-- 
2.5.1
Junio C Hamano· Jan 26, 2016, 18:44 UTC · re: larsxschneider@gmail.com · lore

Re: [PATCH v2 1/2] travis-ci: run previously failed tests first, then slowest to fastest

larsxschneider@gmail.com writes:
Show 17 quoted lines
> From: Lars Schneider <larsxschneider@gmail.com>
>
> The Travis-CI machines are in a clean state in the beginning of every run
> (transient by default). Use the Travis-CI cache feature to make the prove
> state persistent across consecutive Travis-CI runs on the same branch.
> This allows to run previously failed tests first and run remaining tests
> in slowest to fastest order. As a result it is less likely that Travis-CI
> needs to wait for a single test at the end which speeds up the test suite
> execution by ~2 min.
>
> Travis-CI can only cache entire directories. Prove stores the .prove file
> always in the t/ directory but we don't want to cache the entire t/ directory.
> Therefore we create a symlink from $HOME/travis-cache/.prove to t/.prove and
> cache the $HOME/travis-cache directory.
>
> Unfortunately the cache feature is only available (for free) on the
> Travis-CI Linux environment.

Thanks. I wouldn't be confused with the above description like before.

I see systems other than Linux listed in os: section; presumably our $HOME would not persist and we wouldn't even have $HOME/travis-cache directory when we run on them, so "mkdir -p" would be both necessary and save to have whether the "cache" feature is available or not. And the "ln -s" would be the same way--either we symlink to a file that hasn't been born (either on a platform without "cache" feature, or the first run on Linux) and that is perfectly fine.

And all the systems listed in os: section supports symlinks, so the "mkdir -p && ln -s" is safe.

Looks sensible.  Thanks.
Show 38 quoted lines
>
> Suggested-by: Jeff King <peff@peff.net>
> Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
> ---
>  .travis.yml | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index c3bf9c6..610881e 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -1,5 +1,9 @@
>  language: c
>  
> +cache:
> +  directories:
> +    - $HOME/travis-cache
> +
>  os:
>    - linux
>    - osx
> @@ -18,7 +22,7 @@ env:
>      - P4_VERSION="15.2"
>      - GIT_LFS_VERSION="1.1.0"
>      - DEFAULT_TEST_TARGET=prove
> -    - GIT_PROVE_OPTS="--timer --jobs 3"
> +    - GIT_PROVE_OPTS="--timer --jobs 3 --state=failed,slow,save"
>      - GIT_TEST_OPTS="--verbose --tee"
>      - CFLAGS="-g -O2 -Wall -Werror"
>      - GIT_TEST_CLONE_2GB=YesPlease
> @@ -67,6 +71,8 @@ before_install:
>      p4 -V | grep Rev.;
>      echo "$(tput setaf 6)Git-LFS Version$(tput sgr0)";
>      git-lfs version;
> +    mkdir -p $HOME/travis-cache;
> +    ln -s $HOME/travis-cache/.prove t/.prove;
>  
>  before_script: make --jobs=2
larsxschneider@gmail.com· Jan 26, 2016, 09:53 UTC · re: larsxschneider@gmail.com · lore

[PATCH v2 2/2] travis-ci: explicity use container-based infrastructure

From: Lars Schneider <larsxschneider@gmail.com>

Set `sudo: false` to explicitly use the (faster) container-based infrastructure for the Travis-CI Linux build.

More info: https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
 .travis.yml | 2 ++
 1 file changed, 2 insertions(+)
Show changes to .travis.yml +2 −0
diff --git a/.travis.yml b/.travis.yml
index 610881e..f8b73ec 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,7 @@
 language: c
 
+sudo: false
+
 cache:
   directories:
     - $HOME/travis-cache
-- 
2.5.1

← back to recent threads