threads / patch / 52074

patch, 2 partsFix the speed of the CI (Visual Studio) tests

Subject: [PATCH 0/2] Fix the speed of the CI (Visual Studio) tests

## tl;dr

9 messages between Oct 20, 2019 and Oct 24, 2019. Diffs are folded; open one to read it.

replies: 8people: 3as markdown or json

Johannes Schindelin via GitGitGadget· Oct 20, 2019, 20:39 UTC · lore

I made a mistake when converting the make/prove-based test job to a test-tool run-command testsuite one: I lost the parallelization, resulting in way slower CI runs.

Also, I forgot to build with DEVELOPER=1, i.e. with stricter compile flags.
This pair of patches fixes both issues.
Johannes Schindelin (2):
  ci(visual-studio): use strict compile flags, and optimization
  ci(visual-studio): actually run the tests in parallel
 azure-pipelines.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
base-commit: d966095db01190a2196e31195ea6fa0c722aa732
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-410%2Fdscho%2Faccelerate-ci-vs-test-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-410/dscho/accelerate-ci-vs-test-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/410
-- 
gitgitgadget
Johannes Schindelin via GitGitGadget· Oct 20, 2019, 20:39 UTC · re: Johannes Schindelin via GitGitGadget · lore

[PATCH 2/2] ci(visual-studio): actually run the tests in parallel

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Originally, the CI/PR builds that build and test using Visual Studio were implemented imitating `linux-clang`, i.e. still using the `Makefile`-based build infrastructure.

Later (but still before the patches made their way into git.git's `master`), however, this was changed to generate Visual Studio project files and build the binaries using `MSBuild`, as this reflects more accurately how Visual Studio users would want to build Git (internally, Visual Studio uses `MSBuild`, or at least something very similar).

During that transition, we needed to implement a new way to run the test suite in parallel, as Visual Studio users typically will only have a Git Bash available (which does not ship with `make` nore with support for `prove`): we simply implemented a new test helper to run the test suite.

This helper even knows how to run the tests in parallel, but due to a mistake on this developer's part, it was never turned on in the CI/PR builds. This results in 2x-3x longer run times of the test phase.

Let's use the `--jobs=10` option to fix this.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 azure-pipelines.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to azure-pipelines.yml +1 −1
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 457c6fee31..af2a5ea484 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -255,7 +255,7 @@ jobs:
 
         cd t &&
         PATH=\"`$PWD/helper:`$PATH\" &&
-        test-tool.exe run-command testsuite -V -x --write-junit-xml \
+        test-tool.exe run-command testsuite --jobs=10 -V -x --write-junit-xml \
                 `$(test-tool.exe path-utils slice-tests \
                         `$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE t[0-9]*.sh)
       "@
-- 
gitgitgadget
Eric Sunshine· Oct 20, 2019, 21:23 UTC · re: Johannes Schindelin via GitGitGadget · lore

Re: [PATCH 2/2] ci(visual-studio): actually run the tests in parallel

On Sun, Oct 20, 2019 at 4:39 PM Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com> wrote:

> [...]
> During that transition, we needed to implement a new way to run the test
> suite in parallel, as Visual Studio users typically will only have a Git
> Bash available (which does not ship with `make` nore with support for
s/nore/nor/
Show 9 quoted lines
> `prove`): we simply implemented a new test helper to run the test suite.
>
> This helper even knows how to run the tests in parallel, but due to a
> mistake on this developer's part, it was never turned on in the CI/PR
> builds. This results in 2x-3x longer run times of the test phase.
>
> Let's use the `--jobs=10` option to fix this.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Johannes Schindelin via GitGitGadget· Oct 20, 2019, 20:39 UTC · re: Johannes Schindelin via GitGitGadget · lore

[PATCH 1/2] ci(visual-studio): use strict compile flags, and optimization

From: Johannes Schindelin <johannes.schindelin@gmx.de>

To make full use of the work that went into the Visual Studio build & test jobs in our CI/PR builds, let's turn on strict compiler flags. This will give us the benefit of Visual C's compiler warnings (which, at times, seem to catch things that GCC does not catch, and vice versa).

While at it, also turn on optimization; It does not make sense to produce binaries with debug information, and we can use any ounce of speed that we get (because the test suite is particularly slow on Windows, thanks to the need to run inside a Unix shell, which requires us to use the POSIX emulation layer provided by MSYS2).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 azure-pipelines.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to azure-pipelines.yml +1 −1
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 9f099b9529..457c6fee31 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -157,7 +157,7 @@ jobs:
     displayName: 'Download git-sdk-64-minimal'
   - powershell: |
       & git-sdk-64-minimal\usr\bin\bash.exe -lc @"
-        make vcxproj
+        make NDEBUG=1 DEVELOPER=1 vcxproj
       "@
       if (!$?) { exit(1) }
     displayName: Generate Visual Studio Solution
-- 
gitgitgadget
Johannes Schindelin via GitGitGadget· Oct 21, 2019, 19:59 UTC · re: Johannes Schindelin via GitGitGadget · lore

[PATCH v2 0/2] Fix the speed of the CI (Visual Studio) tests

I made a mistake when converting the make/prove-based test job to a test-tool run-command testsuite one: I lost the parallelization, resulting in way slower CI runs.

Also, I forgot to build with DEVELOPER=1, i.e. with stricter compile flags.
This pair of patches fixes both issues.
Changes since v1:
 * Fixed typo "nore" -> "nor" in the commit message.
Johannes Schindelin (2):
  ci(visual-studio): use strict compile flags, and optimization
  ci(visual-studio): actually run the tests in parallel
 azure-pipelines.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
base-commit: d966095db01190a2196e31195ea6fa0c722aa732
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-410%2Fdscho%2Faccelerate-ci-vs-test-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-410/dscho/accelerate-ci-vs-test-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/410
Range-diff vs v1:
 1:  6fba3b1c76 = 1:  6fba3b1c76 ci(visual-studio): use strict compile flags, and optimization
 2:  e3343d1740 ! 2:  07749ab720 ci(visual-studio): actually run the tests in parallel
     @@ -14,7 +14,7 @@
      
          During that transition, we needed to implement a new way to run the test
          suite in parallel, as Visual Studio users typically will only have a Git
     -    Bash available (which does not ship with `make` nore with support for
     +    Bash available (which does not ship with `make` nor with support for
          `prove`): we simply implemented a new test helper to run the test suite.
      
          This helper even knows how to run the tests in parallel, but due to a
-- 
gitgitgadget
Johannes Schindelin via GitGitGadget· Oct 21, 2019, 19:59 UTC · re: Johannes Schindelin via GitGitGadget · lore

[PATCH v2 1/2] ci(visual-studio): use strict compile flags, and optimization

From: Johannes Schindelin <johannes.schindelin@gmx.de>

To make full use of the work that went into the Visual Studio build & test jobs in our CI/PR builds, let's turn on strict compiler flags. This will give us the benefit of Visual C's compiler warnings (which, at times, seem to catch things that GCC does not catch, and vice versa).

While at it, also turn on optimization; It does not make sense to produce binaries with debug information, and we can use any ounce of speed that we get (because the test suite is particularly slow on Windows, thanks to the need to run inside a Unix shell, which requires us to use the POSIX emulation layer provided by MSYS2).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 azure-pipelines.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to azure-pipelines.yml +1 −1
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 9f099b9529..457c6fee31 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -157,7 +157,7 @@ jobs:
     displayName: 'Download git-sdk-64-minimal'
   - powershell: |
       & git-sdk-64-minimal\usr\bin\bash.exe -lc @"
-        make vcxproj
+        make NDEBUG=1 DEVELOPER=1 vcxproj
       "@
       if (!$?) { exit(1) }
     displayName: Generate Visual Studio Solution
-- 
gitgitgadget
Johannes Schindelin via GitGitGadget· Oct 21, 2019, 19:59 UTC · re: Johannes Schindelin via GitGitGadget · lore

[PATCH v2 2/2] ci(visual-studio): actually run the tests in parallel

From: Johannes Schindelin <johannes.schindelin@gmx.de>

Originally, the CI/PR builds that build and test using Visual Studio were implemented imitating `linux-clang`, i.e. still using the `Makefile`-based build infrastructure.

Later (but still before the patches made their way into git.git's `master`), however, this was changed to generate Visual Studio project files and build the binaries using `MSBuild`, as this reflects more accurately how Visual Studio users would want to build Git (internally, Visual Studio uses `MSBuild`, or at least something very similar).

During that transition, we needed to implement a new way to run the test suite in parallel, as Visual Studio users typically will only have a Git Bash available (which does not ship with `make` nor with support for `prove`): we simply implemented a new test helper to run the test suite.

This helper even knows how to run the tests in parallel, but due to a mistake on this developer's part, it was never turned on in the CI/PR builds. This results in 2x-3x longer run times of the test phase.

Let's use the `--jobs=10` option to fix this.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 azure-pipelines.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Show changes to azure-pipelines.yml +1 −1
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 457c6fee31..af2a5ea484 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -255,7 +255,7 @@ jobs:
 
         cd t &&
         PATH=\"`$PWD/helper:`$PATH\" &&
-        test-tool.exe run-command testsuite -V -x --write-junit-xml \
+        test-tool.exe run-command testsuite --jobs=10 -V -x --write-junit-xml \
                 `$(test-tool.exe path-utils slice-tests \
                         `$SYSTEM_JOBPOSITIONINPHASE `$SYSTEM_TOTALJOBSINPHASE t[0-9]*.sh)
       "@
-- 
gitgitgadget
Junio C Hamano· Oct 23, 2019, 02:06 UTC · re: Johannes Schindelin via GitGitGadget · lore

Re: [PATCH v2 0/2] Fix the speed of the CI (Visual Studio) tests

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 10 quoted lines
> Changes since v1:
>
>  * Fixed typo "nore" -> "nor" in the commit message.
>
> Johannes Schindelin (2):
>   ci(visual-studio): use strict compile flags, and optimization
>   ci(visual-studio): actually run the tests in parallel
>
>  azure-pipelines.yml | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
Thanks.

I will take the change to 'master' directly, as cooking in 'next' won't give it any extra exposure when the topic touches only this file and the patch comes from those who do exercise azure CI build before sending it out to the list ;-)

Johannes Schindelin· Oct 24, 2019, 23:23 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 0/2] Fix the speed of the CI (Visual Studio) tests

Hi Junio,
On Wed, 23 Oct 2019, Junio C Hamano wrote:
Show 20 quoted lines
> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
>
> > Changes since v1:
> >
> >  * Fixed typo "nore" -> "nor" in the commit message.
> >
> > Johannes Schindelin (2):
> >   ci(visual-studio): use strict compile flags, and optimization
> >   ci(visual-studio): actually run the tests in parallel
> >
> >  azure-pipelines.yml | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
>
> Thanks.
>
> I will take the change to 'master' directly, as cooking in 'next'
> won't give it any extra exposure when the topic touches only this
> file and the patch comes from those who do exercise azure CI build
> before sending it out to the list ;-)
Thank you very much! It is indeed a lot faster now.

Ciao, Dscho

← back to recent threads