[PATCH v2 0/6] ci: replace our Azure Pipeline by GitHub Actions
- From
Đoàn Trần Công Danh <congdanhqx@gmail.com>
- Date
- Apr 1, 2020, 15:55 UTC
- Message-ID
- <cover.1585756350.git.congdanhqx@gmail.com>
- In-Reply-To
- <pull.743.git.git.1585658913.gitgitgadget@gmail.com>
This series is based on work started by Dscho, I started to work with this series since there're merge conflicts with my series at dd/ci-musl-lib, and Dscho said he was busy for a while.
Changes frome Dscho's version: * Based on dd/ci-musl-libc * Move artifact extraction out of Test phase of windows-test * Move ci/print-test-failures.sh out of build-and-test phase * set TERM environment variable if not exist * add linux-musl job * v1 doesn't report failure on linux32, fixed * run linux32 directly inside container * install development file of curl in documentation job because "curl-config" will be called in pu's Makefile + Other approach could be call make CURL_CONFIG=: in test-documentation.sh
Sample run: * of this series: https://github.com/sgn/git/actions/runs/68291472 * pretend compile failure: https://github.com/sgn/git/actions/runs/68292112 * pretend test failure: https://github.com/sgn/git/actions/runs/68293056 * merged to pu: https://github.com/sgn/git/actions/runs/68301122
Johannes Schindelin (4): ci/lib: if CI type is unknown, show the environment variables ci/lib: allow running in GitHub Actions README: add a build badge for the GitHub Actions runs ci: retire the Azure Pipelines definition
Đoàn Trần Công Danh (2): ci/lib: set TERM environment variable if not exist ci: configure GitHub Actions for CI/PR
.github/workflows/main.yml | 314 ++++++++++++++++++++ README.md | 2 +- azure-pipelines.yml | 593 ------------------------------------- ci/lib.sh | 24 +- 4 files changed, 338 insertions(+), 595 deletions(-) create mode 100644 .github/workflows/main.yml delete mode 100644 azure-pipelines.yml
Range-diff against v1:
1: 64d61c3e38 ! 1: 27cb77c257 ci/lib: if CI type is unknown, show the environment variables
@@ Commit message
This should help with adding new CI-specific if-else arms.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
## ci/lib.sh ##
@@ ci/lib.sh: then
2: 4a0ec072cc ! 2: 079038c4a7 ci/lib: allow running in GitHub Actions
@@ Commit message
construct in ci/lib.sh. Let's add one for GitHub Actions.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
## ci/lib.sh ##
@@ ci/lib.sh: save_good_tree () {
-: ---------- > 3: f9d0b6b0d5 ci/lib: set TERM environment variable if not exist
3: e33653fdf0 ! 4: fb714f5889 ci: configure GitHub Actions for CI/PR
@@
## Metadata ##
-Author: Johannes Schindelin <Johannes.Schindelin@gmx.de>
+Author: Đoàn Trần Công Danh <congdanhqx@gmail.com>
## Commit message ##
ci: configure GitHub Actions for CI/PR
This patch adds CI builds via GitHub Actions. While the underlying
technology is at least _very_ similar to that of Azure Pipelines, GitHub
- Actions are much easier to set up than Azure Pipelines: no need to
- install a GitHub App, no need to set up an Azure DevOps account, all you
- need to do is push to your fork on GitHub.
+ Actions are much easier to set up than Azure Pipelines:
+ - no need to install a GitHub App,
+ - no need to set up an Azure DevOps account,
+ - all you need to do is push to your fork on GitHub.
Therefore, it makes a lot of sense for us to have a working GitHub
Actions setup.
- While transmogrifying `azure-pipelines.yml` into
+ While translate-and-modify `azure-pipelines.yml` into
`.github/workflows/main.yml`, we also use the opportunity to accelerate
the step that sets up a minimal subset of Git for Windows' SDK in the
- Windows-build job: we now download a `.tar.xz` stored in Azure Blobs and
- extract it simultaneously (by calling `curl` and piping the result to
- `tar`, decompressing via `xz`, all three utilities being available by
- grace of using Git for Windows' Bash that is installed on the build
- agents). This accelerates that step from ~1m50s to ~7s.
+ Windows-build job:
+ - we now download a `.tar.xz` stored in Azure Blobs and extract it
+ simultaneously (by calling `curl` and piping the result to `tar`,
+ - decompressing via `xz`,
+ - all three utilities are installed together with Git for Windows
Also, we do away with the parts that try to mount a file share on which
`prove` can store data between runs. It is just too complicated to set
up, so it's little return on investment there.
+ Based-on-patch-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
@@ .github/workflows/main.yml (new)
+ with:
+ name: windows-artifacts
+ path: ${{github.workspace}}
-+ - name: Test
++ - name: extract artifact
+ shell: powershell
+ run: |
+ & .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
@@ .github/workflows/main.yml (new)
+ echo No test artifacts found\; skipping >&2
+ exit 0
+ }
-+ tar xf artifacts.tar.gz || exit 1
-+
++ tar xf artifacts.tar.gz
++ "@
++ if (!$?) { exit(1) }
++ - name: Test
++ shell: powershell
++ run: |
++ & .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
+ # Let Git ignore the SDK
+ printf '%s\n' /git-sdk-64-minimal/ >>.git/info/exclude
+
-+ ci/run-test-slice.sh ${{matrix.nr}} 10 || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
++ ci/run-test-slice.sh ${{matrix.nr}} 10
+ "@
+ if (!$?) { exit(1) }
++ - name: ci/print-test-failures.sh
++ if: failure()
++ shell: powershell
++ run: |
++ & .\git-sdk-64-minimal\usr\bin\bash.exe -lc ci/print-test-failures.sh
++ exit(1)
+ vs-build:
+ runs-on: windows-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ - name: ci/run-build-and-test.sh
+ env:
+ CC: clang
++ run: ci/run-build-and-tests.sh
++ - name: ci/print-test-failures.sh
++ if: failure()
++ env:
++ CC: clang
+ run: |
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
++ ci/print-test-failures.sh
++ exit 1
+ linux-gcc:
+ runs-on: ubuntu-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ sudo apt-get -y install git gcc make libssl-dev libcurl4-openssl-dev libexpat-dev tcl tk gettext git-email zlib1g-dev apache2 language-pack-is git-svn gcc-8 &&
+ ci/install-dependencies.sh
+ - name: ci/run-build-and-tests.sh
++ run: ci/run-build-and-tests.sh
++ - name: ci/print-test-failures.sh
++ if: failure()
+ run: |
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
++ ci/print-test-failures.sh
++ exit 1
+ osx-clang:
+ runs-on: macos-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ env:
+ CC: clang
+ run: ci/install-dependencies.sh
-+ - name: ci/run-build-and-tests.sh
++ - name: ci/run-build-and-test.sh
++ env:
++ CC: clang
++ run: ci/run-build-and-tests.sh
++ - name: ci/print-test-failures.sh
++ if: failure()
+ env:
+ CC: clang
+ run: |
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
++ ci/print-test-failures.sh
++ exit 1
+ osx-gcc:
+ runs-on: macos-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ - name: install dependencies
+ run: ci/install-dependencies.sh
+ - name: ci/run-build-and-tests.sh
++ run: ci/run-build-and-tests.sh
++ - name: ci/print-test-failures.sh
++ if: failure()
+ run: |
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
++ ci/print-test-failures.sh
++ exit 1
+ GETTEXT_POISON:
+ runs-on: ubuntu-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ - name: ci/run-build-and-tests.sh
+ env:
+ jobname: GETTEXT_POISON
++ run: ci/run-build-and-tests.sh
++ - name: ci/print-test-failures.sh
++ if: failure()
++ env:
++ jobname: GETTEXT_POISON
+ run: |
-+ ci/run-build-and-tests.sh || {
-+ ci/print-test-failures.sh
-+ exit 1
-+ }
-+ linux32:
++ ci/print-test-failures.sh
++ exit 1
++ linux-musl:
+ runs-on: ubuntu-latest
++ container: alpine
+ steps:
+ - uses: actions/checkout@v1
-+ - name: ci/run-linux32-docker.sh
++ - name: install additional dependencies
++ run: apk add --update ncurses perl-utils
++ - name: prepare /usr/src/git
+ run: |
-+ res=0
-+ sudo GITHUB_ACTIONS="$GITHUB_ACTIONS" RUNNER_OS="$RUNNER_OS" GITHUB_REF="$GITHUB_REF" GITHUB_SHA="$GITHUB_SHA" GITHUB_REPOSITORY="$GITHUB_REPOSITORY" GITHUB_RUN_ID="$GITHUB_RUN_ID" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
++ mkdir -p /usr/src &&
++ rm -rf /usr/src/git &&
++ ln -sf $(pwd) /usr/src/git
++ - name: ci/run-alpine-build.sh
++ run: ci/run-alpine-build.sh $(id -u)
++ - name: ci/print-test-failures.sh
++ if: failure()
++ run: ci/print-test-failures.sh
++ linux32:
++ runs-on: ubuntu-latest
++ container: daald/ubuntu32:xenial
++ steps:
++ - uses: actions/checkout@v1
++ - name: prepare /usr/src/git
++ run:
++ mkdir -p /usr/src &&
++ rm -rf /usr/src/git &&
++ ln -sf $(pwd) /usr/src/git
++ - name: ci/run-linux32-build.sh
++ run: ci/run-linux32-build.sh $(id -u)
++ - name: ci/print-test-failures.sh
++ if: failure()
++ run: ci/print-test-failures.sh
+ static-analysis:
+ runs-on: ubuntu-latest
+ steps:
@@ .github/workflows/main.yml (new)
+ - name: install dependencies
+ run: |
+ sudo apt-get update &&
-+ sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns
++ sudo apt-get install -y asciidoc xmlto asciidoctor docbook-xsl-ns libcurl4-openssl-dev
+ - name: ci/test-documentation.sh
+ env:
+ ALREADY_HAVE_ASCIIDOCTOR: yes.
4: c4f7338184 ! 5: 4310a7c9d6 README: add a build badge for the GitHub Actions runs
@@ Commit message
README: add a build badge for the GitHub Actions runs
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
## README.md ##
@@
5: 9598f73435 ! 6: 2451d4991f ci: retire the Azure Pipelines definition
@@ Commit message
Pipelines would be redundant, and a waste of energy.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
+ [Danh: fix apply conflicts]
Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com>
## README.md ##
@@ azure-pipelines.yml (deleted)
- test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
-
- res=0
-- sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1
+- sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1
-
- sudo chmod a+r t/out/TEST-*.xml
- test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
-
- test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
- exit $res
-- displayName: 'ci/run-linux32-docker.sh'
+- displayName: 'jobname=Linux32 ci/run-docker.sh'
- env:
- GITFILESHAREPWD: $(gitfileshare.pwd)
- - task: PublishTestResults@2
@@ azure-pipelines.yml (deleted)
- PathtoPublish: t/failed-test-artifacts
- ArtifactName: failed-test-artifacts
-
+-- job: linux_musl
+- displayName: linux-musl
+- condition: succeeded()
+- pool:
+- vmImage: ubuntu-latest
+- steps:
+- - bash: |
+- test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1
+-
+- res=0
+- sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1
+-
+- sudo chmod a+r t/out/TEST-*.xml
+- test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts
+-
+- test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1
+- exit $res
+- displayName: 'jobname=linux-musl ci/run-docker.sh'
+- env:
+- GITFILESHAREPWD: $(gitfileshare.pwd)
+- - task: PublishTestResults@2
+- displayName: 'Publish Test Results **/TEST-*.xml'
+- inputs:
+- mergeTestResults: true
+- testRunTitle: 'musl'
+- platform: Linux
+- publishRunAttachments: false
+- condition: succeededOrFailed()
+- - task: PublishBuildArtifacts@1
+- displayName: 'Publish trash directories of failed tests'
+- condition: failed()
+- inputs:
+- PathtoPublish: t/failed-test-artifacts
+- ArtifactName: failed-test-artifacts
+-
-- job: static_analysis
- displayName: StaticAnalysis
- condition: succeeded()-- 2.26.0.334.g6536db25bb