[RFC PATCH v5 00/10] CI: js/ci-github-workflow-markup rebased on "use $GITHUB_ENV"
- From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
- Date
- Apr 21, 2022, 18:36 UTC
- Message-ID
- <RFC-cover-v5-00.10-00000000000-20220421T183001Z-avarab@gmail.com>
- In-Reply-To
- <RFC-cover-v4-0.6-00000000000-20220413T195514Z-avarab@gmail.com>
See the "v3" for the "real" cover-letter: https://lore.kernel.org/git/RFC-cover-v3-0.6-00000000000-20220325T183946Z-avarab@gmail.com/
I submitted a re-roll of the parent series at https://lore.kernel.org/git/cover-v5-00.29-00000000000-20220421T181526Z-avarab@gmail.com which required a conflict resolution for this one.
This version also has outstanding changes I had for using ci/print-test-failures.sh with a parameter instead of the previous copy/pasted and adjusted ci/print-test-failures-github.sh: https://lore.kernel.org/git/RFC-patch-v4-1.6-cc137c69ee1-20220413T195514Z-avarab@gmail.com/
There's some discussion about these two serieses in general, and Johannes intends to re-roll this: https://lore.kernel.org/git/nycvar.QRO.7.76.6.2204211454500.355@tvgsbejvaqbjf.bet/
So whatever Junio ends up (re-)picking up (and I'll reply in some of those threads) hopefully having this version for comparison helps.
Johannes Schindelin (6): ci: make it easier to find failed tests' logs in the GitHub workflow tests: refactor --write-junit-xml code test(junit): avoid line feeds in XML attributes ci: optionally mark up output in the GitHub workflow ci: use `--github-workflow-markup` in the GitHub workflow ci: call `finalize_test_case_output` a little later
Ævar Arnfjörð Bjarmason (4): CI: don't "cd" in ci/print-test-failures.sh CI: add --exit-code to ci/print-test-failures.sh CI: don't include "test-results/" in ci/print-test-failures.sh output CI: stop setting FAILED_TEST_ARTIFACTS N times
.github/workflows/main.yml | 20 +--- ci/lib.sh | 2 +- ci/print-test-failures.sh | 89 ++++++++++++++---- t/test-lib-functions.sh | 4 +- t/test-lib-github-workflow-markup.sh | 50 ++++++++++ t/test-lib-junit.sh | 132 +++++++++++++++++++++++++++ t/test-lib.sh | 128 ++++---------------------- 7 files changed, 282 insertions(+), 143 deletions(-) create mode 100644 t/test-lib-github-workflow-markup.sh create mode 100644 t/test-lib-junit.sh
Range-diff against v4:
-: ----------- > 1: 56910df26e7 CI: don't "cd" in ci/print-test-failures.sh
-: ----------- > 2: caec0b1089a CI: add --exit-code to ci/print-test-failures.sh
-: ----------- > 3: 577a5be34f4 CI: don't include "test-results/" in ci/print-test-failures.sh output
-: ----------- > 4: 1482f840f64 CI: stop setting FAILED_TEST_ARTIFACTS N times
1: cc137c69ee1 ! 5: 4291892fbd3 ci: make it easier to find failed tests' logs in the GitHub workflow
@@ .github/workflows/main.yml: jobs:
- if: failure()
- shell: bash
- run: ci/print-test-failures.sh
-+ run: . /etc/profile && make -C t -e || ci/print-test-failures-github.sh
++ run: . /etc/profile && make -C t -e || ci/print-test-failures.sh
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v2
@@ .github/workflows/main.yml: jobs:
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v2
@@ .github/workflows/main.yml: jobs:
- - run: ci/lib.sh --build
- run: make
- run: ci/lib.sh --test
+ if: success() && matrix.vector.skip-tests != 'yes'
- - run: make test
+ - run: make test || ci/print-test-failures-github.sh
- if: success() && matrix.vector.skip-tests != 'no'
+ if: success() && matrix.vector.skip-tests != 'yes'
- - run: ci/print-test-failures.sh
-- if: failure() && matrix.vector.skip-tests != 'no'
+- if: failure() && matrix.vector.skip-tests != 'yes'
- name: Upload failed tests' directories
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
uses: actions/upload-artifact@v1
- ## ci/print-test-failures-github.sh (new) ##
-@@
-+#!/bin/sh
-+
-+. ${0%/*}/lib-ci-type.sh
-+
-+set -e
-+
+ ## ci/print-test-failures.sh ##
+@@ ci/print-test-failures.sh: set -e
+ . ${0%/*}/lib-ci-type.sh
+ . ${0%/*}/lib-tput.sh
+
+-exit_code=
+case "$CI_TYPE" in
+github-actions)
-+ handle_failed_tests () {
-+ mkdir -p t/failed-test-artifacts
-+ echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV
-+
-+ for test_exit in t/test-results/*.exit
-+ do
-+ test 0 != "$(cat "$test_exit")" || continue
-+
-+ test_name="${test_exit%.exit}"
-+ test_name="${test_name##*/}"
-+ printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
-+ cat "t/test-results/$test_name.out"
-+
-+ trash_dir="t/trash directory.$test_name"
-+ cp "t/test-results/$test_name.out" t/failed-test-artifacts/
-+ tar czf t/failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir"
-+ done
-+ return 1
-+ }
++ exit_code=t
++ github_workflow_markup=t
+ ;;
+*)
-+ echo "Unhandled CI type: $CI_TYPE" >&2
-+ exit 1
++ exit_code=
++ github_workflow_markup=
+ ;;
+esac
+
-+handle_failed_tests
+ while test $# != 0
+ do
+ case "$1" in
+ --exit-code)
+ exit_code=t
+ ;;
++ --no-exit-code)
++ exit_code=
++ ;;
++ --github-workflow-markup)
++ github_workflow_markup=t
++ ;;
++ --no-github-workflow-markup)
++ github_workflow_markup=
++ ;;
+ *)
+ echo "BUG: invalid $0 argument: $1" >&2
+ exit 1
+@@ ci/print-test-failures.sh: do
+ TEST_OUT="${TEST_NAME}.out"
+ TEST_MARKUP="${TEST_NAME}.markup"
+
+- echo "------------------------------------------------------------------------"
+- echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+- echo "------------------------------------------------------------------------"
++ if test -n "$github_workflow_markup"
++ then
++ printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
++ else
++ echo "------------------------------------------------------------------------"
++ echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
++ echo "------------------------------------------------------------------------"
++ fi
+ cat "t/test-results/${TEST_OUT}"
+
+ trash_dir="trash directory.$TEST_NAME"
2: 91f96c4f210 = 6: d9b7d5ddf5a tests: refactor --write-junit-xml code
3: 84c722969d5 = 7: 2d233bbdc22 test(junit): avoid line feeds in XML attributes
4: 8acaa800d3a = 8: 5dd43a1624b ci: optionally mark up output in the GitHub workflow
5: 4499f743dd1 ! 9: 877edff3b4d ci: use `--github-workflow-markup` in the GitHub workflow
@@ ci/lib.sh: MAKEFLAGS="$MAKEFLAGS SKIP_DASHED_BUILT_INS=$SKIP_DASHED_BUILT_INS"
setenv --test GIT_PROVE_OPTS "--timer --jobs $NPROC"
- GIT_TEST_OPTS="--verbose-log -x"
+ GIT_TEST_OPTS="--verbose-log -x --github-workflow-markup"
-+ MAKEFLAGS="$MAKEFLAGS --jobs=10"
test Windows != "$RUNNER_OS" ||
GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
setenv --test GIT_TEST_OPTS "$GIT_TEST_OPTS"
- ## ci/print-test-failures-github.sh ##
-@@ ci/print-test-failures-github.sh: github-actions)
- test_name="${test_exit%.exit}"
- test_name="${test_name##*/}"
- printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n"
-- cat "t/test-results/$test_name.out"
-+ cat "t/test-results/$test_name.markup"
+ ## ci/print-test-failures.sh ##
+@@ ci/print-test-failures.sh: do
+ if test -n "$github_workflow_markup"
+ then
+ printf "\\e[33m\\e[1m=== Failed test: ${TEST_NAME} ===\\e[m\\n"
++ cat "t/test-results/$TEST_MARKUP"
+ else
+ echo "------------------------------------------------------------------------"
+ echo "$(tput setaf 1)${TEST_OUT}...$(tput sgr0)"
+ echo "------------------------------------------------------------------------"
++ cat "t/test-results/${TEST_OUT}"
+ fi
+- cat "t/test-results/${TEST_OUT}"
- trash_dir="t/trash directory.$test_name"
- cp "t/test-results/$test_name.out" t/failed-test-artifacts/
+ trash_dir="trash directory.$TEST_NAME"
+ case "$CI_TYPE" in
6: b291f64821c = 10: 6b278194f9a ci: call `finalize_test_case_output` a little later-- 2.36.0.879.g3659959fcca