# [PATCH 0/3] add travis job for linux with musl libc 42 messages from 2020-03-26 to 2020-04-10. Participants: Đoàn Trần Công Danh, Eric Sunshine, SZEDER Gábor, Junio C Hamano, Danh Doan, Johannes Schindelin. Thread: https://gitlist.dev/t/53103 ## Đoàn Trần Công Danh, 2020-03-26 07:35 Subject: [PATCH 0/3] add travis job for linux with musl libc Message-ID: URL: https://gitlist.dev/e/cover.1585203294.git.congdanhqx%40gmail.com ``` Recently, we've un-broken git for Linux with musl libc, and we have a serie to fix false negative with busybox shell utils. There is a sample travis build for this serie applied on top of master: https://travis-ci.org/github/sgn/git/builds/667111988 And, after merging with my v4 to fix busybox false negative: https://travis-ci.org/github/sgn/git/builds/667112197 Đoàn Trần Công Danh (3): ci: libify logic for usage and checking CI_USER ci: refactor docker runner script travis: build and test on Linux with musl libc and busybox .travis.yml | 10 +++++- azure-pipelines.yml | 4 +-- ci/lib-docker.sh | 37 +++++++++++++++++++++ ci/run-alpine-build.sh | 31 +++++++++++++++++ ci/{run-linux32-docker.sh => run-docker.sh} | 20 +++++++---- ci/run-linux32-build.sh | 35 +------------------ 6 files changed, 94 insertions(+), 43 deletions(-) create mode 100644 ci/lib-docker.sh create mode 100755 ci/run-alpine-build.sh rename ci/{run-linux32-docker.sh => run-docker.sh} (46%) -- 2.26.0.rc2.357.g1e1ba0441d ``` ## Đoàn Trần Công Danh, 2020-03-26 07:35 Subject: [PATCH 1/3] ci: libify logic for usage and checking CI_USER Message-ID: URL: https://gitlist.dev/e/ca87bea4a379dac5d2e68bc18b0084201afe5fc4.1585203294.git.congdanhqx%40gmail.com In-Reply-To: ``` This part of logic will be reused for alpine docker later. Merge those logic into single chunk since they will be used together. While we're at it, add a comment to tell people run with root inside podman container. Signed-off-by: Đoàn Trần Công Danh --- This patch is viewed better with "git diff --color-moved" ci/lib-docker.sh | 37 +++++++++++++++++++++++++++++++++++++ ci/run-linux32-build.sh | 35 +---------------------------------- 2 files changed, 38 insertions(+), 34 deletions(-) create mode 100644 ci/lib-docker.sh diff --git a/ci/lib-docker.sh b/ci/lib-docker.sh new file mode 100644 index 0000000000..ac155ace54 --- /dev/null +++ b/ci/lib-docker.sh @@ -0,0 +1,37 @@ +# Library of functions shared by all CI scripts run inside docker + +if test $# -ne 1 || test -z "$1" +then + echo >&2 "usage: $0 " + exit 1 +fi + +# If this script runs inside a docker container, then all commands are +# usually executed as root. Consequently, the host user might not be +# able to access the test output files. +# If a non 0 host user id is given, then create a user "ci" with that +# user id to make everything accessible to the host user. +HOST_UID=$1 +if test $HOST_UID -eq 0 +then + # Just in case someone does want to run the test suite as root. + # or podman is used in place of docker + CI_USER=root +else + CI_USER=ci + if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID + then + echo "user '$CI_USER' already exists with the requested ID $HOST_UID" + else + useradd -u $HOST_UID $CI_USER + fi + + # Due to a bug the test suite was run as root in the past, so + # a prove state file created back then is only accessible by + # root. Now that bug is fixed, the test suite is run as a + # regular user, but the prove state file coming from Travis + # CI's cache might still be owned by root. + # Make sure that this user has rights to any cached files, + # including an existing prove state file. + test -n "$cache_dir" && chown -R $HOST_UID:$HOST_UID "$cache_dir" +fi diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index e3a193adbc..81296cdd19 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -8,11 +8,7 @@ set -ex -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-linux32-build.sh " - exit 1 -fi +. "${0%/*}/lib-docker.sh" # Update packages to the latest available versions linux32 --32bit i386 sh -c ' @@ -21,35 +17,6 @@ linux32 --32bit i386 sh -c ' libexpat-dev gettext python >/dev/null ' -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi - - # Due to a bug the test suite was run as root in the past, so - # a prove state file created back then is only accessible by - # root. Now that bug is fixed, the test suite is run as a - # regular user, but the prove state file coming from Travis - # CI's cache might still be owned by root. - # Make sure that this user has rights to any cached files, - # including an existing prove state file. - test -n "$cache_dir" && chown -R $HOST_UID:$HOST_UID "$cache_dir" -fi - # Build and test linux32 --32bit i386 su -m -l $CI_USER -c ' set -ex -- 2.26.0.rc2.357.g1e1ba0441d ``` ## Đoàn Trần Công Danh, 2020-03-26 07:35 Subject: [PATCH 2/3] ci: refactor docker runner script Message-ID: <2fada3db3d0c802b45551671dcb75e406e3ed2b6.1585203294.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/2fada3db3d0c802b45551671dcb75e406e3ed2b6.1585203294.git.congdanhqx%40gmail.com In-Reply-To: ``` We will support alpine check in docker later in this serie. While we're at it, tell people to run as root in podman. Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 2 +- azure-pipelines.yml | 4 ++-- ci/{run-linux32-docker.sh => run-docker.sh} | 19 +++++++++++++------ 3 files changed, 16 insertions(+), 9 deletions(-) rename ci/{run-linux32-docker.sh => run-docker.sh} (48%) diff --git a/.travis.yml b/.travis.yml index fc5730b085..32e80e2670 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,7 +32,7 @@ matrix: services: - docker before_install: - script: ci/run-linux32-docker.sh + script: ci/run-docker.sh linux32 - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 675c3a43c9..ef504ff29f 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -489,14 +489,14 @@ jobs: test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 res=0 - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-docker.sh linux32 || res=1 sudo chmod a+r t/out/TEST-*.xml test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 exit $res - displayName: 'ci/run-linux32-docker.sh' + displayName: 'ci/run-docker.sh linux32' env: GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh similarity index 48% rename from ci/run-linux32-docker.sh rename to ci/run-docker.sh index 751acfcf8a..c8dff9d41a 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-docker.sh @@ -1,15 +1,22 @@ #!/bin/sh # -# Download and run Docker image to build and test 32-bit Git +# Download and run Docker image to build and test git # . ${0%/*}/lib.sh -docker pull daald/ubuntu32:xenial +CI_TARGET=${1:-linux32} +case "$CI_TARGET" in +linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; +*) exit 1 ;; +esac + +docker pull "$CI_CONTAINER" # Use the following command to debug the docker build locally: -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial -# root@container:/# /usr/src/git/ci/run-linux32-build.sh +# must be 0 if podman is used in place of docker +# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" +# root@container:/# /usr/src/git/ci/run-$CI_TARGET-build.sh container_cache_dir=/tmp/travis-cache @@ -23,8 +30,8 @@ docker run \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ - daald/ubuntu32:xenial \ - /usr/src/git/ci/run-linux32-build.sh $(id -u $USER) + "$CI_CONTAINER" \ + "/usr/src/git/ci/run-$CI_TARGET-build.sh" $(id -u $USER) check_unignored_build_artifacts -- 2.26.0.rc2.357.g1e1ba0441d ``` ## Đoàn Trần Công Danh, 2020-03-26 07:35 Subject: [PATCH 3/3] travis: build and test on Linux with musl libc and busybox Message-ID: <7d65bb87a25ab35dbf21d972e10a2c3321cc6761.1585203294.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/7d65bb87a25ab35dbf21d972e10a2c3321cc6761.1585203294.git.congdanhqx%40gmail.com In-Reply-To: ``` Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 8 ++++++++ ci/run-alpine-build.sh | 31 +++++++++++++++++++++++++++++++ ci/run-docker.sh | 1 + 3 files changed, 40 insertions(+) create mode 100755 ci/run-alpine-build.sh diff --git a/.travis.yml b/.travis.yml index 32e80e2670..a2927dd120 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,6 +33,14 @@ matrix: - docker before_install: script: ci/run-docker.sh linux32 + - env: jobname=linux-musl-busybox + os: linux + compiler: + addons: + services: + - docker + before_install: + script: ci/run-docker.sh alpine - env: jobname=StaticAnalysis os: linux compiler: diff --git a/ci/run-alpine-build.sh b/ci/run-alpine-build.sh new file mode 100755 index 0000000000..c83df536e4 --- /dev/null +++ b/ci/run-alpine-build.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# +# Build and test Git in Alpine Linux +# +# Usage: +# run-alpine-build.sh +# + +set -ex + +useradd () { + adduser -D "$@" +} + +. "${0%/*}/lib-docker.sh" + +# Update packages to the latest available versions +apk add --update autoconf build-base curl-dev openssl-dev expat-dev \ + gettext pcre2-dev python3 musl-libintl >/dev/null + +# Build and test +su -m -l $CI_USER -c ' + set -ex + cd /usr/src/git + test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove + autoconf + echo "PYTHON_PATH=/usr/bin/python3" >config.mak + ./configure --with-libpcre + make + make test +' diff --git a/ci/run-docker.sh b/ci/run-docker.sh index c8dff9d41a..47affcd6d3 100755 --- a/ci/run-docker.sh +++ b/ci/run-docker.sh @@ -8,6 +8,7 @@ CI_TARGET=${1:-linux32} case "$CI_TARGET" in linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; +alpine) CI_CONTAINER="alpine" ;; *) exit 1 ;; esac -- 2.26.0.rc2.357.g1e1ba0441d ``` ## Eric Sunshine, 2020-03-26 16:06 Subject: Re: [PATCH 2/3] ci: refactor docker runner script Message-ID: URL: https://gitlist.dev/e/CAPig%2BcTajaEU6wFK9wVS77ppeqpwqM95XLMeYRCHUORjBFa3Cg%40mail.gmail.com In-Reply-To: <2fada3db3d0c802b45551671dcb75e406e3ed2b6.1585203294.git.congdanhqx@gmail.com> ``` On Thu, Mar 26, 2020 at 3:35 AM Đoàn Trần Công Danh wrote: > We will support alpine check in docker later in this serie. s/serie/series/ > While we're at it, tell people to run as root in podman. > > Signed-off-by: Đoàn Trần Công Danh ``` ## SZEDER Gábor, 2020-03-28 17:53 Subject: Re: [PATCH 2/3] ci: refactor docker runner script Message-ID: <20200328175329.GD2224@szeder.dev> URL: https://gitlist.dev/e/20200328175329.GD2224%40szeder.dev In-Reply-To: <2fada3db3d0c802b45551671dcb75e406e3ed2b6.1585203294.git.congdanhqx@gmail.com> ``` On Thu, Mar 26, 2020 at 02:35:18PM +0700, Đoàn Trần Công Danh wrote: > We will support alpine check in docker later in this serie. > > While we're at it, tell people to run as root in podman. Why tell that to people? Please clarify what podman is and why should we care. > Signed-off-by: Đoàn Trần Công Danh > --- > .travis.yml | 2 +- > azure-pipelines.yml | 4 ++-- > ci/{run-linux32-docker.sh => run-docker.sh} | 19 +++++++++++++------ > 3 files changed, 16 insertions(+), 9 deletions(-) > rename ci/{run-linux32-docker.sh => run-docker.sh} (48%) > > diff --git a/.travis.yml b/.travis.yml > index fc5730b085..32e80e2670 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -32,7 +32,7 @@ matrix: > services: > - docker > before_install: > - script: ci/run-linux32-docker.sh > + script: ci/run-docker.sh linux32 The name of the 'Linux32' build job starts with a capital 'L'; please be consistent with that. > - env: jobname=StaticAnalysis > os: linux > compiler: > diff --git a/azure-pipelines.yml b/azure-pipelines.yml > index 675c3a43c9..ef504ff29f 100644 > --- a/azure-pipelines.yml > +++ b/azure-pipelines.yml > @@ -489,14 +489,14 @@ jobs: > test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 > > res=0 > - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 > + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-docker.sh linux32 || res=1 > > sudo chmod a+r t/out/TEST-*.xml > test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts > > test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 > exit $res > - displayName: 'ci/run-linux32-docker.sh' > + displayName: 'ci/run-docker.sh linux32' > env: > GITFILESHAREPWD: $(gitfileshare.pwd) > - task: PublishTestResults@2 > diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh > similarity index 48% > rename from ci/run-linux32-docker.sh > rename to ci/run-docker.sh > index 751acfcf8a..c8dff9d41a 100755 > --- a/ci/run-linux32-docker.sh > +++ b/ci/run-docker.sh > @@ -1,15 +1,22 @@ > #!/bin/sh > # > -# Download and run Docker image to build and test 32-bit Git > +# Download and run Docker image to build and test git s/git/Git/ > # > > . ${0%/*}/lib.sh > > -docker pull daald/ubuntu32:xenial > +CI_TARGET=${1:-linux32} > +case "$CI_TARGET" in > +linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; > +*) exit 1 ;; > +esac > + > +docker pull "$CI_CONTAINER" > > # Use the following command to debug the docker build locally: > -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial > -# root@container:/# /usr/src/git/ci/run-linux32-build.sh > +# must be 0 if podman is used in place of docker > +# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" > +# root@container:/# /usr/src/git/ci/run-$CI_TARGET-build.sh > > container_cache_dir=/tmp/travis-cache > > @@ -23,8 +30,8 @@ docker run \ > --env cache_dir="$container_cache_dir" \ > --volume "${PWD}:/usr/src/git" \ > --volume "$cache_dir:$container_cache_dir" \ > - daald/ubuntu32:xenial \ > - /usr/src/git/ci/run-linux32-build.sh $(id -u $USER) > + "$CI_CONTAINER" \ > + "/usr/src/git/ci/run-$CI_TARGET-build.sh" $(id -u $USER) > > check_unignored_build_artifacts > > -- > 2.26.0.rc2.357.g1e1ba0441d > ``` ## Junio C Hamano, 2020-03-29 05:49 Subject: Re: [PATCH 0/3] add travis job for linux with musl libc Message-ID: URL: https://gitlist.dev/e/xmqq4ku7d82k.fsf%40gitster.c.googlers.com In-Reply-To: ``` Đoàn Trần Công Danh writes: > Recently, we've un-broken git for Linux with musl libc, > and we have a serie to fix false negative with busybox shell utils. > > There is a sample travis build for this serie applied on top of master: > https://travis-ci.org/github/sgn/git/builds/667111988 > > And, after merging with my v4 to fix busybox false negative: > https://travis-ci.org/github/sgn/git/builds/667112197 I have this topic near the tip of 'pu' and it seems to be finding issues not in your build with 'master'. I'd expect that we'll be seeing patches to various parts of the system from you or others with musl libc to correct them, after which we can merge the topic down. Thanks. ``` ## Danh Doan, 2020-03-29 06:36 Subject: Re: [PATCH 2/3] ci: refactor docker runner script Message-ID: <20200329063608.GA22082@danh.dev> URL: https://gitlist.dev/e/20200329063608.GA22082%40danh.dev In-Reply-To: <20200328175329.GD2224@szeder.dev> ``` On 2020-03-28 18:53:29+0100, SZEDER Gábor wrote: > On Thu, Mar 26, 2020 at 02:35:18PM +0700, Đoàn Trần Công Danh wrote: > > We will support alpine check in docker later in this serie. > > > > While we're at it, tell people to run as root in podman. > > Why tell that to people? Please clarify what podman is and why should > we care. podman is a docker drop-in placement. I use it instead of docker to develop this series. docker requires a service to be run and user in docker system groups, podman requires neither. root user in podman is mapped to host user. I run into trouble when develop this series in my local machine. > > Signed-off-by: Đoàn Trần Công Danh > > --- > > .travis.yml | 2 +- > > azure-pipelines.yml | 4 ++-- > > ci/{run-linux32-docker.sh => run-docker.sh} | 19 +++++++++++++------ > > 3 files changed, 16 insertions(+), 9 deletions(-) > > rename ci/{run-linux32-docker.sh => run-docker.sh} (48%) > > > > diff --git a/.travis.yml b/.travis.yml > > index fc5730b085..32e80e2670 100644 > > --- a/.travis.yml > > +++ b/.travis.yml > > @@ -32,7 +32,7 @@ matrix: > > services: > > - docker > > before_install: > > - script: ci/run-linux32-docker.sh > > + script: ci/run-docker.sh linux32 > > The name of the 'Linux32' build job starts with a capital 'L'; please > be consistent with that. the old name of the script is run-linux32-docker, I think it's better to rename the job to all lowercase. All other jobs, except Documentation and static analysis are in lowercase. > > > - env: jobname=StaticAnalysis > > os: linux > > compiler: > > diff --git a/azure-pipelines.yml b/azure-pipelines.yml > > index 675c3a43c9..ef504ff29f 100644 > > --- a/azure-pipelines.yml > > +++ b/azure-pipelines.yml > > @@ -489,14 +489,14 @@ jobs: > > test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 > > > > res=0 > > - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 > > + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-docker.sh linux32 || res=1 > > > > sudo chmod a+r t/out/TEST-*.xml > > test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts > > > > test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 > > exit $res > > - displayName: 'ci/run-linux32-docker.sh' > > + displayName: 'ci/run-docker.sh linux32' > > env: > > GITFILESHAREPWD: $(gitfileshare.pwd) > > - task: PublishTestResults@2 > > diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh > > similarity index 48% > > rename from ci/run-linux32-docker.sh > > rename to ci/run-docker.sh > > index 751acfcf8a..c8dff9d41a 100755 > > --- a/ci/run-linux32-docker.sh > > +++ b/ci/run-docker.sh > > @@ -1,15 +1,22 @@ > > #!/bin/sh > > # > > -# Download and run Docker image to build and test 32-bit Git > > +# Download and run Docker image to build and test git > > s/git/Git/ Will change -- Danh ``` ## Đoàn Trần Công Danh, 2020-03-29 10:12 Subject: [PATCH v2 0/4] Travis + Azure jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/cover.1585474409.git.congdanhqx%40gmail.com In-Reply-To: ``` Recently, we've un-broken git for Linux with musl libc, and we have a serie to fix false negative with busybox shell utils. Add a CI job on Travis and Azure to make sure we won't break it again. There is a sample travis build for this serie applied on top of: jt/rebase-allow-duplicate https://travis-ci.org/github/sgn/git/builds/668300819 And, after merging with junio's pu to fix busybox false negative: https://travis-ci.org/github/sgn/git/builds/668300919 https://dev.azure.com/git/git/_build/results?buildId=1910&view=results Change from v1: - fix spelling - run-docker.sh: use "jobname" environment variable instead of passing argument - add linux-musl job on Azure - Add 4th patch for jt/rebase-allow-duplicate (feel free to squash into jt/rebase-allow-duplicate) The first 3 patches could be applied on top of master, but the last patch needs to be applied on top of jt/rebase-allow-duplicate Đoàn Trần Công Danh (4): ci: libify logic for usage and checking CI_USER ci: refactor docker runner script travis: build and test on Linux with musl libc and busybox t3402: use POSIX compliant regex(7) .travis.yml | 10 +++++- azure-pipelines.yml | 39 +++++++++++++++++++-- ci/lib-docker.sh | 37 +++++++++++++++++++ ci/run-alpine-build.sh | 31 ++++++++++++++++ ci/{run-linux32-docker.sh => run-docker.sh} | 26 ++++++++++---- ci/run-linux32-build.sh | 35 +----------------- t/t3402-rebase-merge.sh | 8 ++--- 7 files changed, 139 insertions(+), 47 deletions(-) create mode 100644 ci/lib-docker.sh create mode 100755 ci/run-alpine-build.sh rename ci/{run-linux32-docker.sh => run-docker.sh} (46%) Range-diff against v1: 1: f23f2a563a = 1: 1ec7c2024d ci: libify logic for usage and checking CI_USER 2: 6fd1370678 ! 2: 140e0ef390 ci: refactor docker runner script @@ Metadata ## Commit message ## ci: refactor docker runner script - We will support alpine check in docker later in this serie. + We will support alpine check in docker later in this series. While we're at it, tell people to run as root in podman. @@ .travis.yml: matrix: - docker before_install: - script: ci/run-linux32-docker.sh -+ script: ci/run-docker.sh linux32 ++ script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: @@ azure-pipelines.yml: jobs: res=0 - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 -+ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-docker.sh linux32 || res=1 ++ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1 sudo chmod a+r t/out/TEST-*.xml test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts @@ azure-pipelines.yml: jobs: test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 exit $res - displayName: 'ci/run-linux32-docker.sh' -+ displayName: 'ci/run-docker.sh linux32' ++ displayName: 'jobname=Linux32 ci/run-docker.sh' env: GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 @@ ci/run-docker.sh (new) @@ +#!/bin/sh +# -+# Download and run Docker image to build and test git ++# Download and run Docker image to build and test Git +# + +. ${0%/*}/lib.sh + -+CI_TARGET=${1:-linux32} -+case "$CI_TARGET" in -+linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; -+*) exit 1 ;; ++case "$jobname" in ++Linux32) ++ CI_TARGET=linux32 ++ CI_CONTAINER="daald/ubuntu32:xenial" ++ ;; ++*) ++ exit 1 ;; +esac + +docker pull "$CI_CONTAINER" 3: 2f68e65fb7 ! 3: 6cf6400f2e travis: build and test on Linux with musl libc and busybox @@ .travis.yml @@ .travis.yml: matrix: - docker before_install: - script: ci/run-docker.sh linux32 -+ - env: jobname=linux-musl-busybox + script: ci/run-docker.sh ++ - env: jobname=linux-musl + os: linux + compiler: + addons: + services: + - docker + before_install: -+ script: ci/run-docker.sh alpine ++ script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: + ## azure-pipelines.yml ## +@@ azure-pipelines.yml: jobs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + ++- job: linux_musl ++ displayName: linux-musl ++ condition: succeeded() ++ pool: ++ vmImage: ubuntu-latest ++ steps: ++ - bash: | ++ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 ++ ++ res=0 ++ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1 ++ ++ sudo chmod a+r t/out/TEST-*.xml ++ test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts ++ ++ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 ++ exit $res ++ displayName: 'jobname=linux-musl ci/run-docker.sh' ++ env: ++ GITFILESHAREPWD: $(gitfileshare.pwd) ++ - task: PublishTestResults@2 ++ displayName: 'Publish Test Results **/TEST-*.xml' ++ inputs: ++ mergeTestResults: true ++ testRunTitle: 'musl' ++ platform: Linux ++ publishRunAttachments: false ++ condition: succeededOrFailed() ++ - task: PublishBuildArtifacts@1 ++ displayName: 'Publish trash directories of failed tests' ++ condition: failed() ++ inputs: ++ PathtoPublish: t/failed-test-artifacts ++ ArtifactName: failed-test-artifacts ++ + - job: static_analysis + displayName: StaticAnalysis + condition: succeeded() + ## ci/run-alpine-build.sh (new) ## @@ +#!/bin/sh @@ ci/run-alpine-build.sh (new) +' ## ci/run-docker.sh ## -@@ - CI_TARGET=${1:-linux32} - case "$CI_TARGET" in - linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; -+alpine) CI_CONTAINER="alpine" ;; - *) exit 1 ;; +@@ ci/run-docker.sh: Linux32) + CI_TARGET=linux32 + CI_CONTAINER="daald/ubuntu32:xenial" + ;; ++linux-musl) ++ CI_TARGET=alpine ++ CI_CONTAINER=alpine ++ ;; + *) + exit 1 ;; esac - -: ---------- > 4: a4eacb4362 t3402: use POSIX compliant regex(7) -- 2.26.0.302.g234993491e ``` ## Đoàn Trần Công Danh, 2020-03-29 10:12 Subject: [PATCH v2 1/4] ci: libify logic for usage and checking CI_USER Message-ID: <1ec7c2024d60ce2b6888367fc48942e8febef15f.1585474409.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/1ec7c2024d60ce2b6888367fc48942e8febef15f.1585474409.git.congdanhqx%40gmail.com In-Reply-To: ``` This part of logic will be reused for alpine docker later. Merge those logic into single chunk since they will be used together. While we're at it, add a comment to tell people run with root inside podman container. Signed-off-by: Đoàn Trần Công Danh --- ci/lib-docker.sh | 37 +++++++++++++++++++++++++++++++++++++ ci/run-linux32-build.sh | 35 +---------------------------------- 2 files changed, 38 insertions(+), 34 deletions(-) create mode 100644 ci/lib-docker.sh diff --git a/ci/lib-docker.sh b/ci/lib-docker.sh new file mode 100644 index 0000000000..ac155ace54 --- /dev/null +++ b/ci/lib-docker.sh @@ -0,0 +1,37 @@ +# Library of functions shared by all CI scripts run inside docker + +if test $# -ne 1 || test -z "$1" +then + echo >&2 "usage: $0 " + exit 1 +fi + +# If this script runs inside a docker container, then all commands are +# usually executed as root. Consequently, the host user might not be +# able to access the test output files. +# If a non 0 host user id is given, then create a user "ci" with that +# user id to make everything accessible to the host user. +HOST_UID=$1 +if test $HOST_UID -eq 0 +then + # Just in case someone does want to run the test suite as root. + # or podman is used in place of docker + CI_USER=root +else + CI_USER=ci + if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID + then + echo "user '$CI_USER' already exists with the requested ID $HOST_UID" + else + useradd -u $HOST_UID $CI_USER + fi + + # Due to a bug the test suite was run as root in the past, so + # a prove state file created back then is only accessible by + # root. Now that bug is fixed, the test suite is run as a + # regular user, but the prove state file coming from Travis + # CI's cache might still be owned by root. + # Make sure that this user has rights to any cached files, + # including an existing prove state file. + test -n "$cache_dir" && chown -R $HOST_UID:$HOST_UID "$cache_dir" +fi diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index e3a193adbc..81296cdd19 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -8,11 +8,7 @@ set -ex -if test $# -ne 1 || test -z "$1" -then - echo >&2 "usage: run-linux32-build.sh " - exit 1 -fi +. "${0%/*}/lib-docker.sh" # Update packages to the latest available versions linux32 --32bit i386 sh -c ' @@ -21,35 +17,6 @@ linux32 --32bit i386 sh -c ' libexpat-dev gettext python >/dev/null ' -# If this script runs inside a docker container, then all commands are -# usually executed as root. Consequently, the host user might not be -# able to access the test output files. -# If a non 0 host user id is given, then create a user "ci" with that -# user id to make everything accessible to the host user. -HOST_UID=$1 -if test $HOST_UID -eq 0 -then - # Just in case someone does want to run the test suite as root. - CI_USER=root -else - CI_USER=ci - if test "$(id -u $CI_USER 2>/dev/null)" = $HOST_UID - then - echo "user '$CI_USER' already exists with the requested ID $HOST_UID" - else - useradd -u $HOST_UID $CI_USER - fi - - # Due to a bug the test suite was run as root in the past, so - # a prove state file created back then is only accessible by - # root. Now that bug is fixed, the test suite is run as a - # regular user, but the prove state file coming from Travis - # CI's cache might still be owned by root. - # Make sure that this user has rights to any cached files, - # including an existing prove state file. - test -n "$cache_dir" && chown -R $HOST_UID:$HOST_UID "$cache_dir" -fi - # Build and test linux32 --32bit i386 su -m -l $CI_USER -c ' set -ex -- 2.26.0.302.g234993491e ``` ## Đoàn Trần Công Danh, 2020-03-29 10:12 Subject: [PATCH v2 2/4] ci: refactor docker runner script Message-ID: <140e0ef39064d0194e05cad6d9f48e8e5b03aacc.1585474409.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/140e0ef39064d0194e05cad6d9f48e8e5b03aacc.1585474409.git.congdanhqx%40gmail.com In-Reply-To: ``` We will support alpine check in docker later in this series. While we're at it, tell people to run as root in podman. Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 2 +- azure-pipelines.yml | 4 ++-- ci/{run-linux32-docker.sh => run-docker.sh} | 22 +++++++++++++++------ 3 files changed, 19 insertions(+), 9 deletions(-) rename ci/{run-linux32-docker.sh => run-docker.sh} (48%) diff --git a/.travis.yml b/.travis.yml index fc5730b085..069aeeff3c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,7 +32,7 @@ matrix: services: - docker before_install: - script: ci/run-linux32-docker.sh + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index af2a5ea484..f6dcc35ad4 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -478,14 +478,14 @@ jobs: test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 res=0 - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1 sudo chmod a+r t/out/TEST-*.xml test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 exit $res - displayName: 'ci/run-linux32-docker.sh' + displayName: 'jobname=Linux32 ci/run-docker.sh' env: GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh similarity index 48% rename from ci/run-linux32-docker.sh rename to ci/run-docker.sh index 751acfcf8a..be698817cb 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-docker.sh @@ -1,15 +1,25 @@ #!/bin/sh # -# Download and run Docker image to build and test 32-bit Git +# Download and run Docker image to build and test Git # . ${0%/*}/lib.sh -docker pull daald/ubuntu32:xenial +case "$jobname" in +Linux32) + CI_TARGET=linux32 + CI_CONTAINER="daald/ubuntu32:xenial" + ;; +*) + exit 1 ;; +esac + +docker pull "$CI_CONTAINER" # Use the following command to debug the docker build locally: -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial -# root@container:/# /usr/src/git/ci/run-linux32-build.sh +# must be 0 if podman is used in place of docker +# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" +# root@container:/# /usr/src/git/ci/run-$CI_TARGET-build.sh container_cache_dir=/tmp/travis-cache @@ -23,8 +33,8 @@ docker run \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ - daald/ubuntu32:xenial \ - /usr/src/git/ci/run-linux32-build.sh $(id -u $USER) + "$CI_CONTAINER" \ + "/usr/src/git/ci/run-$CI_TARGET-build.sh" $(id -u $USER) check_unignored_build_artifacts -- 2.26.0.302.g234993491e ``` ## Đoàn Trần Công Danh, 2020-03-29 10:12 Subject: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: <6cf6400f2e0f7336e9b13ddbf412101c4ab48dc5.1585474409.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/6cf6400f2e0f7336e9b13ddbf412101c4ab48dc5.1585474409.git.congdanhqx%40gmail.com In-Reply-To: ``` Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 8 ++++++++ azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++++++ ci/run-alpine-build.sh | 31 +++++++++++++++++++++++++++++++ ci/run-docker.sh | 4 ++++ 4 files changed, 78 insertions(+) create mode 100755 ci/run-alpine-build.sh diff --git a/.travis.yml b/.travis.yml index 069aeeff3c..0cfc3c3428 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,6 +33,14 @@ matrix: - docker before_install: script: ci/run-docker.sh + - env: jobname=linux-musl + os: linux + compiler: + addons: + services: + - docker + before_install: + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index f6dcc35ad4..615289167b 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -503,6 +503,41 @@ jobs: PathtoPublish: t/failed-test-artifacts ArtifactName: failed-test-artifacts +- job: linux_musl + displayName: linux-musl + condition: succeeded() + pool: + vmImage: ubuntu-latest + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + res=0 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1 + + sudo chmod a+r t/out/TEST-*.xml + test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 + exit $res + displayName: 'jobname=linux-musl ci/run-docker.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'musl' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + - job: static_analysis displayName: StaticAnalysis condition: succeeded() diff --git a/ci/run-alpine-build.sh b/ci/run-alpine-build.sh new file mode 100755 index 0000000000..c83df536e4 --- /dev/null +++ b/ci/run-alpine-build.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# +# Build and test Git in Alpine Linux +# +# Usage: +# run-alpine-build.sh +# + +set -ex + +useradd () { + adduser -D "$@" +} + +. "${0%/*}/lib-docker.sh" + +# Update packages to the latest available versions +apk add --update autoconf build-base curl-dev openssl-dev expat-dev \ + gettext pcre2-dev python3 musl-libintl >/dev/null + +# Build and test +su -m -l $CI_USER -c ' + set -ex + cd /usr/src/git + test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove + autoconf + echo "PYTHON_PATH=/usr/bin/python3" >config.mak + ./configure --with-libpcre + make + make test +' diff --git a/ci/run-docker.sh b/ci/run-docker.sh index be698817cb..f203db03cf 100755 --- a/ci/run-docker.sh +++ b/ci/run-docker.sh @@ -10,6 +10,10 @@ Linux32) CI_TARGET=linux32 CI_CONTAINER="daald/ubuntu32:xenial" ;; +linux-musl) + CI_TARGET=alpine + CI_CONTAINER=alpine + ;; *) exit 1 ;; esac -- 2.26.0.302.g234993491e ``` ## Đoàn Trần Công Danh, 2020-03-29 10:12 Subject: [PATCH v2 4/4] t3402: use POSIX compliant regex(7) Message-ID: URL: https://gitlist.dev/e/a4eacb4362f29857bb40907e6326d1b0c12d3eb6.1585474409.git.congdanhqx%40gmail.com In-Reply-To: ``` `\?` is undefined for POSIX BRE, from: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_02 > The interpretation of an ordinary character preceded > by an unescaped ( '\\' ) is undefined, except for: > - The characters ')', '(', '{', and '}' > - The digits 1 to 9 inclusive > - A character inside a bracket expression This test is failing with busybox grep. Fix it by using character class. Signed-off-by: Đoàn Trần Công Danh --- Needs to be applied on top of jt/rebase-allow-duplicate Cc: Jonathan Tan t/t3402-rebase-merge.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/t/t3402-rebase-merge.sh b/t/t3402-rebase-merge.sh index 290c79e0f6..e8bab8102d 100755 --- a/t/t3402-rebase-merge.sh +++ b/t/t3402-rebase-merge.sh @@ -228,15 +228,15 @@ test_expect_success '--skip-cherry-pick-detection refrains from reading unneeded git -C client rev-list --objects --all --missing=print >missing_list && MERGE_BASE_BLOB=$(git -C server rev-parse master^^:file.txt) && ADD_11_BLOB=$(git -C server rev-parse master^:file.txt) && - grep "\\?$MERGE_BASE_BLOB" missing_list && - grep "\\?$ADD_11_BLOB" missing_list && + grep "[?]$MERGE_BASE_BLOB" missing_list && + grep "[?]$ADD_11_BLOB" missing_list && git -C client rebase --merge --skip-cherry-pick-detection origin/master && # The blob from the merge base had to be fetched, but not "add 11" git -C client rev-list --objects --all --missing=print >missing_list && - ! grep "\\?$MERGE_BASE_BLOB" missing_list && - grep "\\?$ADD_11_BLOB" missing_list + ! grep "[?]$MERGE_BASE_BLOB" missing_list && + grep "[?]$ADD_11_BLOB" missing_list ' test_done -- 2.26.0.302.g234993491e ``` ## Junio C Hamano, 2020-03-29 16:23 Subject: Re: [PATCH v2 0/4] Travis + Azure jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/xmqqmu7zb04q.fsf%40gitster.c.googlers.com In-Reply-To: ``` Đoàn Trần Công Danh writes: > Change from v1: > - fix spelling > - run-docker.sh: use "jobname" environment variable instead of passing argument > - add linux-musl job on Azure > - Add 4th patch for jt/rebase-allow-duplicate (feel free to squash into > jt/rebase-allow-duplicate) > > The first 3 patches could be applied on top of master, > but the last patch needs to be applied on top of jt/rebase-allow-duplicate This note was very helpful. Very much appreciated. Please keep this a three-patch series ([1/4], [2/4] and [3/4] become [1/3], [2/3] and [3/3]), and make the fourth one a separate fix to the other topic. Even if we were not going to take this topic, the last one is an independently useful improvement. I'll update jt/rebase-allow-duplicate with the last one, so no real harm done, but keeping the topics separate on the list would help reduce confusion. Thanks. ``` ## SZEDER Gábor, 2020-04-01 21:51 Subject: Re: [PATCH v2 2/4] ci: refactor docker runner script Message-ID: <20200401215121.GG2224@szeder.dev> URL: https://gitlist.dev/e/20200401215121.GG2224%40szeder.dev In-Reply-To: <140e0ef39064d0194e05cad6d9f48e8e5b03aacc.1585474409.git.congdanhqx@gmail.com> ``` On Sun, Mar 29, 2020 at 05:12:30PM +0700, Đoàn Trần Công Danh wrote: > We will support alpine check in docker later in this series. > > While we're at it, tell people to run as root in podman. Ok, I'll try to be more specific :) Please clarify *in the commit message* why we should tell this to people, i.e. what podman is and why should we care. ``` ## SZEDER Gábor, 2020-04-01 22:18 Subject: Re: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: <20200401221835.GH2224@szeder.dev> URL: https://gitlist.dev/e/20200401221835.GH2224%40szeder.dev In-Reply-To: <6cf6400f2e0f7336e9b13ddbf412101c4ab48dc5.1585474409.git.congdanhqx@gmail.com> ``` On Sun, Mar 29, 2020 at 05:12:31PM +0700, Đoàn Trần Công Danh wrote: > Signed-off-by: Đoàn Trần Công Danh > diff --git a/ci/run-alpine-build.sh b/ci/run-alpine-build.sh > new file mode 100755 > index 0000000000..c83df536e4 > --- /dev/null > +++ b/ci/run-alpine-build.sh > @@ -0,0 +1,31 @@ > +#!/bin/sh > +# > +# Build and test Git in Alpine Linux > +# > +# Usage: > +# run-alpine-build.sh > +# > + > +set -ex > + > +useradd () { > + adduser -D "$@" > +} > + > +. "${0%/*}/lib-docker.sh" > + > +# Update packages to the latest available versions > +apk add --update autoconf build-base curl-dev openssl-dev expat-dev \ > + gettext pcre2-dev python3 musl-libintl >/dev/null In 'ci/run-docker.sh' we run 'docker run' with a bunch of '--env ...' options to make some important environment variables available inside the container. At this point in this script all those variables are set to the expected values, but ... > +# Build and test > +su -m -l $CI_USER -c ' ... but here, for some reason, those environment variables are not set anymore. This is bad, because this CI job then builds Git sequentially, runs the tests sequentially, runs the tests with 'make' instead of 'prove', and runs the tests without '-V -x'. IOW, it's slow, it produces a lot of useless output, it doesn't report all the failures, and doesn't tell us anything about the failures. > + set -ex > + cd /usr/src/git > + test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove > + autoconf > + echo "PYTHON_PATH=/usr/bin/python3" >config.mak > + ./configure --with-libpcre The recommended way to build Git is without autoconf and configure. You can set the PYTHON_PATH and USE_LIBPCRE Makefile knobs in MAKEFLAGS in 'ci/lib.sh', but to be able to access MAKEFLAGS in the container you'll need to build this patch series on top of: https://public-inbox.org/git/20200401212151.15164-1-szeder.dev@gmail.com/ > + make > + make test > +' > diff --git a/ci/run-docker.sh b/ci/run-docker.sh > index be698817cb..f203db03cf 100755 > --- a/ci/run-docker.sh > +++ b/ci/run-docker.sh > @@ -10,6 +10,10 @@ Linux32) > CI_TARGET=linux32 > CI_CONTAINER="daald/ubuntu32:xenial" > ;; > +linux-musl) > + CI_TARGET=alpine > + CI_CONTAINER=alpine > + ;; > *) > exit 1 ;; > esac > -- > 2.26.0.302.g234993491e > ``` ## Danh Doan, 2020-04-02 01:42 Subject: Re: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: <20200402014202.GC2764@danh.dev> URL: https://gitlist.dev/e/20200402014202.GC2764%40danh.dev In-Reply-To: <20200401221835.GH2224@szeder.dev> ``` On 2020-04-02 00:18:35+0200, SZEDER Gábor wrote: > On Sun, Mar 29, 2020 at 05:12:31PM +0700, Đoàn Trần Công Danh wrote: > > Signed-off-by: Đoàn Trần Công Danh > > > diff --git a/ci/run-alpine-build.sh b/ci/run-alpine-build.sh > > new file mode 100755 > > index 0000000000..c83df536e4 > > --- /dev/null > > +++ b/ci/run-alpine-build.sh > > @@ -0,0 +1,31 @@ > > +#!/bin/sh > > +# > > +# Build and test Git in Alpine Linux > > +# > > +# Usage: > > +# run-alpine-build.sh > > +# > > + > > +set -ex > > + > > +useradd () { > > + adduser -D "$@" > > +} > > + > > +. "${0%/*}/lib-docker.sh" > > + > > +# Update packages to the latest available versions > > +apk add --update autoconf build-base curl-dev openssl-dev expat-dev \ > > + gettext pcre2-dev python3 musl-libintl >/dev/null > > In 'ci/run-docker.sh' we run 'docker run' with a bunch of '--env ...' > options to make some important environment variables available > inside the container. At this point in this script all those > variables are set to the expected values, but ... > > > +# Build and test > > +su -m -l $CI_USER -c ' > > ... but here, for some reason, those environment variables are not set > anymore. This is bad, because this CI job then builds Git > sequentially, runs the tests sequentially, runs the tests with 'make' > instead of 'prove', and runs the tests without '-V -x'. IOW, it's > slow, it produces a lot of useless output, it doesn't report all the > failures, and doesn't tell us anything about the failures. At this point, I tempted to change this to su -m -l $CI_USER -c /usr/src/git/ci/run-build-and-tests.sh instead. But, after digging into ci/lib.sh, I found too many setup for "$CI_*", let choose your path instead. > > + set -ex > > + cd /usr/src/git > > + test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove > > + autoconf > > + echo "PYTHON_PATH=/usr/bin/python3" >config.mak > > + ./configure --with-libpcre > > The recommended way to build Git is without autoconf and configure. > You can set the PYTHON_PATH and USE_LIBPCRE Makefile knobs in > MAKEFLAGS in 'ci/lib.sh', but to be able to access MAKEFLAGS in the > container you'll need to build this patch series on top of: > > https://public-inbox.org/git/20200401212151.15164-1-szeder.dev@gmail.com/ > > > + make > > + make test > > +' > > diff --git a/ci/run-docker.sh b/ci/run-docker.sh > > index be698817cb..f203db03cf 100755 > > --- a/ci/run-docker.sh > > +++ b/ci/run-docker.sh > > @@ -10,6 +10,10 @@ Linux32) > > CI_TARGET=linux32 > > CI_CONTAINER="daald/ubuntu32:xenial" > > ;; > > +linux-musl) > > + CI_TARGET=alpine > > + CI_CONTAINER=alpine > > + ;; > > *) > > exit 1 ;; > > esac > > -- > > 2.26.0.302.g234993491e > > -- Danh ``` ## Đoàn Trần Công Danh, 2020-04-02 13:03 Subject: [PATCH v3 0/6] Travis + Azure jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/cover.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` Recently, we've un-broken git for Linux with musl libc, and we have a serie to fix false negative with busybox shell utils. Add a CI job on Travis and Azure to make sure we won't break it again. This is a nearly rewrite of this series, because there're GitHub Action allow running directly inside container. => I rewrite this series to prepare as much as possible for the GitHub Action series. => No range-diff The first patch is coming from Szeder, Junio hasn't picked it up yet. And, this series depends on it. Sample build without busybox fix series: https://travis-ci.org/github/sgn/git/builds/670097222 With busybox fix: https://travis-ci.org/github/sgn/git/builds/670103249 SZEDER Gábor (1): ci: make MAKEFLAGS available inside the Docker container in the Linux32 job Đoàn Trần Công Danh (5): ci/lib-docker: preserve required environment variables ci/linux32: parameterise command to switch arch ci: refactor docker runner script ci/linux32: libify install-dependencies step travis: build and test on Linux with musl libc and busybox .travis.yml | 10 ++++- azure-pipelines.yml | 39 ++++++++++++++++++- ci/install-docker-dependencies.sh | 18 +++++++++ ci/lib.sh | 8 ++++ ...n-linux32-build.sh => run-docker-build.sh} | 39 +++++++++++++------ ci/{run-linux32-docker.sh => run-docker.sh} | 28 ++++++++++--- 6 files changed, 121 insertions(+), 21 deletions(-) create mode 100755 ci/install-docker-dependencies.sh rename ci/{run-linux32-build.sh => run-docker-build.sh} (63%) rename ci/{run-linux32-docker.sh => run-docker.sh} (43%) -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 1/6] ci: make MAKEFLAGS available inside the Docker container in the Linux32 job Message-ID: <2fdce60075935f8ff94bc117747f1768f41ed435.1585832270.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/2fdce60075935f8ff94bc117747f1768f41ed435.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` From: SZEDER Gábor Once upon a time we ran 'make --jobs=2 ...' to build Git, its documentation, or to apply Coccinelle semantic patches. Then commit eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) came along, and started using the MAKEFLAGS environment variable to centralize setting the number of parallel jobs in 'ci/libs.sh'. Alas, it forgot to update 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container running the 32 bit Linux job, and, consequently, since then that job builds Git sequentially, and it ignores any Makefile knobs that we might set in MAKEFLAGS (though we don't set any for the 32 bit Linux job at the moment). So update the 'docker run' invocation in 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container as well. Set CC=gcc for the 32 bit Linux job, because that's the compiler installed in the 32 bit Linux Docker image that we use (Travis CI nowadays sets CC=clang by default, but clang is not installed in this image). Signed-off-by: SZEDER Gábor Signed-off-by: Đoàn Trần Công Danh --- ci/lib.sh | 3 +++ ci/run-linux32-docker.sh | 1 + 2 files changed, 4 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index c3a8cd2104..d637825222 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -198,6 +198,9 @@ osx-clang|osx-gcc) GIT_TEST_GETTEXT_POISON) export GIT_TEST_GETTEXT_POISON=true ;; +Linux32) + CC=gcc + ;; esac MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index 751acfcf8a..ebb18fa747 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -20,6 +20,7 @@ docker run \ --env GIT_PROVE_OPTS \ --env GIT_TEST_OPTS \ --env GIT_TEST_CLONE_2GB \ + --env MAKEFLAGS \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 2/6] ci/lib-docker: preserve required environment variables Message-ID: URL: https://gitlist.dev/e/b7b079f559a17b6d6cef037afd6ce023df8f90b0.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` We're using "su -m" to preserve environment variables in the shell run by "su". But, that options will be ignored while "-l" (aka "--login") is specified. Since we don't have interest in all environment variables, pass only those necessary variables to the inner script. Signed-off-by: Đoàn Trần Công Danh --- ci/run-linux32-build.sh | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index e3a193adbc..7f985615c2 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -51,10 +51,17 @@ else fi # Build and test -linux32 --32bit i386 su -m -l $CI_USER -c ' +linux32 --32bit i386 su -m -l $CI_USER -c " set -ex + export DEVELOPER='$DEVELOPER' + export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' + export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' + export GIT_TEST_OPTS='$GIT_TEST_OPTS' + export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' + export MAKEFLAGS='$MAKEFLAGS' + export cache_dir='$cache_dir' cd /usr/src/git - test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove + test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove make make test -' +" -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 3/6] ci/linux32: parameterise command to switch arch Message-ID: <8c8cf3eb24d3099fcf95807a10c5e1b9b0256a98.1585832270.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/8c8cf3eb24d3099fcf95807a10c5e1b9b0256a98.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` In a later patch, the remaining of this command will be re-used for the CI job for linux with musl libc. Allow customisation of the emulator, now. Signed-off-by: Đoàn Trần Công Danh --- ci/run-linux32-build.sh | 13 +++++++++++-- ci/run-linux32-docker.sh | 2 ++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index 7f985615c2..44bb332f64 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -14,8 +14,17 @@ then exit 1 fi +case "$jobname" in +Linux32) + switch_cmd="linux32 --32bit i386" + ;; +*) + exit 1 + ;; +esac + # Update packages to the latest available versions -linux32 --32bit i386 sh -c ' +command $switch_cmd sh -c ' apt update >/dev/null && apt install -y build-essential libcurl4-openssl-dev libssl-dev \ libexpat-dev gettext python >/dev/null @@ -51,7 +60,7 @@ else fi # Build and test -linux32 --32bit i386 su -m -l $CI_USER -c " +command $switch_cmd su -m -l $CI_USER -c " set -ex export DEVELOPER='$DEVELOPER' export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index ebb18fa747..54186b6aa7 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -9,6 +9,7 @@ docker pull daald/ubuntu32:xenial # Use the following command to debug the docker build locally: # $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial +# root@container:/# export jobname= # root@container:/# /usr/src/git/ci/run-linux32-build.sh container_cache_dir=/tmp/travis-cache @@ -21,6 +22,7 @@ docker run \ --env GIT_TEST_OPTS \ --env GIT_TEST_CLONE_2GB \ --env MAKEFLAGS \ + --env jobname \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 4/6] ci: refactor docker runner script Message-ID: <22cc7960c3a3a74ad3cd8e7e9ea36d3952249702.1585832270.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/22cc7960c3a3a74ad3cd8e7e9ea36d3952249702.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` We will support alpine check in docker later in this series. While we're at it, tell people to run as root in podman, if podman is used as drop-in replacement for docker, because podman will map host-user to container's root, therefore, mapping their permission. Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 2 +- azure-pipelines.yml | 4 ++-- ...n-linux32-build.sh => run-docker-build.sh} | 6 ++--- ci/{run-linux32-docker.sh => run-docker.sh} | 22 ++++++++++++++----- 4 files changed, 22 insertions(+), 12 deletions(-) rename ci/{run-linux32-build.sh => run-docker-build.sh} (93%) rename ci/{run-linux32-docker.sh => run-docker.sh} (53%) diff --git a/.travis.yml b/.travis.yml index fc5730b085..069aeeff3c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,7 +32,7 @@ matrix: services: - docker before_install: - script: ci/run-linux32-docker.sh + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 675c3a43c9..11413f66f8 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -489,14 +489,14 @@ jobs: test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 res=0 - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1 sudo chmod a+r t/out/TEST-*.xml test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 exit $res - displayName: 'ci/run-linux32-docker.sh' + displayName: 'jobname=Linux32 ci/run-docker.sh' env: GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 diff --git a/ci/run-linux32-build.sh b/ci/run-docker-build.sh similarity index 93% rename from ci/run-linux32-build.sh rename to ci/run-docker-build.sh index 44bb332f64..a05b48c559 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-docker-build.sh @@ -1,16 +1,16 @@ #!/bin/sh # -# Build and test Git in a 32-bit environment +# Build and test Git inside container # # Usage: -# run-linux32-build.sh +# run-docker-build.sh # set -ex if test $# -ne 1 || test -z "$1" then - echo >&2 "usage: run-linux32-build.sh " + echo >&2 "usage: run-docker-build.sh " exit 1 fi diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh similarity index 53% rename from ci/run-linux32-docker.sh rename to ci/run-docker.sh index 54186b6aa7..3881f99b53 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-docker.sh @@ -1,16 +1,26 @@ #!/bin/sh # -# Download and run Docker image to build and test 32-bit Git +# Download and run Docker image to build and test Git # . ${0%/*}/lib.sh -docker pull daald/ubuntu32:xenial +case "$jobname" in +Linux32) + CI_CONTAINER="daald/ubuntu32:xenial" + ;; +*) + exit 1 + ;; +esac + +docker pull "$CI_CONTAINER" # Use the following command to debug the docker build locally: -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial +# must be 0 if podman is used as drop-in replacement for docker +# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" # root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-linux32-build.sh +# root@container:/# /usr/src/git/ci/run-docker-build.sh container_cache_dir=/tmp/travis-cache @@ -26,8 +36,8 @@ docker run \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ - daald/ubuntu32:xenial \ - /usr/src/git/ci/run-linux32-build.sh $(id -u $USER) + "$CI_CONTAINER" \ + /usr/src/git/ci/run-docker-build.sh $(id -u $USER) check_unignored_build_artifacts -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 5/6] ci/linux32: libify install-dependencies step Message-ID: <2e0d54f81e4b997fd469cb6c0d04820140079a7c.1585832270.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/2e0d54f81e4b997fd469cb6c0d04820140079a7c.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` In a later patch, we will add new Travis Job for linux-musl. Most of other code in this file could be reuse for that job. Move the code to install dependencies to a common script. Should we add new CI system that can run directly in container, we can reuse this script for installation step. Signed-off-by: Đoàn Trần Công Danh --- ci/install-docker-dependencies.sh | 14 ++++++++++++++ ci/run-docker-build.sh | 7 +------ 2 files changed, 15 insertions(+), 6 deletions(-) create mode 100755 ci/install-docker-dependencies.sh diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh new file mode 100755 index 0000000000..a104c61d29 --- /dev/null +++ b/ci/install-docker-dependencies.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# +# Install dependencies required to build and test Git inside container +# + +case "$jobname" in +Linux32) + linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; +esac diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh index a05b48c559..4a153492ba 100755 --- a/ci/run-docker-build.sh +++ b/ci/run-docker-build.sh @@ -23,12 +23,7 @@ Linux32) ;; esac -# Update packages to the latest available versions -command $switch_cmd sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev libssl-dev \ - libexpat-dev gettext python >/dev/null -' +"${0%/*}/install-docker-dependencies.sh" # If this script runs inside a docker container, then all commands are # usually executed as root. Consequently, the host user might not be -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-02 13:04 Subject: [PATCH v3 6/6] travis: build and test on Linux with musl libc and busybox Message-ID: URL: https://gitlist.dev/e/b61ed50cf63371858c74efc19797342107fdff3d.1585832270.git.congdanhqx%40gmail.com In-Reply-To: ``` Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 8 +++++++ azure-pipelines.yml | 35 +++++++++++++++++++++++++++++++ ci/install-docker-dependencies.sh | 4 ++++ ci/lib.sh | 5 +++++ ci/run-docker-build.sh | 4 ++++ ci/run-docker.sh | 3 +++ 6 files changed, 59 insertions(+) diff --git a/.travis.yml b/.travis.yml index 069aeeff3c..0cfc3c3428 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,6 +33,14 @@ matrix: - docker before_install: script: ci/run-docker.sh + - env: jobname=linux-musl + os: linux + compiler: + addons: + services: + - docker + before_install: + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 11413f66f8..84ecad76ec 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -514,6 +514,41 @@ jobs: PathtoPublish: t/failed-test-artifacts ArtifactName: failed-test-artifacts +- job: linux_musl + displayName: linux-musl + condition: succeeded() + pool: + vmImage: ubuntu-latest + steps: + - bash: | + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 + + res=0 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1 + + sudo chmod a+r t/out/TEST-*.xml + test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts + + test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 + exit $res + displayName: 'jobname=linux-musl ci/run-docker.sh' + env: + GITFILESHAREPWD: $(gitfileshare.pwd) + - task: PublishTestResults@2 + displayName: 'Publish Test Results **/TEST-*.xml' + inputs: + mergeTestResults: true + testRunTitle: 'musl' + platform: Linux + publishRunAttachments: false + condition: succeededOrFailed() + - task: PublishBuildArtifacts@1 + displayName: 'Publish trash directories of failed tests' + condition: failed() + inputs: + PathtoPublish: t/failed-test-artifacts + ArtifactName: failed-test-artifacts + - job: static_analysis displayName: StaticAnalysis condition: succeeded() diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh index a104c61d29..26a6689766 100755 --- a/ci/install-docker-dependencies.sh +++ b/ci/install-docker-dependencies.sh @@ -11,4 +11,8 @@ Linux32) libssl-dev libexpat-dev gettext python >/dev/null ' ;; +linux-musl) + apk add --update build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null + ;; esac diff --git a/ci/lib.sh b/ci/lib.sh index d637825222..87cd29bab6 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -201,6 +201,11 @@ GIT_TEST_GETTEXT_POISON) Linux32) CC=gcc ;; +linux-musl) + CC=gcc + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes" + MAKEFLAGS="$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes" + ;; esac MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh index 4a153492ba..8d47a5fda3 100755 --- a/ci/run-docker-build.sh +++ b/ci/run-docker-build.sh @@ -18,6 +18,10 @@ case "$jobname" in Linux32) switch_cmd="linux32 --32bit i386" ;; +linux-musl) + switch_cmd= + useradd () { adduser -D "$@"; } + ;; *) exit 1 ;; diff --git a/ci/run-docker.sh b/ci/run-docker.sh index 3881f99b53..37fa372052 100755 --- a/ci/run-docker.sh +++ b/ci/run-docker.sh @@ -9,6 +9,9 @@ case "$jobname" in Linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; +linux-musl) + CI_CONTAINER=alpine + ;; *) exit 1 ;; -- 2.26.0.334.g6536db25bb ``` ## Junio C Hamano, 2020-04-02 17:53 Subject: Re: [PATCH v3 0/6] Travis + Azure jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/xmqqsghl3hb4.fsf%40gitster.c.googlers.com In-Reply-To: ``` Đoàn Trần Công Danh writes: > This is a nearly rewrite of this series, because there're GitHub Action > allow running directly inside container. > > => I rewrite this series to prepare as much as possible for the GitHub > Action series. > ... > .travis.yml | 10 ++++- > azure-pipelines.yml | 39 ++++++++++++++++++- > ci/install-docker-dependencies.sh | 18 +++++++++ > ci/lib.sh | 8 ++++ > ...n-linux32-build.sh => run-docker-build.sh} | 39 +++++++++++++------ > ci/{run-linux32-docker.sh => run-docker.sh} | 28 ++++++++++--- > 6 files changed, 121 insertions(+), 21 deletions(-) > create mode 100755 ci/install-docker-dependencies.sh > rename ci/{run-linux32-build.sh => run-docker-build.sh} (63%) > rename ci/{run-linux32-docker.sh => run-docker.sh} (43%) Thanks. The above diffstat makes me wonder if it makes more sense to do the topic from Dscho first to migrate existing CI targets to GitHub Actions and then add musl job to the ci suite on top? That way, we won't have to worry about azure-pipelines.yml at all here. ``` ## Danh Doan, 2020-04-03 00:23 Subject: Re: [PATCH v3 0/6] Travis + Azure jobs for linux with musl libc Message-ID: <20200403002324.GA29543@danh.dev> URL: https://gitlist.dev/e/20200403002324.GA29543%40danh.dev In-Reply-To: ``` On 2020-04-02 10:53:35-0700, Junio C Hamano wrote: > Đoàn Trần Công Danh writes: > > > This is a nearly rewrite of this series, because there're GitHub Action > > allow running directly inside container. > > > > => I rewrite this series to prepare as much as possible for the GitHub > > Action series. > > ... > > .travis.yml | 10 ++++- > > azure-pipelines.yml | 39 ++++++++++++++++++- > > ci/install-docker-dependencies.sh | 18 +++++++++ > > ci/lib.sh | 8 ++++ > > ...n-linux32-build.sh => run-docker-build.sh} | 39 +++++++++++++------ > > ci/{run-linux32-docker.sh => run-docker.sh} | 28 ++++++++++--- > > 6 files changed, 121 insertions(+), 21 deletions(-) > > create mode 100755 ci/install-docker-dependencies.sh > > rename ci/{run-linux32-build.sh => run-docker-build.sh} (63%) > > rename ci/{run-linux32-docker.sh => run-docker.sh} (43%) > > Thanks. The above diffstat makes me wonder if it makes more sense > to do the topic from Dscho first to migrate existing CI targets to > GitHub Actions and then add musl job to the ci suite on top? That > way, we won't have to worry about azure-pipelines.yml at all here. You can ignore the change to azure-pipelines.yml in 6/6 to reduce noise about Azure (it'll be deleted by next series anyway). And declare that this series is working for Travis only (as same intention of v1). New diffstat: ---------------8<------------------ .travis.yml | 10 +++++- azure-pipelines.yml | 4 +-- ci/install-docker-dependencies.sh | 18 +++++++++++ ci/lib.sh | 8 +++++ ci/{run-linux32-build.sh => run-docker-build.sh} | 39 ++++++++++++++++-------- ci/{run-linux32-docker.sh => run-docker.sh} | 28 +++++++++++++---- 6 files changed, 86 insertions(+), 21 deletions(-) ---------------->8---------------- In _my_ opinion, I still prefer have this series first. But, if we prefer to have GitHub Action first: - We'll need 5/6 moved to that series - In the rebased of this series, we'll change about 10 lines in GitHub Action yml. If people think it's better that way, please tell me, I could re-order it. -- Danh ``` ## SZEDER Gábor, 2020-04-03 08:22 Subject: Re: [PATCH v3 2/6] ci/lib-docker: preserve required environment variables Message-ID: <20200403082254.GJ2224@szeder.dev> URL: https://gitlist.dev/e/20200403082254.GJ2224%40szeder.dev In-Reply-To: ``` On Thu, Apr 02, 2020 at 08:04:01PM +0700, Đoàn Trần Công Danh wrote: > We're using "su -m" to preserve environment variables in the shell run > by "su". But, that options will be ignored while "-l" (aka "--login") is > specified. This is not true. See any previous runs of the 32 bit Linux job, which worked as expected, because none of these environment variables were cleared. > Since we don't have interest in all environment variables, > pass only those necessary variables to the inner script. > > Signed-off-by: Đoàn Trần Công Danh > --- > ci/run-linux32-build.sh | 13 ++++++++++--- > 1 file changed, 10 insertions(+), 3 deletions(-) > > diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh > index e3a193adbc..7f985615c2 100755 > --- a/ci/run-linux32-build.sh > +++ b/ci/run-linux32-build.sh > @@ -51,10 +51,17 @@ else > fi > > # Build and test > -linux32 --32bit i386 su -m -l $CI_USER -c ' > +linux32 --32bit i386 su -m -l $CI_USER -c " > set -ex > + export DEVELOPER='$DEVELOPER' > + export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' > + export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' > + export GIT_TEST_OPTS='$GIT_TEST_OPTS' > + export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' > + export MAKEFLAGS='$MAKEFLAGS' > + export cache_dir='$cache_dir' > cd /usr/src/git > - test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove > + test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove > make > make test > -' > +" > -- > 2.26.0.334.g6536db25bb > ``` ## Danh Doan, 2020-04-03 10:09 Subject: Re: [PATCH v3 2/6] ci/lib-docker: preserve required environment variables Message-ID: <20200403100931.GA32101@danh.dev> URL: https://gitlist.dev/e/20200403100931.GA32101%40danh.dev In-Reply-To: <20200403082254.GJ2224@szeder.dev> ``` On 2020-04-03 10:22:54+0200, SZEDER Gábor wrote: > On Thu, Apr 02, 2020 at 08:04:01PM +0700, Đoàn Trần Công Danh wrote: > > We're using "su -m" to preserve environment variables in the shell run > > by "su". But, that options will be ignored while "-l" (aka "--login") is > > specified. > > This is not true. See any previous runs of the 32 bit Linux job, > which worked as expected, because none of these environment variables > were cleared. Different su have different behavior when combine "-m" and "-l" util-linux's su has this as far as 60541961f, (docs: improve grammar, wording and formatting of su man page, 2013-10-12) -m, -p, --preserve-environment Preserve the entire environment, i.e., it does not set HOME, SHELL, USER nor LOGNAME. This option is ignored if the option --login is specified. https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/login-utils/su.1#n120 Ubuntu (our Linux32 builder), ships su by shadow-utils: Note that the default behavior for the environment is the following: The $HOME, $SHELL, $USER, $LOGNAME, $PATH, and $IFS environment variables are reset. If --login is not used, the environment is copied, except for the variables above. If --login is used, the $TERM, $COLORTERM, $DISPLAY, and $XAUTHORITY environment variables are copied if they were set. There're no mentions of other variables, I _think_ our Linux32 works by accident. Alpine ships su from busybox, this su ignores "-m" if "-l" is set. -- Danh ``` ## SZEDER Gábor, 2020-04-03 19:55 Subject: Re: [PATCH v3 2/6] ci/lib-docker: preserve required environment variables Message-ID: <20200403195551.GB7859@szeder.dev> URL: https://gitlist.dev/e/20200403195551.GB7859%40szeder.dev In-Reply-To: <20200403100931.GA32101@danh.dev> ``` On Fri, Apr 03, 2020 at 05:09:31PM +0700, Danh Doan wrote: > On 2020-04-03 10:22:54+0200, SZEDER Gábor wrote: > > On Thu, Apr 02, 2020 at 08:04:01PM +0700, Đoàn Trần Công Danh wrote: > > > We're using "su -m" to preserve environment variables in the shell run > > > by "su". But, that options will be ignored while "-l" (aka "--login") is > > > specified. > > > > This is not true. See any previous runs of the 32 bit Linux job, > > which worked as expected, because none of these environment variables > > were cleared. > > Different su have different behavior when combine "-m" and "-l" > > util-linux's su has this as far as 60541961f, (docs: improve grammar, > wording and formatting of su man page, 2013-10-12) > > -m, -p, --preserve-environment > Preserve the entire environment, i.e., it does not set HOME, > SHELL, USER nor LOGNAME. This option is ignored if the option > --login is specified. > > https://git.kernel.org/pub/scm/utils/util-linux/util-linux.git/tree/login-utils/su.1#n120 > > Ubuntu (our Linux32 builder), ships su by shadow-utils: > > Note that the default behavior for the environment is the following: > The $HOME, $SHELL, $USER, $LOGNAME, $PATH, > and $IFS environment variables are reset. > > If --login is not used, the environment is copied, > except for the variables above. > > If --login is used, the $TERM, $COLORTERM, $DISPLAY, and > $XAUTHORITY environment variables are copied if they were set. > > There're no mentions of other variables, I _think_ our Linux32 works > by accident. We do know which image we use, and we do know how its 'su' behaves. I think relying on that is fine, and it's not just "works by accident". > Alpine ships su from busybox, this su ignores "-m" if "-l" is set. Then the commit message should specifically mention these behavior differences between different 'su' variants. ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 1/6] ci: make MAKEFLAGS available inside the Docker container in the Linux32 job Message-ID: <2fdce60075935f8ff94bc117747f1768f41ed435.1585961568.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/2fdce60075935f8ff94bc117747f1768f41ed435.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` From: SZEDER Gábor Once upon a time we ran 'make --jobs=2 ...' to build Git, its documentation, or to apply Coccinelle semantic patches. Then commit eaa62291ff (ci: inherit --jobs via MAKEFLAGS in run-build-and-tests, 2019-01-27) came along, and started using the MAKEFLAGS environment variable to centralize setting the number of parallel jobs in 'ci/libs.sh'. Alas, it forgot to update 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container running the 32 bit Linux job, and, consequently, since then that job builds Git sequentially, and it ignores any Makefile knobs that we might set in MAKEFLAGS (though we don't set any for the 32 bit Linux job at the moment). So update the 'docker run' invocation in 'ci/run-linux32-docker.sh' to make MAKEFLAGS available inside the Docker container as well. Set CC=gcc for the 32 bit Linux job, because that's the compiler installed in the 32 bit Linux Docker image that we use (Travis CI nowadays sets CC=clang by default, but clang is not installed in this image). Signed-off-by: SZEDER Gábor Signed-off-by: Đoàn Trần Công Danh --- ci/lib.sh | 3 +++ ci/run-linux32-docker.sh | 1 + 2 files changed, 4 insertions(+) diff --git a/ci/lib.sh b/ci/lib.sh index c3a8cd2104..d637825222 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -198,6 +198,9 @@ osx-clang|osx-gcc) GIT_TEST_GETTEXT_POISON) export GIT_TEST_GETTEXT_POISON=true ;; +Linux32) + CC=gcc + ;; esac MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index 751acfcf8a..ebb18fa747 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -20,6 +20,7 @@ docker run \ --env GIT_PROVE_OPTS \ --env GIT_TEST_OPTS \ --env GIT_TEST_CLONE_2GB \ + --env MAKEFLAGS \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 0/6] Travis jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/cover.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` Cc: Junio C Hamano , SZEDER Gábor , Eric Sunshine , Johannes Schindelin Recently, we've un-broken git for Linux with musl libc, and we have a serie to fix false negative with busybox shell utils. Add a CI job on Travis and Azure to make sure we won't break it again. Change from v3: * reword 2/6: adding su behavior in util-linux and busybox * 6/6: Drop change to azure-pipelines.yml, and declare this series will be used for Travis only. Since Azure will be replaced by GitHub Action in a later series. Hi Junio, The series for GitHub Actions will need to be rebased on this series again. 6/6 in that seriess will have UD conflicts. Please "git rm azure-pipelines.yml" to fix conflicts. SZEDER Gábor (1): ci: make MAKEFLAGS available inside the Docker container in the Linux32 job Đoàn Trần Công Danh (5): ci/lib-docker: preserve required environment variables ci/linux32: parameterise command to switch arch ci: refactor docker runner script ci/linux32: libify install-dependencies step travis: build and test on Linux with musl libc and busybox .travis.yml | 10 ++++- azure-pipelines.yml | 4 +- ci/install-docker-dependencies.sh | 18 +++++++++ ci/lib.sh | 8 ++++ ...n-linux32-build.sh => run-docker-build.sh} | 39 +++++++++++++------ ci/{run-linux32-docker.sh => run-docker.sh} | 28 ++++++++++--- 6 files changed, 86 insertions(+), 21 deletions(-) create mode 100755 ci/install-docker-dependencies.sh rename ci/{run-linux32-build.sh => run-docker-build.sh} (63%) rename ci/{run-linux32-docker.sh => run-docker.sh} (43%) Range-diff against v3: 1: 2fdce60075 = 1: 2fdce60075 ci: make MAKEFLAGS available inside the Docker container in the Linux32 job 2: b7b079f559 ! 2: db574b3ff9 ci/lib-docker: preserve required environment variables @@ Commit message We're using "su -m" to preserve environment variables in the shell run by "su". But, that options will be ignored while "-l" (aka "--login") is - specified. + specified in util-linux and busybox's su. + + In a later patch this script will be reused for checking Git for Linux + with musl libc on Alpine Linux, Alpine Linux uses "su" from busybox. Since we don't have interest in all environment variables, pass only those necessary variables to the inner script. 3: 8c8cf3eb24 = 3: a13715245f ci/linux32: parameterise command to switch arch 4: 22cc7960c3 = 4: b5de868c1e ci: refactor docker runner script 5: 2e0d54f81e = 5: c39451ffe5 ci/linux32: libify install-dependencies step 6: b61ed50cf6 ! 6: 231affae83 travis: build and test on Linux with musl libc and busybox @@ .travis.yml: matrix: os: linux compiler: - ## azure-pipelines.yml ## -@@ azure-pipelines.yml: jobs: - PathtoPublish: t/failed-test-artifacts - ArtifactName: failed-test-artifacts - -+- job: linux_musl -+ displayName: linux-musl -+ condition: succeeded() -+ pool: -+ vmImage: ubuntu-latest -+ steps: -+ - bash: | -+ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 -+ -+ res=0 -+ sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=linux-musl bash -lxc ci/run-docker.sh || res=1 -+ -+ sudo chmod a+r t/out/TEST-*.xml -+ test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts -+ -+ test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 -+ exit $res -+ displayName: 'jobname=linux-musl ci/run-docker.sh' -+ env: -+ GITFILESHAREPWD: $(gitfileshare.pwd) -+ - task: PublishTestResults@2 -+ displayName: 'Publish Test Results **/TEST-*.xml' -+ inputs: -+ mergeTestResults: true -+ testRunTitle: 'musl' -+ platform: Linux -+ publishRunAttachments: false -+ condition: succeededOrFailed() -+ - task: PublishBuildArtifacts@1 -+ displayName: 'Publish trash directories of failed tests' -+ condition: failed() -+ inputs: -+ PathtoPublish: t/failed-test-artifacts -+ ArtifactName: failed-test-artifacts -+ - - job: static_analysis - displayName: StaticAnalysis - condition: succeeded() - ## ci/install-docker-dependencies.sh ## @@ ci/install-docker-dependencies.sh: Linux32) libssl-dev libexpat-dev gettext python >/dev/null -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 2/6] ci/lib-docker: preserve required environment variables Message-ID: URL: https://gitlist.dev/e/db574b3ff9b0a2aff0f3c669e70379bb30d9e8d9.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` We're using "su -m" to preserve environment variables in the shell run by "su". But, that options will be ignored while "-l" (aka "--login") is specified in util-linux and busybox's su. In a later patch this script will be reused for checking Git for Linux with musl libc on Alpine Linux, Alpine Linux uses "su" from busybox. Since we don't have interest in all environment variables, pass only those necessary variables to the inner script. Signed-off-by: Đoàn Trần Công Danh --- ci/run-linux32-build.sh | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index e3a193adbc..7f985615c2 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -51,10 +51,17 @@ else fi # Build and test -linux32 --32bit i386 su -m -l $CI_USER -c ' +linux32 --32bit i386 su -m -l $CI_USER -c " set -ex + export DEVELOPER='$DEVELOPER' + export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' + export GIT_PROVE_OPTS='$GIT_PROVE_OPTS' + export GIT_TEST_OPTS='$GIT_TEST_OPTS' + export GIT_TEST_CLONE_2GB='$GIT_TEST_CLONE_2GB' + export MAKEFLAGS='$MAKEFLAGS' + export cache_dir='$cache_dir' cd /usr/src/git - test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove + test -n '$cache_dir' && ln -s '$cache_dir/.prove' t/.prove make make test -' +" -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 3/6] ci/linux32: parameterise command to switch arch Message-ID: URL: https://gitlist.dev/e/a13715245f4587ea53d2ecbca75b3f25f9c15818.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` In a later patch, the remaining of this command will be re-used for the CI job for linux with musl libc. Allow customisation of the emulator, now. Signed-off-by: Đoàn Trần Công Danh --- ci/run-linux32-build.sh | 13 +++++++++++-- ci/run-linux32-docker.sh | 2 ++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh index 7f985615c2..44bb332f64 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-linux32-build.sh @@ -14,8 +14,17 @@ then exit 1 fi +case "$jobname" in +Linux32) + switch_cmd="linux32 --32bit i386" + ;; +*) + exit 1 + ;; +esac + # Update packages to the latest available versions -linux32 --32bit i386 sh -c ' +command $switch_cmd sh -c ' apt update >/dev/null && apt install -y build-essential libcurl4-openssl-dev libssl-dev \ libexpat-dev gettext python >/dev/null @@ -51,7 +60,7 @@ else fi # Build and test -linux32 --32bit i386 su -m -l $CI_USER -c " +command $switch_cmd su -m -l $CI_USER -c " set -ex export DEVELOPER='$DEVELOPER' export DEFAULT_TEST_TARGET='$DEFAULT_TEST_TARGET' diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh index ebb18fa747..54186b6aa7 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-linux32-docker.sh @@ -9,6 +9,7 @@ docker pull daald/ubuntu32:xenial # Use the following command to debug the docker build locally: # $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial +# root@container:/# export jobname= # root@container:/# /usr/src/git/ci/run-linux32-build.sh container_cache_dir=/tmp/travis-cache @@ -21,6 +22,7 @@ docker run \ --env GIT_TEST_OPTS \ --env GIT_TEST_CLONE_2GB \ --env MAKEFLAGS \ + --env jobname \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 4/6] ci: refactor docker runner script Message-ID: URL: https://gitlist.dev/e/b5de868c1e354153bec78d892e13a6faf03c4f31.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` We will support alpine check in docker later in this series. While we're at it, tell people to run as root in podman, if podman is used as drop-in replacement for docker, because podman will map host-user to container's root, therefore, mapping their permission. Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 2 +- azure-pipelines.yml | 4 ++-- ...n-linux32-build.sh => run-docker-build.sh} | 6 ++--- ci/{run-linux32-docker.sh => run-docker.sh} | 22 ++++++++++++++----- 4 files changed, 22 insertions(+), 12 deletions(-) rename ci/{run-linux32-build.sh => run-docker-build.sh} (93%) rename ci/{run-linux32-docker.sh => run-docker.sh} (53%) diff --git a/.travis.yml b/.travis.yml index fc5730b085..069aeeff3c 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,7 +32,7 @@ matrix: services: - docker before_install: - script: ci/run-linux32-docker.sh + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/azure-pipelines.yml b/azure-pipelines.yml index 675c3a43c9..11413f66f8 100644 --- a/azure-pipelines.yml +++ b/azure-pipelines.yml @@ -489,14 +489,14 @@ jobs: test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare "$GITFILESHAREPWD" "$HOME/test-cache" || exit 1 res=0 - sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" bash -lxc ci/run-linux32-docker.sh || res=1 + sudo AGENT_OS="$AGENT_OS" BUILD_BUILDNUMBER="$BUILD_BUILDNUMBER" BUILD_REPOSITORY_URI="$BUILD_REPOSITORY_URI" BUILD_SOURCEBRANCH="$BUILD_SOURCEBRANCH" BUILD_SOURCEVERSION="$BUILD_SOURCEVERSION" SYSTEM_PHASENAME="$SYSTEM_PHASENAME" SYSTEM_TASKDEFINITIONSURI="$SYSTEM_TASKDEFINITIONSURI" SYSTEM_TEAMPROJECT="$SYSTEM_TEAMPROJECT" CC=$CC MAKEFLAGS="$MAKEFLAGS" jobname=Linux32 bash -lxc ci/run-docker.sh || res=1 sudo chmod a+r t/out/TEST-*.xml test ! -d t/failed-test-artifacts || sudo chmod a+r t/failed-test-artifacts test "$GITFILESHAREPWD" = '$(gitfileshare.pwd)' || sudo umount "$HOME/test-cache" || res=1 exit $res - displayName: 'ci/run-linux32-docker.sh' + displayName: 'jobname=Linux32 ci/run-docker.sh' env: GITFILESHAREPWD: $(gitfileshare.pwd) - task: PublishTestResults@2 diff --git a/ci/run-linux32-build.sh b/ci/run-docker-build.sh similarity index 93% rename from ci/run-linux32-build.sh rename to ci/run-docker-build.sh index 44bb332f64..a05b48c559 100755 --- a/ci/run-linux32-build.sh +++ b/ci/run-docker-build.sh @@ -1,16 +1,16 @@ #!/bin/sh # -# Build and test Git in a 32-bit environment +# Build and test Git inside container # # Usage: -# run-linux32-build.sh +# run-docker-build.sh # set -ex if test $# -ne 1 || test -z "$1" then - echo >&2 "usage: run-linux32-build.sh " + echo >&2 "usage: run-docker-build.sh " exit 1 fi diff --git a/ci/run-linux32-docker.sh b/ci/run-docker.sh similarity index 53% rename from ci/run-linux32-docker.sh rename to ci/run-docker.sh index 54186b6aa7..3881f99b53 100755 --- a/ci/run-linux32-docker.sh +++ b/ci/run-docker.sh @@ -1,16 +1,26 @@ #!/bin/sh # -# Download and run Docker image to build and test 32-bit Git +# Download and run Docker image to build and test Git # . ${0%/*}/lib.sh -docker pull daald/ubuntu32:xenial +case "$jobname" in +Linux32) + CI_CONTAINER="daald/ubuntu32:xenial" + ;; +*) + exit 1 + ;; +esac + +docker pull "$CI_CONTAINER" # Use the following command to debug the docker build locally: -# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/bash daald/ubuntu32:xenial +# must be 0 if podman is used as drop-in replacement for docker +# $ docker run -itv "${PWD}:/usr/src/git" --entrypoint /bin/sh "$CI_CONTAINER" # root@container:/# export jobname= -# root@container:/# /usr/src/git/ci/run-linux32-build.sh +# root@container:/# /usr/src/git/ci/run-docker-build.sh container_cache_dir=/tmp/travis-cache @@ -26,8 +36,8 @@ docker run \ --env cache_dir="$container_cache_dir" \ --volume "${PWD}:/usr/src/git" \ --volume "$cache_dir:$container_cache_dir" \ - daald/ubuntu32:xenial \ - /usr/src/git/ci/run-linux32-build.sh $(id -u $USER) + "$CI_CONTAINER" \ + /usr/src/git/ci/run-docker-build.sh $(id -u $USER) check_unignored_build_artifacts -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 5/6] ci/linux32: libify install-dependencies step Message-ID: URL: https://gitlist.dev/e/c39451ffe56489a7550ab66fcef6ed132159ee6c.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` In a later patch, we will add new Travis Job for linux-musl. Most of other code in this file could be reuse for that job. Move the code to install dependencies to a common script. Should we add new CI system that can run directly in container, we can reuse this script for installation step. Signed-off-by: Đoàn Trần Công Danh --- ci/install-docker-dependencies.sh | 14 ++++++++++++++ ci/run-docker-build.sh | 7 +------ 2 files changed, 15 insertions(+), 6 deletions(-) create mode 100755 ci/install-docker-dependencies.sh diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh new file mode 100755 index 0000000000..a104c61d29 --- /dev/null +++ b/ci/install-docker-dependencies.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# +# Install dependencies required to build and test Git inside container +# + +case "$jobname" in +Linux32) + linux32 --32bit i386 sh -c ' + apt update >/dev/null && + apt install -y build-essential libcurl4-openssl-dev \ + libssl-dev libexpat-dev gettext python >/dev/null + ' + ;; +esac diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh index a05b48c559..4a153492ba 100755 --- a/ci/run-docker-build.sh +++ b/ci/run-docker-build.sh @@ -23,12 +23,7 @@ Linux32) ;; esac -# Update packages to the latest available versions -command $switch_cmd sh -c ' - apt update >/dev/null && - apt install -y build-essential libcurl4-openssl-dev libssl-dev \ - libexpat-dev gettext python >/dev/null -' +"${0%/*}/install-docker-dependencies.sh" # If this script runs inside a docker container, then all commands are # usually executed as root. Consequently, the host user might not be -- 2.26.0.334.g6536db25bb ``` ## Đoàn Trần Công Danh, 2020-04-04 01:08 Subject: [PATCH v4 6/6] travis: build and test on Linux with musl libc and busybox Message-ID: <231affae8376039c68248efa972c37ef92b79d90.1585961568.git.congdanhqx@gmail.com> URL: https://gitlist.dev/e/231affae8376039c68248efa972c37ef92b79d90.1585961568.git.congdanhqx%40gmail.com In-Reply-To: ``` Signed-off-by: Đoàn Trần Công Danh --- .travis.yml | 8 ++++++++ ci/install-docker-dependencies.sh | 4 ++++ ci/lib.sh | 5 +++++ ci/run-docker-build.sh | 4 ++++ ci/run-docker.sh | 3 +++ 5 files changed, 24 insertions(+) diff --git a/.travis.yml b/.travis.yml index 069aeeff3c..0cfc3c3428 100644 --- a/.travis.yml +++ b/.travis.yml @@ -33,6 +33,14 @@ matrix: - docker before_install: script: ci/run-docker.sh + - env: jobname=linux-musl + os: linux + compiler: + addons: + services: + - docker + before_install: + script: ci/run-docker.sh - env: jobname=StaticAnalysis os: linux compiler: diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh index a104c61d29..26a6689766 100755 --- a/ci/install-docker-dependencies.sh +++ b/ci/install-docker-dependencies.sh @@ -11,4 +11,8 @@ Linux32) libssl-dev libexpat-dev gettext python >/dev/null ' ;; +linux-musl) + apk add --update build-base curl-dev openssl-dev expat-dev gettext \ + pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null + ;; esac diff --git a/ci/lib.sh b/ci/lib.sh index d637825222..87cd29bab6 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -201,6 +201,11 @@ GIT_TEST_GETTEXT_POISON) Linux32) CC=gcc ;; +linux-musl) + CC=gcc + MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/python3 USE_LIBPCRE2=Yes" + MAKEFLAGS="$MAKEFLAGS NO_REGEX=Yes ICONV_OMITS_BOM=Yes" + ;; esac MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" diff --git a/ci/run-docker-build.sh b/ci/run-docker-build.sh index 4a153492ba..8d47a5fda3 100755 --- a/ci/run-docker-build.sh +++ b/ci/run-docker-build.sh @@ -18,6 +18,10 @@ case "$jobname" in Linux32) switch_cmd="linux32 --32bit i386" ;; +linux-musl) + switch_cmd= + useradd () { adduser -D "$@"; } + ;; *) exit 1 ;; diff --git a/ci/run-docker.sh b/ci/run-docker.sh index 3881f99b53..37fa372052 100755 --- a/ci/run-docker.sh +++ b/ci/run-docker.sh @@ -9,6 +9,9 @@ case "$jobname" in Linux32) CI_CONTAINER="daald/ubuntu32:xenial" ;; +linux-musl) + CI_CONTAINER=alpine + ;; *) exit 1 ;; -- 2.26.0.334.g6536db25bb ``` ## Junio C Hamano, 2020-04-05 20:39 Subject: Re: [PATCH v4 0/6] Travis jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/xmqq5zedy8dj.fsf%40gitster.c.googlers.com In-Reply-To: ``` Đoàn Trần Công Danh writes: > The series for GitHub Actions will need to be rebased on this series again. > 6/6 in that seriess will have UD conflicts. > Please "git rm azure-pipelines.yml" to fix conflicts. If you want to keep doing this, please take over the ownership of both series and build one on top of the other. I asked you and Dscho to coordinate and work together, but Dscho seems to be comfortable with the idea of letting you touch his series, so doing so would still count as you two working together ;-). I do not have a strong opinion which parts should come first (it is something that can be decided between you two which way is cleaner). Thanks. ``` ## Johannes Schindelin, 2020-04-07 14:53 Subject: Re: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: URL: https://gitlist.dev/e/nycvar.QRO.7.76.6.2004071651060.46%40tvgsbejvaqbjf.bet In-Reply-To: <20200401221835.GH2224@szeder.dev> ``` Hi Gábor, On Thu, 2 Apr 2020, SZEDER Gábor wrote: > On Sun, Mar 29, 2020 at 05:12:31PM +0700, Đoàn Trần Công Danh wrote: > > Signed-off-by: Đoàn Trần Công Danh > > > + set -ex > > + cd /usr/src/git > > + test -n "$cache_dir" && ln -s "$cache_dir/.prove" t/.prove > > + autoconf > > + echo "PYTHON_PATH=/usr/bin/python3" >config.mak > > + ./configure --with-libpcre > > The recommended way to build Git is without autoconf and configure. That is news to me. My understanding still is that `make` is the recommended way to build Git, and `./configure` is only for those who want to use autoconf. It seems that the `INSTALL` file agrees with my understanding: -- snip -- Git installation Normally you can just do "make" followed by "make install", and that will install the git programs in your own ~/bin/ directory. If you want to do a global install, you can do $ make prefix=/usr all doc info ;# as yourself # make prefix=/usr install install-doc install-html install-info # ;# as root (or prefix=/usr/local, of course). Just like any program suite that uses $prefix, the built results have some paths encoded, which are derived from $prefix, so "make all; make prefix=/usr install" would not work. The beginning of the Makefile documents many variables that affect the way git is built. You can override them either from the command line, or in a config.mak file. Alternatively you can use autoconf generated ./configure script to set up install paths (via config.mak.autogen), so you can write instead $ make configure ;# as yourself $ ./configure --prefix=/usr ;# as yourself $ make all doc ;# as yourself # make install install-doc install-html;# as root -- snap -- If you think that I am wrong, I invite you to change the recommendation by proposing a patch to `INSTALL`, to change the current recommendation. Ciao, Dscho ``` ## Johannes Schindelin, 2020-04-07 14:55 Subject: Re: [PATCH v4 0/6] Travis jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/nycvar.QRO.7.76.6.2004071654170.46%40tvgsbejvaqbjf.bet In-Reply-To: ``` Hi Junio, On Sun, 5 Apr 2020, Junio C Hamano wrote: > Đoàn Trần Công Danh writes: > > > The series for GitHub Actions will need to be rebased on this series again. > > 6/6 in that seriess will have UD conflicts. > > Please "git rm azure-pipelines.yml" to fix conflicts. > > If you want to keep doing this, please take over the ownership of > both series and build one on top of the other. I asked you and > Dscho to coordinate and work together, but Dscho seems to be > comfortable with the idea of letting you touch his series, so doing > so would still count as you two working together ;-). I do not have > a strong opinion which parts should come first (it is something that > can be decided between you two which way is cleaner). For the record, Danh and I _are_ working together, and yes, I am totally comfortable with him taking the lead on submitting the patch series' iterations. Thanks, Dscho ``` ## Junio C Hamano, 2020-04-07 19:25 Subject: Re: [PATCH v4 0/6] Travis jobs for linux with musl libc Message-ID: URL: https://gitlist.dev/e/xmqqd08jt7xn.fsf%40gitster.c.googlers.com In-Reply-To: ``` Johannes Schindelin writes: > Hi Junio, > > On Sun, 5 Apr 2020, Junio C Hamano wrote: > >> Đoàn Trần Công Danh writes: >> >> > The series for GitHub Actions will need to be rebased on this series again. >> > 6/6 in that seriess will have UD conflicts. >> > Please "git rm azure-pipelines.yml" to fix conflicts. >> >> If you want to keep doing this, please take over the ownership of >> both series and build one on top of the other. I asked you and >> Dscho to coordinate and work together, but Dscho seems to be >> comfortable with the idea of letting you touch his series, so doing >> so would still count as you two working together ;-). I do not have >> a strong opinion which parts should come first (it is something that >> can be decided between you two which way is cleaner). > > For the record, Danh and I _are_ working together, and yes, I am totally > comfortable with him taking the lead on submitting the patch series' > iterations. Good. Thanks, both. ``` ## Junio C Hamano, 2020-04-07 21:35 Subject: Re: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: URL: https://gitlist.dev/e/xmqqh7xvrncc.fsf%40gitster.c.googlers.com In-Reply-To: ``` Johannes Schindelin writes: >> The recommended way to build Git is without autoconf and configure. > > That is news to me. > > My understanding still is that `make` is the recommended way to build Git, > and `./configure` is only for those who want to use autoconf. > > It seems that the `INSTALL` file agrees with my understanding: Did you misread the sentence you quoted? It says "without", not "with", so I think you two are on the same page. ``` ## Johannes Schindelin, 2020-04-10 13:38 Subject: Re: [PATCH v2 3/4] travis: build and test on Linux with musl libc and busybox Message-ID: URL: https://gitlist.dev/e/nycvar.QRO.7.76.6.2004101538190.46%40tvgsbejvaqbjf.bet In-Reply-To: ``` Hi Junio, On Tue, 7 Apr 2020, Junio C Hamano wrote: > Johannes Schindelin writes: > > >> The recommended way to build Git is without autoconf and configure. > > > > That is news to me. > > > > My understanding still is that `make` is the recommended way to build Git, > > and `./configure` is only for those who want to use autoconf. > > > > It seems that the `INSTALL` file agrees with my understanding: > > Did you misread the sentence you quoted? It says "without", not > "with", so I think you two are on the same page. Yes, I misread it. Thanks for pointing that out. Ciao, Dscho ```