Re: [PATCH v3 7/8] ci: install test dependencies for linux-musl

From: Patrick Steinhardt <ps@pks.im>
Date: Oct 30, 2023, 13:22 UTC
Message-ID: <ZT-uGSu-ftFRrn-X@tanuki>
In-Reply-To: <ZT-l4qDDNloizjxl@tanuki>

On Mon, Oct 30, 2023 at 01:47:30PM +0100, Patrick Steinhardt wrote:

Show 67 quoted lines

> On Mon, Oct 30, 2023 at 01:15:10PM +0100, Patrick Steinhardt wrote:
> > The linux-musl CI job executes tests on Alpine Linux, which is based on
> > musl libc instead of glibc. We're missing some test dependencies though,
> > which causes us to skip a subset of tests.
> > 
> > Install these test dependencies to increase our test coverage on this
> > platform. There are still some missing test dependecies, but these do
> > not have a corresponding package in the Alpine repositories:
> > 
> >     - p4 and p4d, both parts of the Perforce version control system.
> > 
> >     - cvsps, which generates patch sets for CVS.
> > 
> >     - Subversion and the SVN::Core Perl library, the latter of which is
> >       not available in the Alpine repositories. While the tool itself is
> >       available, all Subversion-related tests are skipped without the
> >       SVN::Core Perl library anyway.
> > 
> > Furthermore, in order to make the Apache-based tests work, this commit
> > also adds the Alpine-specific modules path of it to the list of known
> > paths.
> > 
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  ci/install-docker-dependencies.sh | 3 ++-
> >  t/lib-httpd.sh                    | 3 ++-
> >  2 files changed, 4 insertions(+), 2 deletions(-)
> > 
> > diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
> > index d0bc19d3bb3..05dde5c5d40 100755
> > --- a/ci/install-docker-dependencies.sh
> > +++ b/ci/install-docker-dependencies.sh
> > @@ -17,7 +17,8 @@ linux32)
> >  	;;
> >  linux-musl)
> >  	apk add --update build-base curl-dev openssl-dev expat-dev gettext \
> > -		pcre2-dev python3 musl-libintl perl-utils ncurses >/dev/null
> > +		pcre2-dev python3 musl-libintl perl-utils ncurses \
> > +		apache2 bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
> 
> Meh. I just noticed that I missed a few other dependencies to make
> Apache2 work:
> 
> diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
> index 5e28adf55b..ce910e3f3c 100755
> --- a/ci/install-docker-dependencies.sh
> +++ b/ci/install-docker-dependencies.sh
> @@ -21,7 +21,8 @@ linux32)
>  linux-musl)
>  	apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
>  		pcre2-dev python3 musl-libintl perl-utils ncurses \
> -		apache2 bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
> +		apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav \
> +		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
>  	;;
>  linux-*)
>  	apt update -q &&
> 
> But once fixed, tests do indeed start to fail:
> 
> t5540-http-push-webdav.sh                        (Wstat: 256 (exited 1) Tests: 20 Failed: 11)
>   Failed tests:  5-11, 13, 15-16, 18
>   Non-zero exit status: 1
> 
> Seems like another thing to fix in a separate patch series.
> 
> Patrick

I've been digging a bit, and the issue comes from the the DAV module indeed:

``` [Mon Oct 30 12:36:19.776149 2023] [dav_fs:crit] [pid 275752] (20019)DSO load failed: AH00576: The DBM driver could not be loaded [Mon Oct 30 12:36:19.776168 2023] [dav:error] [pid 275752] [client 127.0.0.1:51388] Could not LOCK /dumb/test_repo.git/info/refs due to a failed precondition (e.g. other locks). [500, #0] [Mon Oct 30 12:36:19.776174 2023] [dav:error] [pid 275752] [client 127.0.0.1:51388] The locks could not be queried for verification against a possible "If:" header. [500, #0] [Mon Oct 30 12:36:19.776177 2023] [dav:error] [pid 275752] [client 127.0.0.1:51388] Could not open the lock database. [500, #400] [Mon Oct 30 12:36:19.776181 2023] [dav:error] [pid 275752] (20019)DSO load failed: [client 127.0.0.1:51388] Could not open property database. [500, #1] ```

This seems to be a known limitation in Alpine Linux as they do not package apr-util-dbm_db anymore due to license incompatibilities with with Berkely DB [1], and the WebDAV module does rely on it to provide locking.

In the best case we'd be able to detect this limitation and skip those tests automatically so that we can at least execute all the other Apache tests. But again, this rather feels like something we should do as a follow up rather than as part of this series.

Patrick

[1]: https://gitlab.alpinelinux.org/alpine/aports/-/issues/12534

Show 22 quoted lines

> >  	;;
> >  pedantic)
> >  	dnf -yq update >/dev/null &&
> > diff --git a/t/lib-httpd.sh b/t/lib-httpd.sh
> > index 2fb1b2ae561..9791f94b16f 100644
> > --- a/t/lib-httpd.sh
> > +++ b/t/lib-httpd.sh
> > @@ -67,7 +67,8 @@ for DEFAULT_HTTPD_MODULE_PATH in '/usr/libexec/apache2' \
> >  				 '/usr/lib/apache2/modules' \
> >  				 '/usr/lib64/httpd/modules' \
> >  				 '/usr/lib/httpd/modules' \
> > -				 '/usr/libexec/httpd'
> > +				 '/usr/libexec/httpd' \
> > +				 '/usr/lib/apache2'
> >  do
> >  	if test -d "$DEFAULT_HTTPD_MODULE_PATH"
> >  	then
> > -- 
> > 2.42.0
> > 
> 
>