git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 08/13] ci: merge scripts which install dependencies

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 11, 2024, 09:10 UTC
Message-ID
<7748f87f8cd23826adbbfbaa7616d8b5be346c42.1712825204.git.ps@pks.im>
In-Reply-To
<cover.1712825204.git.ps@pks.im>

We have two different scripts which install dependencies, one for dockerized jobs and one for non-dockerized ones. Naturally, these scripts have quite some duplication. Furthermore, either of these scripts is missing some test dependencies that the respective other script has, thus reducing test coverage.

Merge those two scripts such that there is a single source of truth for test dependencies, only.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 .github/workflows/main.yml        |  2 +-
 .gitlab-ci.yml                    |  4 +--
 ci/install-dependencies.sh        | 32 ++++++++++++++++++---
 ci/install-docker-dependencies.sh | 46 -------------------------------
 4 files changed, 31 insertions(+), 53 deletions(-)
 delete mode 100755 ci/install-docker-dependencies.sh
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 71cd4e5486..5838986895 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -359,7 +359,7 @@ jobs:
       if: matrix.vector.jobname != 'linux32'
     - uses: actions/checkout@v1 # cannot be upgraded because Node.js Actions aren't supported in this container
       if: matrix.vector.jobname == 'linux32'
-    - run: ci/install-docker-dependencies.sh
+    - run: ci/install-dependencies.sh
     - run: ci/run-build-and-tests.sh
     - name: print test failures
       if: failure() && env.FAILED_TEST_ARTIFACTS != ''
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3a0ef4d4d4..ba65f50aac 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -12,7 +12,7 @@ test:linux:
   variables:
     CUSTOM_PATH: "/custom"
   before_script:
-    - ./ci/install-docker-dependencies.sh
+    - ./ci/install-dependencies.sh
   script:
     - useradd builder --create-home
     - chown -R builder "${CI_PROJECT_DIR}"
@@ -100,7 +100,7 @@ static-analysis:
   variables:
     jobname: StaticAnalysis
   before_script:
-    - ./ci/install-docker-dependencies.sh
+    - ./ci/install-dependencies.sh
   script:
     - ./ci/run-static-analysis.sh
     - ./ci/check-directional-formatting.bash
diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
index bafe37f2d1..e673797115 100755
--- a/ci/install-dependencies.sh
+++ b/ci/install-dependencies.sh
@@ -5,6 +5,8 @@
 
 . ${0%/*}/lib.sh
 
+begin_group "Install dependencies"
+
 P4WHENCE=https://cdist2.perforce.com/perforce/r21.2
 LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION
 
@@ -20,14 +22,27 @@ then
 fi
 
 case "$distro" in
+alpine-*)
+	apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
+		pcre2-dev python3 musl-libintl perl-utils ncurses \
+		apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
+		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
+	;;
+fedora-*)
+	dnf -yq update >/dev/null &&
+	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
+	;;
 ubuntu-*)
+	# Required so that apt doesn't wait for user input on certain packages.
+	export DEBIAN_FRONTEND=noninteractive
+
 	sudo apt-get -q update
 	sudo apt-get -q -y install \
-		language-pack-is libsvn-perl apache2 \
-		make libssl-dev libcurl4-openssl-dev libexpat-dev \
+		language-pack-is libsvn-perl apache2 cvs cvsps git gnupg subversion \
+		make libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo \
 		tcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \
-		libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl \
-		$CC_PACKAGE $PYTHON_PACKAGE
+		libemail-valid-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \
+		${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE
 
 	mkdir --parents "$CUSTOM_PATH"
 	wget --quiet --directory-prefix="$CUSTOM_PATH" \
@@ -39,6 +54,13 @@ ubuntu-*)
 		-C "$CUSTOM_PATH" --strip-components=1 "git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs"
 	rm "git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
 	;;
+ubuntu32-*)
+	sudo linux32 --32bit i386 sh -c '
+		apt update >/dev/null &&
+		apt install -y build-essential libcurl4-openssl-dev \
+			libssl-dev libexpat-dev gettext python >/dev/null
+	'
+	;;
 macos-*)
 	export HOMEBREW_NO_AUTO_UPDATE=1 HOMEBREW_NO_INSTALL_CLEANUP=1
 	# Uncomment this if you want to run perf tests:
@@ -98,3 +120,5 @@ then
 else
 	echo >&2 "WARNING: git-lfs wasn't installed, see above for clues why"
 fi
+
+end_group "Install dependencies"
diff --git a/ci/install-docker-dependencies.sh b/ci/install-docker-dependencies.sh
deleted file mode 100755
index eb2c9e1eca..0000000000
--- a/ci/install-docker-dependencies.sh
+++ /dev/null
@@ -1,46 +0,0 @@
-#!/bin/sh
-#
-# Install dependencies required to build and test Git inside container
-#
-
-. ${0%/*}/lib.sh
-
-begin_group "Install dependencies"
-
-case "$jobname" in
-linux32)
-	linux32 --32bit i386 sh -c '
-		apt update >/dev/null &&
-		apt install -y build-essential libcurl4-openssl-dev \
-			libssl-dev libexpat-dev gettext python >/dev/null
-	'
-	;;
-linux-musl)
-	apk add --update shadow sudo build-base curl-dev openssl-dev expat-dev gettext \
-		pcre2-dev python3 musl-libintl perl-utils ncurses \
-		apache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \
-		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
-	;;
-linux-*|StaticAnalysis)
-	# Required so that apt doesn't wait for user input on certain packages.
-	export DEBIAN_FRONTEND=noninteractive
-
-	apt update -q &&
-	apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \
-		libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \
-		perl-modules liberror-perl libauthen-sasl-perl libemail-valid-perl \
-		libdbd-sqlite3-perl libio-socket-ssl-perl libnet-smtp-ssl-perl ${CC_PACKAGE:-${CC:-gcc}} \
-		apache2 cvs cvsps gnupg libcgi-pm-perl subversion
-
-	if test "$jobname" = StaticAnalysis
-	then
-		apt install -q -y coccinelle
-	fi
-	;;
-pedantic)
-	dnf -yq update >/dev/null &&
-	dnf -yq install make gcc findutils diffutils perl python3 gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null
-	;;
-esac
-
-end_group "Install dependencies"
-- 
2.44.GIT
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 80 of 106 in “t: exercise Git/JGit reftable compatibility”
  1. 00/12 t: exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 4, 2024
  2. 01/12 ci: rename "runs_on_pool" to "distro"Patrick Steinhardt, Apr 4, 2024
  3. 02/12 ci: expose distro name in dockerized GitHub jobsPatrick Steinhardt, Apr 4, 2024
  4. 03/12 ci: allow skipping sudo on dockerized jobsPatrick Steinhardt, Apr 4, 2024
  5. 04/12 ci: drop duplicate package installation for "linux-gcc-default"Patrick Steinhardt, Apr 4, 2024
  6. 05/12 ci: convert "install-dependencies.sh" to use "/bin/sh"Patrick Steinhardt, Apr 4, 2024
  7. 06/12 ci: merge custom PATH directoriesPatrick Steinhardt, Apr 4, 2024
  8. 07/12 ci: merge scripts which install dependenciesPatrick Steinhardt, Apr 4, 2024
  9. 08/12 ci: make Perforce binaries executable for all usersPatrick Steinhardt, Apr 4, 2024
  10. Josh SteadmonApr 5, 2024
  11. Patrick SteinhardtApr 8, 2024
  12. 09/12 ci: install JGit dependencyPatrick Steinhardt, Apr 4, 2024
  13. 10/12 t06xx: always execute backend-specific testsPatrick Steinhardt, Apr 4, 2024
  14. 11/12 t0610: fix non-portable variable assignmentPatrick Steinhardt, Apr 4, 2024
  15. Eric SunshineApr 5, 2024
  16. Patrick SteinhardtApr 5, 2024
  17. Junio C HamanoApr 5, 2024
  18. CodingGuidelines: quote assigned value with "local" and "export"Junio C Hamano, Apr 5, 2024
  19. Junio C HamanoApr 5, 2024
  20. Jeff KingApr 5, 2024
  21. Junio C HamanoApr 5, 2024
  22. Junio C HamanoApr 5, 2024
  23. Jeff KingApr 7, 2024
  24. Patrick SteinhardtApr 5, 2024
  25. 12/12 t0612: add tests to exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 4, 2024
  26. Han-Wen NienhuysApr 4, 2024
  27. Patrick SteinhardtApr 5, 2024
  28. Han-Wen NienhuysApr 6, 2024
  29. Patrick SteinhardtApr 6, 2024
  30. Patrick SteinhardtApr 8, 2024
  31. Patrick SteinhardtApr 4, 2024
  32. 00/12 t: exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 8, 2024
  33. 02/12 ci: expose distro name in dockerized GitHub jobsPatrick Steinhardt, Apr 8, 2024
  34. 03/12 ci: allow skipping sudo on dockerized jobsPatrick Steinhardt, Apr 8, 2024
  35. Toon claesApr 10, 2024
  36. Junio C HamanoApr 10, 2024
  37. Patrick SteinhardtApr 11, 2024
  38. 04/12 ci: drop duplicate package installation for "linux-gcc-default"Patrick Steinhardt, Apr 8, 2024
  39. 05/12 ci: convert "install-dependencies.sh" to use "/bin/sh"Patrick Steinhardt, Apr 8, 2024
  40. Justin ToblerApr 10, 2024
  41. Patrick SteinhardtApr 11, 2024
  42. 06/12 ci: merge custom PATH directoriesPatrick Steinhardt, Apr 8, 2024
  43. 07/12 ci: merge scripts which install dependenciesPatrick Steinhardt, Apr 8, 2024
  44. 08/12 ci: make Perforce binaries executable for all usersPatrick Steinhardt, Apr 8, 2024
  45. 09/12 ci: install JGit dependencyPatrick Steinhardt, Apr 8, 2024
  46. 10/12 t06xx: always execute backend-specific testsPatrick Steinhardt, Apr 8, 2024
  47. 11/12 t0610: fix non-portable variable assignmentPatrick Steinhardt, Apr 8, 2024
  48. Eric SunshineApr 8, 2024
  49. Patrick SteinhardtApr 8, 2024
  50. 12/12 t0612: add tests to exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 8, 2024
  51. Eric SunshineApr 8, 2024
  52. Junio C HamanoApr 8, 2024
  53. Eric SunshineApr 8, 2024
  54. Eric SunshineApr 8, 2024
  55. Patrick SteinhardtApr 8, 2024
  56. Jeff KingApr 8, 2024
  57. Junio C HamanoApr 8, 2024
  58. Junio C HamanoApr 8, 2024
  59. Justin ToblerApr 10, 2024
  60. Patrick SteinhardtApr 11, 2024
  61. 01/12 ci: rename "runs_on_pool" to "distro"Patrick Steinhardt, Apr 8, 2024
  62. Junio C HamanoApr 9, 2024
  63. Patrick SteinhardtApr 9, 2024
  64. Junio C HamanoApr 9, 2024
  65. Patrick SteinhardtApr 9, 2024
  66. 0/2 t0610: fix umask testsPatrick Steinhardt, Apr 9, 2024
  67. 1/2 t0610: make `--shared=` tests reusablePatrick Steinhardt, Apr 9, 2024
  68. 2/2 t0610: execute git-pack-refs(1) with specified umaskPatrick Steinhardt, Apr 9, 2024
  69. Justin ToblerApr 10, 2024
  70. 00/13 t: exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 11, 2024
  71. 01/13 ci: rename "runs_on_pool" to "distro"Patrick Steinhardt, Apr 11, 2024
  72. 02/13 ci: expose distro name in dockerized GitHub jobsPatrick Steinhardt, Apr 11, 2024
  73. 03/13 ci: skip sudo when we are already rootPatrick Steinhardt, Apr 11, 2024
  74. 04/13 ci: drop duplicate package installation for "linux-gcc-default"Patrick Steinhardt, Apr 11, 2024
  75. 05/13 ci: convert "install-dependencies.sh" to use "/bin/sh"Patrick Steinhardt, Apr 11, 2024
  76. 06/13 ci: merge custom PATH directoriesPatrick Steinhardt, Apr 11, 2024
  77. 07/13 ci: fix setup of custom path for GitLab CIPatrick Steinhardt, Apr 11, 2024
  78. Eric SunshineApr 11, 2024
  79. Patrick SteinhardtApr 11, 2024
  80. 08/13 ci: merge scripts which install dependenciesPatrick Steinhardt, Apr 11, 2024
  81. 09/13 ci: make Perforce binaries executable for all usersPatrick Steinhardt, Apr 11, 2024
  82. 10/13 ci: install JGit dependencyPatrick Steinhardt, Apr 11, 2024
  83. 11/13 t06xx: always execute backend-specific testsPatrick Steinhardt, Apr 11, 2024
  84. 12/13 t0610: fix non-portable variable assignmentPatrick Steinhardt, Apr 11, 2024
  85. 13/13 t0612: add tests to exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 11, 2024
  86. 00/13 t: exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 12, 2024
  87. 01/13 ci: rename "runs_on_pool" to "distro"Patrick Steinhardt, Apr 12, 2024
  88. 02/13 ci: expose distro name in dockerized GitHub jobsPatrick Steinhardt, Apr 12, 2024
  89. 03/13 ci: skip sudo when we are already rootPatrick Steinhardt, Apr 12, 2024
  90. 04/13 ci: drop duplicate package installation for "linux-gcc-default"Patrick Steinhardt, Apr 12, 2024
  91. 05/13 ci: convert "install-dependencies.sh" to use "/bin/sh"Patrick Steinhardt, Apr 12, 2024
  92. 06/13 ci: merge custom PATH directoriesPatrick Steinhardt, Apr 12, 2024
  93. 07/13 ci: fix setup of custom path for GitLab CIPatrick Steinhardt, Apr 12, 2024
  94. 08/13 ci: merge scripts which install dependenciesPatrick Steinhardt, Apr 12, 2024
  95. 09/13 ci: make Perforce binaries executable for all usersPatrick Steinhardt, Apr 12, 2024
  96. 10/13 ci: install JGit dependencyPatrick Steinhardt, Apr 12, 2024
  97. 11/13 t06xx: always execute backend-specific testsPatrick Steinhardt, Apr 12, 2024
  98. 12/13 t0610: fix non-portable variable assignmentPatrick Steinhardt, Apr 12, 2024
  99. 13/13 t0612: add tests to exercise Git/JGit reftable compatibilityPatrick Steinhardt, Apr 12, 2024
  100. Justin ToblerMay 3, 2024
  101. Patrick SteinhardtMay 3, 2024
  102. Patrick SteinhardtMay 3, 2024
  103. Justin ToblerMay 3, 2024
  104. Patrick SteinhardtMay 3, 2024
  105. Justin ToblerMay 4, 2024
  106. Patrick SteinhardtMay 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.