git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] ci: skip unavailable external software

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 25, 2025, 04:19 UTC
Message-ID
<aAsNUYUKJZbrMCf2@pks.im>
In-Reply-To
<xmqqmsc5rw94.fsf@gitster.g>
On Thu, Apr 24, 2025 at 04:10:47PM -0700, Junio C Hamano wrote:
Show 22 quoted lines
> The ci/install-dependencies.sh script used in a very early phase of
> our CI jobs downloads Perforce, Git-LFS, and JGit, used for running
> the test scripts.  The test framework is prepared to properly skip
> the tests that depend on these external software, but the CI script
> is unnecessarily strict (due to its use of "set -e" in ci/lib.sh)
> and fails the entire CI run before even starting to test the rest of
> the system.
> 
> Notice a failure to download to any of these external software, but
> keep going.  We need to be careful about cleaning after a failed
> wget, as a later part of the script that does:
> 
>         if type jgit >/dev/null 2>&1
>         then
>                 echo "$(tput setaf 6)JGit Version$(tput sgr0)"
>                 jgit version
>         else
>                 echo >&2 "WARNING: JGit wasn't installed, see above for clues why"
>         fi
> 
> will (surprise!) succeed running "type jgit", and then fail with
> "jgit version", taking the whole thing down due to "set -e".

Yeah, I think this is a sensible direction to go. It is unfortunate that this may lead to silent breakage of these dependencies unless somebody explicitly looks for those warnings. But that feels like the lesser evil compared to failing the whole pipeline.

Show 24 quoted lines
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  ci/install-dependencies.sh | 31 ++++++++++++++++++++++---------
>  1 file changed, 22 insertions(+), 9 deletions(-)
> 
> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh
> index 0df74610d0..e51304c3b0 100755
> --- a/ci/install-dependencies.sh
> +++ b/ci/install-dependencies.sh
> @@ -66,16 +66,29 @@ ubuntu-*|i386/ubuntu-*|debian-*)
>  		mkdir --parents "$CUSTOM_PATH"
>  
>  		wget --quiet --directory-prefix="$CUSTOM_PATH" \
> -			"$P4WHENCE/bin.linux26x86_64/p4d" "$P4WHENCE/bin.linux26x86_64/p4"
> -		chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4"
> -
> -		wget --quiet "$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz"
> +			"$P4WHENCE/bin.linux26x86_64/p4d" \
> +			"$P4WHENCE/bin.linux26x86_64/p4" &&
> +		chmod a+x "$CUSTOM_PATH/p4d" "$CUSTOM_PATH/p4" || {
> +			rm -f "$CUSTOM_PATH/p4"
> +			rm -f "$CUSTOM_PATH/p4d"
> +			echo >&2 "P4 download (optional) failed"
> +		}

I think it would be preferable to only handle failure of wget as chmod shouldn't ever fail if wget was successful. The same is true for the other downloads -- let's be as strict as possible but allow failure of those actions that depend on the network.

Patrick
Previous: Junio C HamanoNext: Junio C Hamano
Message 3 of 16 in “repo.eclipse.org outage breaking all our linux CI jobs”
  1. Junio C HamanoApr 24, 2025
  2. ci: skip unavailable external softwareJunio C Hamano, Apr 24, 2025
  3. Patrick SteinhardtApr 25, 2025
  4. Junio C HamanoApr 25, 2025
  5. Patrick SteinhardtApr 25, 2025
  6. Junio C HamanoApr 25, 2025
  7. Johannes SchindelinApr 25, 2025
  8. Junio C HamanoApr 25, 2025
  9. 0/2 ci: update unavailable external software handlingJunio C Hamano, Apr 25, 2025
  10. 1/2 ci: update the message for unavailble third-party softwareJunio C Hamano, Apr 25, 2025
  11. 2/2 ci: download JGit from maven, not eclipse.orgJunio C Hamano, Apr 25, 2025
  12. shejialuoApr 25, 2025
  13. Junio C HamanoApr 25, 2025
  14. shejialuoApr 26, 2025
  15. Patrick SteinhardtApr 28, 2025
  16. shejialuoApr 28, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.