v2travis-ci: build and test Git on Windows

Most Git developers work on Linux and they have no way to know if their changes would break the Git for Windows build. Let's fix that by adding a job to TravisCI that builds and tests Git on Windows. Unfortunately, TravisCI does not support Windows.

Therefore, we did the following:
* Johannes Schindelin set up a Visual Studio Team Services build
  sponsored by Microsoft and made it accessible via an Azure Function
  that speaks a super-simple API. We made TravisCI use this API to
  trigger a build, wait until its completion, and print the build and
  test results.
* A Windows build and test run takes up to 3h and TravisCI has a timeout
  after 50min for Open Source projects. Since the TravisCI job does not
  use heavy CPU/memory/etc. resources, the friendly TravisCI folks
  extended the job timeout for git/git to 3h.

Things, that would need to be done:
* Someone with write access to https://travis-ci.org/git/git would need
  to add the secret token as "GFW_CI_TOKEN" variable in the TravisCI
  repository setting [1]. Afterwards the build should just work.

Things, that might need to be done:
* The Windows box can only process a single build at a time. A second
  Windows build would need to wait until the first finishes. This
  waiting time and the build time after the wait could exceed the 3h
  threshold. If this is a problem, then it is likely to happen every day
  as usually multiple branches are pushed at the same time (pu/next/
  master/maint). I cannot test this as my TravisCI account has the 50min
  timeout. One solution could be to limit the number of concurrent
  TravisCI jobs [2].

[1] https://docs.travis-ci.com/user/environment-variables#Defining-Variables-in-Repository-Settings [2] https://docs.travis-ci.com/user/customizing-the-build#Limiting-Concurrent-Builds

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---

Hi,

I think I addressed all issues from the v1 review (see interdiff below)
with one exception. The script still uses bash instead of sh. Something
about this does not work in sh:
    --output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >cat >&3)

Does anyone know how to make this sh compatible?

Thanks, Lars

Notes:
    Base Ref: master
    Web-Diff: https://github.com/larsxschneider/git/commit/7c527f3571
    Checkout: git fetch https://github.com/larsxschneider/git travisci/win-v2 && git checkout 7c527f3571

    Interdiff (v1..v2):

    diff --git a/.travis.yml b/.travis.yml
    index a7e98ae519..c757a111ce 100644
    --- a/.travis.yml
    +++ b/.travis.yml
    @@ -48,7 +48,7 @@ matrix:
           script:
             - >
               test "$TRAVIS_REPO_SLUG" != "git/git" ||
    -          ci/run-windows-build.sh $GFW_CI_TOKEN $TRAVIS_BRANCH $(git rev-parse HEAD)
    +          ci/run-windows-build.sh $TRAVIS_BRANCH $(git rev-parse HEAD)
           after_failure:
         - env: Linux32
           os: linux
    diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh
    index 324a9ea4e6..234a235c77 100755
    --- a/ci/run-windows-build.sh
    +++ b/ci/run-windows-build.sh
    @@ -1,30 +1,49 @@
     #!/usr/bin/env bash
     #
     # Script to trigger the a Git for Windows build and test run.
    -# Pass a token, the branch (only branches on https://github.com/git/git)
    -# are supported), and a commit hash.
    +# Set the $GFW_CI_TOKEN as environment variable.
    +# Pass the branch (only branches on https://github.com/git/git are
    +# supported) and a commit hash.
     #

    -[ $# -eq 3 ] || (echo "Unexpected number of parameters" && exit 1)
    +test $# -ne 2 && echo "Unexpected number of parameters" && exit 1
    +test -z "$GFW_CI_TOKEN" && echo "GFW_CI_TOKEN not defined" && exit

    -TOKEN=$1
    -BRANCH=$2
    -COMMIT=$3
    +BRANCH=$1
    +COMMIT=$2

     gfwci () {
    -	curl \
    -		-H "Authentication: Bearer $TOKEN" \
    -		--silent --retry 5 \
    -		"https://git-for-windows-ci.azurewebsites.net/api/TestNow?$1" |
    -	sed "$(printf '1s/^\xef\xbb\xbf//')"  # Remove the Byte Order Mark
    +	local CURL_ERROR_CODE HTTP_CODE
    +	exec 3>&1
    +	HTTP_CODE=$(curl \
    +		-H "Authentication: Bearer $GFW_CI_TOKEN" \
    +		--silent --retry 5 --write-out '%{HTTP_CODE}' \
    +		--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >cat >&3) \
    +		"https://git-for-windows-ci.azurewebsites.net/api/TestNow?$1" \
    +	)
    +	CURL_ERROR_CODE=$?
    +	if test $CURL_ERROR_CODE -ne 0
    +	then
    +		return $CURL_ERROR_CODE
    +	fi
    +	if test "$HTTP_CODE" -ge 400 && test "$HTTP_CODE" -lt 600
    +	then
    +		return 127
    +	fi
     }

     # Trigger build job
     BUILD_ID=$(gfwci "action=trigger&branch=$BRANCH&commit=$COMMIT&skipTests=false")
    +if test $? -ne 0
    +then
    +	echo "Unable to trigger Visual Studio Team Services Build"
    +	echo "$BUILD_ID"
    +	exit 1
    +fi

     # Check if the $BUILD_ID contains a number
     case $BUILD_ID in
    -	''|*[!0-9]*) echo $BUILD_ID && exit 1
    +''|*[!0-9]*) echo "Unexpected build number: $BUILD_ID" && exit 1
     esac

     echo "Visual Studio Team Services Build #${BUILD_ID}"
    @@ -36,13 +55,13 @@ while true
     do
     	LAST_STATUS=$STATUS
     	STATUS=$(gfwci "action=status&buildId=$BUILD_ID")
    -	[ "$STATUS" == "$LAST_STATUS" ] || printf "\nStatus: $STATUS "
    +	test "$STATUS" = "$LAST_STATUS" || printf "\nStatus: $STATUS "
     	printf "."

    -	case $STATUS in
    +	case "$STATUS" in
     	inProgress|postponed|notStarted) sleep 10               ;; # continue
     	         "completed: succeeded") RESULT="success"; break;; # success
    -		                              *) echo "Unknown: $STATUS"; break;; # failure
    +	*) echo "Unhandled status: $STATUS";               break;; # failure
     	esac
     done

    @@ -52,4 +71,5 @@ echo ""
     gfwci "action=log&buildId=$BUILD_ID" | cut -c 30-

     # Set exit code for TravisCI
    -[ "$RESULT" == "success" ]
    +test "$RESULT" = "success"
    +

    \0

 .travis.yml             | 11 ++++++++
 ci/run-windows-build.sh | 75 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
 create mode 100755 ci/run-windows-build.sh

On Fri, 24 Mar 2017, Lars Schneider wrote:

Forgive me for bursting in and possibly repeating what you've already discussed. I just read about the limitations for doing windows builds via travis so I thought I'd at least let you know that you can avoid those limitations without too much work:

Two alternative approaches would be:

1. use appveyor.com, as that is a Travis-like service for Windows. We do our
    windows-builds in the curl project using that.

2. run your own buildbot and submit data using the regular github hook and
    have buildbot submit the results back (it has a plugin that can do that).
    We do solaris-builds in the curl project using that method (thanks to
    opencsw.org) and some additional windows-builds thanks to private
    individuals.

You are not bursting in - I am happy to get your feedback! Thanks a lot for CURL BTW :-)

The Git for Windows build and tests are *really* resources intensive and they take a lot of setup time. AFAIK we would run into timeouts with AppVeyor. Maybe Sebastian or Dscho know details?

We could do that! However, the idea was to have the entire build status for all platforms in one place.

- Lars

On Fri, 24 Mar 2017, Lars Schneider wrote:

As the status is (or at least can be) passed back to github, all CI jobs are shown at the same place: below each commit and each pull-request...

On Fri, Mar 24, 2017 at 1:35 PM, Lars Schneider <larsxschneider@gmail.com> wrote:

At lot of setup time, in terms of installing the Git for Windows SDK, could probably be saved by only installing the required packages on top of MSYS2 that's already installed in AppVeyor nodes [1].

[1] https://www.appveyor.com/docs/build-environment/#mingw-msys-cygwin

Hi,

On Fri, 24 Mar 2017, Sebastian Schuberth wrote:

And that is indeed what Git for Windows' Git fork has configured. Sadly, the job timed out too often for me, even if I disabled the tests.

As of recent, things turned more green, with the build taking only about 3 minutes. Of course, re-enabling the tests would let us run smack into the 20 minutes time out, *and* we still would have to disable the tests that fail because MSYS2's maintainer ignored the PRs trying to integrate Git for Windows' patches to the MSYS2 runtime.

Example for a green job: https://ci.appveyor.com/project/dscho/git/build/1.0.676

Ciao, Johannes

On Fri, Mar 24, 2017 at 12:37:47PM +0100, Lars Schneider wrote:

Process substitution is a bash-ism. Just looking at this snippet I would ask why you aren't just using stdout, but from the whole diff it looks like you're using --write-out and want to keep the streams separate.

The POSIX shell way would be to make your own named pipe rather than anonymous one:

  mkfifo data
  sed ... <data &
  curl ... --output data
  rm data

But I don't think relying on bash is that big a deal, as long as the script is clearly marked with the correct #!-line (which it is).

The ">cat" isn't doing AFAICT, though.

-Peff

Lars Schneider <larsxschneider@gmail.com> writes:

Making this runnable under any shell is not a goal; if your script does rely on that bash-ism, then leave it as-is and mark it with #!/bin/bash or somesuch.

Thanks.