[PATCH v2 0/3] ci: upgrade to the latest Azure Pipelines agent pools

At least the Windows agents we use will go away on March 23rd, 2020 [https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/] , in favor of newer, faster, better ones. We might just as well use the latter already long before the cut-off date.

While at it, let's also update to a newer macOS pool. This necessitates two distinct fixes, though:

 * Our ci/ scripts were ill-prepared to deal with System Integrity
   Protection [https://support.apple.com/en-us/HT204899]. This resulted in
   the p4d -V call seemingly "hanging" (Narrator's voice: it did not "hang",
   it actually opened a GUI dialog to ask the user whether it was okay to
   execute this quarantined executable).
   
   
 * Apparently nobody ever bothered running the Apache2-based tests (t5616,
   t5702 and t5703, probably more) on macOS. I say that because they don't
   work, over-relying on GNU-specific behavior of sed. I rewrote those sed 
   calls to use Perl instead.
   
   

Changes since v1:

 * Avoid blaming BSD for what is (at least nowadays) squarely a macOS issue.
 * Targeting maint

Johannes Schindelin (3):
  t/lib-httpd: avoid using macOS' sed
  ci: prevent `perforce` from being quarantined
  Azure Pipeline: switch to the latest agent pools

 azure-pipelines.yml                | 37 +++++++++++++++++++----------
 ci/install-dependencies.sh         |  4 ++--
 t/lib-httpd.sh                     |  2 +-
 t/lib-httpd/apache.conf            |  6 ++---
 t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++
 t/lib-httpd/apply-one-time-sed.sh  | 24 -------------------
 t/t5537-fetch-shallow.sh           | 10 ++++----
 t/t5616-partial-clone.sh           | 38 +++++++++++++++++-------------
 t/t5702-protocol-v2.sh             | 12 +++++-----
 t/t5703-upload-pack-ref-in-want.sh |  6 ++---
 10 files changed, 93 insertions(+), 73 deletions(-)
 create mode 100644 t/lib-httpd/apply-one-time-perl.sh
 delete mode 100644 t/lib-httpd/apply-one-time-sed.sh

base-commit: c522f061d551c9bb8684a7c3859b2ece4499b56b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-714%2Fdscho%2Fazure-pipelines-latest-pools-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-714/dscho/azure-pipelines-latest-pools-v2
Pull-Request: https://github.com/git/git/pull/714

Range-diff vs v1:

 1:  9900cacbfef ! 1:  af28d115ded t/lib-httpd: avoid using BSD's sed
     @@ -1,8 +1,8 @@
      Author: Johannes Schindelin <johannes.schindelin@gmx.de>
      
     -    t/lib-httpd: avoid using BSD's sed
     +    t/lib-httpd: avoid using macOS' sed
      
     -    Among other differences relative to GNU sed, BSD sed always ends its
     +    Among other differences relative to GNU sed, macOS' sed always ends its
          output with a trailing newline, even if the input did not have such a
          trailing newline.
      
     @@ -14,7 +14,7 @@
          The reason is that we use `sed` in those tests to filter the response of
          the web server. Apart from the fact that we use GNU constructs (such as
          using a space after the `c` command instead of a backslash and a
     -    newline), we have another problem: BSD sed LF-only newlines while
     +    newline), we have another problem: macOS' sed LF-only newlines while
          webservers are supposed to use CR/LF ones.
      
          Even worse, t5616 uses `sed` to replace a binary part of the response
     @@ -25,11 +25,11 @@
          hex-encode it, then calls on `sed` to prefix every hex digit pair with a
          `\x` in order to construct the text that the `c` statement of the `sed`
          invocation is supposed to insert. So we call Perl and sed to construct a
     -    sed statement. The final nail in the coffin is that BSD sed does not
     +    sed statement. The final nail in the coffin is that macOS' sed does not
          even interpret those `\x<hex>` constructs.
      
          Let's just replace all of that by Perl snippets. With Perl, at least, we
     -    do not have to deal with GNU vs BSD semantics, we do not have to worry
     +    do not have to deal with GNU vs macOS semantics, we do not have to worry
          about unwanted trailing newlines, and we do not have to spawn commands
          to construct arguments for other commands to be spawned (i.e. we can
          avoid a whole lot of shell scripting complexity).
     @@ -159,7 +159,7 @@
       	# Craft a situation in which the server sends back an unshallow request
       	# with an empty packfile. This is done by refetching with a shorter
      @@
     - 	printf "$(test_oid sed)" \
     + 	printf "s/0034shallow %s/0036unshallow %s/" \
       	       "$(git -C "$REPO" rev-parse HEAD)" \
       	       "$(git -C "$REPO" rev-parse HEAD^)" \
      -	       >"$HTTPD_ROOT_PATH/one-time-sed" &&
 2:  8739f8bac0f = 2:  5e5fcf3b76b ci: prevent `perforce` from being quarantined
 3:  f141c295fd8 = 3:  6d2f8e7e70e Azure Pipeline: switch to the latest agent pools