{"thread":{"id":"52893","subject":"[PATCH 0/3] ci: upgrade to the latest Azure Pipelines agent pools","startedAt":"2020-02-26T20:09:40Z","lastAt":"2020-02-28T23:57:19Z","messageCount":16,"participants":["Johannes Schindelin via GitGitGadget","Ed Maste","Junio C Hamano","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"392561","messageId":"pull.714.git.git.1582747775.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":null,"subject":"[PATCH 0/3] ci: upgrade to the latest Azure Pipelines agent pools","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-26T20:09:32Z","receivedAt":"2020-02-26T20:09:40Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"At least the Windows agents we use will go away on March 23rd, 2020\n[https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/]\n, in favor of newer, faster, better ones. We might just as well use the\nlatter already long before the cut-off date.\n\nWhile at it, let's also update to a newer macOS pool. This necessitates two\ndistinct fixes, though:\n\n * Our ci/ scripts were ill-prepared to deal with System Integrity\n   Protection [https://support.apple.com/en-us/HT204899]. This resulted in\n   the p4d -V call seemingly \"hanging\" (Narrator's voice: it did not \"hang\",\n   it actually opened a GUI dialog to ask the user whether it was okay to\n   execute this quarantined executable).\n   \n   \n * Apparently nobody ever bothered running the Apache2-based tests (t5616,\n   t5702 and t5703, probably more) on macOS. I say that because they don't\n   work, over-relying on GNU-specific behavior of sed. I rewrote those sed \n   calls to use Perl instead.\n\nJohannes Schindelin (3):\n  t/lib-httpd: avoid using BSD's sed\n  ci: prevent `perforce` from being quarantined\n  Azure Pipeline: switch to the latest agent pools\n\n azure-pipelines.yml                | 37 +++++++++++++++++++----------\n ci/install-dependencies.sh         |  4 ++--\n t/lib-httpd.sh                     |  2 +-\n t/lib-httpd/apache.conf            |  6 ++---\n t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++\n t/lib-httpd/apply-one-time-sed.sh  | 24 -------------------\n t/t5537-fetch-shallow.sh           | 10 ++++----\n t/t5616-partial-clone.sh           | 38 +++++++++++++++++-------------\n t/t5702-protocol-v2.sh             | 12 +++++-----\n t/t5703-upload-pack-ref-in-want.sh |  6 ++---\n 10 files changed, 93 insertions(+), 73 deletions(-)\n create mode 100644 t/lib-httpd/apply-one-time-perl.sh\n delete mode 100644 t/lib-httpd/apply-one-time-sed.sh\n\n\nbase-commit: 2d2118b814c11f509e1aa76cb07110f7231668dc\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-714%2Fdscho%2Fazure-pipelines-latest-pools-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-714/dscho/azure-pipelines-latest-pools-v1\nPull-Request: https://github.com/git/git/pull/714\n-- \ngitgitgadget\n"},{"id":"392562","messageId":"f141c295fd84e619ab84bdaf440a0bff966b2861.1582747775.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.git.git.1582747775.gitgitgadget@gmail.com","subject":"[PATCH 3/3] Azure Pipeline: switch to the latest agent pools","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-26T20:09:35Z","receivedAt":"2020-02-26T20:09:42Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nIt would seem that at least the `vs2015-win2012r2` pool (which we use\nvia its old name, `Hosted`) is about to be phased out. Let's switch\nbefore that.\n\nWhile at it, use the newer pool names as suggested at\nhttps://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n azure-pipelines.yml | 37 +++++++++++++++++++++++++------------\n 1 file changed, 25 insertions(+), 12 deletions(-)\n\ndiff --git a/azure-pipelines.yml b/azure-pipelines.yml\nindex af2a5ea4845..675c3a43c9c 100644\n--- a/azure-pipelines.yml\n+++ b/azure-pipelines.yml\n@@ -5,7 +5,8 @@ jobs:\n - job: windows_build\n   displayName: Windows Build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   steps:\n   - powershell: |\n@@ -61,7 +62,8 @@ jobs:\n   displayName: Windows Test\n   dependsOn: windows_build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   strategy:\n     parallel: 10\n@@ -133,7 +135,8 @@ jobs:\n - job: vs_build\n   displayName: Visual Studio Build\n   condition: succeeded()\n-  pool: Hosted VS2017\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   steps:\n   - powershell: |\n@@ -181,6 +184,7 @@ jobs:\n       platform: x64\n       configuration: Release\n       maximumCpuCount: 4\n+      msbuildArguments: /p:PlatformToolset=v142\n   - powershell: |\n       & compat\\vcbuild\\vcpkg_copy_dlls.bat release\n       if (!$?) { exit(1) }\n@@ -224,7 +228,8 @@ jobs:\n   displayName: Visual Studio Test\n   dependsOn: vs_build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   strategy:\n     parallel: 10\n@@ -292,7 +297,8 @@ jobs:\n - job: linux_clang\n   displayName: linux-clang\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -330,7 +336,8 @@ jobs:\n - job: linux_gcc\n   displayName: linux-gcc\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -367,7 +374,8 @@ jobs:\n - job: osx_clang\n   displayName: osx-clang\n   condition: succeeded()\n-  pool: Hosted macOS\n+  pool:\n+    vmImage: macOS-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -402,7 +410,8 @@ jobs:\n - job: osx_gcc\n   displayName: osx-gcc\n   condition: succeeded()\n-  pool: Hosted macOS\n+  pool:\n+    vmImage: macOS-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -435,7 +444,8 @@ jobs:\n - job: gettext_poison\n   displayName: GETTEXT_POISON\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -472,7 +482,8 @@ jobs:\n - job: linux32\n   displayName: Linux32\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -506,7 +517,8 @@ jobs:\n - job: static_analysis\n   displayName: StaticAnalysis\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -526,7 +538,8 @@ jobs:\n - job: documentation\n   displayName: Documentation\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n-- \ngitgitgadget\n"},{"id":"392563","messageId":"9900cacbfefb46610114702e6d48d8020f3f2866.1582747775.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.git.git.1582747775.gitgitgadget@gmail.com","subject":"[PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-26T20:09:33Z","receivedAt":"2020-02-26T20:09:43Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nAmong other differences relative to GNU sed, BSD sed always ends its\noutput with a trailing newline, even if the input did not have such a\ntrailing newline.\n\nSurprisingly, this makes three httpd-based tests fail on macOS: t5616,\nt5702 and t5703. (\"Surprisingly\" because those tests have been around\nfor some time, but apparently nobody runs them on macOS with a working\nApache2 setup.)\n\nThe reason is that we use `sed` in those tests to filter the response of\nthe web server. Apart from the fact that we use GNU constructs (such as\nusing a space after the `c` command instead of a backslash and a\nnewline), we have another problem: BSD sed LF-only newlines while\nwebservers are supposed to use CR/LF ones.\n\nEven worse, t5616 uses `sed` to replace a binary part of the response\nwith a new binary part (kind of hoping that the replaced binary part\ndoes not contain a 0x0a byte which would be interpreted as a newline).\n\nTo that end, it calls on Perl to read the binary pack file and\nhex-encode it, then calls on `sed` to prefix every hex digit pair with a\n`\\x` in order to construct the text that the `c` statement of the `sed`\ninvocation is supposed to insert. So we call Perl and sed to construct a\nsed statement. The final nail in the coffin is that BSD sed does not\neven interpret those `\\x<hex>` constructs.\n\nLet's just replace all of that by Perl snippets. With Perl, at least, we\ndo not have to deal with GNU vs BSD semantics, we do not have to worry\nabout unwanted trailing newlines, and we do not have to spawn commands\nto construct arguments for other commands to be spawned (i.e. we can\navoid a whole lot of shell scripting complexity).\n\nThe upshot is that this fixes t5616, t5702 and t5703 on macOS with\nApache2.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n t/lib-httpd.sh                     |  2 +-\n t/lib-httpd/apache.conf            |  6 ++---\n t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++\n t/lib-httpd/apply-one-time-sed.sh  | 24 -------------------\n t/t5537-fetch-shallow.sh           | 10 ++++----\n t/t5616-partial-clone.sh           | 38 +++++++++++++++++-------------\n t/t5702-protocol-v2.sh             | 12 +++++-----\n t/t5703-upload-pack-ref-in-want.sh |  6 ++---\n 8 files changed, 66 insertions(+), 59 deletions(-)\n create mode 100644 t/lib-httpd/apply-one-time-perl.sh\n delete mode 100644 t/lib-httpd/apply-one-time-sed.sh\n\ndiff --git a/t/lib-httpd.sh b/t/lib-httpd.sh\nindex 656997b4d66..1449ee95e9e 100644\n--- a/t/lib-httpd.sh\n+++ b/t/lib-httpd.sh\n@@ -132,7 +132,7 @@ prepare_httpd() {\n \tinstall_script broken-smart-http.sh\n \tinstall_script error-smart-http.sh\n \tinstall_script error.sh\n-\tinstall_script apply-one-time-sed.sh\n+\tinstall_script apply-one-time-perl.sh\n \n \tln -s \"$LIB_HTTPD_MODULE_PATH\" \"$HTTPD_ROOT_PATH/modules\"\n \ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 5c1c86c193a..994e5290d63 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -113,7 +113,7 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n </LocationMatch>\n-<LocationMatch /one_time_sed/>\n+<LocationMatch /one_time_perl/>\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n </LocationMatch>\n@@ -122,7 +122,7 @@ ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error_smart/ error-smart-http.sh/\n ScriptAlias /error/ error.sh/\n-ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1\n+ScriptAliasMatch /one_time_perl/(.*) apply-one-time-perl.sh/$1\n <Directory ${GIT_EXEC_PATH}>\n \tOptions FollowSymlinks\n </Directory>\n@@ -135,7 +135,7 @@ ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1\n <Files error.sh>\n   Options ExecCGI\n </Files>\n-<Files apply-one-time-sed.sh>\n+<Files apply-one-time-perl.sh>\n \tOptions ExecCGI\n </Files>\n <Files ${GIT_EXEC_PATH}/git-http-backend>\ndiff --git a/t/lib-httpd/apply-one-time-perl.sh b/t/lib-httpd/apply-one-time-perl.sh\nnew file mode 100644\nindex 00000000000..09a0abdff7c\n--- /dev/null\n+++ b/t/lib-httpd/apply-one-time-perl.sh\n@@ -0,0 +1,27 @@\n+#!/bin/sh\n+\n+# If \"one-time-perl\" exists in $HTTPD_ROOT_PATH, run perl on the HTTP response,\n+# using the contents of \"one-time-perl\" as the perl command to be run. If the\n+# response was modified as a result, delete \"one-time-perl\" so that subsequent\n+# HTTP responses are no longer modified.\n+#\n+# This can be used to simulate the effects of the repository changing in\n+# between HTTP request-response pairs.\n+if test -f one-time-perl\n+then\n+\tLC_ALL=C\n+\texport LC_ALL\n+\n+\t\"$GIT_EXEC_PATH/git-http-backend\" >out\n+\tperl -pe \"$(cat one-time-perl)\" out >out_modified\n+\n+\tif cmp -s out out_modified\n+\tthen\n+\t\tcat out\n+\telse\n+\t\tcat out_modified\n+\t\trm one-time-perl\n+\tfi\n+else\n+\t\"$GIT_EXEC_PATH/git-http-backend\"\n+fi\ndiff --git a/t/lib-httpd/apply-one-time-sed.sh b/t/lib-httpd/apply-one-time-sed.sh\ndeleted file mode 100644\nindex bf7689d0202..00000000000\n--- a/t/lib-httpd/apply-one-time-sed.sh\n+++ /dev/null\n@@ -1,24 +0,0 @@\n-#!/bin/sh\n-\n-# If \"one-time-sed\" exists in $HTTPD_ROOT_PATH, run sed on the HTTP response,\n-# using the contents of \"one-time-sed\" as the sed command to be run. If the\n-# response was modified as a result, delete \"one-time-sed\" so that subsequent\n-# HTTP responses are no longer modified.\n-#\n-# This can be used to simulate the effects of the repository changing in\n-# between HTTP request-response pairs.\n-if test -f one-time-sed\n-then\n-\t\"$GIT_EXEC_PATH/git-http-backend\" >out\n-\tsed \"$(cat one-time-sed)\" out >out_modified\n-\n-\tif cmp -s out out_modified\n-\tthen\n-\t\tcat out\n-\telse\n-\t\tcat out_modified\n-\t\trm one-time-sed\n-\tfi\n-else\n-\t\"$GIT_EXEC_PATH/git-http-backend\"\n-fi\ndiff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\nindex 9e16512fe31..4f10057e9f1 100755\n--- a/t/t5537-fetch-shallow.sh\n+++ b/t/t5537-fetch-shallow.sh\n@@ -237,7 +237,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n \tgit -C \"$REPO\" config protocol.version 2 &&\n \tgit -C client config protocol.version 2 &&\n \n-\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_sed/repo\" master:a_branch &&\n+\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_perl/repo\" master:a_branch &&\n \n \t# Craft a situation in which the server sends back an unshallow request\n \t# with an empty packfile. This is done by refetching with a shorter\n@@ -246,13 +246,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n \tprintf \"$(test_oid sed)\" \\\n \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n-\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\t       >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \ttest_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \\\n-\t\tfetch --depth=1 \"$HTTPD_URL/one_time_sed/repo\" \\\n+\t\tfetch --depth=1 \"$HTTPD_URL/one_time_perl/repo\" \\\n \t\tmaster:a_branch &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \t# Ensure that the resulting repo is consistent, despite our failure to\n \t# fetch.\ndiff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh\nindex 9a9178fd281..0eb5b1c47b1 100755\n--- a/t/t5616-partial-clone.sh\n+++ b/t/t5616-partial-clone.sh\n@@ -398,14 +398,18 @@ intersperse () {\n \tsed 's/\\(..\\)/'$1'\\1/g'\n }\n \n-# Create a one-time-sed command to replace the existing packfile with $1.\n+# Create a one-time-perl command to replace the existing packfile with $1.\n replace_packfile () {\n \t# The protocol requires that the packfile be sent in sideband 1, hence\n \t# the extra \\x01 byte at the beginning.\n-\tprintf \"1,/packfile/!c %04x\\\\\\\\x01%s0000\" \\\n-\t\t\"$(($(wc -c <$1) + 5))\" \\\n-\t\t\"$(hex_unpack <$1 | intersperse '\\\\x')\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\"\n+\tcp $1 \"$HTTPD_ROOT_PATH/one-time-pack\" &&\n+\techo 'if (/packfile/) {\n+\t\tprint;\n+\t\tmy $length = -s \"one-time-pack\";\n+\t\tprintf \"%04x\\x01\", $length + 5;\n+\t\tprint `cat one-time-pack` . \"0000\";\n+\t\tlast\n+\t}' >\"$HTTPD_ROOT_PATH/one-time-perl\"\n }\n \n test_expect_success 'upon cloning, check that all refs point to objects' '\n@@ -429,16 +433,16 @@ test_expect_success 'upon cloning, check that all refs point to objects' '\n \t# \\x01 byte at the beginning.\n \treplace_packfile incomplete.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \ttest_must_fail git -c protocol.version=2 clone \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo 2>err &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo 2>err &&\n \n \ttest_i18ngrep \"did not send all necessary objects\" err &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n test_expect_success 'when partial cloning, tolerate server not sending target of tag' '\n@@ -469,17 +473,17 @@ test_expect_success 'when partial cloning, tolerate server not sending target of\n \t# \\x01 byte at the beginning.\n \treplace_packfile incomplete.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \n \t# Exercise to make sure it works.\n \tgit -c protocol.version=2 clone \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo 2> err &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo 2> err &&\n \t! grep \"missing object referenced by\" err &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n test_expect_success 'tolerate server sending REF_DELTA against missing promisor objects' '\n@@ -502,7 +506,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \n \t# Clone. The client has deltabase_have but not deltabase_missing.\n \tgit -c protocol.version=2 clone --no-checkout \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo &&\n \tgit -C repo hash-object -w -- \"$SERVER/have.txt\" &&\n \n \t# Sanity check to ensure that the client does not have\n@@ -543,7 +547,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \n \treplace_packfile thin.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \n@@ -556,8 +560,8 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \tgrep \"want $(cat deltabase_missing)\" trace &&\n \t! grep \"want $(cat deltabase_have)\" trace &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n # DO NOT add non-httpd-specific tests here, because the last part of this\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 7fd7102c874..5039e66dc47 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -712,11 +712,11 @@ test_expect_success 'when server sends \"ready\", expect DELIM' '\n \n \t# After \"ready\" in the acknowledgments section, pretend that a FLUSH\n \t# (0000) was sent instead of a DELIM (0001).\n-\tprintf \"/ready/,$ s/0001/0000/\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\tprintf \"\\$ready = 1 if /ready/; \\$ready && s/0001/0000/\" \\\n+\t\t>\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \ttest_must_fail git -C http_child -c protocol.version=2 \\\n-\t\tfetch \"$HTTPD_URL/one_time_sed/http_parent\" 2> err &&\n+\t\tfetch \"$HTTPD_URL/one_time_perl/http_parent\" 2> err &&\n \ttest_i18ngrep \"expected packfile to be sent after .ready.\" err\n '\n \n@@ -737,12 +737,12 @@ test_expect_success 'when server does not send \"ready\", expect FLUSH' '\n \n \t# After the acknowledgments section, pretend that a DELIM\n \t# (0001) was sent instead of a FLUSH (0000).\n-\tprintf \"/acknowledgments/,$ s/0000/0001/\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\tprintf \"\\$ack = 1 if /acknowledgments/; \\$ack && s/0000/0001/\" \\\n+\t\t>\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \ttest_must_fail env GIT_TRACE_PACKET=\"$(pwd)/log\" git -C http_child \\\n \t\t-c protocol.version=2 \\\n-\t\tfetch \"$HTTPD_URL/one_time_sed/http_parent\" 2> err &&\n+\t\tfetch \"$HTTPD_URL/one_time_perl/http_parent\" 2> err &&\n \tgrep \"fetch< .*acknowledgments\" log &&\n \t! grep \"fetch< .*ready\" log &&\n \ttest_i18ngrep \"expected no other sections to be sent after no .ready.\" err\ndiff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh\nindex 8aeeaac5091..7fba3063bf9 100755\n--- a/t/t5703-upload-pack-ref-in-want.sh\n+++ b/t/t5703-upload-pack-ref-in-want.sh\n@@ -314,7 +314,7 @@ test_expect_success 'setup repos for change-while-negotiating test' '\n \t\ttest_commit m3 &&\n \t\tgit tag -d m2 m3\n \t) &&\n-\tgit -C \"$LOCAL_PRISTINE\" remote set-url origin \"http://127.0.0.1:$LIB_HTTPD_PORT/one_time_sed/repo\" &&\n+\tgit -C \"$LOCAL_PRISTINE\" remote set-url origin \"http://127.0.0.1:$LIB_HTTPD_PORT/one_time_perl/repo\" &&\n \tgit -C \"$LOCAL_PRISTINE\" config protocol.version 2\n '\n \n@@ -327,7 +327,7 @@ inconsistency () {\n \t# RPCs during a single negotiation.\n \toid1=$(git -C \"$REPO\" rev-parse $1) &&\n \toid2=$(git -C \"$REPO\" rev-parse $2) &&\n-\techo \"s/$oid1/$oid2/\" >\"$HTTPD_ROOT_PATH/one-time-sed\"\n+\techo \"s/$oid1/$oid2/\" >\"$HTTPD_ROOT_PATH/one-time-perl\"\n }\n \n test_expect_success 'server is initially ahead - no ref in want' '\n@@ -379,7 +379,7 @@ test_expect_success 'server loses a ref - ref in want' '\n \tgit -C \"$REPO\" config uploadpack.allowRefInWant true &&\n \trm -rf local &&\n \tcp -r \"$LOCAL_PRISTINE\" local &&\n-\techo \"s/master/raster/\" >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\techo \"s/master/raster/\" >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \ttest_must_fail git -C local fetch 2>err &&\n \n \ttest_i18ngrep \"fatal: remote error: unknown ref refs/heads/raster\" err\n-- \ngitgitgadget\n\n"},{"id":"392564","messageId":"8739f8bac0f1c0a97ac16255807514bf7eba0020.1582747775.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.git.git.1582747775.gitgitgadget@gmail.com","subject":"[PATCH 2/3] ci: prevent `perforce` from being quarantined","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-26T20:09:34Z","receivedAt":"2020-02-26T20:09:43Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nThe most recent Azure Pipelines macOS agents enable what Apple calls\n\"System Integrity Protection\". This makes `p4d -V` hang: there is some\nsort of GUI dialog waiting for the user to acknowledge that the copied\nbinaries are legit and may be executed, but on build agents, there is no\nuser who could acknowledge that.\n\nLet's ask Homebrew specifically to _not_ quarantine the Perforce\nbinaries.\n\nHelped-by: Aleksandr Chebotov\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n ci/install-dependencies.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex cd59855d73d..497fd32ca83 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -40,11 +40,11 @@ osx-clang|osx-gcc)\n \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n \tbrew install $BREW_INSTALL_PACKAGES\n \tbrew link --force gettext\n-\tbrew cask install perforce || {\n+\tbrew cask install --no-quarantine perforce || {\n \t\t# Update the definitions and try again\n \t\tcask_repo=\"$(brew --repository)\"/Library/Taps/homebrew/homebrew-cask &&\n \t\tgit -C \"$cask_repo\" pull --no-stat &&\n-\t\tbrew cask install perforce\n+\t\tbrew cask install --no-quarantine perforce\n \t} ||\n \tbrew install caskroom/cask/perforce\n \tcase \"$jobname\" in\n-- \ngitgitgadget\n\n"},{"id":"392567","messageId":"CAPyFy2ArnEFe33GFLsbcOEV37DZ=VJpF-Sckn-V_tJUrToir=w@mail.gmail.com","threadId":"52893","inReplyTo":"9900cacbfefb46610114702e6d48d8020f3f2866.1582747775.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Ed Maste","fromEmail":"emaste@freebsd.org","sentAt":"2020-02-26T20:20:44Z","receivedAt":"2020-02-26T20:21:00Z","isPatch":true,"sender":{"key":"emaste@freebsd.org","avatar":"https://avatars.githubusercontent.com/u/1034582?v=4"},"body":"On Wed, 26 Feb 2020 at 15:09, Johannes Schindelin via GitGitGadget\n<gitgitgadget@gmail.com> wrote:\n>\n> From: Johannes Schindelin <johannes.schindelin@gmx.de>\n>\n> Among other differences relative to GNU sed, BSD sed always ends its\n> output with a trailing newline, even if the input did not have such a\n> trailing newline.\n>\n> Surprisingly, this makes three httpd-based tests fail on macOS: t5616,\n> t5702 and t5703. (\"Surprisingly\" because those tests have been around\n> for some time, but apparently nobody runs them on macOS with a working\n> Apache2 setup.)\n\nHmm, this is interesting - all tests (that are executed) are passing\non FreeBSD, in CI.\n\nI tried on FreeBSD and do not see a trailing newline added; I'm not\nsure how sed behaves on other BSDs. However, you probably want to\nrefer to macOS sed rather than BSD sed in the commit.\n"},{"id":"392568","messageId":"xmqqimjthz8e.fsf@gitster-ct.c.googlers.com","threadId":"52893","inReplyTo":"9900cacbfefb46610114702e6d48d8020f3f2866.1582747775.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-02-26T20:26:09Z","receivedAt":"2020-02-26T20:26:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Johannes Schindelin via GitGitGadget\" <gitgitgadget@gmail.com>\nwrites:\n\n> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\n> index 9e16512fe31..4f10057e9f1 100755\n> --- a/t/t5537-fetch-shallow.sh\n> +++ b/t/t5537-fetch-shallow.sh\n> @@ -237,7 +237,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n>  \tgit -C \"$REPO\" config protocol.version 2 &&\n>  \tgit -C client config protocol.version 2 &&\n>  \n> -\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_sed/repo\" master:a_branch &&\n> +\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_perl/repo\" master:a_branch &&\n>  \n>  \t# Craft a situation in which the server sends back an unshallow request\n>  \t# with an empty packfile. This is done by refetching with a shorter\n> @@ -246,13 +246,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n>  \tprintf \"$(test_oid sed)\" \\\n\nHmm, shouldn't the test-oid token \"sed\" whose value is set up in the\nsetup section of this test script also be renamed to \"perl\"?  Or, if\nwe are actively taking advantage of the fact that the syntax of the\nreplacement operator is the same between the languages, perhaps \"sed\"\nis better renamed to something more language agnostic and reflects\nthe purpose/reason why we extend the packet header by two bytes with\nthe one-time munging process?\n\n>  \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n>  \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n> -\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n> +\t       >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n\nOther than that, this step looked quite sensible.  Thanks.\n"},{"id":"392582","messageId":"xmqqsgixgfal.fsf@gitster-ct.c.googlers.com","threadId":"52893","inReplyTo":"xmqqimjthz8e.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-02-26T22:22:10Z","receivedAt":"2020-02-26T22:22:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> \"Johannes Schindelin via GitGitGadget\" <gitgitgadget@gmail.com>\n> writes:\n>\n>> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\n>> index 9e16512fe31..4f10057e9f1 100755\n>> --- a/t/t5537-fetch-shallow.sh\n>> +++ b/t/t5537-fetch-shallow.sh\n>> @@ -237,7 +237,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n>>  \tgit -C \"$REPO\" config protocol.version 2 &&\n>>  \tgit -C client config protocol.version 2 &&\n>>  \n>> -\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_sed/repo\" master:a_branch &&\n>> +\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_perl/repo\" master:a_branch &&\n>>  \n>>  \t# Craft a situation in which the server sends back an unshallow request\n>>  \t# with an empty packfile. This is done by refetching with a shorter\n>> @@ -246,13 +246,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n>>  \tprintf \"$(test_oid sed)\" \\\n>\n> Hmm, shouldn't the test-oid token \"sed\" whose value is set up in the\n> setup section of this test script also be renamed to \"perl\"?  Or, if\n> we are actively taking advantage of the fact that the syntax of the\n> replacement operator is the same between the languages, perhaps \"sed\"\n> is better renamed to something more language agnostic and reflects\n> the purpose/reason why we extend the packet header by two bytes with\n> the one-time munging process?\n>\n>>  \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n>>  \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n>> -\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n>> +\t       >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n>\n> Other than that, this step looked quite sensible.  Thanks.\n\nHmm, is it because you wanted to backport this down to 'maint'\n(otherwise, your tests will start failing in a month) that you left\nthe \"test_oid sed\" thing untouched?  If so, that makes sort-of\nsense.\n\nI expect that the series will be rerolled, if only for s/BSD/macOS/\nmentioned elsewhere in the thread, but in the meantime, I'll rebase\nthem on 'maint' \"as a practice\" while queuing.\n\n"},{"id":"392609","messageId":"5e5fcf3b76b58cb71546f8a2e61c934f64afca98.1582809793.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.v2.git.git.1582809793.gitgitgadget@gmail.com","subject":"[PATCH v2 2/3] ci: prevent `perforce` from being quarantined","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-27T13:23:12Z","receivedAt":"2020-02-27T13:23:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nThe most recent Azure Pipelines macOS agents enable what Apple calls\n\"System Integrity Protection\". This makes `p4d -V` hang: there is some\nsort of GUI dialog waiting for the user to acknowledge that the copied\nbinaries are legit and may be executed, but on build agents, there is no\nuser who could acknowledge that.\n\nLet's ask Homebrew specifically to _not_ quarantine the Perforce\nbinaries.\n\nHelped-by: Aleksandr Chebotov\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n ci/install-dependencies.sh | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex cd59855d73d..497fd32ca83 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -40,11 +40,11 @@ osx-clang|osx-gcc)\n \ttest -z \"$BREW_INSTALL_PACKAGES\" ||\n \tbrew install $BREW_INSTALL_PACKAGES\n \tbrew link --force gettext\n-\tbrew cask install perforce || {\n+\tbrew cask install --no-quarantine perforce || {\n \t\t# Update the definitions and try again\n \t\tcask_repo=\"$(brew --repository)\"/Library/Taps/homebrew/homebrew-cask &&\n \t\tgit -C \"$cask_repo\" pull --no-stat &&\n-\t\tbrew cask install perforce\n+\t\tbrew cask install --no-quarantine perforce\n \t} ||\n \tbrew install caskroom/cask/perforce\n \tcase \"$jobname\" in\n-- \ngitgitgadget\n\n"},{"id":"392610","messageId":"pull.714.v2.git.git.1582809793.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.git.git.1582747775.gitgitgadget@gmail.com","subject":"[PATCH v2 0/3] ci: upgrade to the latest Azure Pipelines agent pools","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-27T13:23:10Z","receivedAt":"2020-02-27T13:23:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"At least the Windows agents we use will go away on March 23rd, 2020\n[https://devblogs.microsoft.com/devops/removing-older-images-in-azure-pipelines-hosted-pools/]\n, in favor of newer, faster, better ones. We might just as well use the\nlatter already long before the cut-off date.\n\nWhile at it, let's also update to a newer macOS pool. This necessitates two\ndistinct fixes, though:\n\n * Our ci/ scripts were ill-prepared to deal with System Integrity\n   Protection [https://support.apple.com/en-us/HT204899]. This resulted in\n   the p4d -V call seemingly \"hanging\" (Narrator's voice: it did not \"hang\",\n   it actually opened a GUI dialog to ask the user whether it was okay to\n   execute this quarantined executable).\n   \n   \n * Apparently nobody ever bothered running the Apache2-based tests (t5616,\n   t5702 and t5703, probably more) on macOS. I say that because they don't\n   work, over-relying on GNU-specific behavior of sed. I rewrote those sed \n   calls to use Perl instead.\n   \n   \n\nChanges since v1:\n\n * Avoid blaming BSD for what is (at least nowadays) squarely a macOS issue.\n * Targeting maint\n\nJohannes Schindelin (3):\n  t/lib-httpd: avoid using macOS' sed\n  ci: prevent `perforce` from being quarantined\n  Azure Pipeline: switch to the latest agent pools\n\n azure-pipelines.yml                | 37 +++++++++++++++++++----------\n ci/install-dependencies.sh         |  4 ++--\n t/lib-httpd.sh                     |  2 +-\n t/lib-httpd/apache.conf            |  6 ++---\n t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++\n t/lib-httpd/apply-one-time-sed.sh  | 24 -------------------\n t/t5537-fetch-shallow.sh           | 10 ++++----\n t/t5616-partial-clone.sh           | 38 +++++++++++++++++-------------\n t/t5702-protocol-v2.sh             | 12 +++++-----\n t/t5703-upload-pack-ref-in-want.sh |  6 ++---\n 10 files changed, 93 insertions(+), 73 deletions(-)\n create mode 100644 t/lib-httpd/apply-one-time-perl.sh\n delete mode 100644 t/lib-httpd/apply-one-time-sed.sh\n\n\nbase-commit: c522f061d551c9bb8684a7c3859b2ece4499b56b\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-714%2Fdscho%2Fazure-pipelines-latest-pools-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-714/dscho/azure-pipelines-latest-pools-v2\nPull-Request: https://github.com/git/git/pull/714\n\nRange-diff vs v1:\n\n 1:  9900cacbfef ! 1:  af28d115ded t/lib-httpd: avoid using BSD's sed\n     @@ -1,8 +1,8 @@\n      Author: Johannes Schindelin <johannes.schindelin@gmx.de>\n      \n     -    t/lib-httpd: avoid using BSD's sed\n     +    t/lib-httpd: avoid using macOS' sed\n      \n     -    Among other differences relative to GNU sed, BSD sed always ends its\n     +    Among other differences relative to GNU sed, macOS' sed always ends its\n          output with a trailing newline, even if the input did not have such a\n          trailing newline.\n      \n     @@ -14,7 +14,7 @@\n          The reason is that we use `sed` in those tests to filter the response of\n          the web server. Apart from the fact that we use GNU constructs (such as\n          using a space after the `c` command instead of a backslash and a\n     -    newline), we have another problem: BSD sed LF-only newlines while\n     +    newline), we have another problem: macOS' sed LF-only newlines while\n          webservers are supposed to use CR/LF ones.\n      \n          Even worse, t5616 uses `sed` to replace a binary part of the response\n     @@ -25,11 +25,11 @@\n          hex-encode it, then calls on `sed` to prefix every hex digit pair with a\n          `\\x` in order to construct the text that the `c` statement of the `sed`\n          invocation is supposed to insert. So we call Perl and sed to construct a\n     -    sed statement. The final nail in the coffin is that BSD sed does not\n     +    sed statement. The final nail in the coffin is that macOS' sed does not\n          even interpret those `\\x<hex>` constructs.\n      \n          Let's just replace all of that by Perl snippets. With Perl, at least, we\n     -    do not have to deal with GNU vs BSD semantics, we do not have to worry\n     +    do not have to deal with GNU vs macOS semantics, we do not have to worry\n          about unwanted trailing newlines, and we do not have to spawn commands\n          to construct arguments for other commands to be spawned (i.e. we can\n          avoid a whole lot of shell scripting complexity).\n     @@ -159,7 +159,7 @@\n       \t# Craft a situation in which the server sends back an unshallow request\n       \t# with an empty packfile. This is done by refetching with a shorter\n      @@\n     - \tprintf \"$(test_oid sed)\" \\\n     + \tprintf \"s/0034shallow %s/0036unshallow %s/\" \\\n       \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n       \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n      -\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n 2:  8739f8bac0f = 2:  5e5fcf3b76b ci: prevent `perforce` from being quarantined\n 3:  f141c295fd8 = 3:  6d2f8e7e70e Azure Pipeline: switch to the latest agent pools\n\n-- \ngitgitgadget\n"},{"id":"392611","messageId":"6d2f8e7e70e19df7e1693b13e64121eabee038f2.1582809793.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.v2.git.git.1582809793.gitgitgadget@gmail.com","subject":"[PATCH v2 3/3] Azure Pipeline: switch to the latest agent pools","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-27T13:23:13Z","receivedAt":"2020-02-27T13:23:20Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nIt would seem that at least the `vs2015-win2012r2` pool (which we use\nvia its old name, `Hosted`) is about to be phased out. Let's switch\nbefore that.\n\nWhile at it, use the newer pool names as suggested at\nhttps://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n azure-pipelines.yml | 37 +++++++++++++++++++++++++------------\n 1 file changed, 25 insertions(+), 12 deletions(-)\n\ndiff --git a/azure-pipelines.yml b/azure-pipelines.yml\nindex af2a5ea4845..675c3a43c9c 100644\n--- a/azure-pipelines.yml\n+++ b/azure-pipelines.yml\n@@ -5,7 +5,8 @@ jobs:\n - job: windows_build\n   displayName: Windows Build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   steps:\n   - powershell: |\n@@ -61,7 +62,8 @@ jobs:\n   displayName: Windows Test\n   dependsOn: windows_build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   strategy:\n     parallel: 10\n@@ -133,7 +135,8 @@ jobs:\n - job: vs_build\n   displayName: Visual Studio Build\n   condition: succeeded()\n-  pool: Hosted VS2017\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   steps:\n   - powershell: |\n@@ -181,6 +184,7 @@ jobs:\n       platform: x64\n       configuration: Release\n       maximumCpuCount: 4\n+      msbuildArguments: /p:PlatformToolset=v142\n   - powershell: |\n       & compat\\vcbuild\\vcpkg_copy_dlls.bat release\n       if (!$?) { exit(1) }\n@@ -224,7 +228,8 @@ jobs:\n   displayName: Visual Studio Test\n   dependsOn: vs_build\n   condition: succeeded()\n-  pool: Hosted\n+  pool:\n+    vmImage: windows-latest\n   timeoutInMinutes: 240\n   strategy:\n     parallel: 10\n@@ -292,7 +297,8 @@ jobs:\n - job: linux_clang\n   displayName: linux-clang\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -330,7 +336,8 @@ jobs:\n - job: linux_gcc\n   displayName: linux-gcc\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -367,7 +374,8 @@ jobs:\n - job: osx_clang\n   displayName: osx-clang\n   condition: succeeded()\n-  pool: Hosted macOS\n+  pool:\n+    vmImage: macOS-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -402,7 +410,8 @@ jobs:\n - job: osx_gcc\n   displayName: osx-gcc\n   condition: succeeded()\n-  pool: Hosted macOS\n+  pool:\n+    vmImage: macOS-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -435,7 +444,8 @@ jobs:\n - job: gettext_poison\n   displayName: GETTEXT_POISON\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -472,7 +482,8 @@ jobs:\n - job: linux32\n   displayName: Linux32\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -506,7 +517,8 @@ jobs:\n - job: static_analysis\n   displayName: StaticAnalysis\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n@@ -526,7 +538,8 @@ jobs:\n - job: documentation\n   displayName: Documentation\n   condition: succeeded()\n-  pool: Hosted Ubuntu 1604\n+  pool:\n+    vmImage: ubuntu-latest\n   steps:\n   - bash: |\n        test \"$GITFILESHAREPWD\" = '$(gitfileshare.pwd)' || ci/mount-fileshare.sh //gitfileshare.file.core.windows.net/test-cache gitfileshare \"$GITFILESHAREPWD\" \"$HOME/test-cache\" || exit 1\n-- \ngitgitgadget\n"},{"id":"392612","messageId":"af28d115ded3c6366dacd501e2309c05c6c1ac2a.1582809793.git.gitgitgadget@gmail.com","threadId":"52893","inReplyTo":"pull.714.v2.git.git.1582809793.gitgitgadget@gmail.com","subject":"[PATCH v2 1/3] t/lib-httpd: avoid using macOS' sed","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-02-27T13:23:11Z","receivedAt":"2020-02-27T13:23:24Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin <johannes.schindelin@gmx.de>\n\nAmong other differences relative to GNU sed, macOS' sed always ends its\noutput with a trailing newline, even if the input did not have such a\ntrailing newline.\n\nSurprisingly, this makes three httpd-based tests fail on macOS: t5616,\nt5702 and t5703. (\"Surprisingly\" because those tests have been around\nfor some time, but apparently nobody runs them on macOS with a working\nApache2 setup.)\n\nThe reason is that we use `sed` in those tests to filter the response of\nthe web server. Apart from the fact that we use GNU constructs (such as\nusing a space after the `c` command instead of a backslash and a\nnewline), we have another problem: macOS' sed LF-only newlines while\nwebservers are supposed to use CR/LF ones.\n\nEven worse, t5616 uses `sed` to replace a binary part of the response\nwith a new binary part (kind of hoping that the replaced binary part\ndoes not contain a 0x0a byte which would be interpreted as a newline).\n\nTo that end, it calls on Perl to read the binary pack file and\nhex-encode it, then calls on `sed` to prefix every hex digit pair with a\n`\\x` in order to construct the text that the `c` statement of the `sed`\ninvocation is supposed to insert. So we call Perl and sed to construct a\nsed statement. The final nail in the coffin is that macOS' sed does not\neven interpret those `\\x<hex>` constructs.\n\nLet's just replace all of that by Perl snippets. With Perl, at least, we\ndo not have to deal with GNU vs macOS semantics, we do not have to worry\nabout unwanted trailing newlines, and we do not have to spawn commands\nto construct arguments for other commands to be spawned (i.e. we can\navoid a whole lot of shell scripting complexity).\n\nThe upshot is that this fixes t5616, t5702 and t5703 on macOS with\nApache2.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n t/lib-httpd.sh                     |  2 +-\n t/lib-httpd/apache.conf            |  6 ++---\n t/lib-httpd/apply-one-time-perl.sh | 27 +++++++++++++++++++++\n t/lib-httpd/apply-one-time-sed.sh  | 24 -------------------\n t/t5537-fetch-shallow.sh           | 10 ++++----\n t/t5616-partial-clone.sh           | 38 +++++++++++++++++-------------\n t/t5702-protocol-v2.sh             | 12 +++++-----\n t/t5703-upload-pack-ref-in-want.sh |  6 ++---\n 8 files changed, 66 insertions(+), 59 deletions(-)\n create mode 100644 t/lib-httpd/apply-one-time-perl.sh\n delete mode 100644 t/lib-httpd/apply-one-time-sed.sh\n\ndiff --git a/t/lib-httpd.sh b/t/lib-httpd.sh\nindex 656997b4d66..1449ee95e9e 100644\n--- a/t/lib-httpd.sh\n+++ b/t/lib-httpd.sh\n@@ -132,7 +132,7 @@ prepare_httpd() {\n \tinstall_script broken-smart-http.sh\n \tinstall_script error-smart-http.sh\n \tinstall_script error.sh\n-\tinstall_script apply-one-time-sed.sh\n+\tinstall_script apply-one-time-perl.sh\n \n \tln -s \"$LIB_HTTPD_MODULE_PATH\" \"$HTTPD_ROOT_PATH/modules\"\n \ndiff --git a/t/lib-httpd/apache.conf b/t/lib-httpd/apache.conf\nindex 5c1c86c193a..994e5290d63 100644\n--- a/t/lib-httpd/apache.conf\n+++ b/t/lib-httpd/apache.conf\n@@ -113,7 +113,7 @@ Alias /auth/dumb/ www/auth/dumb/\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n </LocationMatch>\n-<LocationMatch /one_time_sed/>\n+<LocationMatch /one_time_perl/>\n \tSetEnv GIT_EXEC_PATH ${GIT_EXEC_PATH}\n \tSetEnv GIT_HTTP_EXPORT_ALL\n </LocationMatch>\n@@ -122,7 +122,7 @@ ScriptAliasMatch /smart_*[^/]*/(.*) ${GIT_EXEC_PATH}/git-http-backend/$1\n ScriptAlias /broken_smart/ broken-smart-http.sh/\n ScriptAlias /error_smart/ error-smart-http.sh/\n ScriptAlias /error/ error.sh/\n-ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1\n+ScriptAliasMatch /one_time_perl/(.*) apply-one-time-perl.sh/$1\n <Directory ${GIT_EXEC_PATH}>\n \tOptions FollowSymlinks\n </Directory>\n@@ -135,7 +135,7 @@ ScriptAliasMatch /one_time_sed/(.*) apply-one-time-sed.sh/$1\n <Files error.sh>\n   Options ExecCGI\n </Files>\n-<Files apply-one-time-sed.sh>\n+<Files apply-one-time-perl.sh>\n \tOptions ExecCGI\n </Files>\n <Files ${GIT_EXEC_PATH}/git-http-backend>\ndiff --git a/t/lib-httpd/apply-one-time-perl.sh b/t/lib-httpd/apply-one-time-perl.sh\nnew file mode 100644\nindex 00000000000..09a0abdff7c\n--- /dev/null\n+++ b/t/lib-httpd/apply-one-time-perl.sh\n@@ -0,0 +1,27 @@\n+#!/bin/sh\n+\n+# If \"one-time-perl\" exists in $HTTPD_ROOT_PATH, run perl on the HTTP response,\n+# using the contents of \"one-time-perl\" as the perl command to be run. If the\n+# response was modified as a result, delete \"one-time-perl\" so that subsequent\n+# HTTP responses are no longer modified.\n+#\n+# This can be used to simulate the effects of the repository changing in\n+# between HTTP request-response pairs.\n+if test -f one-time-perl\n+then\n+\tLC_ALL=C\n+\texport LC_ALL\n+\n+\t\"$GIT_EXEC_PATH/git-http-backend\" >out\n+\tperl -pe \"$(cat one-time-perl)\" out >out_modified\n+\n+\tif cmp -s out out_modified\n+\tthen\n+\t\tcat out\n+\telse\n+\t\tcat out_modified\n+\t\trm one-time-perl\n+\tfi\n+else\n+\t\"$GIT_EXEC_PATH/git-http-backend\"\n+fi\ndiff --git a/t/lib-httpd/apply-one-time-sed.sh b/t/lib-httpd/apply-one-time-sed.sh\ndeleted file mode 100644\nindex bf7689d0202..00000000000\n--- a/t/lib-httpd/apply-one-time-sed.sh\n+++ /dev/null\n@@ -1,24 +0,0 @@\n-#!/bin/sh\n-\n-# If \"one-time-sed\" exists in $HTTPD_ROOT_PATH, run sed on the HTTP response,\n-# using the contents of \"one-time-sed\" as the sed command to be run. If the\n-# response was modified as a result, delete \"one-time-sed\" so that subsequent\n-# HTTP responses are no longer modified.\n-#\n-# This can be used to simulate the effects of the repository changing in\n-# between HTTP request-response pairs.\n-if test -f one-time-sed\n-then\n-\t\"$GIT_EXEC_PATH/git-http-backend\" >out\n-\tsed \"$(cat one-time-sed)\" out >out_modified\n-\n-\tif cmp -s out out_modified\n-\tthen\n-\t\tcat out\n-\telse\n-\t\tcat out_modified\n-\t\trm one-time-sed\n-\tfi\n-else\n-\t\"$GIT_EXEC_PATH/git-http-backend\"\n-fi\ndiff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\nindex 97a67728ca9..dd9cd4b51a2 100755\n--- a/t/t5537-fetch-shallow.sh\n+++ b/t/t5537-fetch-shallow.sh\n@@ -233,7 +233,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n \tgit -C \"$REPO\" config protocol.version 2 &&\n \tgit -C client config protocol.version 2 &&\n \n-\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_sed/repo\" master:a_branch &&\n+\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_perl/repo\" master:a_branch &&\n \n \t# Craft a situation in which the server sends back an unshallow request\n \t# with an empty packfile. This is done by refetching with a shorter\n@@ -242,13 +242,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n \tprintf \"s/0034shallow %s/0036unshallow %s/\" \\\n \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n-\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\t       >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \ttest_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \\\n-\t\tfetch --depth=1 \"$HTTPD_URL/one_time_sed/repo\" \\\n+\t\tfetch --depth=1 \"$HTTPD_URL/one_time_perl/repo\" \\\n \t\tmaster:a_branch &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \t# Ensure that the resulting repo is consistent, despite our failure to\n \t# fetch.\ndiff --git a/t/t5616-partial-clone.sh b/t/t5616-partial-clone.sh\nindex 9a9178fd281..0eb5b1c47b1 100755\n--- a/t/t5616-partial-clone.sh\n+++ b/t/t5616-partial-clone.sh\n@@ -398,14 +398,18 @@ intersperse () {\n \tsed 's/\\(..\\)/'$1'\\1/g'\n }\n \n-# Create a one-time-sed command to replace the existing packfile with $1.\n+# Create a one-time-perl command to replace the existing packfile with $1.\n replace_packfile () {\n \t# The protocol requires that the packfile be sent in sideband 1, hence\n \t# the extra \\x01 byte at the beginning.\n-\tprintf \"1,/packfile/!c %04x\\\\\\\\x01%s0000\" \\\n-\t\t\"$(($(wc -c <$1) + 5))\" \\\n-\t\t\"$(hex_unpack <$1 | intersperse '\\\\x')\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\"\n+\tcp $1 \"$HTTPD_ROOT_PATH/one-time-pack\" &&\n+\techo 'if (/packfile/) {\n+\t\tprint;\n+\t\tmy $length = -s \"one-time-pack\";\n+\t\tprintf \"%04x\\x01\", $length + 5;\n+\t\tprint `cat one-time-pack` . \"0000\";\n+\t\tlast\n+\t}' >\"$HTTPD_ROOT_PATH/one-time-perl\"\n }\n \n test_expect_success 'upon cloning, check that all refs point to objects' '\n@@ -429,16 +433,16 @@ test_expect_success 'upon cloning, check that all refs point to objects' '\n \t# \\x01 byte at the beginning.\n \treplace_packfile incomplete.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \ttest_must_fail git -c protocol.version=2 clone \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo 2>err &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo 2>err &&\n \n \ttest_i18ngrep \"did not send all necessary objects\" err &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n test_expect_success 'when partial cloning, tolerate server not sending target of tag' '\n@@ -469,17 +473,17 @@ test_expect_success 'when partial cloning, tolerate server not sending target of\n \t# \\x01 byte at the beginning.\n \treplace_packfile incomplete.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \n \t# Exercise to make sure it works.\n \tgit -c protocol.version=2 clone \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo 2> err &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo 2> err &&\n \t! grep \"missing object referenced by\" err &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n test_expect_success 'tolerate server sending REF_DELTA against missing promisor objects' '\n@@ -502,7 +506,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \n \t# Clone. The client has deltabase_have but not deltabase_missing.\n \tgit -c protocol.version=2 clone --no-checkout \\\n-\t\t--filter=blob:none $HTTPD_URL/one_time_sed/server repo &&\n+\t\t--filter=blob:none $HTTPD_URL/one_time_perl/server repo &&\n \tgit -C repo hash-object -w -- \"$SERVER/have.txt\" &&\n \n \t# Sanity check to ensure that the client does not have\n@@ -543,7 +547,7 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \n \treplace_packfile thin.pack &&\n \n-\t# Use protocol v2 because the sed command looks for the \"packfile\"\n+\t# Use protocol v2 because the perl command looks for the \"packfile\"\n \t# section header.\n \ttest_config -C \"$SERVER\" protocol.version 2 &&\n \n@@ -556,8 +560,8 @@ test_expect_success 'tolerate server sending REF_DELTA against missing promisor\n \tgrep \"want $(cat deltabase_missing)\" trace &&\n \t! grep \"want $(cat deltabase_have)\" trace &&\n \n-\t# Ensure that the one-time-sed script was used.\n-\t! test -e \"$HTTPD_ROOT_PATH/one-time-sed\"\n+\t# Ensure that the one-time-perl script was used.\n+\t! test -e \"$HTTPD_ROOT_PATH/one-time-perl\"\n '\n \n # DO NOT add non-httpd-specific tests here, because the last part of this\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 7fd7102c874..5039e66dc47 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -712,11 +712,11 @@ test_expect_success 'when server sends \"ready\", expect DELIM' '\n \n \t# After \"ready\" in the acknowledgments section, pretend that a FLUSH\n \t# (0000) was sent instead of a DELIM (0001).\n-\tprintf \"/ready/,$ s/0001/0000/\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\tprintf \"\\$ready = 1 if /ready/; \\$ready && s/0001/0000/\" \\\n+\t\t>\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \ttest_must_fail git -C http_child -c protocol.version=2 \\\n-\t\tfetch \"$HTTPD_URL/one_time_sed/http_parent\" 2> err &&\n+\t\tfetch \"$HTTPD_URL/one_time_perl/http_parent\" 2> err &&\n \ttest_i18ngrep \"expected packfile to be sent after .ready.\" err\n '\n \n@@ -737,12 +737,12 @@ test_expect_success 'when server does not send \"ready\", expect FLUSH' '\n \n \t# After the acknowledgments section, pretend that a DELIM\n \t# (0001) was sent instead of a FLUSH (0000).\n-\tprintf \"/acknowledgments/,$ s/0000/0001/\" \\\n-\t\t>\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\tprintf \"\\$ack = 1 if /acknowledgments/; \\$ack && s/0000/0001/\" \\\n+\t\t>\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \n \ttest_must_fail env GIT_TRACE_PACKET=\"$(pwd)/log\" git -C http_child \\\n \t\t-c protocol.version=2 \\\n-\t\tfetch \"$HTTPD_URL/one_time_sed/http_parent\" 2> err &&\n+\t\tfetch \"$HTTPD_URL/one_time_perl/http_parent\" 2> err &&\n \tgrep \"fetch< .*acknowledgments\" log &&\n \t! grep \"fetch< .*ready\" log &&\n \ttest_i18ngrep \"expected no other sections to be sent after no .ready.\" err\ndiff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh\nindex 1424fabd4aa..e1eec7d7bb7 100755\n--- a/t/t5703-upload-pack-ref-in-want.sh\n+++ b/t/t5703-upload-pack-ref-in-want.sh\n@@ -313,7 +313,7 @@ test_expect_success 'setup repos for change-while-negotiating test' '\n \t\ttest_commit m3 &&\n \t\tgit tag -d m2 m3\n \t) &&\n-\tgit -C \"$LOCAL_PRISTINE\" remote set-url origin \"http://127.0.0.1:$LIB_HTTPD_PORT/one_time_sed/repo\" &&\n+\tgit -C \"$LOCAL_PRISTINE\" remote set-url origin \"http://127.0.0.1:$LIB_HTTPD_PORT/one_time_perl/repo\" &&\n \tgit -C \"$LOCAL_PRISTINE\" config protocol.version 2\n '\n \n@@ -326,7 +326,7 @@ inconsistency () {\n \t# RPCs during a single negotiation.\n \toid1=$(git -C \"$REPO\" rev-parse $1) &&\n \toid2=$(git -C \"$REPO\" rev-parse $2) &&\n-\techo \"s/$oid1/$oid2/\" >\"$HTTPD_ROOT_PATH/one-time-sed\"\n+\techo \"s/$oid1/$oid2/\" >\"$HTTPD_ROOT_PATH/one-time-perl\"\n }\n \n test_expect_success 'server is initially ahead - no ref in want' '\n@@ -378,7 +378,7 @@ test_expect_success 'server loses a ref - ref in want' '\n \tgit -C \"$REPO\" config uploadpack.allowRefInWant true &&\n \trm -rf local &&\n \tcp -r \"$LOCAL_PRISTINE\" local &&\n-\techo \"s/master/raster/\" >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n+\techo \"s/master/raster/\" >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n \ttest_must_fail git -C local fetch 2>err &&\n \n \ttest_i18ngrep \"fatal: remote error: unknown ref refs/heads/raster\" err\n-- \ngitgitgadget\n\n"},{"id":"392617","messageId":"nycvar.QRO.7.76.6.2002271640320.46@tvgsbejvaqbjf.bet","threadId":"52893","inReplyTo":"CAPyFy2ArnEFe33GFLsbcOEV37DZ=VJpF-Sckn-V_tJUrToir=w@mail.gmail.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-02-27T15:40:38Z","receivedAt":"2020-02-27T15:40:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ed,\n\nOn Wed, 26 Feb 2020, Ed Maste wrote:\n\n> On Wed, 26 Feb 2020 at 15:09, Johannes Schindelin via GitGitGadget\n> <gitgitgadget@gmail.com> wrote:\n> >\n> > From: Johannes Schindelin <johannes.schindelin@gmx.de>\n> >\n> > Among other differences relative to GNU sed, BSD sed always ends its\n> > output with a trailing newline, even if the input did not have such a\n> > trailing newline.\n> >\n> > Surprisingly, this makes three httpd-based tests fail on macOS: t5616,\n> > t5702 and t5703. (\"Surprisingly\" because those tests have been around\n> > for some time, but apparently nobody runs them on macOS with a working\n> > Apache2 setup.)\n>\n> Hmm, this is interesting - all tests (that are executed) are passing\n> on FreeBSD, in CI.\n>\n> I tried on FreeBSD and do not see a trailing newline added; I'm not\n> sure how sed behaves on other BSDs. However, you probably want to\n> refer to macOS sed rather than BSD sed in the commit.\n\nMy bad. I looked at StackOverflow and there the claim was that all BSD\nseds behave that way.\n\nOf course, SO always lags behind by a couple years (although even such old\nthreads are often very useful), so it is possible that _old_ BSD sed\nbehaved that way.\n\nIn any case, I adjusted the commit message.\n\nRelated, I saw that Cirrus CI offers FreeBSD builds, maybe you'd be\ninterested in supporting that out of the box in\nhttps://github.com/git/git?\n\nCiao,\nDscho\n"},{"id":"392618","messageId":"nycvar.QRO.7.76.6.2002271640520.46@tvgsbejvaqbjf.bet","threadId":"52893","inReplyTo":"xmqqsgixgfal.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-02-27T15:42:15Z","receivedAt":"2020-02-27T15:42:23Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Junio,\n\nOn Wed, 26 Feb 2020, Junio C Hamano wrote:\n\n> Junio C Hamano <gitster@pobox.com> writes:\n>\n> > \"Johannes Schindelin via GitGitGadget\" <gitgitgadget@gmail.com>\n> > writes:\n> >\n> >> diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh\n> >> index 9e16512fe31..4f10057e9f1 100755\n> >> --- a/t/t5537-fetch-shallow.sh\n> >> +++ b/t/t5537-fetch-shallow.sh\n> >> @@ -237,7 +237,7 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n> >>  \tgit -C \"$REPO\" config protocol.version 2 &&\n> >>  \tgit -C client config protocol.version 2 &&\n> >>\n> >> -\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_sed/repo\" master:a_branch &&\n> >> +\tgit -C client fetch --depth=2 \"$HTTPD_URL/one_time_perl/repo\" master:a_branch &&\n> >>\n> >>  \t# Craft a situation in which the server sends back an unshallow request\n> >>  \t# with an empty packfile. This is done by refetching with a shorter\n> >> @@ -246,13 +246,13 @@ test_expect_success 'shallow fetches check connectivity before writing shallow f\n> >>  \tprintf \"$(test_oid sed)\" \\\n> >\n> > Hmm, shouldn't the test-oid token \"sed\" whose value is set up in the\n> > setup section of this test script also be renamed to \"perl\"?\n\nOoops...\n\n> > Or, if we are actively taking advantage of the fact that the syntax of\n> > the replacement operator is the same between the languages, perhaps\n> > \"sed\" is better renamed to something more language agnostic and\n> > reflects the purpose/reason why we extend the packet header by two\n> > bytes with the one-time munging process?\n> >\n> >>  \t       \"$(git -C \"$REPO\" rev-parse HEAD)\" \\\n> >>  \t       \"$(git -C \"$REPO\" rev-parse HEAD^)\" \\\n> >> -\t       >\"$HTTPD_ROOT_PATH/one-time-sed\" &&\n> >> +\t       >\"$HTTPD_ROOT_PATH/one-time-perl\" &&\n> >\n> > Other than that, this step looked quite sensible.  Thanks.\n>\n> Hmm, is it because you wanted to backport this down to 'maint'\n> (otherwise, your tests will start failing in a month) that you left\n> the \"test_oid sed\" thing untouched?  If so, that makes sort-of\n> sense.\n\nThat's a good point. I target `maint` in v2, and offered an add-on patch\nmeant to be applied on top of the merge into `master` (or `next`, or\n`pu`).\n\n> I expect that the series will be rerolled, if only for s/BSD/macOS/\n> mentioned elsewhere in the thread, but in the meantime, I'll rebase\n> them on 'maint' \"as a practice\" while queuing.\n\nThanks ;-)\n\nCiao,\nDscho\n"},{"id":"392630","messageId":"CAPyFy2BZoJAuugAbmgsrbzjM1pNu7o9Wzusz1qV7No8Hr+T4ww@mail.gmail.com","threadId":"52893","inReplyTo":"nycvar.QRO.7.76.6.2002271640320.46@tvgsbejvaqbjf.bet","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Ed Maste","fromEmail":"emaste@freebsd.org","sentAt":"2020-02-27T17:39:01Z","receivedAt":"2020-02-27T17:39:17Z","isPatch":true,"sender":{"key":"emaste@freebsd.org","avatar":"https://avatars.githubusercontent.com/u/1034582?v=4"},"body":"On Thu, 27 Feb 2020 at 10:40, Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n>\n> My bad. I looked at StackOverflow and there the claim was that all BSD\n> seds behave that way.\n>\n> Of course, SO always lags behind by a couple years (although even such old\n> threads are often very useful), so it is possible that _old_ BSD sed\n> behaved that way.\n\nYeah, I wondered about the different behaviour, and asked on Twitter\nabout it. You're right, it is historical BSD behaviour and NetBSD at\nleast still appends the newline. FreeBSD changed this in 2014 -\nhttp://bugs.freebsd.org/160745.\n\n> Related, I saw that Cirrus CI offers FreeBSD builds, maybe you'd be\n> interested in supporting that out of the box in\n> https://github.com/git/git?\n\nIndeed - there is a .cirrus.yml in git now which builds and runs tests\n(on FreeBSD 12.1). I'll look into working with the GitHub organization\nowners for git and gitgitgadget to see about allowing Cirrus to access\nthe repositories.\n"},{"id":"392636","messageId":"nycvar.QRO.7.76.6.2002272046060.46@tvgsbejvaqbjf.bet","threadId":"52893","inReplyTo":"CAPyFy2BZoJAuugAbmgsrbzjM1pNu7o9Wzusz1qV7No8Hr+T4ww@mail.gmail.com","subject":"Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-02-27T19:46:26Z","receivedAt":"2020-02-27T19:46:36Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ed,\n\nOn Thu, 27 Feb 2020, Ed Maste wrote:\n\n> On Thu, 27 Feb 2020 at 10:40, Johannes Schindelin\n> <Johannes.Schindelin@gmx.de> wrote:\n> >\n> > My bad. I looked at StackOverflow and there the claim was that all BSD\n> > seds behave that way.\n> >\n> > Of course, SO always lags behind by a couple years (although even such old\n> > threads are often very useful), so it is possible that _old_ BSD sed\n> > behaved that way.\n>\n> Yeah, I wondered about the different behaviour, and asked on Twitter\n> about it. You're right, it is historical BSD behaviour and NetBSD at\n> least still appends the newline. FreeBSD changed this in 2014 -\n> http://bugs.freebsd.org/160745.\n\nThank you for digging into this!\n\n> > Related, I saw that Cirrus CI offers FreeBSD builds, maybe you'd be\n> > interested in supporting that out of the box in\n> > https://github.com/git/git?\n>\n> Indeed - there is a .cirrus.yml in git now which builds and runs tests\n> (on FreeBSD 12.1). I'll look into working with the GitHub organization\n> owners for git and gitgitgadget to see about allowing Cirrus to access\n> the repositories.\n\nOy, I had forgotten that you worked on this. I enabled this in the git and\nin the gitgitgadget orgs. The next pushes/PRs should benefit from this.\n\nThanks,\nDscho\n"},{"id":"392702","messageId":"nycvar.QRO.7.76.6.2002290054540.11433@tvgsbejvaqbjf.bet","threadId":"52893","inReplyTo":"nycvar.QRO.7.76.6.2002272046060.46@tvgsbejvaqbjf.bet","subject":"CI/PR builds on FreeBSD, was Re: [PATCH 1/3] t/lib-httpd: avoid using BSD's sed","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2020-02-28T23:57:07Z","receivedAt":"2020-02-28T23:57:19Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ed,\n\nOn Thu, 27 Feb 2020, Johannes Schindelin wrote:\n\n> On Thu, 27 Feb 2020, Ed Maste wrote:\n>\n> > On Thu, 27 Feb 2020 at 10:40, Johannes Schindelin\n> > <Johannes.Schindelin@gmx.de> wrote:\n> >\n> > > Related, I saw that Cirrus CI offers FreeBSD builds, maybe you'd be\n> > > interested in supporting that out of the box in\n> > > https://github.com/git/git?\n> >\n> > Indeed - there is a .cirrus.yml in git now which builds and runs tests\n> > (on FreeBSD 12.1). I'll look into working with the GitHub organization\n> > owners for git and gitgitgadget to see about allowing Cirrus to access\n> > the repositories.\n>\n> Oy, I had forgotten that you worked on this. I enabled this in the git\n> and in the gitgitgadget orgs. The next pushes/PRs should benefit from\n> this.\n\nThis works now. When you click on the green checkmarks (and the one red X)\nat https://github.com/git/git/branches/active, you will see the\n`freebsd_12` build.\n\nCiao,\nDscho\n"}]}