{"thread":{"id":"51810","subject":"[PATCH] ci: install P4 from package to fix build error","startedAt":"2019-09-06T10:27:28Z","lastAt":"2019-09-12T11:10:00Z","messageCount":6,"participants":["SZEDER Gábor","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"381949","messageId":"20190906102711.6401-1-szeder.dev@gmail.com","threadId":"51810","inReplyTo":null,"subject":"[PATCH] ci: install P4 from package to fix build error","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-06T10:27:11Z","receivedAt":"2019-09-06T10:27:28Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"To test 'git-p4' in the Linux Clang and GCC build jobs we used to\ninstall the 'p4' and 'p4d' binaries by directly downloading those\nbinaries from a Perforce filehost.  This has worked just fine ever\nsince we started using Travis CI [1], but during the last day or so\nthat filehost appeared to be gone: while its hostname still resolves,\nthe host doesn't seem to reply to any download request, it doesn't\neven refuse the connection, and eventually our build jobs time out\n[2].\n\nNow, this might be just a temporary glitch, but I'm afraid that it\nisn't.  The \"Helix Core Server Administrator Guide\" [3] describes two\nways to install these binaries on Linux, and none of them mentions the\nfilehost that we've been downloading from in the past:\n\n  - non-package installation: open the website's download page in your\n    web browser, select OS and platform, click on the download link,\n    and eventually you get a .tar.gz archive containing, among other\n    things, the necessary 'p4' and 'p4d' binaries.\n\n    Although we could use the URL of this archive to download it in\n    our CI scripts with 'wget', nobody said that that URL remains\n    stable, and we would still need to extract the archive and copy\n    the binaries to $PATH.\n\n  - package installation for various distros, including Ubuntu 16.04\n    (i.e. the Ubuntu version used both in our Travis CI and Azure\n    Pipelines builds): add a package repository and its pubkey,\n    'apt-get update && apt-get install', and ready to go.\n\nLet's install P4 from the package repository, because this approach\nseems to be simpler and more future proof.\n\nNote that we used to install an old P4 version (2016.2) in the Linux\nbuild jobs, but with this change we'll install the most recent version\navailable in the Perforce package repository (currently 2019.1).\n\n[1] 522354d70f (Add Travis CI support, 2015-11-27).\n[2] https://travis-ci.org/git/git/jobs/581429927#L422\n[3] https://www.perforce.com/manuals/p4sag/Content/P4SAG/chapter.install.html\n\nSigned-off-by: SZEDER Gábor <szeder.dev@gmail.com>\n---\n ci/install-dependencies.sh | 14 +++++---------\n ci/lib.sh                  |  8 +++-----\n 2 files changed, 8 insertions(+), 14 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 8cc72503cb..0df48365dc 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -5,27 +5,23 @@\n \n . ${0%/*}/lib.sh\n \n-P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION\n LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION\n \n case \"$jobname\" in\n linux-clang|linux-gcc)\n+\twget -qO - https://package.perforce.com/perforce.pubkey | sudo apt-key add -\n+\techo \"deb http://package.perforce.com/apt/ubuntu xenial release\" >perforce.list\n+\tsudo mv perforce.list /etc/apt/sources.list.d/\n \tsudo apt-add-repository -y \"ppa:ubuntu-toolchain-r/test\"\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install language-pack-is libsvn-perl apache2\n+\tsudo apt-get -q -y install language-pack-is libsvn-perl apache2 \\\n+\t\thelix-p4d\n \tcase \"$jobname\" in\n \tlinux-gcc)\n \t\tsudo apt-get -q -y install gcc-8\n \t\t;;\n \tesac\n \n-\tmkdir --parents \"$P4_PATH\"\n-\tpushd \"$P4_PATH\"\n-\t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4d\"\n-\t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4\"\n-\t\tchmod u+x p4d\n-\t\tchmod u+x p4\n-\tpopd\n \tmkdir --parents \"$GIT_LFS_PATH\"\n \tpushd \"$GIT_LFS_PATH\"\n \t\twget --quiet \"$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz\"\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex 44db2d5cbb..efcccfee6f 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -162,17 +162,15 @@ linux-clang|linux-gcc)\n \n \texport GIT_TEST_HTTPD=YesPlease\n \n-\t# The Linux build installs the defined dependency versions below.\n-\t# The OS X build installs much more recent versions, whichever\n+\t# The Linux build installs the defined dependency version below.\n+\t# The OS X build installs much more recent version, whichever\n \t# were recorded in the Homebrew database upon creating the OS X\n \t# image.\n \t# Keep that in mind when you encounter a broken OS X build!\n-\texport LINUX_P4_VERSION=\"16.2\"\n \texport LINUX_GIT_LFS_VERSION=\"1.5.2\"\n \n-\tP4_PATH=\"$HOME/custom/p4\"\n \tGIT_LFS_PATH=\"$HOME/custom/git-lfs\"\n-\texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n+\texport PATH=\"$GIT_LFS_PATH:$PATH\"\n \t;;\n osx-clang|osx-gcc)\n \tif [ \"$jobname\" = osx-gcc ]\n-- \n2.23.0.331.g4e51dcdf11\n\n"},{"id":"381950","messageId":"20190906105825.GD32087@szeder.dev","threadId":"51810","inReplyTo":"20190906102711.6401-1-szeder.dev@gmail.com","subject":"Re: [PATCH] ci: install P4 from package to fix build error","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-06T10:58:25Z","receivedAt":"2019-09-06T10:58:31Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote:\n> To test 'git-p4' in the Linux Clang and GCC build jobs we used to\n> install the 'p4' and 'p4d' binaries by directly downloading those\n> binaries from a Perforce filehost.  This has worked just fine ever\n> since we started using Travis CI [1], but during the last day or so\n> that filehost appeared to be gone: while its hostname still resolves,\n> the host doesn't seem to reply to any download request, it doesn't\n> even refuse the connection, and eventually our build jobs time out\n> [2].\n> \n> Now, this might be just a temporary glitch, but I'm afraid that it\n> isn't.\n\nWell, now would you believe it, while I was testing this patch (I even\nmade a gitgitgadget PR to run it on Azure Pipelines! :) and touching\nup its log message the good old Perforce filehost sprang back to life,\nand the CI build jobs now succeed again even without this patch.\n\n> Let's install P4 from the package repository, because this approach\n> seems to be simpler and more future proof.\n> \n> Note that we used to install an old P4 version (2016.2) in the Linux\n> build jobs, but with this change we'll install the most recent version\n> available in the Perforce package repository (currently 2019.1).\n\nSo I'm not quite sure whether we really want this patch.  It depends\non how important it is to test 'git-p4' with an old P4 version, but I\ndon't really have an opinion on that.\n\n"},{"id":"382105","messageId":"nycvar.QRO.7.76.6.1909100049180.5377@tvgsbejvaqbjf.bet","threadId":"51810","inReplyTo":"20190906105825.GD32087@szeder.dev","subject":"Re: [PATCH] ci: install P4 from package to fix build error","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-09-09T22:51:01Z","receivedAt":"2019-09-09T22:51:24Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\n\nOn Fri, 6 Sep 2019, SZEDER Gábor wrote:\n\n> On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote:\n> > To test 'git-p4' in the Linux Clang and GCC build jobs we used to\n> > install the 'p4' and 'p4d' binaries by directly downloading those\n> > binaries from a Perforce filehost.  This has worked just fine ever\n> > since we started using Travis CI [1], but during the last day or so\n> > that filehost appeared to be gone: while its hostname still resolves,\n> > the host doesn't seem to reply to any download request, it doesn't\n> > even refuse the connection, and eventually our build jobs time out\n> > [2].\n> >\n> > Now, this might be just a temporary glitch, but I'm afraid that it\n> > isn't.\n>\n> Well, now would you believe it, while I was testing this patch (I even\n> made a gitgitgadget PR to run it on Azure Pipelines! :) and touching\n> up its log message the good old Perforce filehost sprang back to life,\n> and the CI build jobs now succeed again even without this patch.\n\nSorry for being so slow with granting you access to GitGitGadget. FWIW\n_anybody_ who already was granted access can issue `/allow` commands, it\nis not just me.\n\n> > Let's install P4 from the package repository, because this approach\n> > seems to be simpler and more future proof.\n> >\n> > Note that we used to install an old P4 version (2016.2) in the Linux\n> > build jobs, but with this change we'll install the most recent version\n> > available in the Perforce package repository (currently 2019.1).\n>\n> So I'm not quite sure whether we really want this patch.  It depends\n> on how important it is to test 'git-p4' with an old P4 version, but I\n> don't really have an opinion on that.\n\nI'd rather have that patch. It seems to be a much better idea to use the\npackage management system than to rely on one host, especially when said\nhost already displayed hiccups.\n\nCiao,\nDscho\n"},{"id":"382126","messageId":"20190910122455.GH32087@szeder.dev","threadId":"51810","inReplyTo":"nycvar.QRO.7.76.6.1909100049180.5377@tvgsbejvaqbjf.bet","subject":"Re: [PATCH] ci: install P4 from package to fix build error","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-10T12:24:55Z","receivedAt":"2019-09-10T12:25:02Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote:\n> On Fri, 6 Sep 2019, SZEDER Gábor wrote:\n> \n> > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote:\n> > > To test 'git-p4' in the Linux Clang and GCC build jobs we used to\n> > > install the 'p4' and 'p4d' binaries by directly downloading those\n> > > binaries from a Perforce filehost.  This has worked just fine ever\n> > > since we started using Travis CI [1], but during the last day or so\n> > > that filehost appeared to be gone: while its hostname still resolves,\n> > > the host doesn't seem to reply to any download request, it doesn't\n> > > even refuse the connection, and eventually our build jobs time out\n> > > [2].\n> > >\n> > > Now, this might be just a temporary glitch, but I'm afraid that it\n> > > isn't.\n> >\n> > Well, now would you believe it, while I was testing this patch (I even\n> > made a gitgitgadget PR to run it on Azure Pipelines! :) and touching\n> > up its log message the good old Perforce filehost sprang back to life,\n> > and the CI build jobs now succeed again even without this patch.\n> \n> Sorry for being so slow with granting you access to GitGitGadget. FWIW\n> _anybody_ who already was granted access can issue `/allow` commands, it\n> is not just me.\n\nNo problem; I was only interested in the results of the Azure\nPipelines build, and that seemed to go well even without access.\n\n> > > Let's install P4 from the package repository, because this approach\n> > > seems to be simpler and more future proof.\n> > >\n> > > Note that we used to install an old P4 version (2016.2) in the Linux\n> > > build jobs, but with this change we'll install the most recent version\n> > > available in the Perforce package repository (currently 2019.1).\n> >\n> > So I'm not quite sure whether we really want this patch.  It depends\n> > on how important it is to test 'git-p4' with an old P4 version, but I\n> > don't really have an opinion on that.\n> \n> I'd rather have that patch. It seems to be a much better idea to use the\n> package management system than to rely on one host, especially when said\n> host already displayed hiccups.\n\nWell, I'm not so sure.  As far as I remember this was the first time\nthat this Perforce filehost was inaccessible and a simple \"Restart\njob\" could not rectify the situation, because it was inaccessible for\nabout a day or more.  OTOH, transient errors or timeouts from 'apt-get\nupdate' or 'install' from the official Ubuntu package repositories are\nnot uncommon (at least on Travis CI), although in those cases it's\nusually enough to just restart the errored job.\n\n"},{"id":"382165","messageId":"nycvar.QRO.7.76.6.1909111710050.5377@tvgsbejvaqbjf.bet","threadId":"51810","inReplyTo":"20190910122455.GH32087@szeder.dev","subject":"Re: [PATCH] ci: install P4 from package to fix build error","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-09-11T15:12:31Z","receivedAt":"2019-09-11T15:12:54Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 10 Sep 2019, SZEDER Gábor wrote:\n\n> On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote:\n> > On Fri, 6 Sep 2019, SZEDER Gábor wrote:\n> >\n> > > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote:\n>\n> > > > Let's install P4 from the package repository, because this\n> > > > approach seems to be simpler and more future proof.\n> > > >\n> > > > Note that we used to install an old P4 version (2016.2) in the\n> > > > Linux build jobs, but with this change we'll install the most\n> > > > recent version available in the Perforce package repository\n> > > > (currently 2019.1).\n> > >\n> > > So I'm not quite sure whether we really want this patch.  It\n> > > depends on how important it is to test 'git-p4' with an old P4\n> > > version, but I don't really have an opinion on that.\n> >\n> > I'd rather have that patch. It seems to be a much better idea to use\n> > the package management system than to rely on one host, especially\n> > when said host already displayed hiccups.\n>\n> Well, I'm not so sure.  As far as I remember this was the first time\n> that this Perforce filehost was inaccessible and a simple \"Restart\n> job\" could not rectify the situation, because it was inaccessible for\n> about a day or more.\n\nRight.\n\n> OTOH, transient errors or timeouts from 'apt-get update' or 'install'\n> from the official Ubuntu package repositories are not uncommon (at\n> least on Travis CI), although in those cases it's usually enough to\n> just restart the errored job.\n\nMy impression precisely. I trust the Ubuntu package servers to have\ntransient, _short-lived_ problems ;-)\n\nCiao,\nDscho\n"},{"id":"382208","messageId":"20190912110955.GM32087@szeder.dev","threadId":"51810","inReplyTo":"nycvar.QRO.7.76.6.1909111710050.5377@tvgsbejvaqbjf.bet","subject":"Re: [PATCH] ci: install P4 from package to fix build error","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2019-09-12T11:09:55Z","receivedAt":"2019-09-12T11:10:00Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Sep 11, 2019 at 05:12:31PM +0200, Johannes Schindelin wrote:\n> On Tue, 10 Sep 2019, SZEDER Gábor wrote:\n> \n> > On Tue, Sep 10, 2019 at 12:51:01AM +0200, Johannes Schindelin wrote:\n> > > On Fri, 6 Sep 2019, SZEDER Gábor wrote:\n> > >\n> > > > On Fri, Sep 06, 2019 at 12:27:11PM +0200, SZEDER Gábor wrote:\n> >\n> > > > > Let's install P4 from the package repository, because this\n> > > > > approach seems to be simpler and more future proof.\n> > > > >\n> > > > > Note that we used to install an old P4 version (2016.2) in the\n> > > > > Linux build jobs, but with this change we'll install the most\n> > > > > recent version available in the Perforce package repository\n> > > > > (currently 2019.1).\n> > > >\n> > > > So I'm not quite sure whether we really want this patch.  It\n> > > > depends on how important it is to test 'git-p4' with an old P4\n> > > > version, but I don't really have an opinion on that.\n> > >\n> > > I'd rather have that patch. It seems to be a much better idea to use\n> > > the package management system than to rely on one host, especially\n> > > when said host already displayed hiccups.\n> >\n> > Well, I'm not so sure.  As far as I remember this was the first time\n> > that this Perforce filehost was inaccessible and a simple \"Restart\n> > job\" could not rectify the situation, because it was inaccessible for\n> > about a day or more.\n> \n> Right.\n> \n> > OTOH, transient errors or timeouts from 'apt-get update' or 'install'\n> > from the official Ubuntu package repositories are not uncommon (at\n> > least on Travis CI), although in those cases it's usually enough to\n> > just restart the errored job.\n> \n> My impression precisely. I trust the Ubuntu package servers to have\n> transient, _short-lived_ problems ;-)\n\nHeh, that's true, but note that we won't install P4 from the Ubuntu\npackage servers, but from the Perforce package repository, so I doubt\nwe can expect any improvements in this regard.\n\nOTOH, instead of issuing only two HTTP requests to download those two\nbinaries we'll issue a total of 9 requests (1 pubkey, 2 package lists,\n6 packages) that could go wrong.\n\nI'm undecided, and at the very least the proposed commit message\nshould be updated.\n\n"}]}