{"thread":{"id":"47090","subject":"[PATCH] travis-ci: fix running P4 and Git LFS tests in Linux build jobs","startedAt":"2017-11-01T11:55:54Z","lastAt":"2017-12-28T11:17:54Z","messageCount":44,"participants":["SZEDER Gábor","Lars Schneider","Johannes Schindelin","Eric Sunshine","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"331560","messageId":"20171101115535.15074-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":null,"subject":"[PATCH] travis-ci: fix running P4 and Git LFS tests in Linux build jobs","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-11-01T11:55:35Z","receivedAt":"2017-11-01T11:55:54Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Linux build jobs on Travis CI skip the P4 and Git LFS tests since\ncommit 657343a60 (travis-ci: move Travis CI code into dedicated\nscripts, 2017-09-10), claiming there are no P4 or Git LFS installed.\n\nThe reason is that P4 and Git LFS binaries are not installed to a\ndirectory in the default $PATH, but their directories are prepended to\n$PATH. This worked just fine before said commit, because $PATH was\nset in a scriptlet embedded in our '.travis.yml', thus its new value\nwas visible during the rest of the build job. However, after these\nembedded scriptlets were moved into dedicated scripts executed in\nseparate shell processes, any variable set in one of those scripts is\nonly visible in that single script but not in any of the others. In\nthis case, 'ci/install-dependencies.sh' downloads P4 and Git LFS and\nmodifies $PATH, but to no effect, because 'ci/run-tests.sh' only sees\nTravis CI's default $PATH.\n\nMove adjusting $PATH to 'ci/lib-travisci.sh', which is sourced in all\nother 'ci/' scripts, so all those scripts will see the updated $PATH\nvalue.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/install-dependencies.sh | 10 ++++------\n ci/lib-travisci.sh | 8 ++++++++\n 2 files changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex a29246af3..5bd06fe90 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -12,20 +12,18 @@ case \"${TRAVIS_OS_NAME:-linux}\" in\n linux)\n \texport GIT_TEST_HTTPD=YesPlease\n \n-\tmkdir --parents custom/p4\n-\tpushd custom/p4\n+\tmkdir --parents \"$P4_PATH\"\n+\tpushd \"$P4_PATH\"\n \t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4d\"\n \t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4\"\n \t\tchmod u+x p4d\n \t\tchmod u+x p4\n-\t\texport PATH=\"$(pwd):$PATH\"\n \tpopd\n-\tmkdir --parents custom/git-lfs\n-\tpushd custom/git-lfs\n+\tmkdir --parents \"$GIT_LFS_PATH\"\n+\tpushd \"$GIT_LFS_PATH\"\n \t\twget --quiet \"$LFSWHENCE/git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz\"\n \t\ttar --extract --gunzip --file \"git-lfs-linux-amd64-$LINUX_GIT_LFS_VERSION.tar.gz\"\n \t\tcp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .\n-\t\texport PATH=\"$(pwd):$PATH\"\n \tpopd\n \t;;\n osx)\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex b3ed0a0dd..ac05f1f46 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -26,3 +26,11 @@ skip_branch_tip_with_tag () {\n set -e\n \n skip_branch_tip_with_tag\n+\n+case \"${TRAVIS_OS_NAME:-linux}\" in\n+linux)\n+\tP4_PATH=\"$(pwd)/custom/p4\"\n+\tGIT_LFS_PATH=\"$(pwd)/custom/git-lfs\"\n+\texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n+\t;;\n+esac\n-- \n2.15.0.67.gb67a46776\n\n"},{"id":"334643","messageId":"20171211233446.10596-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171101115535.15074-1-szeder.dev@gmail.com","subject":"[PATCH 0/4] travis-ci: clean up setting environment variables","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-11T23:34:42Z","receivedAt":"2017-12-11T23:35:26Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"(Was: [PATCH] travis-ci: fix running P4 and Git LFS tests in Linux build\njobs)\n\nOn Wed, Nov 1, 2017 at 12:55 PM, SZEDER Gábor wrote:\n> However, after these\n> embedded scriptlets were moved into dedicated scripts executed in\n> separate shell processes, any variable set in one of those scripts is\n> only visible in that single script but not in any of the others.\n\n> diff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\n> index a29246af3..5bd06fe90 100755\n> --- a/ci/install-dependencies.sh\n> +++ b/ci/install-dependencies.sh\n> @@ -12,20 +12,18 @@ case \"${TRAVIS_OS_NAME:-linux}\" in\n> linux)\n> export GIT_TEST_HTTPD=YesPlease\n\nAstute readers ;) might have been wondering what's the deal with this\nenvironment variable in the patch context, since this won't have any\neffect outside of this script, either. Alas, it's not as easy as moving\nsetting GIT_TEST_HTTPD to 'ci/lib-travisci.sh', like the quoted patch\ndid with paths to P4 and Git LFS, because then it would be set for the 32\nbit Linux build job which runs everything as root, thus can't run https\ntests.\n\nThis patch series deals with this by adding means that 'ci/*' scripts\ncan use to identify which build job they are taking part in (patch 3),\nand then setting environment variables in 'ci/lib-travisci.sh', where we\nhave now more freedom to set a specific variable only for specific build\njobs (patches 3 and 4).\n\n\nThis patch series conflicts with the last patch in Thomas' split index\nfix series.\n\n https://public-inbox.org/git/20171210212202.28231-4-t.gummerer@gmail.com/\n\nThe conflict is not overly difficult, but to resolve it we should first\ncome up with a reasonable job name for that build job, e.g. something\nlike \"misc-knobs\" or whatever, because the combination\n\"GETTEXT_POISON-SPLIT_INDEX\" is just too long to exist and won't scale\nif we enable further knobs in the future.\n\n\nSZEDER Gábor (4):\n travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output\n travis-ci: introduce a $jobname variable for 'ci/*' scripts\n travis-ci: move setting environment variables to 'ci/lib-travisci.sh'\n travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh'\n\n .travis.yml | 26 +++++---------------------\n ci/install-dependencies.sh | 8 +++-----\n ci/lib-travisci.sh | 34 +++++++++++++++++++++++++++++++---\n 3 files changed, 39 insertions(+), 29 deletions(-)\n\n-- \n2.15.1.421.gc469ca1de\n\n"},{"id":"334644","messageId":"20171211233446.10596-2-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-1-szeder.dev@gmail.com","subject":"[PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-11T23:34:43Z","receivedAt":"2017-12-11T23:35:29Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"While the build logic was embedded in our '.travis.yml', Travis CI\nused to produce a nice trace log including all commands executed in\nthose embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\ncode into dedicated scripts, 2017-09-10), however, we only see the\nname of the dedicated scripts, but not what those scripts are actually\ndoing, resulting in a less useful trace log. A patch later in this\nseries will move setting environment variables from '.travis.yml' to\nthe 'ci/*' scripts, so not even those will be included in the trace\nlog.\n\nUse 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other\n'ci/*' scripts, so we get trace log about the commands executed in all\nof those scripts.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/lib-travisci.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex ac05f1f46..a0c8ae03f 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {\n \n # Set 'exit on error' for all CI scripts to let the caller know that\n # something went wrong\n-set -e\n+set -ex\n \n skip_branch_tip_with_tag\n \n-- \n2.15.1.421.gc469ca1de\n\n"},{"id":"334645","messageId":"20171211233446.10596-3-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-1-szeder.dev@gmail.com","subject":"[PATCH 2/4] travis-ci: introduce a $jobname variable for 'ci/*' scripts","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-11T23:34:44Z","receivedAt":"2017-12-11T23:35:34Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"A couple of 'ci/*' scripts are shared between different build jobs:\n'ci/lib-travisci.sh', being a common library, is sourced from almost\nevery script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'\nand 'ci/run-tests.sh' are shared between the \"regular\" GCC and Clang\nLinux and OSX build jobs, and the latter two scripts are used in the\nGETTEXT_POISON Linux build job as well.\n\nOur builds could benefit from these shared scripts being able to\neasily tell which build job they are taking part in. Now, it's\nalready quite easy to tell apart Linux vs OSX and GCC vs Clang build\njobs, but it gets trickier with all the additional Linux-based build\njobs included explicitly in the build matrix.\n\nUnfortunately, Travis CI doesn't provide much help in this regard.\nThe closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of\nwhich is two dot-separated integers, where the second integer\nindicates a particular build job. While it would be possible to use\nthat second number to identify the build job in our shared scripts, it\ndoesn't seem like a good idea to rely on that:\n\n - Though the build job numbering sequence seems to be stable so far,\n Travis CI's documentation doesn't explicitly states that it is\n indeed stable and will remain so in the future. And even if it\n were stable,\n\n - if we were to remove or insert a build job in the middle, then the\n job numbers of all subsequent build jobs would change accordingly.\n\nSo roll our own means of simple build job identification and introduce\nthe $jobname environment variable in our builds, setting it in the\nenvironments of the explicitly included jobs in '.travis.yml', while\nconstructing one in 'ci/lib-travisci.sh' as the combination of the OS\nand compiler name for the GCC and Clang Linux and OSX build jobs. Use\n$jobname instead of $TRAVIS_OS_NAME in scripts taking different\nactions based on the OS and build job (when installing P4 and Git LFS\ndependencies and including them in $PATH). The following two patches\nwill also rely on $jobname.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 10 +++++-----\n ci/install-dependencies.sh | 6 +++---\n ci/lib-travisci.sh | 9 +++++++--\n 3 files changed, 15 insertions(+), 10 deletions(-)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 281f101f3..88435e11c 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -39,12 +39,12 @@ env:\n \n matrix:\n include:\n- - env: GETTEXT_POISON=YesPlease\n+ - env: jobname=GETTEXT_POISON GETTEXT_POSION=YesPlease\n os: linux\n compiler:\n addons:\n before_install:\n- - env: Windows\n+ - env: jobname=Windows\n os: linux\n compiler:\n addons:\n@@ -55,7 +55,7 @@ matrix:\n test \"$TRAVIS_REPO_SLUG\" != \"git/git\" ||\n ci/run-windows-build.sh $TRAVIS_BRANCH $(git rev-parse HEAD)\n after_failure:\n- - env: Linux32\n+ - env: jobname=Linux32\n os: linux\n compiler:\n services:\n@@ -63,7 +63,7 @@ matrix:\n before_install:\n before_script:\n script: ci/run-linux32-docker.sh\n- - env: Static Analysis\n+ - env: jobname=StaticAnalysis\n os: linux\n compiler:\n addons:\n@@ -74,7 +74,7 @@ matrix:\n before_script:\n script: ci/run-static-analysis.sh\n after_failure:\n- - env: Documentation\n+ - env: jobname=Documentation\n os: linux\n compiler:\n addons:\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 5bd06fe90..468788566 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -8,8 +8,8 @@\n P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION\n LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION\n \n-case \"${TRAVIS_OS_NAME:-linux}\" in\n-linux)\n+case \"$jobname\" in\n+linux-clang|linux-gcc)\n \texport GIT_TEST_HTTPD=YesPlease\n \n \tmkdir --parents \"$P4_PATH\"\n@@ -26,7 +26,7 @@ linux)\n \t\tcp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .\n \tpopd\n \t;;\n-osx)\n+osx-clang|osx-gcc)\n \tbrew update --quiet\n \t# Uncomment this if you want to run perf tests:\n \t# brew install gnu-time\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex a0c8ae03f..b60e93797 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -27,8 +27,13 @@ set -ex\n \n skip_branch_tip_with_tag\n \n-case \"${TRAVIS_OS_NAME:-linux}\" in\n-linux)\n+if test -z \"$jobname\"\n+then\n+\tjobname=\"$TRAVIS_OS_NAME-$CC\"\n+fi\n+\n+case \"$jobname\" in\n+linux-clang|linux-gcc)\n \tP4_PATH=\"$(pwd)/custom/p4\"\n \tGIT_LFS_PATH=\"$(pwd)/custom/git-lfs\"\n \texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n-- \n2.15.1.421.gc469ca1de\n\n"},{"id":"334646","messageId":"20171211233446.10596-4-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-1-szeder.dev@gmail.com","subject":"[PATCH 3/4] travis-ci: move setting environment variables to 'ci/lib-travisci.sh'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-11T23:34:45Z","receivedAt":"2017-12-11T23:35:40Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Our '.travis.yml's 'env.global' section sets a bunch of environment\nvariables for all build jobs, though none of them actually affects all\nbuild jobs. It's convenient for us, and in most cases it works just\nfine, because irrelevant environment variables are simply ignored.\n\nHowever, $GIT_SKIP_TESTS is an exception: it tells the test harness to\nskip the two test scripts that are prone to occasional failures on\nOSX, but as it's set for all build jobs those tests are not run in any\nof the build jobs that are capable to run them reliably, either.\n\nTherefore $GIT_SKIP_TESTS should only be set in the OSX build jobs,\nbut those build jobs are included in the build matrix implicitly (i.e.\nby combining the matrix keys 'os' and 'compiler'), and there is no way\nto set an environment variable only for a subset of those implicit\nbuild jobs. (Unless we were to add new scriptlets to '.travis.yml',\nwhich is exactly the opposite direction that we took with commit\n657343a60 (travis-ci: move Travis CI code into dedicated scripts,\n2017-09-10)).\n\nSo move setting $GIT_SKIP_TESTS to 'ci/lib-travisci.sh', where it can\ntrivially be set only for the OSX build jobs.\n\nFurthermore, move setting all other environment variables from\n'.travis.yml' to 'ci/lib-travisci.sh', too, because a couple of\nenvironment variables are already set there, and this way all\nenvironment variables will be set in the same place. All the logic\ncontrolling our builds is already in the 'ci/*' scripts anyway, so\nthere is really no good reason to keep the environment variables\nseparately.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 18 +-----------------\n ci/lib-travisci.sh | 21 +++++++++++++++++++++\n 2 files changed, 22 insertions(+), 17 deletions(-)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 88435e11c..7c9aa0557 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -21,25 +21,9 @@ addons:\n - git-svn\n - apache2\n \n-env:\n- global:\n- - DEVELOPER=1\n- # The Linux build installs the defined dependency versions below.\n- # The OS X build installs the latest available versions. Keep that\n- # in mind when you encounter a broken OS X build!\n- - LINUX_P4_VERSION=\"16.2\"\n- - LINUX_GIT_LFS_VERSION=\"1.5.2\"\n- - DEFAULT_TEST_TARGET=prove\n- - GIT_PROVE_OPTS=\"--timer --jobs 3 --state=failed,slow,save\"\n- - GIT_TEST_OPTS=\"--verbose-log\"\n- - GIT_TEST_CLONE_2GB=YesPlease\n- # t9810 occasionally fails on Travis CI OS X\n- # t9816 occasionally fails with \"TAP out of sequence errors\" on Travis CI OS X\n- - GIT_SKIP_TESTS=\"t9810 t9816\"\n-\n matrix:\n include:\n- - env: jobname=GETTEXT_POISON GETTEXT_POSION=YesPlease\n+ - env: jobname=GETTEXT_POISON\n os: linux\n compiler:\n addons:\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex b60e93797..e85571298 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -32,10 +32,31 @@ then\n \tjobname=\"$TRAVIS_OS_NAME-$CC\"\n fi\n \n+export DEVELOPER=1\n+export DEFAULT_TEST_TARGET=prove\n+export GIT_PROVE_OPTS=\"--timer --jobs 3 --state=failed,slow,save\"\n+export GIT_TEST_OPTS=\"--verbose-log\"\n+export GIT_TEST_CLONE_2GB=YesPlease\n+\n case \"$jobname\" in\n linux-clang|linux-gcc)\n+\t# The Linux build installs the defined dependency versions below.\n+\t# The OS X build installs the latest available versions. Keep that\n+\t# in mind when you encounter a broken OS X build!\n+\texport LINUX_P4_VERSION=\"16.2\"\n+\texport LINUX_GIT_LFS_VERSION=\"1.5.2\"\n+\n \tP4_PATH=\"$(pwd)/custom/p4\"\n \tGIT_LFS_PATH=\"$(pwd)/custom/git-lfs\"\n \texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n \t;;\n+osx-clang|osx-gcc)\n+\t# t9810 occasionally fails on Travis CI OS X\n+\t# t9816 occasionally fails with \"TAP out of sequence errors\" on\n+\t# Travis CI OS X\n+\texport GIT_SKIP_TESTS=\"t9810 t9816\"\n+\t;;\n+GETTEXT_POISON)\n+\texport GETTEXT_POISON=YesPlease\n+\t;;\n esac\n-- \n2.15.1.421.gc469ca1de\n\n"},{"id":"334647","messageId":"20171211233446.10596-5-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-1-szeder.dev@gmail.com","subject":"[PATCH 4/4] travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-11T23:34:46Z","receivedAt":"2017-12-11T23:35:45Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Commit 657343a60 (travis-ci: move Travis CI code into dedicated\nscripts, 2017-09-10) converted '.travis.yml's default 'before_install'\nscriptlet to the 'ci/install-dependencies.sh' script, and while doing\nso moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang\nLinux build jobs to that script. This is wrong for two reasons:\n\n - The purpose of that script is, as its name suggests, to install\n dependencies, not to set any environment variables influencing\n which tests should be run (though, arguably, this was already an\n issue with the original 'before_install' scriptlet).\n\n - Setting the variable has no effect anymore, because that script is\n run in a separate shell process, and the variable won't be visible\n in any of the other scripts, notably in 'ci/run-tests.sh'\n responsible for, well, running the tests.\n\nLuckily, this didn't have a negative effect on our Travis CI build\njobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to\n\"auto\" and a functioning web server was installed in those Linux build\njobs, so the httpd tests were run anyway.\n\nApparently the httpd tests run just fine without GIT_TEST_HTTPD being\nset, therefore we could simply remove this environment variable.\nHowever, if a bug were to creep in to change the Travis CI build\nenvironment to run the tests as root or to not install Apache, then\nthe httpd tests would be skipped and the build job would still\nsucceed. We would only notice if someone actually were to look\nthrough the build job's trace log; but who would look at the trace log\nof a successful build job?!\n\nSince httpd tests are important, we do want to run them and we want to\nbe loudly reminded if they can't be run. Therefore, move setting\nGIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs\nto 'ci/lib-travisci.sh' to ensure that the build job fails when the\nhttpd tests can't be run. (We could set it in 'ci/run-tests.sh' just\nas well, but it's better to keep all environment variables in one\nplace in 'ci/lib-travisci.sh'.)\n\nSigned-off-by: SZEDER Gábor \n---\n ci/install-dependencies.sh | 2 --\n ci/lib-travisci.sh | 2 ++\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 468788566..75a9fd247 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -10,8 +10,6 @@ LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VE\n \n case \"$jobname\" in\n linux-clang|linux-gcc)\n-\texport GIT_TEST_HTTPD=YesPlease\n-\n \tmkdir --parents \"$P4_PATH\"\n \tpushd \"$P4_PATH\"\n \t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4d\"\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex e85571298..331d3eb3a 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -40,6 +40,8 @@ export GIT_TEST_CLONE_2GB=YesPlease\n \n case \"$jobname\" in\n linux-clang|linux-gcc)\n+\texport GIT_TEST_HTTPD=YesPlease\n+\n \t# The Linux build installs the defined dependency versions below.\n \t# The OS X build installs the latest available versions. Keep that\n \t# in mind when you encounter a broken OS X build!\n-- \n2.15.1.421.gc469ca1de\n\n"},{"id":"334686","messageId":"9AAC2BCC-AA06-46F9-8E6E-1D99CD959FDD@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-2-szeder.dev@gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-12T18:00:23Z","receivedAt":"2017-12-12T18:00:34Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 12 Dec 2017, at 00:34, SZEDER Gábor wrote:\n> \n> While the build logic was embedded in our '.travis.yml', Travis CI\n> used to produce a nice trace log including all commands executed in\n> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n> code into dedicated scripts, 2017-09-10), however, we only see the\n> name of the dedicated scripts, but not what those scripts are actually\n> doing, resulting in a less useful trace log. A patch later in this\n> series will move setting environment variables from '.travis.yml' to\n> the 'ci/*' scripts, so not even those will be included in the trace\n> log.\n> \n> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other\n> 'ci/*' scripts, so we get trace log about the commands executed in all\n> of those scripts.\n\nI kind of did that intentionally to avoid clutter in the logs.\nHowever, I also agree with your reasoning. Therefore, the change\nlooks good to me!\n\nThanks,\nLars\n\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> ci/lib-travisci.sh | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n> \n> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n> index ac05f1f46..a0c8ae03f 100755\n> --- a/ci/lib-travisci.sh\n> +++ b/ci/lib-travisci.sh\n> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {\n> \n> # Set 'exit on error' for all CI scripts to let the caller know that\n> # something went wrong\n> -set -e\n> +set -ex\n> \n> skip_branch_tip_with_tag\n> \n> -- \n> 2.15.1.421.gc469ca1de\n> \n\n"},{"id":"334689","messageId":"CAM0VKjkVw8QeDErDg9aXcQ1sAgY34eBEeA0QJGamvEEBOG8y9w@mail.gmail.com","threadId":"47090","inReplyTo":"9AAC2BCC-AA06-46F9-8E6E-1D99CD959FDD@gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-12T18:43:18Z","receivedAt":"2017-12-12T18:43:25Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider\n wrote:\n>\n>> On 12 Dec 2017, at 00:34, SZEDER Gábor wrote:\n>>\n>> While the build logic was embedded in our '.travis.yml', Travis CI\n>> used to produce a nice trace log including all commands executed in\n>> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n>> code into dedicated scripts, 2017-09-10), however, we only see the\n>> name of the dedicated scripts, but not what those scripts are actually\n>> doing, resulting in a less useful trace log. A patch later in this\n>> series will move setting environment variables from '.travis.yml' to\n>> the 'ci/*' scripts, so not even those will be included in the trace\n>> log.\n>>\n>> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other\n>> 'ci/*' scripts, so we get trace log about the commands executed in all\n>> of those scripts.\n>\n> I kind of did that intentionally to avoid clutter in the logs.\n> However, I also agree with your reasoning. Therefore, the change\n> looks good to me!\n\nGreat, 'cause I'm starting to have second thoughts about this change :)\n\nIt sure helped a lot while I worked on this patch series and a couple of\nother Travis CI related patches (will submit them later)... OTOH it\ndefinitely creates clutter in the trace log. The worst offender might\nbe 'ci/print-test-failures.sh', which iterates over all\n't/test-results/*.exit' files to find which tests failed and to show\ntheir output, and 'set -x' makes every iteration visible. And we have\nabout 800 tests, which means 800 iterations. Yuck.\n\nPerhaps we should use other means to show what's going on instead, e.g.\nuse more 'echo's and '--verbose' options, or just avoid using '--quiet'.\nAnd if some brave souls really want to tweak '.travis.yml' or the 'ci/*'\nscripts, then they can set 'set -x' for themselves during development...\nas the patch below shows it's easy enough, just a single character :)\n\n\nGábor\n\n\n>>\n>> Signed-off-by: SZEDER Gábor \n>> ---\n>> ci/lib-travisci.sh | 2 +-\n>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>\n>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n>> index ac05f1f46..a0c8ae03f 100755\n>> --- a/ci/lib-travisci.sh\n>> +++ b/ci/lib-travisci.sh\n>> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {\n>>\n>> # Set 'exit on error' for all CI scripts to let the caller know that\n>> # something went wrong\n>> -set -e\n>> +set -ex\n>>\n>> skip_branch_tip_with_tag\n>>\n>> --\n>> 2.15.1.421.gc469ca1de\n>>\n>\n"},{"id":"334785","messageId":"77E6AE23-44FF-42E0-94FF-C4B35D913D5B@gmail.com","threadId":"47090","inReplyTo":"CAM0VKjkVw8QeDErDg9aXcQ1sAgY34eBEeA0QJGamvEEBOG8y9w@mail.gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-13T23:10:17Z","receivedAt":"2017-12-13T23:10:25Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 12 Dec 2017, at 19:43, SZEDER Gábor wrote:\n> \n> On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider\n> wrote:\n>> \n>>> On 12 Dec 2017, at 00:34, SZEDER Gábor wrote:\n>>> \n>>> While the build logic was embedded in our '.travis.yml', Travis CI\n>>> used to produce a nice trace log including all commands executed in\n>>> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n>>> code into dedicated scripts, 2017-09-10), however, we only see the\n>>> name of the dedicated scripts, but not what those scripts are actually\n>>> doing, resulting in a less useful trace log. A patch later in this\n>>> series will move setting environment variables from '.travis.yml' to\n>>> the 'ci/*' scripts, so not even those will be included in the trace\n>>> log.\n>>> \n>>> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other\n>>> 'ci/*' scripts, so we get trace log about the commands executed in all\n>>> of those scripts.\n>> \n>> I kind of did that intentionally to avoid clutter in the logs.\n>> However, I also agree with your reasoning. Therefore, the change\n>> looks good to me!\n> \n> Great, 'cause I'm starting to have second thoughts about this change :)\n> \n> It sure helped a lot while I worked on this patch series and a couple of\n> other Travis CI related patches (will submit them later)... OTOH it\n> definitely creates clutter in the trace log. The worst offender might\n> be 'ci/print-test-failures.sh', which iterates over all\n> 't/test-results/*.exit' files to find which tests failed and to show\n> their output, and 'set -x' makes every iteration visible. And we have\n> about 800 tests, which means 800 iterations. Yuck.\n> \n> Perhaps we should use other means to show what's going on instead, e.g.\n> use more 'echo's and '--verbose' options, or just avoid using '--quiet'.\n> And if some brave souls really want to tweak '.travis.yml' or the 'ci/*'\n> scripts, then they can set 'set -x' for themselves during development...\n> as the patch below shows it's easy enough, just a single character :)\n\nHm... in that case. Would it be an option to \"set -x\" only in the header\nof \"install-dependencies.sh\"?\n\nIn \"lib-travisci.sh\" we could keep your \"set -x\" and execute\n\"set +x\" at the end of the file. Wouldn't that give us the \ninteresting traces without much clutter (e.g. what is $PATH etc)?\n\n- Lars\n\n\n> \n> \n> Gábor\n> \n> \n>>> \n>>> Signed-off-by: SZEDER Gábor \n>>> ---\n>>> ci/lib-travisci.sh | 2 +-\n>>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>> \n>>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n>>> index ac05f1f46..a0c8ae03f 100755\n>>> --- a/ci/lib-travisci.sh\n>>> +++ b/ci/lib-travisci.sh\n>>> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {\n>>> \n>>> # Set 'exit on error' for all CI scripts to let the caller know that\n>>> # something went wrong\n>>> -set -e\n>>> +set -ex\n>>> \n>>> skip_branch_tip_with_tag\n>>> \n>>> --\n>>> 2.15.1.421.gc469ca1de\n>>> \n>> \n\n"},{"id":"334865","messageId":"CAM0VKjkv0XqHf8s94Wj2DHsD49gTP0M6HU2AY=36Jj-5SYhS9A@mail.gmail.com","threadId":"47090","inReplyTo":"77E6AE23-44FF-42E0-94FF-C4B35D913D5B@gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-14T23:51:07Z","receivedAt":"2017-12-14T23:51:13Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Thu, Dec 14, 2017 at 12:10 AM, Lars Schneider\n wrote:\n>\n>> On 12 Dec 2017, at 19:43, SZEDER Gábor wrote:\n>>\n>> On Tue, Dec 12, 2017 at 7:00 PM, Lars Schneider\n>> wrote:\n>>>\n>>>> On 12 Dec 2017, at 00:34, SZEDER Gábor wrote:\n>>>>\n>>>> While the build logic was embedded in our '.travis.yml', Travis CI\n>>>> used to produce a nice trace log including all commands executed in\n>>>> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n>>>> code into dedicated scripts, 2017-09-10), however, we only see the\n>>>> name of the dedicated scripts, but not what those scripts are actually\n>>>> doing, resulting in a less useful trace log. A patch later in this\n>>>> series will move setting environment variables from '.travis.yml' to\n>>>> the 'ci/*' scripts, so not even those will be included in the trace\n>>>> log.\n>>>>\n>>>> Use 'set -x' in 'ci/lib-travisci.sh', which is sourced in most other\n>>>> 'ci/*' scripts, so we get trace log about the commands executed in all\n>>>> of those scripts.\n>>>\n>>> I kind of did that intentionally to avoid clutter in the logs.\n>>> However, I also agree with your reasoning. Therefore, the change\n>>> looks good to me!\n>>\n>> Great, 'cause I'm starting to have second thoughts about this change :)\n>>\n>> It sure helped a lot while I worked on this patch series and a couple of\n>> other Travis CI related patches (will submit them later)... OTOH it\n>> definitely creates clutter in the trace log. The worst offender might\n>> be 'ci/print-test-failures.sh', which iterates over all\n>> 't/test-results/*.exit' files to find which tests failed and to show\n>> their output, and 'set -x' makes every iteration visible. And we have\n>> about 800 tests, which means 800 iterations. Yuck.\n>>\n>> Perhaps we should use other means to show what's going on instead, e.g.\n>> use more 'echo's and '--verbose' options, or just avoid using '--quiet'.\n>> And if some brave souls really want to tweak '.travis.yml' or the 'ci/*'\n>> scripts, then they can set 'set -x' for themselves during development...\n>> as the patch below shows it's easy enough, just a single character :)\n>\n> Hm... in that case. Would it be an option to \"set -x\" only in the header\n> of \"install-dependencies.sh\"?\n>\n> In \"lib-travisci.sh\" we could keep your \"set -x\" and execute\n> \"set +x\" at the end of the file. Wouldn't that give us the\n> interesting traces without much clutter (e.g. what is $PATH etc)?\n\nHmm, that's an idea worth considering.\n\nScripts like 'run-build.sh', 'run-tests.sh' and 'run-static-analysis.sh'\ndo basically nothing more than run make with different targets, so on\none hand 'set -x' doesn't cause any clutter in the trace log, on the\nother hand there is no benefit from it either.\n'run-linux32-docker.sh' runs docker (the command) twice, so it's\nbasically in the same boat.\n\nI think both 'lib-travisci.sh' and 'install-dependencies.sh' benefit\nfrom 'set -x'.\nSo does 'test-documentation.sh': it executes about 15 commands, among\nthem a bunch of 'test -s ' which fail quietly. With 'set -x' we\nwould see the last executed command and know that that's the one that\nfailed.\n\nAs mentioned above, 'print-test-failures.sh' definitely suffers from\n'set -x'.\n\nThere is a lot going on in 'run-windows-build.sh', so the output of 'set\n-x' might be useful or might be considered too much clutter, I don't\nknow. I put Dscho on Cc, I think it's mainly his call.\n\nSo it seems that there are more scripts that would benefit from tracing\nexecuted command using 'set -x' than scripts that would suffer because\nof it, and it doesn't matter for the rest. This means we could issue a\n'set -x' in 'lib-travisci.sh' and disable it only in\n'print-test-failures.sh'.\n\nThere is one thing that triggers my OCD: whenever we echo something, it\nends up being duplicated in the trace log, e.g.:\n\n + echo foo bar baz\n foo bar baz\n\nWe could workaround it by writing 'echo \"\" >/dev/null', but it\nfeels hackish and I'm not sure it's worth it.\n\n\nGábor\n\n>>>> Signed-off-by: SZEDER Gábor \n>>>> ---\n>>>> ci/lib-travisci.sh | 2 +-\n>>>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>>>\n>>>> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n>>>> index ac05f1f46..a0c8ae03f 100755\n>>>> --- a/ci/lib-travisci.sh\n>>>> +++ b/ci/lib-travisci.sh\n>>>> @@ -23,7 +23,7 @@ skip_branch_tip_with_tag () {\n>>>>\n>>>> # Set 'exit on error' for all CI scripts to let the caller know that\n>>>> # something went wrong\n>>>> -set -e\n>>>> +set -ex\n>>>>\n>>>> skip_branch_tip_with_tag\n>>>>\n>>>> --\n>>>> 2.15.1.421.gc469ca1de\n>>>>\n>>>\n>\n"},{"id":"334875","messageId":"alpine.DEB.2.21.1.1712151308230.406@MININT-6BKU6QN.europe.corp.microsoft.com","threadId":"47090","inReplyTo":"CAM0VKjkv0XqHf8s94Wj2DHsD49gTP0M6HU2AY=36Jj-5SYhS9A@mail.gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-12-15T12:10:54Z","receivedAt":"2017-12-15T12:11:03Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\n> There is a lot going on in 'run-windows-build.sh', so the output of 'set\n> -x' might be useful or might be considered too much clutter, I don't\n> know. I put Dscho on Cc, I think it's mainly his call.\n\nCertainly it might be useful.\n\nHowever, please make sure that the secret token is not leaked that way.\nLike, *really* sure. Due to the failure of Git to use a portable and\nperformant test suite, it does take about 90 minutes to build and test a\nrevision, therefore it would be very easy to DOS my build system, and I\nreally, really need it not to be DOSed because I use it in my day job, too.\n\nCiao,\nDscho\n"},{"id":"334878","messageId":"CAM0VKjmpJ_J+GjJ3PyU_pxsWx85L6cPa2tLM0xJx3cJu2T87Zg@mail.gmail.com","threadId":"47090","inReplyTo":"alpine.DEB.2.21.1.1712151308230.406@MININT-6BKU6QN.europe.corp.microsoft.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-15T13:06:02Z","receivedAt":"2017-12-15T13:06:11Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Fri, Dec 15, 2017 at 1:10 PM, Johannes Schindelin\n wrote:\n> Hi,\n>\n>> There is a lot going on in 'run-windows-build.sh', so the output of 'set\n>> -x' might be useful or might be considered too much clutter, I don't\n>> know. I put Dscho on Cc, I think it's mainly his call.\n>\n> Certainly it might be useful.\n>\n> However, please make sure that the secret token is not leaked that way.\n> Like, *really* sure. Due to the failure of Git to use a portable and\n> performant test suite, it does take about 90 minutes to build and test a\n> revision, therefore it would be very easy to DOS my build system, and I\n> really, really need it not to be DOSed because I use it in my day job, too.\n\nUgh, I was completely unaware of this issue, and the first version of\nthis patch is already in 'pu'... **runs to check the trace logs**\n\nGreat, it seems we are in luck, as the secret token was specified as an\nencrypted environment variable in git/git repository settings on Travis\nCI. It's redacted in the trace log and I only see lines like these:\n\n Setting environment variables from repository settings\n $ export GFW_CI_TOKEN=[secure]\n\n +test -z [secure]\n\n +++curl -H 'Authentication: Bearer [secure]' --silent --retry 5\n--write-out '%{HTTP_CODE}' --output /dev/fd/63\n'https://git-for-windows-ci.azurewebsites.net/api/TestNow?action=trigger&branch=pu&commit=1229713f78cd2883798e95f33c19c81b523413fd&skipTests=false'\n\n https://travis-ci.org/git/git/jobs/316791071\n\nPhew.\n\nHowever, I don't feel competent enough with Travis CI's encrypted\nenvironment variables to say that this qualifies as \"*really* sure\"\n\"that the secret token is not leaked\".\nAnyway, note, that that '$ export GFW_CI_TOKEN=[secure]' line is already\npresent in all 'git/git' trace logs independently of this 'set -x'\nchange in question.\n\nGábor\n"},{"id":"334885","messageId":"alpine.DEB.2.21.1.1712151632140.406@MININT-6BKU6QN.europe.corp.microsoft.com","threadId":"47090","inReplyTo":"CAM0VKjmpJ_J+GjJ3PyU_pxsWx85L6cPa2tLM0xJx3cJu2T87Zg@mail.gmail.com","subject":"Re: [PATCH 1/4] travis-ci: use 'set -x' in 'ci/*' scripts for extra tracing output","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-12-15T15:32:55Z","receivedAt":"2017-12-15T15:33:02Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Fri, 15 Dec 2017, SZEDER Gábor wrote:\n\n> On Fri, Dec 15, 2017 at 1:10 PM, Johannes Schindelin\n> wrote:\n> >\n> >> There is a lot going on in 'run-windows-build.sh', so the output of\n> >> 'set -x' might be useful or might be considered too much clutter, I\n> >> don't know. I put Dscho on Cc, I think it's mainly his call.\n> >\n> > Certainly it might be useful.\n> >\n> > However, please make sure that the secret token is not leaked that\n> > way. Like, *really* sure. Due to the failure of Git to use a portable\n> > and performant test suite, it does take about 90 minutes to build and\n> > test a revision, therefore it would be very easy to DOS my build\n> > system, and I really, really need it not to be DOSed because I use it\n> > in my day job, too.\n> \n> Ugh, I was completely unaware of this issue, and the first version of\n> this patch is already in 'pu'... **runs to check the trace logs**\n> \n> Great, it seems we are in luck, as the secret token was specified as an\n> encrypted environment variable in git/git repository settings on Travis\n> CI. It's redacted in the trace log [...]\n\nThat's good enough.\n\nThanks for making sure,\nDscho"},{"id":"334928","messageId":"20171216125418.10743-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171211233446.10596-1-szeder.dev@gmail.com","subject":"[PATCH v2 0/8] Travis CI cleanups","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:54:17Z","receivedAt":"2017-12-16T12:54:40Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"This is a reroll of 'sg/travis-fixes'.\n\nChanges since the previous round:\n\n - Patch 1 got updated following the discussion:\n\n - I went with enabling tracing executed commands everywhere,\n including the Windows build job, except where we know it causes way\n too much clutter, which is currently only\n 'ci/print-test-failures.sh'.\n\n - Also enable this tracing in 'ci/run-linux32-build.sh', which\n doesn't source 'ci/lib-travisci.sh' as it's run inside a Docker\n container.\n\n - The commit message got updated accordingly, including a note about\n the Windows build job's secret token.\n I would like to get an Acked-by: from Dscho on this patch before it\n gets merged.\n\n - Patches 5-8 are new. They are various fixes/cleanups unrelated to\n the Travis CI scriptification, but I don't think it's worth to have\n them in separate patch series.\n\nSZEDER Gábor (8):\n travis-ci: use 'set -x' in select 'ci/*' scripts for extra tracing\n travis-ci: introduce a $jobname variable for 'ci/*' scripts\n travis-ci: move setting environment variables to 'ci/lib-travisci.sh'\n travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh'\n travis-ci: don't install default addon packages for the 32 bit Linux\n build\n travis-ci: don't install 'language-pack-is' package\n travis-ci: save prove state for the 32 bit Linux build\n travis-ci: only print test failures if there are test results\n available\n\n .travis.yml | 28 ++++++----------------------\n ci/install-dependencies.sh | 8 +++-----\n ci/lib-travisci.sh | 38 ++++++++++++++++++++++++++++++++++----\n ci/print-test-failures.sh | 9 +++++++++\n ci/run-linux32-build.sh | 3 +++\n ci/run-linux32-docker.sh | 1 +\n 6 files changed, 56 insertions(+), 31 deletions(-)\n\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334929","messageId":"20171216125418.10743-2-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-1-szeder.dev@gmail.com","subject":"[PATCH v2 1/8] travis-ci: use 'set -x' in select 'ci/*' scripts for extra tracing","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:54:18Z","receivedAt":"2017-12-16T12:54:43Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"While the build logic was embedded in our '.travis.yml', Travis CI\nused to produce a nice trace log including all commands executed in\nthose embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\ncode into dedicated scripts, 2017-09-10), however, we only see the\nname of the dedicated scripts, but not what those scripts are actually\ndoing, resulting in a less useful trace log about e.g. installing\ndependencies. A patch later in this series will move setting\nenvironment variables from '.travis.yml' to 'ci/lib-travisci.sh', so\nnot even those will be included in the trace log. Unrelated to\n657343a60, 'ci/test-documentation.sh' runs a bunch of 'test -s '\nchecks which would fail quietly if something were wrong, leaving no\nclue about which one of those checks triggered the failure.\n\nUse 'set -x' in 'ci/lib-travisci.sh' to get more detailed trace log\nabout the commands executed in the 'ci/*' scripts. Use it in\n'ci/run-linux32-build.sh' as well, which is run in a Docker container\nand therefore doesn't source 'ci/lib-travisci.sh'. The secret token\nused for the Windows builds is specified as an encrypted environment\nvariable in git/git repository settings on Travis CI and it's redacted\nin the trace logs even with 'set -x'. However, disable this tracing\nin 'ci/print-test-failures.sh', as it produces far too much noise in\nthe output of that script.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/lib-travisci.sh | 6 ++++--\n ci/print-test-failures.sh | 3 +++\n ci/run-linux32-build.sh | 2 ++\n 3 files changed, 9 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex ac05f1f46..2d0d1d613 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -22,8 +22,10 @@ skip_branch_tip_with_tag () {\n }\n \n # Set 'exit on error' for all CI scripts to let the caller know that\n-# something went wrong\n-set -e\n+# something went wrong.\n+# Set tracing executed commands, primarily setting environment variables\n+# and installing dependencies.\n+set -ex\n \n skip_branch_tip_with_tag\n \ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex 8c8973cbf..f757e616c 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -5,6 +5,9 @@\n \n . ${0%/*}/lib-travisci.sh\n \n+# Tracing executed commands would produce too much noise in this script.\n+set +x\n+\n for TEST_EXIT in t/test-results/*.exit\n do\n \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\ndiff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\nindex e30fb2cdd..a8518eddf 100755\n--- a/ci/run-linux32-build.sh\n+++ b/ci/run-linux32-build.sh\n@@ -6,6 +6,8 @@\n # run-linux32-build.sh [host-user-id]\n #\n \n+set -x\n+\n # Update packages to the latest available versions\n linux32 --32bit i386 sh -c '\n apt update >/dev/null &&\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334930","messageId":"20171216125532.10826-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 2/8] travis-ci: introduce a $jobname variable for 'ci/*' scripts","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:55:32Z","receivedAt":"2017-12-16T12:55:46Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"A couple of 'ci/*' scripts are shared between different build jobs:\n'ci/lib-travisci.sh', being a common library, is sourced from almost\nevery script, while 'ci/install-dependencies.sh', 'ci/run-build.sh'\nand 'ci/run-tests.sh' are shared between the \"regular\" GCC and Clang\nLinux and OSX build jobs, and the latter two scripts are used in the\nGETTEXT_POISON Linux build job as well.\n\nOur builds could benefit from these shared scripts being able to\neasily tell which build job they are taking part in. Now, it's\nalready quite easy to tell apart Linux vs OSX and GCC vs Clang build\njobs, but it gets trickier with all the additional Linux-based build\njobs included explicitly in the build matrix.\n\nUnfortunately, Travis CI doesn't provide much help in this regard.\nThe closest we've got is the $TRAVIS_JOB_NUMBER variable, the value of\nwhich is two dot-separated integers, where the second integer\nindicates a particular build job. While it would be possible to use\nthat second number to identify the build job in our shared scripts, it\ndoesn't seem like a good idea to rely on that:\n\n - Though the build job numbering sequence seems to be stable so far,\n Travis CI's documentation doesn't explicitly states that it is\n indeed stable and will remain so in the future. And even if it\n were stable,\n\n - if we were to remove or insert a build job in the middle, then the\n job numbers of all subsequent build jobs would change accordingly.\n\nSo roll our own means of simple build job identification and introduce\nthe $jobname environment variable in our builds, setting it in the\nenvironments of the explicitly included jobs in '.travis.yml', while\nconstructing one in 'ci/lib-travisci.sh' as the combination of the OS\nand compiler name for the GCC and Clang Linux and OSX build jobs. Use\n$jobname instead of $TRAVIS_OS_NAME in scripts taking different\nactions based on the OS and build job (when installing P4 and Git LFS\ndependencies and including them in $PATH). The following two patches\nwill also rely on $jobname.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 10 +++++-----\n ci/install-dependencies.sh | 6 +++---\n ci/lib-travisci.sh | 9 +++++++--\n 3 files changed, 15 insertions(+), 10 deletions(-)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 281f101f3..88435e11c 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -39,12 +39,12 @@ env:\n \n matrix:\n include:\n- - env: GETTEXT_POISON=YesPlease\n+ - env: jobname=GETTEXT_POISON GETTEXT_POSION=YesPlease\n os: linux\n compiler:\n addons:\n before_install:\n- - env: Windows\n+ - env: jobname=Windows\n os: linux\n compiler:\n addons:\n@@ -55,7 +55,7 @@ matrix:\n test \"$TRAVIS_REPO_SLUG\" != \"git/git\" ||\n ci/run-windows-build.sh $TRAVIS_BRANCH $(git rev-parse HEAD)\n after_failure:\n- - env: Linux32\n+ - env: jobname=Linux32\n os: linux\n compiler:\n services:\n@@ -63,7 +63,7 @@ matrix:\n before_install:\n before_script:\n script: ci/run-linux32-docker.sh\n- - env: Static Analysis\n+ - env: jobname=StaticAnalysis\n os: linux\n compiler:\n addons:\n@@ -74,7 +74,7 @@ matrix:\n before_script:\n script: ci/run-static-analysis.sh\n after_failure:\n- - env: Documentation\n+ - env: jobname=Documentation\n os: linux\n compiler:\n addons:\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 5bd06fe90..468788566 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -8,8 +8,8 @@\n P4WHENCE=http://filehost.perforce.com/perforce/r$LINUX_P4_VERSION\n LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VERSION\n \n-case \"${TRAVIS_OS_NAME:-linux}\" in\n-linux)\n+case \"$jobname\" in\n+linux-clang|linux-gcc)\n \texport GIT_TEST_HTTPD=YesPlease\n \n \tmkdir --parents \"$P4_PATH\"\n@@ -26,7 +26,7 @@ linux)\n \t\tcp git-lfs-$LINUX_GIT_LFS_VERSION/git-lfs .\n \tpopd\n \t;;\n-osx)\n+osx-clang|osx-gcc)\n \tbrew update --quiet\n \t# Uncomment this if you want to run perf tests:\n \t# brew install gnu-time\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex 2d0d1d613..4b3c5fdd0 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -29,8 +29,13 @@ set -ex\n \n skip_branch_tip_with_tag\n \n-case \"${TRAVIS_OS_NAME:-linux}\" in\n-linux)\n+if test -z \"$jobname\"\n+then\n+\tjobname=\"$TRAVIS_OS_NAME-$CC\"\n+fi\n+\n+case \"$jobname\" in\n+linux-clang|linux-gcc)\n \tP4_PATH=\"$(pwd)/custom/p4\"\n \tGIT_LFS_PATH=\"$(pwd)/custom/git-lfs\"\n \texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334931","messageId":"20171216125702.10901-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 3/8] travis-ci: move setting environment variables to 'ci/lib-travisci.sh'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:57:02Z","receivedAt":"2017-12-16T12:57:17Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Our '.travis.yml's 'env.global' section sets a bunch of environment\nvariables for all build jobs, though none of them actually affects all\nbuild jobs. It's convenient for us, and in most cases it works just\nfine, because irrelevant environment variables are simply ignored.\n\nHowever, $GIT_SKIP_TESTS is an exception: it tells the test harness to\nskip the two test scripts that are prone to occasional failures on\nOSX, but as it's set for all build jobs those tests are not run in any\nof the build jobs that are capable to run them reliably, either.\n\nTherefore $GIT_SKIP_TESTS should only be set in the OSX build jobs,\nbut those build jobs are included in the build matrix implicitly (i.e.\nby combining the matrix keys 'os' and 'compiler'), and there is no way\nto set an environment variable only for a subset of those implicit\nbuild jobs. (Unless we were to add new scriptlets to '.travis.yml',\nwhich is exactly the opposite direction that we took with commit\n657343a60 (travis-ci: move Travis CI code into dedicated scripts,\n2017-09-10)).\n\nSo move setting $GIT_SKIP_TESTS to 'ci/lib-travisci.sh', where it can\ntrivially be set only for the OSX build jobs.\n\nFurthermore, move setting all other environment variables from\n'.travis.yml' to 'ci/lib-travisci.sh', too, because a couple of\nenvironment variables are already set there, and this way all\nenvironment variables will be set in the same place. All the logic\ncontrolling our builds is already in the 'ci/*' scripts anyway, so\nthere is really no good reason to keep the environment variables\nseparately.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 18 +-----------------\n ci/lib-travisci.sh | 21 +++++++++++++++++++++\n 2 files changed, 22 insertions(+), 17 deletions(-)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 88435e11c..7c9aa0557 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -21,25 +21,9 @@ addons:\n - git-svn\n - apache2\n \n-env:\n- global:\n- - DEVELOPER=1\n- # The Linux build installs the defined dependency versions below.\n- # The OS X build installs the latest available versions. Keep that\n- # in mind when you encounter a broken OS X build!\n- - LINUX_P4_VERSION=\"16.2\"\n- - LINUX_GIT_LFS_VERSION=\"1.5.2\"\n- - DEFAULT_TEST_TARGET=prove\n- - GIT_PROVE_OPTS=\"--timer --jobs 3 --state=failed,slow,save\"\n- - GIT_TEST_OPTS=\"--verbose-log\"\n- - GIT_TEST_CLONE_2GB=YesPlease\n- # t9810 occasionally fails on Travis CI OS X\n- # t9816 occasionally fails with \"TAP out of sequence errors\" on Travis CI OS X\n- - GIT_SKIP_TESTS=\"t9810 t9816\"\n-\n matrix:\n include:\n- - env: jobname=GETTEXT_POISON GETTEXT_POSION=YesPlease\n+ - env: jobname=GETTEXT_POISON\n os: linux\n compiler:\n addons:\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex 4b3c5fdd0..c5c5cb1bf 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -34,10 +34,31 @@ then\n \tjobname=\"$TRAVIS_OS_NAME-$CC\"\n fi\n \n+export DEVELOPER=1\n+export DEFAULT_TEST_TARGET=prove\n+export GIT_PROVE_OPTS=\"--timer --jobs 3 --state=failed,slow,save\"\n+export GIT_TEST_OPTS=\"--verbose-log\"\n+export GIT_TEST_CLONE_2GB=YesPlease\n+\n case \"$jobname\" in\n linux-clang|linux-gcc)\n+\t# The Linux build installs the defined dependency versions below.\n+\t# The OS X build installs the latest available versions. Keep that\n+\t# in mind when you encounter a broken OS X build!\n+\texport LINUX_P4_VERSION=\"16.2\"\n+\texport LINUX_GIT_LFS_VERSION=\"1.5.2\"\n+\n \tP4_PATH=\"$(pwd)/custom/p4\"\n \tGIT_LFS_PATH=\"$(pwd)/custom/git-lfs\"\n \texport PATH=\"$GIT_LFS_PATH:$P4_PATH:$PATH\"\n \t;;\n+osx-clang|osx-gcc)\n+\t# t9810 occasionally fails on Travis CI OS X\n+\t# t9816 occasionally fails with \"TAP out of sequence errors\" on\n+\t# Travis CI OS X\n+\texport GIT_SKIP_TESTS=\"t9810 t9816\"\n+\t;;\n+GETTEXT_POISON)\n+\texport GETTEXT_POISON=YesPlease\n+\t;;\n esac\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334932","messageId":"20171216125740.10978-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 4/8] travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh'","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:57:40Z","receivedAt":"2017-12-16T12:57:49Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Commit 657343a60 (travis-ci: move Travis CI code into dedicated\nscripts, 2017-09-10) converted '.travis.yml's default 'before_install'\nscriptlet to the 'ci/install-dependencies.sh' script, and while doing\nso moved setting GIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang\nLinux build jobs to that script. This is wrong for two reasons:\n\n - The purpose of that script is, as its name suggests, to install\n dependencies, not to set any environment variables influencing\n which tests should be run (though, arguably, this was already an\n issue with the original 'before_install' scriptlet).\n\n - Setting the variable has no effect anymore, because that script is\n run in a separate shell process, and the variable won't be visible\n in any of the other scripts, notably in 'ci/run-tests.sh'\n responsible for, well, running the tests.\n\nLuckily, this didn't have a negative effect on our Travis CI build\njobs, because GIT_TEST_HTTPD is a tri-state variable defaulting to\n\"auto\" and a functioning web server was installed in those Linux build\njobs, so the httpd tests were run anyway.\n\nApparently the httpd tests run just fine without GIT_TEST_HTTPD being\nset, therefore we could simply remove this environment variable.\nHowever, if a bug were to creep in to change the Travis CI build\nenvironment to run the tests as root or to not install Apache, then\nthe httpd tests would be skipped and the build job would still\nsucceed. We would only notice if someone actually were to look\nthrough the build job's trace log; but who would look at the trace log\nof a successful build job?!\n\nSince httpd tests are important, we do want to run them and we want to\nbe loudly reminded if they can't be run. Therefore, move setting\nGIT_TEST_HTTPD=YesPlease for the 64-bit GCC and Clang Linux build jobs\nto 'ci/lib-travisci.sh' to ensure that the build job fails when the\nhttpd tests can't be run. (We could set it in 'ci/run-tests.sh' just\nas well, but it's better to keep all environment variables in one\nplace in 'ci/lib-travisci.sh'.)\n\nSigned-off-by: SZEDER Gábor \n---\n ci/install-dependencies.sh | 2 --\n ci/lib-travisci.sh | 2 ++\n 2 files changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex 468788566..75a9fd247 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -10,8 +10,6 @@ LFSWHENCE=https://github.com/github/git-lfs/releases/download/v$LINUX_GIT_LFS_VE\n \n case \"$jobname\" in\n linux-clang|linux-gcc)\n-\texport GIT_TEST_HTTPD=YesPlease\n-\n \tmkdir --parents \"$P4_PATH\"\n \tpushd \"$P4_PATH\"\n \t\twget --quiet \"$P4WHENCE/bin.linux26x86_64/p4d\"\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex c5c5cb1bf..348fe3c3c 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -42,6 +42,8 @@ export GIT_TEST_CLONE_2GB=YesPlease\n \n case \"$jobname\" in\n linux-clang|linux-gcc)\n+\texport GIT_TEST_HTTPD=YesPlease\n+\n \t# The Linux build installs the defined dependency versions below.\n \t# The OS X build installs the latest available versions. Keep that\n \t# in mind when you encounter a broken OS X build!\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334933","messageId":"20171216125750.11048-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 5/8] travis-ci: don't install default addon packages for the 32 bit Linux build","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:57:50Z","receivedAt":"2017-12-16T12:57:58Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"The 32 bit Linux build job compiles Git and runs the test suite in a\nDocker container, while the additional packages (apache2, git-svn,\nlanguage-pack-is) are installed on the host, therefore don't have\nany effect and are unnecessary.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 7c9aa0557..4684b3f4f 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -42,6 +42,7 @@ matrix:\n - env: jobname=Linux32\n os: linux\n compiler:\n+ addons:\n services:\n - docker\n before_install:\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334934","messageId":"20171216125758.11120-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:57:58Z","receivedAt":"2017-12-16T12:58:09Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"Ever since we have started to use Travis CI in 522354d70 (Add Travis\nCI support, 2015-11-27), our 64 bit Linux build jobs install the\n'languate-pack-is' package. That commit doesn't discuss why it was\ndeemed necessary back then, but Travis CI can build and test Git\nwithout that package just fine, even that commit introducing Travis CI\nsupport.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 4684b3f4f..ea11b5af6 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -17,7 +17,6 @@ compiler:\n addons:\n apt:\n packages:\n- - language-pack-is\n - git-svn\n - apache2\n \n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334935","messageId":"20171216125806.11193-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 7/8] travis-ci: save prove state for the 32 bit Linux build","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:58:06Z","receivedAt":"2017-12-16T12:58:13Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"This change follows suit of 6272ed319 (travis-ci: run previously\nfailed tests first, then slowest to fastest, 2016-01-26), which did\nthis for the Linux and OSX build jobs. Travis CI build jobs run the\ntests parallel, which is sligtly faster when tests are run in slowest\nto fastest order, shortening the overall runtime of this build job by\nabout a minute / 10%.\n\nNote, that the 32 bit Linux build job runs the tests suite in a Docker\ncontainer and we have to share the Travis CI cache directory with the\ncontainer as a second volume. Otherwise we couldn't use a symlink\npointing to the prove state file in the cache directory, because\nthat's outside of the directory hierarchy accessible from within the\ncontainer.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/run-linux32-build.sh | 1 +\n ci/run-linux32-docker.sh | 1 +\n 2 files changed, 2 insertions(+)\n\ndiff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\nindex a8518eddf..c19c50c1c 100755\n--- a/ci/run-linux32-build.sh\n+++ b/ci/run-linux32-build.sh\n@@ -27,6 +27,7 @@ test -z $HOST_UID || (CI_USER=\"ci\" && useradd -u $HOST_UID $CI_USER) &&\n # Build and test\n linux32 --32bit i386 su -m -l $CI_USER -c '\n cd /usr/src/git &&\n+ ln -s /tmp/travis-cache/.prove t/.prove &&\n make --jobs=2 &&\n make --quiet test\n '\ndiff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh\nindex 0edf63acf..3a8b2ba42 100755\n--- a/ci/run-linux32-docker.sh\n+++ b/ci/run-linux32-docker.sh\n@@ -19,5 +19,6 @@ docker run \\\n \t--env GIT_TEST_OPTS \\\n \t--env GIT_TEST_CLONE_2GB \\\n \t--volume \"${PWD}:/usr/src/git\" \\\n+\t--volume \"${HOME}/travis-cache:/tmp/travis-cache\" \\\n \tdaald/ubuntu32:xenial \\\n \t/usr/src/git/ci/run-linux32-build.sh $(id -u $USER)\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334936","messageId":"20171216125819.11263-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"[PATCH v2 8/8] travis-ci: only print test failures if there are test results available","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T12:58:19Z","receivedAt":"2017-12-16T12:58:27Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"When a build job running the test suite fails, our\n'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'\nfiles to find failed tests and prints their verbose output. However,\nif a build job were to fail before it ever gets to run the test suite,\nthen there will be no files to match the above pattern and the shell\nwill take the pattern literally, resulting in errors like this in the\ntrace log:\n\n cat: t/test-results/*.exit: No such file or directory\n ------------------------------------------------------------------------\n t/test-results/*.out...\n ------------------------------------------------------------------------\n cat: t/test-results/*.out: No such file or directory\n\nCheck upfront and proceed only if there are any such files present.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/print-test-failures.sh | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex f757e616c..1a3d74d47 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -8,6 +8,12 @@\n # Tracing executed commands would produce too much noise in this script.\n set +x\n \n+if test t/test-results/*.exit = \"t/test-results/*.exit\"\n+then\n+\techo \"Build job failed before the tests could have been run\"\n+\texit\n+fi\n+\n for TEST_EXIT in t/test-results/*.exit\n do\n \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334937","messageId":"alpine.DEB.2.21.1.1712161743330.406@MININT-6BKU6QN.europe.corp.microsoft.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"Re: [PATCH v2 1/8] travis-ci: use 'set -x' in select 'ci/*' scripts for extra tracing","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-12-16T16:43:51Z","receivedAt":"2017-12-16T16:44:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Gábor,\n\nOn Sat, 16 Dec 2017, SZEDER Gábor wrote:\n\n> While the build logic was embedded in our '.travis.yml', Travis CI\n> used to produce a nice trace log including all commands executed in\n> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n> code into dedicated scripts, 2017-09-10), however, we only see the\n> name of the dedicated scripts, but not what those scripts are actually\n> doing, resulting in a less useful trace log about e.g. installing\n> dependencies. A patch later in this series will move setting\n> environment variables from '.travis.yml' to 'ci/lib-travisci.sh', so\n> not even those will be included in the trace log. Unrelated to\n> 657343a60, 'ci/test-documentation.sh' runs a bunch of 'test -s '\n> checks which would fail quietly if something were wrong, leaving no\n> clue about which one of those checks triggered the failure.\n> \n> Use 'set -x' in 'ci/lib-travisci.sh' to get more detailed trace log\n> about the commands executed in the 'ci/*' scripts. Use it in\n> 'ci/run-linux32-build.sh' as well, which is run in a Docker container\n> and therefore doesn't source 'ci/lib-travisci.sh'. The secret token\n> used for the Windows builds is specified as an encrypted environment\n> variable in git/git repository settings on Travis CI and it's redacted\n> in the trace logs even with 'set -x'. However, disable this tracing\n> in 'ci/print-test-failures.sh', as it produces far too much noise in\n> the output of that script.\n\nACK,\nDscho"},{"id":"334938","messageId":"CAPig+cTL5hU6R=L3M2i0kfM6_zMLvcHWvNooAr8BU3VQ=X0_Ww@mail.gmail.com","threadId":"47090","inReplyTo":"20171216125819.11263-1-szeder.dev@gmail.com","subject":"Re: [PATCH v2 8/8] travis-ci: only print test failures if there are test results available","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2017-12-16T18:32:07Z","receivedAt":"2017-12-16T18:32:13Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Dec 16, 2017 at 7:58 AM, SZEDER Gábor wrote:\n> When a build job running the test suite fails, our\n> 'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'\n> files to find failed tests and prints their verbose output. However,\n> if a build job were to fail before it ever gets to run the test suite,\n> then there will be no files to match the above pattern and the shell\n> will take the pattern literally, resulting in errors like this in the\n> trace log:\n>\n> cat: t/test-results/*.exit: No such file or directory\n> ------------------------------------------------------------------------\n> t/test-results/*.out...\n> ------------------------------------------------------------------------\n> cat: t/test-results/*.out: No such file or directory\n>\n> Check upfront and proceed only if there are any such files present.\n>\n> Signed-off-by: SZEDER Gábor \n> ---\n> diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\n> @@ -8,6 +8,12 @@\n> +if test t/test-results/*.exit = \"t/test-results/*.exit\"\n\nIsn't the above going to cause 'test' to error out?\n\n $ mkdir -p t/test-results\n $ >t/test-results/a.exit\n $ >t/test-results/b.exit\n $ test t/test-results/*.exit = 't/test-results/*.exit'\n -bash: test: too many arguments\n\nI'd think you'd want to capture the result of the expansion of\nt/test-result/*.exit as a string and compare that instead.\n\n> +then\n> + echo \"Build job failed before the tests could have been run\"\n> + exit\n> +fi\n"},{"id":"334944","messageId":"20171216224839.11372-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"CAPig+cTL5hU6R=L3M2i0kfM6_zMLvcHWvNooAr8BU3VQ=X0_Ww@mail.gmail.com","subject":"Re: [PATCH v2 8/8] travis-ci: only print test failures if there are","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-16T22:48:39Z","receivedAt":"2017-12-16T22:49:07Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"\n> On Sat, Dec 16, 2017 at 7:58 AM, SZEDER G=C3=A1bor w=\n> rote:\n\n> > +if test t/test-results/*.exit =3D \"t/test-results/*.exit\"\n> \n> Isn't the above going to cause 'test' to error out?\n>\n> $ mkdir -p t/test-results\n> $ >t/test-results/a.exit\n> $ >t/test-results/b.exit\n> $ test t/test-results/*.exit =3D 't/test-results/*.exit'\n> -bash: test: too many arguments\n\nIndeed it does, though Travis CI's /bin/sh gives a different error\nmessage. Thanks for pointin it out, I didn't notice the error msg,\nbecause the script as a whole still did the right thing and then I\ndidn't look close enough.\n\n> I'd think you'd want to capture the result of the expansion of\n> t/test-result/*.exit as a string and compare that instead.\n\nI'm not sure how the result of the expansion could be captured in the\nshell, because the shell performs the expansion only just before it\nexecutes a command. And if we do have to execute a command anyway,\nthen we can simply rely on the command exiting with an error code upon\nnot finding any files, and there's no need to capture the expansion or\nfor the comparison for that matter.\n\nSo I propose this updated patch, using 'ls' instead of 'test':\n\n\n -- >8 --\n\nSubject: [PATCH v2.1 8/8] travis-ci: only print test failures if there are test results available\n\nWhen a build job running the test suite fails, our\n'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'\nfiles to find failed tests and prints their verbose output. However,\nif a build job were to fail before it ever gets to run the test suite,\nthen there will be no files to match the above pattern and the shell\nwill take the pattern literally, resulting in errors like this in the\ntrace log:\n\n cat: t/test-results/*.exit: No such file or directory\n ------------------------------------------------------------------------\n t/test-results/*.out...\n ------------------------------------------------------------------------\n cat: t/test-results/*.out: No such file or directory\n\nCheck upfront and proceed only if there are any such files present.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/print-test-failures.sh | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex f757e616c..b4a62ef98 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -8,6 +8,12 @@\n # Tracing executed commands would produce too much noise in this script.\n set +x\n \n+if ! ls t/test-results/*.exit >/dev/null 2>/dev/null\n+then\n+\techo \"Build job failed before the tests could have been run\"\n+\texit\n+fi\n+\n for TEST_EXIT in t/test-results/*.exit\n do\n \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n-- \n2.15.1.429.ga000dd9c7\n\n"},{"id":"334945","messageId":"CAPig+cR2FE1Gtw5LWUVHgiVwA1++YXKjLgkeN+3MjNzP1=Tguw@mail.gmail.com","threadId":"47090","inReplyTo":"20171216224839.11372-1-szeder.dev@gmail.com","subject":"Re: [PATCH v2 8/8] travis-ci: only print test failures if there are","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2017-12-17T00:02:58Z","receivedAt":"2017-12-17T00:03:08Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sat, Dec 16, 2017 at 5:48 PM, SZEDER Gábor wrote:\n> On Sat, Dec 16, 2017, Eric Sunshine wrote:\n>> I'd think you'd want to capture the result of the expansion of\n>> t/test-result/*.exit as a string and compare that instead.\n>\n> I'm not sure how the result of the expansion could be captured in the\n> shell, because the shell performs the expansion only just before it\n> executes a command. And if we do have to execute a command anyway,\n> then we can simply rely on the command exiting with an error code upon\n> not finding any files, and there's no need to capture the expansion or\n> for the comparison for that matter.\n> So I propose this updated patch, using 'ls' instead of 'test':\n\nTo capture the expansion as a string, I was thinking along the lines of:\n\n x=$(echo t/test-results/*.exit)\n\nbut that's a minor point. Checking the explicit exit code of a\ncommand, as you suggest, is probably cleaner.\n"},{"id":"334990","messageId":"8F53EF33-6FDA-484C-91A4-49CF24C0B417@gmail.com","threadId":"47090","inReplyTo":"20171216125758.11120-1-szeder.dev@gmail.com","subject":"Re: [PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-18T21:33:56Z","receivedAt":"2017-12-18T21:34:04Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 16 Dec 2017, at 13:57, SZEDER Gábor wrote:\n> \n> Ever since we have started to use Travis CI in 522354d70 (Add Travis\n> CI support, 2015-11-27), our 64 bit Linux build jobs install the\n> 'languate-pack-is' package. That commit doesn't discuss why it was\n> deemed necessary back then, but Travis CI can build and test Git\n> without that package just fine, even that commit introducing Travis CI\n> support.\n\nIf I remember correctly then we had to install the Icelandic\nlanguage pack for the i18n tests (e.g. t0204):\n\nhttps://github.com/git/git/blob/master/t/lib-gettext.sh\nhttps://packages.ubuntu.com/trusty/language-pack-is\n\nHowever, I checked your Travis-CI and I cannot spot any errors:\nhttps://travis-ci.org/szeder/git/jobs/317494789\n\nI am a bit puzzled. Maybe the Icelandic language pack was not part\nof the Ubuntu image that was used when we introduced Travis CI?\n\nThe Ubuntu default image was 12.04 back then:\nhttps://travis-ci.org/git/git/jobs/100241871\n\nNowadays it is 14.04.\n\n@Avar: \nDo you know what kind of errors we should expect if the language \npack is not installed?\n\nThanks,\nLars\n\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> .travis.yml | 1 -\n> 1 file changed, 1 deletion(-)\n> \n> diff --git a/.travis.yml b/.travis.yml\n> index 4684b3f4f..ea11b5af6 100644\n> --- a/.travis.yml\n> +++ b/.travis.yml\n> @@ -17,7 +17,6 @@ compiler:\n> addons:\n> apt:\n> packages:\n> - - language-pack-is\n> - git-svn\n> - apache2\n> \n> -- \n> 2.15.1.429.ga000dd9c7\n> \n\n"},{"id":"334992","messageId":"E20A2A7B-232D-4873-A026-E8CB104A84BF@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-1-szeder.dev@gmail.com","subject":"Re: [PATCH v2 0/8] Travis CI cleanups","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-18T21:46:02Z","receivedAt":"2017-12-18T21:46:08Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 16 Dec 2017, at 13:54, SZEDER Gábor wrote:\n> \n> This is a reroll of 'sg/travis-fixes'.\n> \n> Changes since the previous round:\n> \n> - Patch 1 got updated following the discussion:\n> \n> - I went with enabling tracing executed commands everywhere,\n> including the Windows build job, except where we know it causes way\n> too much clutter, which is currently only\n> 'ci/print-test-failures.sh'.\n> \n> - Also enable this tracing in 'ci/run-linux32-build.sh', which\n> doesn't source 'ci/lib-travisci.sh' as it's run inside a Docker\n> container.\n> \n> - The commit message got updated accordingly, including a note about\n> the Windows build job's secret token.\n> I would like to get an Acked-by: from Dscho on this patch before it\n> gets merged.\n> \n> - Patches 5-8 are new. They are various fixes/cleanups unrelated to\n> the Travis CI scriptification, but I don't think it's worth to have\n> them in separate patch series.\n\nThis cleanup series looks very good. ACK. Thank you, Gabor!\n\nThe is the only potential nit I see:\nhttps://public-inbox.org/git/8F53EF33-6FDA-484C-91A4-49CF24C0B417@gmail.com/\n\n- Lars\n\n> \n> SZEDER Gábor (8):\n> travis-ci: use 'set -x' in select 'ci/*' scripts for extra tracing\n> travis-ci: introduce a $jobname variable for 'ci/*' scripts\n> travis-ci: move setting environment variables to 'ci/lib-travisci.sh'\n> travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh'\n> travis-ci: don't install default addon packages for the 32 bit Linux\n> build\n> travis-ci: don't install 'language-pack-is' package\n> travis-ci: save prove state for the 32 bit Linux build\n> travis-ci: only print test failures if there are test results\n> available\n> \n> .travis.yml | 28 ++++++----------------------\n> ci/install-dependencies.sh | 8 +++-----\n> ci/lib-travisci.sh | 38 ++++++++++++++++++++++++++++++++++----\n> ci/print-test-failures.sh | 9 +++++++++\n> ci/run-linux32-build.sh | 3 +++\n> ci/run-linux32-docker.sh | 1 +\n> 6 files changed, 56 insertions(+), 31 deletions(-)\n> \n> -- \n> 2.15.1.429.ga000dd9c7\n> \n\n"},{"id":"334994","messageId":"B2E7DCD0-E4E8-4898-ABC5-5C2E70284CFA@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-2-szeder.dev@gmail.com","subject":"Re: [PATCH v2 1/8] travis-ci: use 'set -x' in select 'ci/*' scripts for extra tracing","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-18T21:53:56Z","receivedAt":"2017-12-18T21:54:03Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 16 Dec 2017, at 13:54, SZEDER Gábor wrote:\n> \n> While the build logic was embedded in our '.travis.yml', Travis CI\n> used to produce a nice trace log including all commands executed in\n> those embedded scriptlets. Since 657343a60 (travis-ci: move Travis CI\n> code into dedicated scripts, 2017-09-10), however, we only see the\n> name of the dedicated scripts, but not what those scripts are actually\n> doing, resulting in a less useful trace log about e.g. installing\n> dependencies. A patch later in this series will move setting\n> environment variables from '.travis.yml' to 'ci/lib-travisci.sh', so\n> not even those will be included in the trace log. Unrelated to\n> 657343a60, 'ci/test-documentation.sh' runs a bunch of 'test -s '\n> checks which would fail quietly if something were wrong, leaving no\n> clue about which one of those checks triggered the failure.\n> \n> Use 'set -x' in 'ci/lib-travisci.sh' to get more detailed trace log\n> about the commands executed in the 'ci/*' scripts. Use it in\n> 'ci/run-linux32-build.sh' as well, which is run in a Docker container\n> and therefore doesn't source 'ci/lib-travisci.sh'. The secret token\n> used for the Windows builds is specified as an encrypted environment\n> variable in git/git repository settings on Travis CI and it's redacted\n> in the trace logs even with 'set -x'. However, disable this tracing\n> in 'ci/print-test-failures.sh', as it produces far too much noise in\n> the output of that script.\n\nI ACK too soon in my other email ;-)\n\nCan you disable the trace log for the loop in \"run-windows-build.sh\":\nhttps://github.com/git/git/blob/52015aaf9d19c97b52c47c7046058e6d029ff856/ci/run-windows-build.sh#L75-L88\n\nOtherwise the Travis output look like this:\nhttps://travis-ci.org/git/git/jobs/316791071\n\n- Lars\n\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> ci/lib-travisci.sh | 6 ++++--\n> ci/print-test-failures.sh | 3 +++\n> ci/run-linux32-build.sh | 2 ++\n> 3 files changed, 9 insertions(+), 2 deletions(-)\n> \n> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n> index ac05f1f46..2d0d1d613 100755\n> --- a/ci/lib-travisci.sh\n> +++ b/ci/lib-travisci.sh\n> @@ -22,8 +22,10 @@ skip_branch_tip_with_tag () {\n> }\n> \n> # Set 'exit on error' for all CI scripts to let the caller know that\n> -# something went wrong\n> -set -e\n> +# something went wrong.\n> +# Set tracing executed commands, primarily setting environment variables\n> +# and installing dependencies.\n> +set -ex\n> \n> skip_branch_tip_with_tag\n> \n> diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\n> index 8c8973cbf..f757e616c 100755\n> --- a/ci/print-test-failures.sh\n> +++ b/ci/print-test-failures.sh\n> @@ -5,6 +5,9 @@\n> \n> . ${0%/*}/lib-travisci.sh\n> \n> +# Tracing executed commands would produce too much noise in this script.\n> +set +x\n> +\n> for TEST_EXIT in t/test-results/*.exit\n> do\n> \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n> diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\n> index e30fb2cdd..a8518eddf 100755\n> --- a/ci/run-linux32-build.sh\n> +++ b/ci/run-linux32-build.sh\n> @@ -6,6 +6,8 @@\n> # run-linux32-build.sh [host-user-id]\n> #\n> \n> +set -x\n> +\n> # Update packages to the latest available versions\n> linux32 --32bit i386 sh -c '\n> apt update >/dev/null &&\n> -- \n> 2.15.1.429.ga000dd9c7\n> \n\n"},{"id":"334996","messageId":"CAM0VKj=xVfCd5qR4jXwFGDxSABgyAeZY5qJxUP3GHvUE7gjw6A@mail.gmail.com","threadId":"47090","inReplyTo":"8F53EF33-6FDA-484C-91A4-49CF24C0B417@gmail.com","subject":"Re: [PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-18T22:04:41Z","receivedAt":"2017-12-18T22:04:54Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Dec 18, 2017 at 10:33 PM, Lars Schneider\n wrote:\n>\n>> On 16 Dec 2017, at 13:57, SZEDER Gábor wrote:\n>>\n>> Ever since we have started to use Travis CI in 522354d70 (Add Travis\n>> CI support, 2015-11-27), our 64 bit Linux build jobs install the\n>> 'languate-pack-is' package. That commit doesn't discuss why it was\n>> deemed necessary back then, but Travis CI can build and test Git\n>> without that package just fine, even that commit introducing Travis CI\n>> support.\n>\n> If I remember correctly then we had to install the Icelandic\n> language pack for the i18n tests (e.g. t0204):\n>\n> https://github.com/git/git/blob/master/t/lib-gettext.sh\n> https://packages.ubuntu.com/trusty/language-pack-is\n>\n> However, I checked your Travis-CI and I cannot spot any errors:\n> https://travis-ci.org/szeder/git/jobs/317494789\n\nAh, now I start to understand. I was only looking for errors, too, but\nthere are none, because the tests requiring the language pack are\nprotected by the appropriate prerequisites. On my box, without and\nwith the language pack:\n\n $ ./t0204-gettext-reencode-sanity.sh\n # lib-gettext: No is_IS UTF-8 locale available\n # lib-gettext: No is_IS ISO-8859-1 locale available\n ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files /\nIcelandic (missing GETTEXT_LOCALE)\n ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files /\nRunes (missing GETTEXT_LOCALE)\n ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files /\nIcelandic (missing GETTEXT_ISO_LOCALE)\n ok 4 # skip gettext: impossible ISO-8859-1 output (missing GETTEXT_ISO_LOCALE)\n ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)\n ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing\nGETTEXT_ISO_LOCALE)\n ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)\n ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing\nGETTEXT_ISO_LOCALE)\n # passed all 8 test(s)\n\n $ sudo apt-get install language-pack-is\n [...]\n $ ./t0204-gettext-reencode-sanity.sh\n # lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale\n # lib-gettext: No is_IS ISO-8859-1 locale available\n ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic\n ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes\n ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files /\nIcelandic (missing GETTEXT_ISO_LOCALE)\n ok 4 # skip gettext: impossible ISO-8859-1 output (missing GETTEXT_ISO_LOCALE)\n ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8\n ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing\nGETTEXT_ISO_LOCALE)\n ok 7 - gettext.c: git init UTF-8 -> UTF-8\n ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing\nGETTEXT_ISO_LOCALE)\n # passed all 8 test(s)\n 1..8\n\nI'd expect something like this in the Travis CI build jobs as well, but\nprove hides the detailed output.\n\nIt seems we would loose coverage with this patch, so it should be\ndropped.\n\n> I am a bit puzzled. Maybe the Icelandic language pack was not part\n> of the Ubuntu image that was used when we introduced Travis CI?\n>\n> The Ubuntu default image was 12.04 back then:\n> https://travis-ci.org/git/git/jobs/100241871\n>\n> Nowadays it is 14.04.\n>\n> @Avar:\n> Do you know what kind of errors we should expect if the language\n> pack is not installed?\n>\n> Thanks,\n> Lars\n>\n>>\n>> Signed-off-by: SZEDER Gábor \n>> ---\n>> .travis.yml | 1 -\n>> 1 file changed, 1 deletion(-)\n>>\n>> diff --git a/.travis.yml b/.travis.yml\n>> index 4684b3f4f..ea11b5af6 100644\n>> --- a/.travis.yml\n>> +++ b/.travis.yml\n>> @@ -17,7 +17,6 @@ compiler:\n>> addons:\n>> apt:\n>> packages:\n>> - - language-pack-is\n>> - git-svn\n>> - apache2\n>>\n>> --\n>> 2.15.1.429.ga000dd9c7\n>>\n>\n"},{"id":"334998","messageId":"AB053B56-5318-47D4-8ADE-0A72B9860BED@gmail.com","threadId":"47090","inReplyTo":"CAM0VKj=xVfCd5qR4jXwFGDxSABgyAeZY5qJxUP3GHvUE7gjw6A@mail.gmail.com","subject":"Re: [PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-18T22:17:39Z","receivedAt":"2017-12-18T22:17:47Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 18 Dec 2017, at 23:04, SZEDER Gábor wrote:\n> \n> On Mon, Dec 18, 2017 at 10:33 PM, Lars Schneider\n> wrote:\n>> \n>>> On 16 Dec 2017, at 13:57, SZEDER Gábor wrote:\n>>> \n>>> Ever since we have started to use Travis CI in 522354d70 (Add Travis\n>>> CI support, 2015-11-27), our 64 bit Linux build jobs install the\n>>> 'languate-pack-is' package. That commit doesn't discuss why it was\n>>> deemed necessary back then, but Travis CI can build and test Git\n>>> without that package just fine, even that commit introducing Travis CI\n>>> support.\n>> \n>> If I remember correctly then we had to install the Icelandic\n>> language pack for the i18n tests (e.g. t0204):\n>> \n>> https://github.com/git/git/blob/master/t/lib-gettext.sh\n>> https://packages.ubuntu.com/trusty/language-pack-is\n>> \n>> However, I checked your Travis-CI and I cannot spot any errors:\n>> https://travis-ci.org/szeder/git/jobs/317494789\n> \n> Ah, now I start to understand. I was only looking for errors, too, but\n> there are none, because the tests requiring the language pack are\n> protected by the appropriate prerequisites. On my box, without and\n> with the language pack:\n> \n> $ ./t0204-gettext-reencode-sanity.sh\n> # lib-gettext: No is_IS UTF-8 locale available\n> # lib-gettext: No is_IS ISO-8859-1 locale available\n> ok 1 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files /\n> Icelandic (missing GETTEXT_LOCALE)\n> ok 2 # skip gettext: Emitting UTF-8 from our UTF-8 *.mo files /\n> Runes (missing GETTEXT_LOCALE)\n> ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files /\n> Icelandic (missing GETTEXT_ISO_LOCALE)\n> ok 4 # skip gettext: impossible ISO-8859-1 output (missing GETTEXT_ISO_LOCALE)\n> ok 5 # skip gettext: Fetching a UTF-8 msgid -> UTF-8 (missing GETTEXT_LOCALE)\n> ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> ok 7 # skip gettext.c: git init UTF-8 -> UTF-8 (missing GETTEXT_LOCALE)\n> ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> # passed all 8 test(s)\n> \n> $ sudo apt-get install language-pack-is\n> [...]\n> $ ./t0204-gettext-reencode-sanity.sh\n> # lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale\n> # lib-gettext: No is_IS ISO-8859-1 locale available\n> ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic\n> ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes\n> ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files /\n> Icelandic (missing GETTEXT_ISO_LOCALE)\n> ok 4 # skip gettext: impossible ISO-8859-1 output (missing GETTEXT_ISO_LOCALE)\n> ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8\n> ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> ok 7 - gettext.c: git init UTF-8 -> UTF-8\n> ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> # passed all 8 test(s)\n> 1..8\n> \n> I'd expect something like this in the Travis CI build jobs as well, but\n> prove hides the detailed output.\n\nAhh. That explains it!\n\n\n> It seems we would loose coverage with this patch, so it should be\n> dropped.\n\nYeah. I think we should add a comment to the travis.yml to avoid\nfuture confusion. I'll do it unless you beat me to it with a re-roll.\n\n\n- Lars\n\n> \n>> I am a bit puzzled. Maybe the Icelandic language pack was not part\n>> of the Ubuntu image that was used when we introduced Travis CI?\n>> \n>> The Ubuntu default image was 12.04 back then:\n>> https://travis-ci.org/git/git/jobs/100241871\n>> \n>> Nowadays it is 14.04.\n>> \n>> @Avar:\n>> Do you know what kind of errors we should expect if the language\n>> pack is not installed?\n>> \n>> Thanks,\n>> Lars\n>> \n>>> \n>>> Signed-off-by: SZEDER Gábor \n>>> ---\n>>> .travis.yml | 1 -\n>>> 1 file changed, 1 deletion(-)\n>>> \n>>> diff --git a/.travis.yml b/.travis.yml\n>>> index 4684b3f4f..ea11b5af6 100644\n>>> --- a/.travis.yml\n>>> +++ b/.travis.yml\n>>> @@ -17,7 +17,6 @@ compiler:\n>>> addons:\n>>> apt:\n>>> packages:\n>>> - - language-pack-is\n>>> - git-svn\n>>> - apache2\n>>> \n>>> --\n>>> 2.15.1.429.ga000dd9c7\n\n"},{"id":"335001","messageId":"xmqq3747y8b0.fsf@gitster.mtv.corp.google.com","threadId":"47090","inReplyTo":"AB053B56-5318-47D4-8ADE-0A72B9860BED@gmail.com","subject":"Re: [PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-18T22:34:27Z","receivedAt":"2017-12-18T22:34:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n>> It seems we would loose coverage with this patch, so it should be\n>> dropped.\n>\n> Yeah. I think we should add a comment to the travis.yml to avoid\n> future confusion. I'll do it unless you beat me to it with a re-roll.\n\nRather, it should be commented close to where is locale is used in\ntests, and possibly in t/README, I would think. Those who want to\nuse t/*, not necessarily using Travis, would benefit from the hint,\n\"installing icelandic locale may give you better test coverage\", or\nsomething like that.\n\n\n"},{"id":"335017","messageId":"CAM0VKjnctWRatbjB7EPHH+5zLjZLY+1umgFwz=bcV=g25eTD4w@mail.gmail.com","threadId":"47090","inReplyTo":"CAM0VKj=xVfCd5qR4jXwFGDxSABgyAeZY5qJxUP3GHvUE7gjw6A@mail.gmail.com","subject":"Re: [PATCH v2 6/8] travis-ci: don't install 'language-pack-is' package","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-19T12:22:42Z","receivedAt":"2017-12-19T12:22:50Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Mon, Dec 18, 2017 at 11:04 PM, SZEDER Gábor wrote:\n\n> $ sudo apt-get install language-pack-is\n> [...]\n> $ ./t0204-gettext-reencode-sanity.sh\n> # lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale\n> # lib-gettext: No is_IS ISO-8859-1 locale available\n> ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic\n> ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes\n> ok 3 # skip gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files /\n> Icelandic (missing GETTEXT_ISO_LOCALE)\n> ok 4 # skip gettext: impossible ISO-8859-1 output (missing GETTEXT_ISO_LOCALE)\n> ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8\n> ok 6 # skip gettext: Fetching a UTF-8 msgid -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> ok 7 - gettext.c: git init UTF-8 -> UTF-8\n> ok 8 # skip gettext.c: git init UTF-8 -> ISO-8859-1 (missing\n> GETTEXT_ISO_LOCALE)\n> # passed all 8 test(s)\n> 1..8\n>\n> I'd expect something like this in the Travis CI build jobs as well, but\n> prove hides the detailed output.\n>\n> It seems we would loose coverage with this patch, so it should be\n> dropped.\n\nNot so fast!\n\nNotice how there are still a few tests skipped above, because of missing\nGETTEXT_ISO_LOCALE.\n\n # grep is_IS /etc/locale.gen\n # is_IS ISO-8859-1\n # is_IS.UTF-8 UTF-8\n # sed -i -e 's/^# is_IS/is_IS/' /etc/locale.gen\n # locale-gen\n Generating locales (this might take a while)...\n [...]\n is_IS.ISO-8859-1... done\n is_IS.UTF-8... done\n Generation complete.\n\nBoth UTF-8 and ISO Icelandic locales are generated, good.\n\n $ $ ./t0204-gettext-reencode-sanity.sh\n # lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale\n # lib-gettext: Found 'is_IS.iso88591' as an is_IS ISO-8859-1 locale\n ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic\n ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes\n ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic\n ok 4 - gettext: impossible ISO-8859-1 output\n ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8\n ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1\n ok 7 - gettext.c: git init UTF-8 -> UTF-8\n ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1\n # passed all 8 test(s)\n\nAnd now all those tests are run, great!\nBut look what happens without the language pack:\n\n # dpkg -P language-pack-is{,-base}\n [...]\n # grep is_IS /etc/locale.gen\n is_IS ISO-8859-1\n is_IS.UTF-8 UTF-8\n buzz ~# locale-gen\n Generating locales (this might take a while)...\n [...]\n is_IS.ISO-8859-1... done\n is_IS.UTF-8... done\n Generation complete.\n\nStill both Icelandic locales are generated.\n\n $ ./t0204-gettext-reencode-sanity.sh\n # lib-gettext: Found 'is_IS.utf8' as an is_IS UTF-8 locale\n # lib-gettext: Found 'is_IS.iso88591' as an is_IS ISO-8859-1 locale\n ok 1 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Icelandic\n ok 2 - gettext: Emitting UTF-8 from our UTF-8 *.mo files / Runes\n ok 3 - gettext: Emitting ISO-8859-1 from our UTF-8 *.mo files / Icelandic\n ok 4 - gettext: impossible ISO-8859-1 output\n ok 5 - gettext: Fetching a UTF-8 msgid -> UTF-8\n ok 6 - gettext: Fetching a UTF-8 msgid -> ISO-8859-1\n ok 7 - gettext.c: git init UTF-8 -> UTF-8\n ok 8 - gettext.c: git init UTF-8 -> ISO-8859-1\n # passed all 8 test(s)\n\nAnd still all those tests are run!\n\nI did run all test scripts sourcing 'lib-gettext.sh' with both locales\ngenerated and didn't see any errors, independently from whether the\nlanguage pack was installed or not. I still have a few skipped tests\n(because of no GETTEXT_POISON and something PCRE-related), but those,\ntoo, are independent from the language pack.\n\nSo it seems that we don't need 'language-pack-is' after all; what we do\nneed are the ISO and UTF-8 Icelandic locales. Not sure we can modify\n/etc/locale.gen without sudo in the Travis CI build job, though, and\nI've run out of time to try.\n\n\nGábor\n"},{"id":"335357","messageId":"20171227163603.13313-1-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171216125418.10743-1-szeder.dev@gmail.com","subject":"[PATCH v3 0/4] Rest of the Travis CI fixes","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T16:35:59Z","receivedAt":"2017-12-27T16:36:23Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"These are the rest of the fixes that were in the second half of the v2\npatch series but not already in 'sg/travis-fixes', which was already\nmerged to next, perhaps a bit prematurely.\n\nPatch 1/4 fixes some rough edges related to tracing executed commands\naccording to the discussion.\nPatches 2/4, 3/4 are the same as patches 5/8 and 7/8 in v2, while patch\n4/4 includes the small update I sent out as v2.1 of patch 8/8.\n\nI dropped v2's patch 6/8 (travis-ci: don't install 'language-pack-is'\npackage) for now: these locale-related things require more\ninvestigation.\n\nv2: https://public-inbox.org/git/20171216125418.10743-1-szeder.dev@gmail.com/\nv1: https://public-inbox.org/git/20171211233446.10596-1-szeder.dev@gmail.com/\n\nSZEDER Gábor (4):\n travis-ci: fine tune the use of 'set -x' in 'ci/*' scripts\n travis-ci: don't install default addon packages for the 32 bit Linux\n build\n travis-ci: save prove state for the 32 bit Linux build\n travis-ci: only print test failures if there are test results\n available\n\n .travis.yml | 1 +\n ci/lib-travisci.sh | 4 +++-\n ci/print-test-failures.sh | 9 +++++++++\n ci/run-linux32-build.sh | 3 +++\n ci/run-linux32-docker.sh | 1 +\n ci/run-windows-build.sh | 5 +++++\n 6 files changed, 22 insertions(+), 1 deletion(-)\n\n-- \n2.15.1.500.g54ea76cc4\n\n"},{"id":"335358","messageId":"20171227163603.13313-2-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-1-szeder.dev@gmail.com","subject":"[PATCH v3 1/4] travis-ci: fine tune the use of 'set -x' in 'ci/*' scripts","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T16:36:00Z","receivedAt":"2017-12-27T16:36:29Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"The change in commit 4f2636667 (travis-ci: use 'set -x' in 'ci/*'\nscripts for extra tracing output, 2017-12-12) left a couple of rough\nedges:\n\n - 'ci/run-linux32-build.sh' is executed in a Docker container and\n therefore doesn't source 'ci/lib-travisci.sh', which would enable\n tracing executed commands. Enable 'set -x' in this script, too.\n\n - 'ci/print-test-failures.sh' iterates over all the files containing\n the exit codes of all the execued test scripts. Since there are\n over 800 such files, the loop produces way too much noise with\n tracing executed commands enabled, so disable 'set -x' for this\n script.\n\n - 'ci/run-windows-build.sh' busily waits in a loop for the result of\n the Windows build, producing too much noise with tracing executed\n commands enabled as well. Disable 'set -x' for the duration of\n that loop.\n\nigned-off-by: SZEDER Gábor \n---\n ci/lib-travisci.sh | 4 +++-\n ci/print-test-failures.sh | 3 +++\n ci/run-linux32-build.sh | 2 ++\n ci/run-windows-build.sh | 5 +++++\n 4 files changed, 13 insertions(+), 1 deletion(-)\n\ndiff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\nindex 331d3eb3a..348fe3c3c 100755\n--- a/ci/lib-travisci.sh\n+++ b/ci/lib-travisci.sh\n@@ -22,7 +22,9 @@ skip_branch_tip_with_tag () {\n }\n \n # Set 'exit on error' for all CI scripts to let the caller know that\n-# something went wrong\n+# something went wrong.\n+# Set tracing executed commands, primarily setting environment variables\n+# and installing dependencies.\n set -ex\n \n skip_branch_tip_with_tag\ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex 8c8973cbf..97cc05901 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -5,6 +5,9 @@\n \n . ${0%/*}/lib-travisci.sh\n \n+# Tracing executed commands would produce too much noise in the loop below.\n+set +x\n+\n for TEST_EXIT in t/test-results/*.exit\n do\n \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\ndiff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\nindex e30fb2cdd..a8518eddf 100755\n--- a/ci/run-linux32-build.sh\n+++ b/ci/run-linux32-build.sh\n@@ -6,6 +6,8 @@\n # run-linux32-build.sh [host-user-id]\n #\n \n+set -x\n+\n # Update packages to the latest available versions\n linux32 --32bit i386 sh -c '\n apt update >/dev/null &&\ndiff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh\nindex 8757b3a97..86999268a 100755\n--- a/ci/run-windows-build.sh\n+++ b/ci/run-windows-build.sh\n@@ -69,6 +69,10 @@ esac\n \n echo \"Visual Studio Team Services Build #${BUILD_ID}\"\n \n+# Tracing execued commands would produce too much noise in the waiting\n+# loop below.\n+set +x\n+\n # Wait until build job finished\n STATUS=\n RESULT=\n@@ -90,6 +94,7 @@ done\n # Print log\n echo \"\"\n echo \"\"\n+set -x\n gfwci \"action=log&buildId=$BUILD_ID\" | cut -c 30-\n \n # Set exit code for TravisCI\n-- \n2.15.1.500.g54ea76cc4\n\n"},{"id":"335359","messageId":"20171227163603.13313-3-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-1-szeder.dev@gmail.com","subject":"[PATCH v3 2/4] travis-ci: don't install default addon packages for the 32 bit Linux build","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T16:36:01Z","receivedAt":"2017-12-27T16:36:33Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"The 32 bit Linux build job compiles Git and runs the test suite in a\nDocker container, while the additional packages (apache2, git-svn,\nlanguage-pack-is) are installed on the host, therefore don't have\nany effect and are unnecessary.\n\nSigned-off-by: SZEDER Gábor \n---\n .travis.yml | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 7c9aa0557..4684b3f4f 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -42,6 +42,7 @@ matrix:\n - env: jobname=Linux32\n os: linux\n compiler:\n+ addons:\n services:\n - docker\n before_install:\n-- \n2.15.1.500.g54ea76cc4\n\n"},{"id":"335360","messageId":"20171227163603.13313-4-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-1-szeder.dev@gmail.com","subject":"[PATCH v3 3/4] travis-ci: save prove state for the 32 bit Linux build","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T16:36:02Z","receivedAt":"2017-12-27T16:36:37Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"This change follows suit of 6272ed319 (travis-ci: run previously\nfailed tests first, then slowest to fastest, 2016-01-26), which did\nthis for the Linux and OSX build jobs. Travis CI build jobs run the\ntests parallel, which is sligtly faster when tests are run in slowest\nto fastest order, shortening the overall runtime of this build job by\nabout a minute / 10%.\n\nNote, that the 32 bit Linux build job runs the tests suite in a Docker\ncontainer and we have to share the Travis CI cache directory with the\ncontainer as a second volume. Otherwise we couldn't use a symlink\npointing to the prove state file in the cache directory, because\nthat's outside of the directory hierarchy accessible from within the\ncontainer.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/run-linux32-build.sh | 1 +\n ci/run-linux32-docker.sh | 1 +\n 2 files changed, 2 insertions(+)\n\ndiff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\nindex a8518eddf..c19c50c1c 100755\n--- a/ci/run-linux32-build.sh\n+++ b/ci/run-linux32-build.sh\n@@ -27,6 +27,7 @@ test -z $HOST_UID || (CI_USER=\"ci\" && useradd -u $HOST_UID $CI_USER) &&\n # Build and test\n linux32 --32bit i386 su -m -l $CI_USER -c '\n cd /usr/src/git &&\n+ ln -s /tmp/travis-cache/.prove t/.prove &&\n make --jobs=2 &&\n make --quiet test\n '\ndiff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh\nindex 0edf63acf..3a8b2ba42 100755\n--- a/ci/run-linux32-docker.sh\n+++ b/ci/run-linux32-docker.sh\n@@ -19,5 +19,6 @@ docker run \\\n \t--env GIT_TEST_OPTS \\\n \t--env GIT_TEST_CLONE_2GB \\\n \t--volume \"${PWD}:/usr/src/git\" \\\n+\t--volume \"${HOME}/travis-cache:/tmp/travis-cache\" \\\n \tdaald/ubuntu32:xenial \\\n \t/usr/src/git/ci/run-linux32-build.sh $(id -u $USER)\n-- \n2.15.1.500.g54ea76cc4\n\n"},{"id":"335361","messageId":"20171227163603.13313-5-szeder.dev@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-1-szeder.dev@gmail.com","subject":"[PATCH v3 4/4] travis-ci: only print test failures if there are test results available","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T16:36:03Z","receivedAt":"2017-12-27T16:36:39Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"When a build job running the test suite fails, our\n'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'\nfiles to find failed tests and prints their verbose output. However,\nif a build job were to fail before it ever gets to run the test suite,\nthen there will be no files to match the above pattern and the shell\nwill take the pattern literally, resulting in errors like this in the\ntrace log:\n\n cat: t/test-results/*.exit: No such file or directory\n ------------------------------------------------------------------------\n t/test-results/*.out...\n ------------------------------------------------------------------------\n cat: t/test-results/*.out: No such file or directory\n\nCheck upfront and proceed only if there are any such files present.\n\nSigned-off-by: SZEDER Gábor \n---\n ci/print-test-failures.sh | 6 ++++++\n 1 file changed, 6 insertions(+)\n\ndiff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\nindex 97cc05901..4f261ddc0 100755\n--- a/ci/print-test-failures.sh\n+++ b/ci/print-test-failures.sh\n@@ -8,6 +8,12 @@\n # Tracing executed commands would produce too much noise in the loop below.\n set +x\n \n+if ! ls t/test-results/*.exit >/dev/null 2>/dev/null\n+then\n+\techo \"Build job failed before the tests could have been run\"\n+\texit\n+fi\n+\n for TEST_EXIT in t/test-results/*.exit\n do\n \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n-- \n2.15.1.500.g54ea76cc4\n\n"},{"id":"335374","messageId":"757B917B-1F47-487B-989B-C31BD6F43C61@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-2-szeder.dev@gmail.com","subject":"Re: [PATCH v3 1/4] travis-ci: fine tune the use of 'set -x' in 'ci/*' scripts","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-27T18:35:55Z","receivedAt":"2017-12-27T18:36:01Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 27 Dec 2017, at 17:36, SZEDER Gábor wrote:\n> \n> The change in commit 4f2636667 (travis-ci: use 'set -x' in 'ci/*'\n> scripts for extra tracing output, 2017-12-12) left a couple of rough\n> edges:\n> \n> - 'ci/run-linux32-build.sh' is executed in a Docker container and\n> therefore doesn't source 'ci/lib-travisci.sh', which would enable\n> tracing executed commands. Enable 'set -x' in this script, too.\n> \n> - 'ci/print-test-failures.sh' iterates over all the files containing\n> the exit codes of all the execued test scripts. Since there are\ns/execued/executed/\n\n> over 800 such files, the loop produces way too much noise with\n> tracing executed commands enabled, so disable 'set -x' for this\n> script.\n> \n> - 'ci/run-windows-build.sh' busily waits in a loop for the result of\n> the Windows build, producing too much noise with tracing executed\n> commands enabled as well. Disable 'set -x' for the duration of\n> that loop.\n> \n> igned-off-by: SZEDER Gábor \n> ---\n> ci/lib-travisci.sh | 4 +++-\n> ci/print-test-failures.sh | 3 +++\n> ci/run-linux32-build.sh | 2 ++\n> ci/run-windows-build.sh | 5 +++++\n> 4 files changed, 13 insertions(+), 1 deletion(-)\n> \n> diff --git a/ci/lib-travisci.sh b/ci/lib-travisci.sh\n> index 331d3eb3a..348fe3c3c 100755\n> --- a/ci/lib-travisci.sh\n> +++ b/ci/lib-travisci.sh\n> @@ -22,7 +22,9 @@ skip_branch_tip_with_tag () {\n> }\n> \n> # Set 'exit on error' for all CI scripts to let the caller know that\n> -# something went wrong\n> +# something went wrong.\n> +# Set tracing executed commands, primarily setting environment variables\n> +# and installing dependencies.\n\nMaybe:\n\n # something went wrong. Enable tracing to ease debugging on TravisCI.\n\nI would move the \"primarily setting environment ...\" either to the\ntop of the file or to the respective section below.\n\n\nBut this is a minor nit. The rest of the patch looks very good.\n\nThanks,\nLars\n \n\n> set -ex\n> \n> skip_branch_tip_with_tag\n> diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\n> index 8c8973cbf..97cc05901 100755\n> --- a/ci/print-test-failures.sh\n> +++ b/ci/print-test-failures.sh\n> @@ -5,6 +5,9 @@\n> \n> . ${0%/*}/lib-travisci.sh\n> \n> +# Tracing executed commands would produce too much noise in the loop below.\n> +set +x\n> +\n> for TEST_EXIT in t/test-results/*.exit\n> do\n> \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n> diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\n> index e30fb2cdd..a8518eddf 100755\n> --- a/ci/run-linux32-build.sh\n> +++ b/ci/run-linux32-build.sh\n> @@ -6,6 +6,8 @@\n> # run-linux32-build.sh [host-user-id]\n> #\n> \n> +set -x\n> +\n> # Update packages to the latest available versions\n> linux32 --32bit i386 sh -c '\n> apt update >/dev/null &&\n> diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh\n> index 8757b3a97..86999268a 100755\n> --- a/ci/run-windows-build.sh\n> +++ b/ci/run-windows-build.sh\n> @@ -69,6 +69,10 @@ esac\n> \n> echo \"Visual Studio Team Services Build #${BUILD_ID}\"\n> \n> +# Tracing execued commands would produce too much noise in the waiting\n> +# loop below.\n> +set +x\n> +\n> # Wait until build job finished\n> STATUS=\n> RESULT=\n> @@ -90,6 +94,7 @@ done\n> # Print log\n> echo \"\"\n> echo \"\"\n> +set -x\n> gfwci \"action=log&buildId=$BUILD_ID\" | cut -c 30-\n> \n> # Set exit code for TravisCI\n> -- \n> 2.15.1.500.g54ea76cc4\n> \n\n"},{"id":"335375","messageId":"DF7D99CD-EBF2-40A1-B6F6-28694E213F42@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-3-szeder.dev@gmail.com","subject":"Re: [PATCH v3 2/4] travis-ci: don't install default addon packages for the 32 bit Linux build","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-27T18:41:05Z","receivedAt":"2017-12-27T18:41:11Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 27 Dec 2017, at 17:36, SZEDER Gábor wrote:\n> \n> The 32 bit Linux build job compiles Git and runs the test suite in a\n> Docker container, while the additional packages (apache2, git-svn,\n> language-pack-is) are installed on the host, therefore don't have\n> any effect and are unnecessary.\n\nNice optimization - ACK!\n\nThanks,\nLars\n\n\n\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> .travis.yml | 1 +\n> 1 file changed, 1 insertion(+)\n> \n> diff --git a/.travis.yml b/.travis.yml\n> index 7c9aa0557..4684b3f4f 100644\n> --- a/.travis.yml\n> +++ b/.travis.yml\n> @@ -42,6 +42,7 @@ matrix:\n> - env: jobname=Linux32\n> os: linux\n> compiler:\n> + addons:\n> services:\n> - docker\n> before_install:\n> -- \n> 2.15.1.500.g54ea76cc4\n> \n\n"},{"id":"335377","messageId":"DB47DCB3-DF66-437A-BF0B-4DF1838C2F7F@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-4-szeder.dev@gmail.com","subject":"Re: [PATCH v3 3/4] travis-ci: save prove state for the 32 bit Linux build","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-27T18:46:56Z","receivedAt":"2017-12-27T18:47:13Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 27 Dec 2017, at 17:36, SZEDER Gábor wrote:\n> \n> This change follows suit of 6272ed319 (travis-ci: run previously\n> failed tests first, then slowest to fastest, 2016-01-26), which did\n> this for the Linux and OSX build jobs. Travis CI build jobs run the\n> tests parallel, which is sligtly faster when tests are run in slowest\n> to fastest order, shortening the overall runtime of this build job by\n> about a minute / 10%.\n> \n> Note, that the 32 bit Linux build job runs the tests suite in a Docker\n> container and we have to share the Travis CI cache directory with the\n> container as a second volume. Otherwise we couldn't use a symlink\n> pointing to the prove state file in the cache directory, because\n> that's outside of the directory hierarchy accessible from within the\n> container.\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> ci/run-linux32-build.sh | 1 +\n> ci/run-linux32-docker.sh | 1 +\n> 2 files changed, 2 insertions(+)\n> \n> diff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\n> index a8518eddf..c19c50c1c 100755\n> --- a/ci/run-linux32-build.sh\n> +++ b/ci/run-linux32-build.sh\n> @@ -27,6 +27,7 @@ test -z $HOST_UID || (CI_USER=\"ci\" && useradd -u $HOST_UID $CI_USER) &&\n> # Build and test\n> linux32 --32bit i386 su -m -l $CI_USER -c '\n> cd /usr/src/git &&\n> + ln -s /tmp/travis-cache/.prove t/.prove &&\n> make --jobs=2 &&\n> make --quiet test\n> '\n> diff --git a/ci/run-linux32-docker.sh b/ci/run-linux32-docker.sh\n> index 0edf63acf..3a8b2ba42 100755\n> --- a/ci/run-linux32-docker.sh\n> +++ b/ci/run-linux32-docker.sh\n> @@ -19,5 +19,6 @@ docker run \\\n> \t--env GIT_TEST_OPTS \\\n> \t--env GIT_TEST_CLONE_2GB \\\n> \t--volume \"${PWD}:/usr/src/git\" \\\n> +\t--volume \"${HOME}/travis-cache:/tmp/travis-cache\" \\\n\nI assume \"${HOME}/travis-cache:/usr/src/git/t/.prove\" would not\nwork because that would be a mapping in another mapping?\n\nThanks,\nLars\n\n> \tdaald/ubuntu32:xenial \\\n> \t/usr/src/git/ci/run-linux32-build.sh $(id -u $USER)\n> -- \n> 2.15.1.500.g54ea76cc4\n> \n\n"},{"id":"335381","messageId":"53B4EF08-4511-4668-B7F4-7BC593550E5F@gmail.com","threadId":"47090","inReplyTo":"20171227163603.13313-5-szeder.dev@gmail.com","subject":"Re: [PATCH v3 4/4] travis-ci: only print test failures if there are test results available","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-27T18:52:18Z","receivedAt":"2017-12-27T18:52:23Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 27 Dec 2017, at 17:36, SZEDER Gábor wrote:\n> \n> When a build job running the test suite fails, our\n> 'ci/print-test-failures.sh' script scans all 't/test-results/*.exit'\n> files to find failed tests and prints their verbose output. However,\n> if a build job were to fail before it ever gets to run the test suite,\n> then there will be no files to match the above pattern and the shell\n> will take the pattern literally, resulting in errors like this in the\n> trace log:\n> \n> cat: t/test-results/*.exit: No such file or directory\n> ------------------------------------------------------------------------\n> t/test-results/*.out...\n> ------------------------------------------------------------------------\n> cat: t/test-results/*.out: No such file or directory\n> \n> Check upfront and proceed only if there are any such files present.\n> \n> Signed-off-by: SZEDER Gábor \n> ---\n> ci/print-test-failures.sh | 6 ++++++\n> 1 file changed, 6 insertions(+)\n> \n> diff --git a/ci/print-test-failures.sh b/ci/print-test-failures.sh\n> index 97cc05901..4f261ddc0 100755\n> --- a/ci/print-test-failures.sh\n> +++ b/ci/print-test-failures.sh\n> @@ -8,6 +8,12 @@\n> # Tracing executed commands would produce too much noise in the loop below.\n> set +x\n> \n> +if ! ls t/test-results/*.exit >/dev/null 2>/dev/null\n> +then\n> +\techo \"Build job failed before the tests could have been run\"\n> +\texit\n> +fi\n> +\n\nLook good to me!\n\nMinor nit: I am not 100% sure about the grammar but I am no native speaker\nand can't really tell.\n\n\nThanks,\nLars\n\n\n\n> for TEST_EXIT in t/test-results/*.exit\n> do\n> \tif [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ]\n> -- \n> 2.15.1.500.g54ea76cc4\n> \n\n"},{"id":"335396","messageId":"CAM0VKj=3ZQhjwtUFXFsMPzFrs7CYntrjJOcdQfOZ+0jjVVGVog@mail.gmail.com","threadId":"47090","inReplyTo":"DB47DCB3-DF66-437A-BF0B-4DF1838C2F7F@gmail.com","subject":"Re: [PATCH v3 3/4] travis-ci: save prove state for the 32 bit Linux build","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2017-12-27T21:42:04Z","receivedAt":"2017-12-27T21:42:11Z","isPatch":true,"sender":{"key":"szeder.dev@gmail.com","avatar":"https://avatars.githubusercontent.com/u/116324?v=4"},"body":"On Wed, Dec 27, 2017 at 7:46 PM, Lars Schneider\n wrote:\n>> + --volume \"${HOME}/travis-cache:/tmp/travis-cache\" \\\n>\n> I assume \"${HOME}/travis-cache:/usr/src/git/t/.prove\" would not\n> work because that would be a mapping in another mapping?\n\n't/.prove' is a file, but '.../travis-cache' is a directory. It must\nbe, because Travis CI caches whole directories.\n\nGábor\n"},{"id":"335463","messageId":"A3474967-F081-4CD5-8A7E-81615EC5B3B7@gmail.com","threadId":"47090","inReplyTo":"CAM0VKj=3ZQhjwtUFXFsMPzFrs7CYntrjJOcdQfOZ+0jjVVGVog@mail.gmail.com","subject":"Re: [PATCH v3 3/4] travis-ci: save prove state for the 32 bit Linux build","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-12-28T11:17:47Z","receivedAt":"2017-12-28T11:17:54Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 27 Dec 2017, at 22:42, SZEDER Gábor wrote:\n> \n> On Wed, Dec 27, 2017 at 7:46 PM, Lars Schneider\n> wrote:\n>>> + --volume \"${HOME}/travis-cache:/tmp/travis-cache\" \\\n>> \n>> I assume \"${HOME}/travis-cache:/usr/src/git/t/.prove\" would not\n>> work because that would be a mapping in another mapping?\n> \n> 't/.prove' is a file, but '.../travis-cache' is a directory. It must\n> be, because Travis CI caches whole directories.\n\nOf course. Your solution is the right one.\n\nThanks,\nLars"}]}