{"thread":{"id":"40833","subject":"[PATCH v6 0/6] Add Travis CI support","startedAt":"2015-11-19T08:58:05Z","lastAt":"2015-11-23T21:19:16Z","messageCount":14,"participants":["larsxschneider@gmail.com","Jeff King","Lars Schneider","Luke Diamand"],"isPatch":true,"patchVersion":6,"patchTotal":6},"messages":[{"id":"273491","messageId":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":null,"subject":"[PATCH v6 0/6] Add Travis CI support","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:05Z","receivedAt":"2015-11-19T08:58:05Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\ndiff to v5:\n* check if PID file still exists on P4D cleanup (thanks Luke)\n* fix space/tab formatting error\n* add sleep to timeout loops (thanks Luke)\n* replace 'date +%s' with platform independent Python function (thanks Eric and Luke)\n\nWith the patches of this series the Travis CI test stability increases.\nHowever, as I am \"stress testing\" the Travis CI infrastructure you can\nsee that it is not perfect: https://travis-ci.org/larsxschneider/git/builds\n\nNevertheless, I believe that Travis CI integration has still value as\ncontributors can test their patches easily on Linux and OS X before\nposting them.\n\n@junio / @peff: Do you consider merging this?\n\nThanks,\nLars\n\nLars Schneider (6):\n implement test_might_fail using a refactored test_must_fail\n add \"ok=sigpipe\" to test_must_fail and use it to fix flaky tests\n git-p4: retry kill/cleanup operations in tests with timeout\n git-p4: add p4d timeout in tests\n git-p4: add trap to kill p4d on test exit\n Add Travis CI support\n\n .travis.yml | 131 ++++++++++++++++++++++++++++++++++++++++\n t/lib-git-p4.sh | 71 +++++++++++++++++++---\n t/t5504-fetch-receive-strict.sh | 3 +-\n t/t5516-fetch-push.sh | 6 +-\n t/test-lib-functions.sh | 39 +++++++-----\n 5 files changed, 221 insertions(+), 29 deletions(-)\n create mode 100644 .travis.yml\n\n--\n2.5.1\n"},{"id":"273492","messageId":"1447923491-15330-2-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 1/6] implement test_might_fail using a refactored test_must_fail","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:06Z","receivedAt":"2015-11-19T08:58:06Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nAdd an (optional) first parameter \"ok=\" to test_must_fail\nand return success for \"\". Add \"success\" as\n\"\" and use it to implement \"test_might_fail\". This removes\nredundancies in test-lib-function.sh.\n\nSigned-off-by: Junio C Hamano \nSigned-off-by: Lars Schneider \n---\n t/test-lib-functions.sh | 35 ++++++++++++++++++++---------------\n 1 file changed, 20 insertions(+), 15 deletions(-)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 73e37a1..1e762da 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -582,18 +582,32 @@ test_line_count () {\n # the failure could be due to a segv. We want a controlled failure.\n \n test_must_fail () {\n+\tcase \"$1\" in\n+\tok=*)\n+\t\t_test_ok=${1#ok=}\n+\t\tshift\n+\t\t;;\n+\t*)\n+\t\t_test_ok=\n+\t\t;;\n+\tesac\n \t\"$@\"\n \texit_code=$?\n-\tif test $exit_code = 0; then\n+\tif ! case \",$_test_ok,\" in *,success,*) false;; esac &&\n+\t\ttest $exit_code = 0\n+\tthen\n \t\techo >&2 \"test_must_fail: command succeeded: $*\"\n-\t\treturn 1\n-\telif test $exit_code -gt 129 && test $exit_code -le 192; then\n+\t\treturn 0\n+\telif test $exit_code -gt 129 && test $exit_code -le 192\n+\tthen\n \t\techo >&2 \"test_must_fail: died by signal: $*\"\n \t\treturn 1\n-\telif test $exit_code = 127; then\n+\telif test $exit_code = 127\n+\tthen\n \t\techo >&2 \"test_must_fail: command not found: $*\"\n \t\treturn 1\n-\telif test $exit_code = 126; then\n+\telif test $exit_code = 126\n+\tthen\n \t\techo >&2 \"test_must_fail: valgrind error: $*\"\n \t\treturn 1\n \tfi\n@@ -612,16 +626,7 @@ test_must_fail () {\n # because we want to notice if it fails due to segv.\n \n test_might_fail () {\n-\t\"$@\"\n-\texit_code=$?\n-\tif test $exit_code -gt 129 && test $exit_code -le 192; then\n-\t\techo >&2 \"test_might_fail: died by signal: $*\"\n-\t\treturn 1\n-\telif test $exit_code = 127; then\n-\t\techo >&2 \"test_might_fail: command not found: $*\"\n-\t\treturn 1\n-\tfi\n-\treturn 0\n+\ttest_must_fail ok=success \"$@\"\n }\n \n # Similar to test_must_fail and test_might_fail, but check that a\n-- \n2.5.1\n"},{"id":"273493","messageId":"1447923491-15330-3-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 2/6] add \"ok=sigpipe\" to test_must_fail and use it to fix flaky tests","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:07Z","receivedAt":"2015-11-19T08:58:07Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nt5516 \"75 - deny fetch unreachable SHA1, allowtipsha1inwant=true\" is\nflaky in the following case:\n1. remote upload-pack finds out \"not our ref\"\n2. remote sends a response and closes the pipe\n3. fetch-pack still tries to write commands to the remote upload-pack\n4. write call in wrapper.c dies with SIGPIPE\n\nt5504 \"9 - push with transfer.fsckobjects\" is flaky, too, and returns\nSIGPIPE once in a while. I had to remove the final \"To dst...\" output\ncheck because there is no output if the process dies with SIGPUPE.\n\nAccept such a death-with-sigpipe also as OK when we are expecting a\nfailure.\n\nSigned-off-by: Lars Schneider \n---\n t/t5504-fetch-receive-strict.sh | 3 +--\n t/t5516-fetch-push.sh | 6 +++---\n t/test-lib-functions.sh | 4 ++++\n 3 files changed, 8 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh\nindex 44f3d5f..a9e382c 100755\n--- a/t/t5504-fetch-receive-strict.sh\n+++ b/t/t5504-fetch-receive-strict.sh\n@@ -111,8 +111,7 @@ test_expect_success 'push with transfer.fsckobjects' '\n \t\tcd dst &&\n \t\tgit config transfer.fsckobjects true\n \t) &&\n-\ttest_must_fail git push --porcelain dst master:refs/heads/test >act &&\n-\ttest_cmp exp act\n+\ttest_must_fail ok=sigpipe git push --porcelain dst master:refs/heads/test >act\n '\n \n cat >bogus-commit <<\\EOF\ndiff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh\nindex ec22c98..0a87e19 100755\n--- a/t/t5516-fetch-push.sh\n+++ b/t/t5516-fetch-push.sh\n@@ -1162,15 +1162,15 @@ do\n \t\tmk_empty shallow &&\n \t\t(\n \t\t\tcd shallow &&\n-\t\t\ttest_must_fail git fetch ../testrepo/.git $SHA1_3 &&\n-\t\t\ttest_must_fail git fetch ../testrepo/.git $SHA1_1 &&\n+\t\t\ttest_must_fail ok=sigpipe git fetch ../testrepo/.git $SHA1_3 &&\n+\t\t\ttest_must_fail ok=sigpipe git fetch ../testrepo/.git $SHA1_1 &&\n \t\t\tgit --git-dir=../testrepo/.git config uploadpack.allowreachablesha1inwant true &&\n \t\t\tgit fetch ../testrepo/.git $SHA1_1 &&\n \t\t\tgit cat-file commit $SHA1_1 &&\n \t\t\ttest_must_fail git cat-file commit $SHA1_2 &&\n \t\t\tgit fetch ../testrepo/.git $SHA1_2 &&\n \t\t\tgit cat-file commit $SHA1_2 &&\n-\t\t\ttest_must_fail git fetch ../testrepo/.git $SHA1_3\n+\t\t\ttest_must_fail ok=sigpipe git fetch ../testrepo/.git $SHA1_3\n \t\t)\n \t'\n done\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 1e762da..1fdc58c 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -598,6 +598,10 @@ test_must_fail () {\n \tthen\n \t\techo >&2 \"test_must_fail: command succeeded: $*\"\n \t\treturn 0\n+\telif ! case \",$_test_ok,\" in *,sigpipe,*) false;; esac &&\n+\t\ttest $exit_code = 141\n+\tthen\n+\t\treturn 0\n \telif test $exit_code -gt 129 && test $exit_code -le 192\n \tthen\n \t\techo >&2 \"test_must_fail: died by signal: $*\"\n-- \n2.5.1\n"},{"id":"273496","messageId":"1447923491-15330-4-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 3/6] git-p4: retry kill/cleanup operations in tests with timeout","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:08Z","receivedAt":"2015-11-19T08:58:08Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nIn rare cases kill/cleanup operations in tests fail. Retry these\noperations with a timeout to make the test less flaky.\n\nSigned-off-by: Lars Schneider \n---\n t/lib-git-p4.sh | 42 ++++++++++++++++++++++++++++++++++--------\n 1 file changed, 34 insertions(+), 8 deletions(-)\n\ndiff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh\nindex 7548225..3c9ad9a 100644\n--- a/t/lib-git-p4.sh\n+++ b/t/lib-git-p4.sh\n@@ -6,6 +6,10 @@\n # a subdirectory called \"$git\"\n TEST_NO_CREATE_REPO=NoThanks\n \n+# Some operations require multiple attempts to be successful. Define\n+# here the maximal retry timeout in seconds.\n+RETRY_TIMEOUT=60\n+\n . ./test-lib.sh\n \n if ! test_have_prereq PYTHON\n@@ -36,6 +40,15 @@ native_path() {\n \techo \"$path\"\n }\n \n+# On Solaris the 'date +%s' function is not supported and therefore we\n+# need this replacement.\n+# Attention: This function is not safe again against time offset updates\n+# at runtime (e.g. via NTP). The 'clock_gettime(CLOCK_MONOTONIC)'\n+# function could fix that but it is not in Python until 3.3.\n+time_in_seconds() {\n+\tpython -c 'import time; print int(time.time())'\n+}\n+\n # Try to pick a unique port: guess a large number, then hope\n # no more than one of each test is running.\n #\n@@ -121,22 +134,35 @@ p4_add_user() {\n \tEOF\n }\n \n+retry_until_success() {\n+\ttimeout=$(($(time_in_seconds) + $RETRY_TIMEOUT))\n+\tuntil \"$@\" 2>/dev/null || test $(time_in_seconds) -gt $timeout\n+\tdo\n+\t\tsleep 1\n+\tdone\n+}\n+\n+retry_until_fail() {\n+\ttimeout=$(($(time_in_seconds) + $RETRY_TIMEOUT))\n+\tuntil ! \"$@\" 2>/dev/null || test $(time_in_seconds) -gt $timeout\n+\tdo\n+\t\tsleep 1\n+\tdone\n+}\n+\n kill_p4d() {\n \tpid=$(cat \"$pidfile\")\n-\t# it had better exist for the first kill\n-\tkill $pid &&\n-\tfor i in 1 2 3 4 5 ; do\n-\t\tkill $pid >/dev/null 2>&1 || break\n-\t\tsleep 1\n-\tdone &&\n+\tretry_until_fail kill $pid\n+\tretry_until_fail kill -9 $pid\n \t# complain if it would not die\n \ttest_must_fail kill $pid >/dev/null 2>&1 &&\n \trm -rf \"$db\" \"$cli\" \"$pidfile\"\n }\n \n cleanup_git() {\n-\trm -rf \"$git\" &&\n-\tmkdir \"$git\"\n+\tretry_until_success rm -r \"$git\"\n+\ttest_must_fail test -d \"$git\" &&\n+\tretry_until_success mkdir \"$git\"\n }\n \n marshal_dump() {\n-- \n2.5.1\n"},{"id":"273494","messageId":"1447923491-15330-5-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 4/6] git-p4: add p4d timeout in tests","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:09Z","receivedAt":"2015-11-19T08:58:09Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nIn rare cases p4d seems to hang. This watchdog will kill the p4d\nprocess after 300s in any case. That means each individual git p4 test\nneeds to finish before 300s or it will fail.\n\nSigned-off-by: Lars Schneider \nAcked-by: Luke Diamand \n---\n t/lib-git-p4.sh | 20 +++++++++++++++++++-\n 1 file changed, 19 insertions(+), 1 deletion(-)\n\ndiff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh\nindex 3c9ad9a..acd5578 100644\n--- a/t/lib-git-p4.sh\n+++ b/t/lib-git-p4.sh\n@@ -10,6 +10,10 @@ TEST_NO_CREATE_REPO=NoThanks\n # here the maximal retry timeout in seconds.\n RETRY_TIMEOUT=60\n \n+# Sometimes p4d seems to hang. Terminate the p4d process automatically after\n+# the defined timeout in seconds.\n+P4D_TIMEOUT=300\n+\n . ./test-lib.sh\n \n if ! test_have_prereq PYTHON\n@@ -94,6 +98,19 @@ start_p4d() {\n \t# will be caught with the \"kill -0\" check below.\n \ti=${P4D_START_PATIENCE:-300}\n \tpid=$(cat \"$pidfile\")\n+\n+\ttimeout=$(($(time_in_seconds) + $P4D_TIMEOUT))\n+\twhile true\n+\tdo\n+\t\tif test $(time_in_seconds) -gt $timeout\n+\t\tthen\n+\t\t\tkill -9 $pid\n+\t\t\texit 1\n+\t\tfi\n+\t\tsleep 1\n+\tdone &\n+\twatchdog_pid=$!\n+\n \tready=\n \twhile test $i -gt 0\n \tdo\n@@ -156,7 +173,8 @@ kill_p4d() {\n \tretry_until_fail kill -9 $pid\n \t# complain if it would not die\n \ttest_must_fail kill $pid >/dev/null 2>&1 &&\n-\trm -rf \"$db\" \"$cli\" \"$pidfile\"\n+\trm -rf \"$db\" \"$cli\" \"$pidfile\" &&\n+\tretry_until_fail kill -9 $watchdog_pid\n }\n \n cleanup_git() {\n-- \n2.5.1\n"},{"id":"273495","messageId":"1447923491-15330-6-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 5/6] git-p4: add trap to kill p4d on test exit","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:10Z","receivedAt":"2015-11-19T08:58:10Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nSometimes the \"prove\" test runner hangs on test exit because p4d is\nstill running. Add a trap to always kill \"p4d\" on test exit.\n\nYou can reproduce the problem by commenting \"P4D_TIMEOUT\" in\n\"lib-git-p4.sh\" and running \"prove ./t9800-git-p4-basic.sh\".\n\nSigned-off-by: Lars Schneider \n---\n t/lib-git-p4.sh | 9 +++++++++\n 1 file changed, 9 insertions(+)\n\ndiff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh\nindex acd5578..f9ae1d7 100644\n--- a/t/lib-git-p4.sh\n+++ b/t/lib-git-p4.sh\n@@ -74,6 +74,15 @@ cli=\"$TRASH_DIRECTORY/cli\"\n git=\"$TRASH_DIRECTORY/git\"\n pidfile=\"$TRASH_DIRECTORY/p4d.pid\"\n\n+# Sometimes \"prove\" seems to hang on exit because p4d is still running\n+cleanup() {\n+\tif test -f \"$pidfile\"\n+\tthen\n+\t\tkill -9 $(cat \"$pidfile\") 2>/dev/null && exit 255\n+\tfi\n+}\n+trap cleanup EXIT\n+\n # git p4 submit generates a temp file, which will\n # not get cleaned up if the submission fails. Don't\n # clutter up /tmp on the test machine.\n--\n2.5.1\n"},{"id":"273497","messageId":"1447923491-15330-7-git-send-email-larsxschneider@gmail.com","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"[PATCH v6 6/6] Add Travis CI support","fromName":"","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-19T08:58:11Z","receivedAt":"2015-11-19T08:58:11Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"From: Lars Schneider \n\nThe tests are currently executed on \"Ubuntu 12.04 LTS Server Edition\n64 bit\" and on \"OS X Mavericks\" using gcc and clang.\n\nPerforce and Git-LFS are installed and therefore available for the\nrespective tests.\n\nSigned-off-by: Lars Schneider \n---\n .travis.yml | 131 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 131 insertions(+)\n create mode 100644 .travis.yml\n\ndiff --git a/.travis.yml b/.travis.yml\nnew file mode 100644\nindex 0000000..61c70fa\n--- /dev/null\n+++ b/.travis.yml\n@@ -0,0 +1,131 @@\n+language: c\n+\n+os:\n+ - linux\n+ - osx\n+\n+compiler:\n+ - clang\n+ - gcc\n+\n+addons:\n+ apt:\n+ packages:\n+ - language-pack-is\n+\n+env:\n+ global:\n+ - P4_VERSION=\"15.1\"\n+ - GIT_LFS_VERSION=\"1.0.2\"\n+ - DEFAULT_TEST_TARGET=prove\n+ - GIT_PROVE_OPTS=\"--timer --jobs 3\"\n+ - GIT_TEST_OPTS=\"--verbose --tee\"\n+ - GETTEXT_ISO_LOCALE=YesPlease\n+ - GETTEXT_LOCALE=YesPlease\n+ # - GETTEXT_POISON=YesPlease\n+ - GIT_TEST_CHAIN_LINT=YesPlease\n+ - GIT_TEST_CLONE_2GB=YesPlease\n+ # - GIT_TEST_LONG=YesPlease\n+ matrix:\n+ -\n+ # NO_ICONV=YesPlease\n+ - >\n+ NO_CURL=YesPlease\n+ NO_D_INO_IN_DIRENT=YesPlease\n+ NO_DEFLATE_BOUND=YesPlease\n+ NO_EXPAT=YesPlease\n+ NO_GECOS_IN_PWENT=YesPlease\n+ NO_GETTEXT=YesPlease\n+ NO_HMAC_CTX_CLEANUP=YesPlease\n+ NO_HSTRERROR=YesPlease\n+ NO_INET_NTOP=YesPlease\n+ NO_INET_PTON=YesPlease\n+ NO_INITGROUPS=YesPlease\n+ NO_INTTYPES_H=YesPlease\n+ NO_IPV6=YesPlease\n+ NO_IPV6=YesPlease\n+ NO_LIBGEN_H=YesPlease\n+ NO_MEMMEM=YesPlease\n+ NO_MKDTEMP=YesPlease\n+ NO_MKSTEMPS=YesPlease\n+ NO_MMAP=YesPlease\n+ NO_NSEC=YesPlease\n+ NO_OPENSSL=YesPlease\n+ NO_PERL=YesPlease\n+ NO_PTHREADS=YesPlease\n+ NO_REGEX=YesPlease\n+ NO_SETENV=YesPlease\n+ NO_SETITIMER=YesPlease\n+ NO_SOCKADDR_STORAGE=YesPlease\n+ NO_STRCASESTR=YesPlease\n+ NO_STRLCPY=YesPlease\n+ NO_STRTOUMAX=YesPlease\n+ NO_STRUCT_ITIMERVAL=YesPlease\n+ NO_SYMLINK_HEAD=YesPlease\n+ NO_SYS_POLL_H=YesPlease\n+ NO_SYS_SELECT_H=YesPlease\n+ NO_UINTMAX_T=YesPlease\n+ NO_UNSETENV=YesPlease\n+\n+before_install:\n+ - >\n+ case \"${TRAVIS_OS_NAME:-linux}\" in\n+ linux)\n+ mkdir --parents custom/p4\n+ pushd custom/p4\n+ wget --quiet http://filehost.perforce.com/perforce/r$P4_VERSION/bin.linux26x86_64/p4d\n+ wget --quiet http://filehost.perforce.com/perforce/r$P4_VERSION/bin.linux26x86_64/p4\n+ chmod u+x p4d\n+ chmod u+x p4\n+ export PATH=\"$(pwd):$PATH\"\n+ popd\n+ mkdir --parents custom/git-lfs\n+ pushd custom/git-lfs\n+ wget --quiet https://github.com/github/git-lfs/releases/download/v$GIT_LFS_VERSION/git-lfs-linux-amd64-$GIT_LFS_VERSION.tar.gz\n+ tar --extract --gunzip --file \"git-lfs-linux-amd64-$GIT_LFS_VERSION.tar.gz\"\n+ cp git-lfs-$GIT_LFS_VERSION/git-lfs .\n+ export PATH=\"$(pwd):$PATH\"\n+ popd\n+ ;;\n+ osx)\n+ brew_force_set_latest_binary_hash () {\n+ FORMULA=$1\n+ SHA=$(brew fetch --force $FORMULA 2>&1 | grep ^SHA256: | cut -d ' ' -f 2)\n+ sed -E -i.bak \"s/sha256 \\\"[0-9a-f]{64}\\\"/sha256 \\\"$SHA\\\"/g\" \\\n+ /usr/local/Library/Taps/homebrew/homebrew-binary/$FORMULA.rb\n+ }\n+ brew update --quiet\n+ brew tap homebrew/binary --quiet\n+ brew_force_set_latest_binary_hash perforce\n+ brew_force_set_latest_binary_hash perforce-server\n+ brew install git-lfs perforce-server perforce\n+ ;;\n+ esac;\n+ echo \"$(tput setaf 6)Perfoce Server Version$(tput sgr0)\";\n+ p4d -V | grep Rev.;\n+ echo \"$(tput setaf 6)Perfoce Client Version$(tput sgr0)\";\n+ p4 -V | grep Rev.;\n+ echo \"$(tput setaf 6)Git-LFS Version$(tput sgr0)\";\n+ git-lfs version;\n+\n+before_script: make configure && ./configure && make --jobs=2\n+\n+script: make --quiet test\n+\n+after_failure:\n+ - >\n+ : '<-- Click here to see detailed test output! ';\n+ for TEST_EXIT in t/test-results/*.exit;\n+ do\n+ if [ \"$(cat \"$TEST_EXIT\")\" != \"0\" ];\n+ then\n+ TEST_OUT=\"${TEST_EXIT%exit}out\";\n+ echo \"------------------------------------------------------------------------\";\n+ echo \"$(tput setaf 1)${TEST_OUT}...$(tput sgr0)\";\n+ echo \"------------------------------------------------------------------------\";\n+ cat \"${TEST_OUT}\";\n+ fi;\n+ done;\n+\n+notifications:\n+ email: false\n-- \n2.5.1\n"},{"id":"273502","messageId":"20151119141409.GB9353@sigill.intra.peff.net","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v6 0/6] Add Travis CI support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2015-11-19T14:14:09Z","receivedAt":"2015-11-19T14:14:09Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 19, 2015 at 09:58:05AM +0100, larsxschneider@gmail.com wrote:\n\n> From: Lars Schneider \n> \n> diff to v5:\n> * check if PID file still exists on P4D cleanup (thanks Luke)\n> * fix space/tab formatting error\n> * add sleep to timeout loops (thanks Luke)\n> * replace 'date +%s' with platform independent Python function (thanks Eric and Luke)\n> \n> With the patches of this series the Travis CI test stability increases.\n> However, as I am \"stress testing\" the Travis CI infrastructure you can\n> see that it is not perfect: https://travis-ci.org/larsxschneider/git/builds\n\nI peeked at a few, and it looks like just p4 tests failing now?\n\n> Nevertheless, I believe that Travis CI integration has still value as\n> contributors can test their patches easily on Linux and OS X before\n> posting them.\n> \n> @junio / @peff: Do you consider merging this?\n\nI think I'd prefer to split it into 3 separate topics (de-flaking\ntest_must_fail, p4 test improvements, and the Travis file). Then they\ncan proceed independently. I can take care of that split when applying.\n\n> Lars Schneider (6):\n> implement test_might_fail using a refactored test_must_fail\n\nYou mentioned in the v5 cover that this one was from Junio. Should it be\n\"From: Junio ...\" in the pseudo-header?\n\n> add \"ok=sigpipe\" to test_must_fail and use it to fix flaky tests\n\nLooks OK.\n\n> git-p4: retry kill/cleanup operations in tests with timeout\n> git-p4: add p4d timeout in tests\n> git-p4: add trap to kill p4d on test exit\n\nThese are all fairly gross, and I don't have p4d to test with myself.\nBut if we assume they're all necessary, I suppose it's the best we can\ndo.\n\n> Add Travis CI support\n\nI'll leave some comments directly in response to this one.\n\n-Peff\n"},{"id":"273503","messageId":"20151119143528.GC9353@sigill.intra.peff.net","threadId":"40833","inReplyTo":"1447923491-15330-7-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v6 6/6] Add Travis CI support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2015-11-19T14:35:29Z","receivedAt":"2015-11-19T14:35:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Nov 19, 2015 at 09:58:11AM +0100, larsxschneider@gmail.com wrote:\n\n> From: Lars Schneider \n> \n> The tests are currently executed on \"Ubuntu 12.04 LTS Server Edition\n> 64 bit\" and on \"OS X Mavericks\" using gcc and clang.\n> \n> Perforce and Git-LFS are installed and therefore available for the\n> respective tests.\n\nMy overall impression is that this is a lot more complicated than I was\nexpecting. Can we start with something simpler to gain experience with\nTravis? And then we can add in more complexity later.\n\nFor example:\n\n> +addons:\n> + apt:\n> + packages:\n> + - language-pack-is\n\nI doubt most people are running the t0204 right now. Maybe we should\nstart without it.\n\n> +env:\n> + global:\n> + - P4_VERSION=\"15.1\"\n> + - GIT_LFS_VERSION=\"1.0.2\"\n\nI know p4 is your area of interest, but from a project perspective, it\nseems like an odd choice for the initial set of tests.\n\n> + - DEFAULT_TEST_TARGET=prove\n> + - GIT_PROVE_OPTS=\"--timer --jobs 3\"\n> + - GIT_TEST_OPTS=\"--verbose --tee\"\n\nThese all make sense, I guess.\n\n> + - GETTEXT_ISO_LOCALE=YesPlease\n> + - GETTEXT_LOCALE=YesPlease\n\nWhat are these? I don't think we have any such Makefile knobs. These\nlook like variables that get used internally by the test suite, but we\nshouldn't need to set them.\n\n> + # - GETTEXT_POISON=YesPlease\n\nI'm assuming the \"#\" means this is commented out. Can we just drop such\ncruft?\n\n> + - GIT_TEST_CHAIN_LINT=YesPlease\n\nThis is the default, and we don't need to specify it.\n\n> + - GIT_TEST_CLONE_2GB=YesPlease\n\nIs it a good idea to run such an expensive test?\n\n> + matrix:\n> + -\n> + # NO_ICONV=YesPlease\n\nDitto here on the commenting.\n\n> + - >\n> + NO_CURL=YesPlease\n\nSo if I understand correctly, the point of this list is to test\nalternate configurations. So compiling without curl makes some sense, I\nguess. Though mostly it will just shut off a bunch off tests.\n\n> + NO_D_INO_IN_DIRENT=YesPlease\n\nBut setting platform compat knobs like this seems weird. Nobody sets\nthis manually. Either your platform has it, or it does not. And we are\nalready testing on alternate platforms to cover such things.\n\nIf there's a specific config setup of interest, it makes sense to me to\ntry to increase test coverage there. But it feels like you've just\npicked a random laundry list of Makefile knobs and tweaked them, without\nany sense of whether they even make sense together, or match the\nplatform.\n\nFor instance:\n\n> + NO_STRCASESTR=YesPlease\n> + NO_STRLCPY=YesPlease\n\nI wouldn't not be surprised if these cause problems building on some\nplatforms that _do_ have these functions.\n\n> + echo \"$(tput setaf 6)Perfoce Server Version$(tput sgr0)\";\n> + p4d -V | grep Rev.;\n> + echo \"$(tput setaf 6)Perfoce Client Version$(tput sgr0)\";\n> + p4 -V | grep Rev.;\n\ns/Perfoce/Perforce/ :)\n\n> +before_script: make configure && ./configure && make --jobs=2\n\nHmm. I wonder if we actually want to use autoconf here at all; most devs\ndo not use it, and Git should generally compile out of the box based on\nconfig.mak.uname (and if it doesn't, it would be nice to know).\n\nThere may be some optional packages that autoconf helps with, though.\n\n\nSo overall, I dunno. I do not want to create a bunch of work for you in\nsplitting it up. But I'm hesitant to merge something that has a bunch of\nlines that I'm not sure I understand the reasoning for. :-/\n\nI was hoping the first cut could just be telling Travis to run \"make\ntest\", without much fanfare beyond that. Maybe that's not realistic.\n\n-Peff\n"},{"id":"273524","messageId":"46C55D27-0B59-41AA-84AD-FED095EBDEE4@gmail.com","threadId":"40833","inReplyTo":"20151119141409.GB9353@sigill.intra.peff.net","subject":"Re: [PATCH v6 0/6] Add Travis CI support","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-20T08:46:10Z","receivedAt":"2015-11-20T08:46:10Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 19 Nov 2015, at 15:14, Jeff King wrote:\n\n> On Thu, Nov 19, 2015 at 09:58:05AM +0100, larsxschneider@gmail.com wrote:\n> \n>> From: Lars Schneider \n>> \n>> diff to v5:\n>> * check if PID file still exists on P4D cleanup (thanks Luke)\n>> * fix space/tab formatting error\n>> * add sleep to timeout loops (thanks Luke)\n>> * replace 'date +%s' with platform independent Python function (thanks Eric and Luke)\n>> \n>> With the patches of this series the Travis CI test stability increases.\n>> However, as I am \"stress testing\" the Travis CI infrastructure you can\n>> see that it is not perfect: https://travis-ci.org/larsxschneider/git/builds\n> \n> I peeked at a few, and it looks like just p4 tests failing now?\n\nYes, in particular t9810-git-p4-rcs.sh and t9816-git-p4-locked.sh. I would probably disable these test in Travis CI until I've found a way to make it stable.\n\n> \n>> Nevertheless, I believe that Travis CI integration has still value as\n>> contributors can test their patches easily on Linux and OS X before\n>> posting them.\n>> \n>> @junio / @peff: Do you consider merging this?\n> \n> I think I'd prefer to split it into 3 separate topics (de-flaking\n> test_must_fail, p4 test improvements, and the Travis file). Then they\n> can proceed independently. I can take care of that split when applying.\n\nSounds good to me!\n\n> \n>> Lars Schneider (6):\n>> implement test_might_fail using a refactored test_must_fail\n> \n> You mentioned in the v5 cover that this one was from Junio. Should it be\n> \"From: Junio ...\" in the pseudo-header?\n\nYes, this one was from Junio with a minor fix from my end if I recall correctly. What do you mean by \"pseudo-header\"? The \"email-header\" in the patch file? \n\n> \n>> add \"ok=sigpipe\" to test_must_fail and use it to fix flaky tests\n> \n> Looks OK.\n\n\"Looks OK\" means I can/should add \"Acked-by: Jeff King \" ? Bare with me, I am still learning ;-)\n\n> \n>> git-p4: retry kill/cleanup operations in tests with timeout\n>> git-p4: add p4d timeout in tests\n>> git-p4: add trap to kill p4d on test exit\n> \n> These are all fairly gross, and I don't have p4d to test with myself.\n> But if we assume they're all necessary, I suppose it's the best we can\n> do.\n\nUnfortunately I think they are necessary. However, if someone finds a better way for stable p4d tests then I would be happy to see them go away, again.\n\n> \n>> Add Travis CI support\n> \n> I'll leave some comments directly in response to this one.\n> \n\nThanks for taking the time to review this!\n\n- Lars\n"},{"id":"273525","messageId":"4304B1FC-B80A-459E-B4EA-CBDABB82A796@gmail.com","threadId":"40833","inReplyTo":"20151119143528.GC9353@sigill.intra.peff.net","subject":"Re: [PATCH v6 6/6] Add Travis CI support","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2015-11-20T09:29:47Z","receivedAt":"2015-11-20T09:29:47Z","isPatch":true,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\nOn 19 Nov 2015, at 15:35, Jeff King wrote:\n\n> On Thu, Nov 19, 2015 at 09:58:11AM +0100, larsxschneider@gmail.com wrote:\n> \n>> From: Lars Schneider \n>> \n>> The tests are currently executed on \"Ubuntu 12.04 LTS Server Edition\n>> 64 bit\" and on \"OS X Mavericks\" using gcc and clang.\n>> \n>> Perforce and Git-LFS are installed and therefore available for the\n>> respective tests.\n> \n> My overall impression is that this is a lot more complicated than I was\n> expecting. Can we start with something simpler to gain experience with\n> Travis? And then we can add in more complexity later.\n> \n> For example:\n> \n>> +addons:\n>> + apt:\n>> + packages:\n>> + - language-pack-is\n> \n> I doubt most people are running the t0204 right now. Maybe we should\n> start without it.\n\nWell, I think the entire point of a CI is to run automatically tests that most people are *not* running. Plus I can't recall any problems with language-pack-is and t0204.\n\n> \n>> +env:\n>> + global:\n>> + - P4_VERSION=\"15.1\"\n>> + - GIT_LFS_VERSION=\"1.0.2\"\n> \n> I know p4 is your area of interest, but from a project perspective, it\n> seems like an odd choice for the initial set of tests.\n\nSee above. Most people do not execute the p4 tests because they don't have p4 installed. Therefore these tests have the highest risk to brake unnoticed.\n\n> \n>> + - DEFAULT_TEST_TARGET=prove\n>> + - GIT_PROVE_OPTS=\"--timer --jobs 3\"\n>> + - GIT_TEST_OPTS=\"--verbose --tee\"\n> \n> These all make sense, I guess.\n> \n>> + - GETTEXT_ISO_LOCALE=YesPlease\n>> + - GETTEXT_LOCALE=YesPlease\n> \n> What are these? I don't think we have any such Makefile knobs. These\n> look like variables that get used internally by the test suite, but we\n> shouldn't need to set them.\n\nThese are used in t020*. I enabled them to run the tests and I haven't seen any problems. However, I am no expert in that area. I'll remove them.\n\n> \n>> + # - GETTEXT_POISON=YesPlease\n> \n> I'm assuming the \"#\" means this is commented out. Can we just drop such\n> cruft?\n\nYes, for the final submission of course. I just wanted to make it stick out as I wasn't able to run the tests successfully with this flag. I wonder if someone can explain to me how it works.\n\n> \n>> + - GIT_TEST_CHAIN_LINT=YesPlease\n> \n> This is the default, and we don't need to specify it.\n\nOK\n\n> \n>> + - GIT_TEST_CLONE_2GB=YesPlease\n> \n> Is it a good idea to run such an expensive test?\n\nWell, it's a CI machine that does not block anyone. I doubt that people run these tests on their local machines for exactly the reason you just described. Therefore I would rather run these tests.\n\n> \n>> + matrix:\n>> + -\n>> + # NO_ICONV=YesPlease\n> \n> Ditto here on the commenting.\n\nOK. Setting NO_ICONV brakes a lot of tests. Do we really \"support\" that flag?\n\n> \n>> + - >\n>> + NO_CURL=YesPlease\n> \n> So if I understand correctly, the point of this list is to test\n> alternate configurations. So compiling without curl makes some sense, I\n> guess. Though mostly it will just shut off a bunch off tests.\n> \n>> + NO_D_INO_IN_DIRENT=YesPlease\n> \n> But setting platform compat knobs like this seems weird. Nobody sets\n> this manually. Either your platform has it, or it does not. And we are\n> already testing on alternate platforms to cover such things.\n> \n> If there's a specific config setup of interest, it makes sense to me to\n> try to increase test coverage there. But it feels like you've just\n> picked a random laundry list of Makefile knobs and tweaked them, without\n> any sense of whether they even make sense together, or match the\n> platform.\n> \n> For instance:\n> \n>> + NO_STRCASESTR=YesPlease\n>> + NO_STRLCPY=YesPlease\n> \n> I wouldn't not be surprised if these cause problems building on some\n> platforms that _do_ have these functions.\n\nAgreed. As mentioned in gmane/280963 [1] I am no expert in these flags. Should I remove the \"NO_*\" test configuration all together?\n\n[1] http://article.gmane.org/gmane.comp.version-control.git/280963/match=no_\n\n> \n>> + echo \"$(tput setaf 6)Perfoce Server Version$(tput sgr0)\";\n>> + p4d -V | grep Rev.;\n>> + echo \"$(tput setaf 6)Perfoce Client Version$(tput sgr0)\";\n>> + p4 -V | grep Rev.;\n> \n> s/Perfoce/Perforce/ :)\n\nThanks :)\n\n\n>> +before_script: make configure && ./configure && make --jobs=2\n> \n> Hmm. I wonder if we actually want to use autoconf here at all; most devs\n> do not use it, and Git should generally compile out of the box based on\n> config.mak.uname (and if it doesn't, it would be nice to know).\n> \n> There may be some optional packages that autoconf helps with, though.\n\nCompiling without autoconf works for Linux but not for OS X:\nhttps://travis-ci.org/larsxschneider/git/jobs/92222770\n\n\n> So overall, I dunno. I do not want to create a bunch of work for you in\n> splitting it up. But I'm hesitant to merge something that has a bunch of\n> lines that I'm not sure I understand the reasoning for. :-/\n\nI understand. How about this: I create a new roll where I...\n... remove the \"NO_*\" configuration for now.\n... remove GETTEXT_* flags\n... disable flaky t9810 and t9816 in the Travis CI run\n... disable autoconf and try to fix the OS X issue\n\n> \n> I was hoping the first cut could just be telling Travis to run \"make\n> test\", without much fanfare beyond that. Maybe that's not realistic.\n\n\nThanks,\nLars\n"},{"id":"273542","messageId":"CAE5ih7-q9JpiAW3pU6XWk+z0sGY8ej+UKEaR0bGuBnR0D9+P5A@mail.gmail.com","threadId":"40833","inReplyTo":"46C55D27-0B59-41AA-84AD-FED095EBDEE4@gmail.com","subject":"Re: [PATCH v6 0/6] Add Travis CI support","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-11-20T12:56:04Z","receivedAt":"2015-11-20T12:56:04Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 20 November 2015 at 08:46, Lars Schneider wrote:\n>\n> On 19 Nov 2015, at 15:14, Jeff King wrote:\n>\n>\n>>\n>>> git-p4: retry kill/cleanup operations in tests with timeout\n>>> git-p4: add p4d timeout in tests\n>>> git-p4: add trap to kill p4d on test exit\n>>\n>> These are all fairly gross, and I don't have p4d to test with myself.\n>> But if we assume they're all necessary, I suppose it's the best we can\n>> do.\n>\n> Unfortunately I think they are necessary. However, if someone finds a better way for stable p4d tests then I would be happy to see them go away, again.\n\nI think that's just how p4d is I'm afraid. It doesn't like being\nstopped and started quickly (I guess it's not a normal use-case for\nmost p4 users). I've made various unsuccessful attempts in the past to\nmake these tests work reliably, and Lars' changes are far better than\nanything I ever managed.\n\nLuke\n"},{"id":"273616","messageId":"20151123185943.GE2597@sigill.intra.peff.net","threadId":"40833","inReplyTo":"CAE5ih7-q9JpiAW3pU6XWk+z0sGY8ej+UKEaR0bGuBnR0D9+P5A@mail.gmail.com","subject":"Re: [PATCH v6 0/6] Add Travis CI support","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2015-11-23T18:59:43Z","receivedAt":"2015-11-23T18:59:43Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Nov 20, 2015 at 12:56:04PM +0000, Luke Diamand wrote:\n\n> >>> git-p4: retry kill/cleanup operations in tests with timeout\n> >>> git-p4: add p4d timeout in tests\n> >>> git-p4: add trap to kill p4d on test exit\n> >>\n> >> These are all fairly gross, and I don't have p4d to test with myself.\n> >> But if we assume they're all necessary, I suppose it's the best we can\n> >> do.\n> >\n> > Unfortunately I think they are necessary. However, if someone finds a better way for stable p4d tests then I would be happy to see them go away, again.\n> \n> I think that's just how p4d is I'm afraid. It doesn't like being\n> stopped and started quickly (I guess it's not a normal use-case for\n> most p4 users). I've made various unsuccessful attempts in the past to\n> make these tests work reliably, and Lars' changes are far better than\n> anything I ever managed.\n\nThanks for the extra context. I hope I didn't sound too negative in my\ninitial assessment. It was meant to be \"this is lamentable but probably\nnecessary\".\n\n-Peff\n"},{"id":"273621","messageId":"565382D4.5010200@diamand.org","threadId":"40833","inReplyTo":"1447923491-15330-1-git-send-email-larsxschneider@gmail.com","subject":"Re: [PATCH v6 0/6] Add Travis CI support","fromName":"Luke Diamand","fromEmail":"luke@diamand.org","sentAt":"2015-11-23T21:19:16Z","receivedAt":"2015-11-23T21:19:16Z","isPatch":true,"sender":{"key":"luke@diamand.org","avatar":"https://avatars.githubusercontent.com/u/5330967?v=4"},"body":"On 19/11/15 08:58, larsxschneider@gmail.com wrote:\n> From: Lars Schneider \n>\n> diff to v5:\n> * check if PID file still exists on P4D cleanup (thanks Luke)\n> * fix space/tab formatting error\n> * add sleep to timeout loops (thanks Luke)\n> * replace 'date +%s' with platform independent Python function (thanks Eric and Luke)\n\nThe first three of these (which fix the git-p4 tests) all look good to \nme. With these changes, running the t98* tests in parallel now works, \nwhich it did not used to do. The changes are not pretty but I think \nthat's inevitable.\n\nAck from me.\n\nLuke\n"}]}