{"thread":{"id":"46368","subject":"Reducing redundant build at Travis?","startedAt":"2017-07-12T23:44:19Z","lastAt":"2017-07-26T18:08:30Z","messageCount":12,"participants":["Junio C Hamano","Lars Schneider","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"324324","messageId":"xmqqeftlz0dv.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":null,"subject":"Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-12T23:44:12Z","receivedAt":"2017-07-12T23:44:19Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"I usually try to stay as late as possible to finish all the\nintegration branches in order before pushing out the result; it is\nmore efficient to be able to batch things (for humans). \n\nI however noticed that This often means we would have multiple build\njobs at Travis for branches and builds on Windows often fails\nwaiting for its response. Since I tagged the tip of 'maint', and I\nwanted to give all the build a fair chance to succeed without other\nbuild jobs starving it of resources, I pushed out 'maint' and the\ntag before others, even though I already have all the other\nintegration branches ready.\n\nUnfortunately, https://travis-ci.org/git/git/builds/ shows that it\ndoes not care if it spawned a job to build the tip of 'maint' and\nanother for 'v2.13.3' that point at the same thing.\n\nI do not mind this behaviour terribly for Linux builds that never\nseem to time out, but it is wasteful. Here is what I came up with\nto skip the build and test on a branch tip that is tagged, but I\nsuspect this would not actually work (when told to build a tag, I\nsuspect that it would try to find an exact-match and ends up finding\nitself, refusing to do the work).\n\nI also do not quite like the way that I had to add a check like this\nfor every script: thing in the .travis.yml file (the attached does\nnot even cover MacOS whose build tends to take a lot longer). I\nsuspect that you or others who are better versed with the system may\nhave better idea to have a single centralized check that avoids a\ntip of a branch that is pointed at by a release tag.\n\nThanks.\n\n ci/run-linux32-build.sh | 6 ++++++\n ci/run-windows-build.sh | 6 ++++++\n 2 files changed, 12 insertions(+)\n\ndiff --git a/ci/run-linux32-build.sh b/ci/run-linux32-build.sh\nindex e30fb2cddc..ffb5c8fdf1 100755\n--- a/ci/run-linux32-build.sh\n+++ b/ci/run-linux32-build.sh\n@@ -6,6 +6,12 @@\n # run-linux32-build.sh [host-user-id]\n #\n \n+if TAG=$(git describe --exact-match HEAD 2>/dev/null)\n+then\n+\techo \"The tip of the branch is exactly at $TAG\"\n+\texit 0\n+fi\n+\n # Update packages to the latest available versions\n linux32 --32bit i386 sh -c '\n apt update >/dev/null &&\ndiff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh\nindex 2d98f6b2f9..74d4819d74 100755\n--- a/ci/run-windows-build.sh\n+++ b/ci/run-windows-build.sh\n@@ -12,6 +12,12 @@ test -z \"$GFW_CI_TOKEN\" && echo \"GFW_CI_TOKEN not defined\" && exit\n BRANCH=$1\n COMMIT=$2\n \n+if TAG=$(git describe --exact-match \"$COMMIT\" 2>/dev/null)\n+then\n+\techo \"Tip of $BRANCH exactly at $TAG\"\n+\texit 0\n+fi\n+\n gfwci () {\n \tlocal CURL_ERROR_CODE HTTP_CODE\n \tCONTENT_FILE=$(mktemp -t \"git-windows-ci-XXXXXX\")\n"},{"id":"324450","messageId":"CADJMNYmGR8Ti0S77y2ykPdSZB68DrBb8O0wfX6PaezV1+j2bnw@mail.gmail.com","threadId":"46368","inReplyTo":"xmqqeftlz0dv.fsf@gitster.mtv.corp.google.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-07-13T21:21:34Z","receivedAt":"2017-07-13T21:21:45Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"On Thu, Jul 13, 2017 at 1:44 AM, Junio C Hamano wrote:\n> I usually try to stay as late as possible to finish all the\n> integration branches in order before pushing out the result; it is\n> more efficient to be able to batch things (for humans).\n>\n> I however noticed that This often means we would have multiple build\n> jobs at Travis for branches and builds on Windows often fails\n\nThe Windows build has some kind of problem since June 22.\nSomehow building gitk-git just blocks the build and waits until\nthe timeout. I had no time, yet, to investigate this further.\n\n\n> waiting for its response. Since I tagged the tip of 'maint', and I\n> wanted to give all the build a fair chance to succeed without other\n> build jobs starving it of resources, I pushed out 'maint' and the\n> tag before others, even though I already have all the other\n> integration branches ready.\n>\n> Unfortunately, https://travis-ci.org/git/git/builds/ shows that it\n> does not care if it spawned a job to build the tip of 'maint' and\n> another for 'v2.13.3' that point at the same thing.\n\nThat is indeed suprising and wasteful. Looks like other people\ndid run into the same issue. How about something like this?\nhttps://github.com/mockito/mockito/blob/release/2.x/.travis.yml#L26-L29\n\n\n- Lars\n"},{"id":"324456","messageId":"xmqqiniwt35e.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":"CADJMNYmGR8Ti0S77y2ykPdSZB68DrBb8O0wfX6PaezV1+j2bnw@mail.gmail.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-13T21:53:17Z","receivedAt":"2017-07-13T21:53:24Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n> On Thu, Jul 13, 2017 at 1:44 AM, Junio C Hamano wrote:\n>> I usually try to stay as late as possible to finish all the\n>> integration branches in order before pushing out the result; it is\n>> more efficient to be able to batch things (for humans).\n>>\n>> I however noticed that This often means we would have multiple build\n>> jobs at Travis for branches and builds on Windows often fails\n>\n> The Windows build has some kind of problem since June 22.\n> Somehow building gitk-git just blocks the build and waits until\n> the timeout. I had no time, yet, to investigate this further.\n>\n>\n>> waiting for its response. Since I tagged the tip of 'maint', and I\n>> wanted to give all the build a fair chance to succeed without other\n>> build jobs starving it of resources, I pushed out 'maint' and the\n>> tag before others, even though I already have all the other\n>> integration branches ready.\n>>\n>> Unfortunately, https://travis-ci.org/git/git/builds/ shows that it\n>> does not care if it spawned a job to build the tip of 'maint' and\n>> another for 'v2.13.3' that point at the same thing.\n>\n> That is indeed suprising and wasteful. Looks like other people\n> did run into the same issue. How about something like this?\n> https://github.com/mockito/mockito/blob/release/2.x/.travis.yml#L26-L29\n\nThat unfortunately is exactly what I wanted to avoid.\n\nWe'd want to test tagged releases, and we'd want to test usual\nupdates to integration branches. It just is that sometimes the tips\nof integration branches happen to be at the tagged release, so I'd\nprefer to always build tags but skip a branch build if it happens to\nbe also tagged. After all, none of the integration branches may\ndirectly point at a tagged release when the tag is pushed out.\n\n\n"},{"id":"324499","messageId":"20170714122401.uahk2fzvcjj3nwve@sigill.intra.peff.net","threadId":"46368","inReplyTo":"xmqqiniwt35e.fsf@gitster.mtv.corp.google.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-14T12:24:01Z","receivedAt":"2017-07-14T12:24:08Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Jul 13, 2017 at 02:53:17PM -0700, Junio C Hamano wrote:\n\n> >> Unfortunately, https://travis-ci.org/git/git/builds/ shows that it\n> >> does not care if it spawned a job to build the tip of 'maint' and\n> >> another for 'v2.13.3' that point at the same thing.\n> >\n> > That is indeed suprising and wasteful. Looks like other people\n> > did run into the same issue. How about something like this?\n> > https://github.com/mockito/mockito/blob/release/2.x/.travis.yml#L26-L29\n> \n> That unfortunately is exactly what I wanted to avoid.\n> \n> We'd want to test tagged releases, and we'd want to test usual\n> updates to integration branches. It just is that sometimes the tips\n> of integration branches happen to be at the tagged release, so I'd\n> prefer to always build tags but skip a branch build if it happens to\n> be also tagged. After all, none of the integration branches may\n> directly point at a tagged release when the tag is pushed out.\n\nRight, I think the right solution is some amount of peeling. Recognizing\nthat the commit sha1 is the same, or better yet, not bothering to retest\ntrees which have already been tested.\n\nWe used a hacked-up copy of Jenkins for our in-house CI, and it skips\nalready-tested trees. Besides the discussed cases, this also saves time\nwhen pushing noop rebases (e.g., when you just changed commit messages)\nand noop merges (e.g., if you already back-merged master to your topic,\ntested it, and now do a \"merge --no-ff\" to merge the topic in). I don't\nthink either of those are common in our workflows, though.\n\nIf we had some kind of persistent storage, we could do a quick:\n\n tree=$(git rev-parse HEAD^{tree})\n if test \"$(get_from_storage status-$tree)\" = \"good\"\n then\n\techo \"Already tested $tree, skipping\"\n\texit 0\n fi\n ... run actual tests ...\n put_into_storage status-$tree good\n\nThe \"git test\" script[1] uses this strategy with git-notes as the\nstorage, and I've found it quite useful. I don't think we can rely on\ngit-notes, but I think Travis gives us some storage options. Even just a\nbest-effort cache directory would probably be sufficient (this is an\noptimization, after all).\n\n-Peff\n\n[1] https://github.com/mhagger/git-test\n"},{"id":"324524","messageId":"xmqqbmonrrvr.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":"20170714122401.uahk2fzvcjj3nwve@sigill.intra.peff.net","subject":"Re: Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-14T14:54:16Z","receivedAt":"2017-07-14T14:54:29Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Right, I think the right solution is some amount of peeling. Recognizing\n> that the commit sha1 is the same, or better yet, not bothering to retest\n> trees which have already been tested.\n\nYup, I also have a hack to avoid testing a version that is only\ndifferent in insignificant way (e.g. the only difference being\nGIT-VERSION-GEN or Documentation/RelNotes/*) from an installed one\nin the script I use after each integration attempt I do, which I use\na few times a day (that's \"Meta/Dothem\" if anybody is interested).\n\n> If we had some kind of persistent storage, we could do a quick:\n> ...\n> The \"git test\" script[1] uses this strategy with git-notes as the\n> storage, and I've found it quite useful. I don't think we can rely on\n> git-notes, but I think Travis gives us some storage options. Even just a\n> best-effort cache directory would probably be sufficient (this is an\n> optimization, after all).\n\nWe do seem to use some persistence to order prove tests already, but\nI do not think it helps the common case, where my end-of-day push\npushes out 'maint' and 'v2.13.3' at the same time, because the push\nis made with \"git push --follow-tags $there maint master next pu\"\nand the new tag happens to be at 'maint'. It would be nice if\nTravis runs were sequential, but I often observe that it creates\njobs for these multiple branches and tags pushed at the same time,\nand start running a few of them.\n\n"},{"id":"324533","messageId":"20170714153224.6qnupcsa3pkig76k@sigill.intra.peff.net","threadId":"46368","inReplyTo":"xmqqbmonrrvr.fsf@gitster.mtv.corp.google.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-14T15:32:25Z","receivedAt":"2017-07-14T15:32:31Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jul 14, 2017 at 07:54:16AM -0700, Junio C Hamano wrote:\n\n> > The \"git test\" script[1] uses this strategy with git-notes as the\n> > storage, and I've found it quite useful. I don't think we can rely on\n> > git-notes, but I think Travis gives us some storage options. Even just a\n> > best-effort cache directory would probably be sufficient (this is an\n> > optimization, after all).\n> \n> We do seem to use some persistence to order prove tests already, but\n> I do not think it helps the common case, where my end-of-day push\n> pushes out 'maint' and 'v2.13.3' at the same time, because the push\n> is made with \"git push --follow-tags $there maint master next pu\"\n> and the new tag happens to be at 'maint'. It would be nice if\n> Travis runs were sequential, but I often observe that it creates\n> jobs for these multiple branches and tags pushed at the same time,\n> and start running a few of them.\n\nAh, right, I didn't think about how these are racing. You'd need storage\nwhich allows some kind of atomic operation to \"claim\" the tree as a\nwork-in-progress (and anybody who loses the race to get the lock would\nhave to spin waiting for the winner to tell them the real status).\n\nI don't know if Travis's cache storage is up to that challenge. We could\nprobably build such a lock on top of third-party storage, but things are\nrapidly getting more complex.\n\n-Peff\n"},{"id":"324793","messageId":"4BB4EDA1-58A5-45AD-8BEC-1CB06CDC23DD@gmail.com","threadId":"46368","inReplyTo":"20170714153224.6qnupcsa3pkig76k@sigill.intra.peff.net","subject":"Re: Reducing redundant build at Travis?","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-07-20T08:18:45Z","receivedAt":"2017-07-20T08:18:58Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 14 Jul 2017, at 17:32, Jeff King wrote:\n> \n> On Fri, Jul 14, 2017 at 07:54:16AM -0700, Junio C Hamano wrote:\n> \n>>> The \"git test\" script[1] uses this strategy with git-notes as the\n>>> storage, and I've found it quite useful. I don't think we can rely on\n>>> git-notes, but I think Travis gives us some storage options. Even just a\n>>> best-effort cache directory would probably be sufficient (this is an\n>>> optimization, after all).\n>> \n>> We do seem to use some persistence to order prove tests already, but\n>> I do not think it helps the common case, where my end-of-day push\n>> pushes out 'maint' and 'v2.13.3' at the same time, because the push\n>> is made with \"git push --follow-tags $there maint master next pu\"\n>> and the new tag happens to be at 'maint'. It would be nice if\n>> Travis runs were sequential, but I often observe that it creates\n>> jobs for these multiple branches and tags pushed at the same time,\n>> and start running a few of them.\n> \n> Ah, right, I didn't think about how these are racing. You'd need storage\n> which allows some kind of atomic operation to \"claim\" the tree as a\n> work-in-progress (and anybody who loses the race to get the lock would\n> have to spin waiting for the winner to tell them the real status).\n> \n> I don't know if Travis's cache storage is up to that challenge. We could\n> probably build such a lock on top of third-party storage, but things are\n> rapidly getting more complex.\n\nI think we shouldn't go there because of the complexity. I reached out\nto TravisCI and asked about the \"hash build twice\" problem [1]. Unfortunately,\nI got no response, yet. The issue could also be considered a feature as you\ncould perform different actions in your TravisCI configuration based on\nthe branch name.\n\nI think Junio's original suggestions for the Windows build makes a lot\nof sense because it saves Dscho's compute resources:\n\n--- a/ci/run-windows-build.sh\n+++ b/ci/run-windows-build.sh\n@@ -12,6 +12,12 @@ test -z \"$GFW_CI_TOKEN\" && echo \"GFW_CI_TOKEN not defined\" && exit\nBRANCH=$1\nCOMMIT=$2\n\n+if TAG=$(git describe --exact-match \"$COMMIT\" 2>/dev/null)\n+then\n+\techo \"Tip of $BRANCH exactly at $TAG\"\n+\texit 0\n+fi\n+\ngfwci () {\n\tlocal CURL_ERROR_CODE HTTP_CODE\n\tCONTENT_FILE=$(mktemp -t \"git-windows-ci-XXXXXX\")\n\n\nHowever, I don't think we need to do the same for the builds that\nuse TravisCI resources. If they would be concerned about that then \nthey wouldn't build the same hash twice in the first place.\n\n- Lars\n\n\n[1] https://twitter.com/kit3bus/status/885902189692112896\n\n"},{"id":"324808","messageId":"xmqq4lu7m94h.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":"4BB4EDA1-58A5-45AD-8BEC-1CB06CDC23DD@gmail.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-20T15:16:30Z","receivedAt":"2017-07-20T15:16:38Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n>> On 14 Jul 2017, at 17:32, Jeff King wrote:\n>> \n>> I don't know if Travis's cache storage is up to that challenge. We could\n>> probably build such a lock on top of third-party storage, but things are\n>> rapidly getting more complex.\n>\n> I think we shouldn't go there because of the complexity. I reached out\n> to TravisCI and asked about the \"hash build twice\" problem [1]. Unfortunately,\n> I got no response, yet. The issue could also be considered a feature as you\n> could perform different actions in your TravisCI configuration based on\n> the branch name.\n\nOh, no doubt that it is a feature, and a very useful one at that.\nWith that, we can change actions depending on the branch name in\nsuch a way that normally we do our build and test, but when we are\non a branch (not testing a tag) and its tip is tagged, we become\nno-op to avoid the cost of testing. That is the feature we exactly\nwant.\n\nThe question I had, and wanted a help from you, was if there was a\nway we can write that \"are we on a branch (not testing a tag) and is\nits tip tagged?\" test only once in .travis.yml, even though we have\nquite a many items in the matrix. With the current way .travis.yml\nis laid out, without such a facility, we'd need the logic sprinkled\nto at the beginning of all \"before_install:\" or something like that,\nwhich is not quite optimal.\n\n> I think Junio's original suggestions for the Windows build makes a lot\n> of sense because it saves Dscho's compute resources:\n>\n> --- a/ci/run-windows-build.sh\n> +++ b/ci/run-windows-build.sh\n> @@ -12,6 +12,12 @@ test -z \"$GFW_CI_TOKEN\" && echo \"GFW_CI_TOKEN not defined\" && exit\n> BRANCH=$1\n> COMMIT=$2\n>\n> +if TAG=$(git describe --exact-match \"$COMMIT\" 2>/dev/null)\n> +then\n> +\techo \"Tip of $BRANCH exactly at $TAG\"\n> +\texit 0\n> +fi\n> +\n> gfwci () {\n> \tlocal CURL_ERROR_CODE HTTP_CODE\n> \tCONTENT_FILE=$(mktemp -t \"git-windows-ci-XXXXXX\")\n>\n> However, I don't think we need to do the same for the builds that\n> use TravisCI resources. If they would be concerned about that then \n> they wouldn't build the same hash twice in the first place.\n\nBut I do care ;-) It would be nice for me not to have to wait and\nkeep worrying about MacOSX builds. It also is not nice that\nbranches for other tests are blocked and have to wait only because\n'maint' and 'vX.Y.Z' are both tested even though we know they are\nthe same tree. This is where my question earlier comes from---is\nthere a good way to do the \"not test a branch if its at a tagged\ncommit, because that tag will be tested anyway\" test only at one\nplace in the .travis.yml we have?---because it's not like we only\ncare about Windows.\n\n"},{"id":"324870","messageId":"333FEA3F-BB43-4183-ADD1-14E73E668EA3@gmail.com","threadId":"46368","inReplyTo":"xmqq4lu7m94h.fsf@gitster.mtv.corp.google.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-07-21T16:11:26Z","receivedAt":"2017-07-21T16:11:34Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 20 Jul 2017, at 17:16, Junio C Hamano wrote:\n> \n> Lars Schneider writes:\n> \n>>> On 14 Jul 2017, at 17:32, Jeff King wrote:\n>>> \n>>> I don't know if Travis's cache storage is up to that challenge. We could\n>>> probably build such a lock on top of third-party storage, but things are\n>>> rapidly getting more complex.\n>> \n>> I think we shouldn't go there because of the complexity. I reached out\n>> to TravisCI and asked about the \"hash build twice\" problem [1]. Unfortunately,\n>> I got no response, yet. The issue could also be considered a feature as you\n>> could perform different actions in your TravisCI configuration based on\n>> the branch name.\n> \n> Oh, no doubt that it is a feature, and a very useful one at that.\n> With that, we can change actions depending on the branch name in\n> such a way that normally we do our build and test, but when we are\n> on a branch (not testing a tag) and its tip is tagged, we become\n> no-op to avoid the cost of testing. That is the feature we exactly\n> want.\n> \n> The question I had, and wanted a help from you, was if there was a\n> way we can write that \"are we on a branch (not testing a tag) and is\n> its tip tagged?\" test only once in .travis.yml, even though we have\n> quite a many items in the matrix. With the current way .travis.yml\n> is laid out, without such a facility, we'd need the logic sprinkled\n> to at the beginning of all \"before_install:\" or something like that,\n> which is not quite optimal.\n\nTo answer your question: I don't see an easy solution to the problem.\n\nHowever, maybe it is solvable with a creative usage of \"build-stages\"?\nhttps://blog.travis-ci.com/2017-05-11-introducing-build-stages\n\nSorry,\nLars\n\n"},{"id":"324874","messageId":"xmqqo9sdhh9o.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":"333FEA3F-BB43-4183-ADD1-14E73E668EA3@gmail.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-21T16:44:03Z","receivedAt":"2017-07-21T16:44:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n> To answer your question: I don't see an easy solution to the problem.\n\nThat's OK. Thanks for digging.\n\nI am wondering if the attached would be acceptable as a minimum\nimpact patch to address this issue. \n\nI think I got the \"are we building a tag, or are we building a\nbranch that happens to be at a tag?\" logic right, but I have no idea\nwhat I am writing in the \"script\" sections (I am just assuming that\nthese lines are squashed into a line by removing line-breaks and\nbecome a single loooong shell script), and can certainly use guiding\nhands. I didn't bother skipping the work done in before_script.\n\nThanks.\n\ndiff --git a/.travis.yml b/.travis.yml\nindex 278943d14a..55af619830 100644\n--- a/.travis.yml\n+++ b/.travis.yml\n@@ -53,6 +53,7 @@ matrix:\n script:\n - >\n test \"$TRAVIS_REPO_SLUG\" != \"git/git\" ||\n+ ci/skip-branch-tip-with-tag.sh ||\n ci/run-windows-build.sh $TRAVIS_BRANCH $(git rev-parse HEAD)\n after_failure:\n - env: Linux32\n@@ -65,6 +66,7 @@ matrix:\n before_script:\n script:\n - >\n+ ci/skip-branch-tip-with-tag.sh ||\n docker run\n --interactive\n --env DEVELOPER\n@@ -145,9 +147,10 @@ before_script: make --jobs=2\n \n script:\n - >\n+ ci/skip-branch-tip-with-tag.sh || {\n mkdir -p $HOME/travis-cache;\n ln -s $HOME/travis-cache/.prove t/.prove;\n- make --quiet test;\n+ make --quiet test; }\n \n after_failure:\n - >\ndiff --git a/ci/skip-branch-tip-with-tag.sh b/ci/skip-branch-tip-with-tag.sh\nnew file mode 100755\nindex 0000000000..a57e724b35\n--- /dev/null\n+++ b/ci/skip-branch-tip-with-tag.sh\n@@ -0,0 +1,23 @@\n+#!/bin/sh\n+\n+# Sometimes, a branch is pushed at the same time the tag that points\n+# at the same commit as the tip of the branch is pushed, and building\n+# both at the same time is a waste.\n+#\n+# Travis gives a tagname e.g. v2.14.0 in $TRAVIS_BRANCH when\n+# the build is triggered by a push to a tag. Let's see if\n+# $TRAVIS_BRANCH is exactly at a tag, and if so, if it is \n+# different from $TRAVIS_BRANCH. That way, we can tell if\n+# we are building the tip of a branch that is tagged---and\n+# we can skip the build because we won't be skipping a build\n+# of a tag.\n+\n+if TAG=$(git describe --exact-match \"$TRAVIS_BRANCH\" 2>/dev/null) &&\n+\t$TAG != $TRAVIS_BRANCH\n+then\n+\techo \"Tip of $TRAVIS_BRANCH is exactly at $TAG\"\n+\texit 0\n+else\n+\texit 1\n+fi\n+\n"},{"id":"325129","messageId":"3056CC17-DDD5-406C-AA22-56309DF09DC8@gmail.com","threadId":"46368","inReplyTo":"xmqqo9sdhh9o.fsf@gitster.mtv.corp.google.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Lars Schneider","fromEmail":"larsxschneider@gmail.com","sentAt":"2017-07-26T16:43:47Z","receivedAt":"2017-07-26T16:43:58Z","isPatch":false,"sender":{"key":"larsxschneider@gmail.com","avatar":"https://avatars.githubusercontent.com/u/477434?v=4"},"body":"\n> On 21 Jul 2017, at 18:44, Junio C Hamano wrote:\n> \n> Lars Schneider writes:\n> \n>> To answer your question: I don't see an easy solution to the problem.\n> \n> That's OK. Thanks for digging.\n> \n> I am wondering if the attached would be acceptable as a minimum\n> impact patch to address this issue. \n\nYour patch would still run a number of expensive operations for the\nsame hashes (e.g. static analysis and documentation builds). I started \nto work on a patch that moves all TravisCI logic into scripts located \nin the `ci` folder. These scripts share a `lib-travisci.sh` for common \nfunctions such as `skip_branch_tip_with_tag ()` executed at the\nbeginning of every script.\n\nDoes this sound sensible to you? I am a bit busy with non Git related \nwork right now but I try to post the patch for you to review ASAP.\n\n- Lars\n\n"},{"id":"325134","messageId":"xmqqinifrrzh.fsf@gitster.mtv.corp.google.com","threadId":"46368","inReplyTo":"3056CC17-DDD5-406C-AA22-56309DF09DC8@gmail.com","subject":"Re: Reducing redundant build at Travis?","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-07-26T18:08:18Z","receivedAt":"2017-07-26T18:08:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Schneider writes:\n\n> ... I started \n> to work on a patch that moves all TravisCI logic into scripts located \n> in the `ci` folder. These scripts share a `lib-travisci.sh` for common \n> functions such as `skip_branch_tip_with_tag ()` executed at the\n> beginning of every script.\n>\n> Does this sound sensible to you? I am a bit busy with non Git related \n> work right now but I try to post the patch for you to review ASAP.\n\nAbsolutely fantastic.\n\nTake your time; there is no need to rush and haste makes waste.\n\nThanks for working on this.\n"}]}