{"thread":{"id":"18566","subject":"Test that every revision builds before pushing changes?","startedAt":"2009-03-26T06:29:58Z","lastAt":"2009-03-27T01:30:07Z","messageCount":7,"participants":["Daniel Pittman","Andreas Ericsson","Jeff King","Wincent Colaiuta"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"109490","messageId":"87myb8aja1.fsf@rimspace.net","threadId":"18566","inReplyTo":null,"subject":"Test that every revision builds before pushing changes?","fromName":"Daniel Pittman","fromEmail":"daniel@rimspace.net","sentAt":"2009-03-26T06:29:58Z","receivedAt":"2009-03-26T06:29:58Z","isPatch":false,"sender":{"key":"daniel@rimspace.net","avatar":"https://gravatar.com/avatar/67f2f02d3521b163f6b7a0dec5943d33fbc5a2a6d9702ab3fef99032e59e692f?d=mp&s=160"},"body":"G'day.\n\nI would like to ensure that my commits are fully bisectable before I\ncommit them to an upstream repository, at least to the limits of an\nautomatic tool for testing them.\n\n'git bisect run' is similar: it can automatically locate the breaking in\na test suite, for example, but that doesn't help me in the case of three\ncommits, A (good), B (bad) and C (good, fixing B).\n\nI would much rather, in this case, use rebase to fix B so that it, too,\nbuilds before I push the changes and pollute a public repository with a\nbroken changeset — and make bisect that much harder to use in future.\n\nRegards,\n        Daniel\n"},{"id":"109501","messageId":"49CB39E5.5060000@op5.se","threadId":"18566","inReplyTo":"87myb8aja1.fsf@rimspace.net","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-03-26T08:16:37Z","receivedAt":"2009-03-26T08:16:37Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Daniel Pittman wrote:\n> G'day.\n> \n> I would like to ensure that my commits are fully bisectable before I\n> commit them to an upstream repository, at least to the limits of an\n> automatic tool for testing them.\n> \n> 'git bisect run' is similar: it can automatically locate the breaking in\n> a test suite, for example, but that doesn't help me in the case of three\n> commits, A (good), B (bad) and C (good, fixing B).\n> \n> I would much rather, in this case, use rebase to fix B so that it, too,\n> builds before I push the changes and pollute a public repository with a\n> broken changeset — and make bisect that much harder to use in future.\n> \n\nYou can do that, but it requires manual work too. The trick is to make\nthe release branch immutable on the public repository and use topic\nbranches with per-developer namespaces. The per-developer namespace\nthing is actually important, as it leaves the freedom to rewind and\nrecreate topics to the developers (which shared branches do not).\n\nThe manual step comes at merge-time; Someone has to be responsible for\nmerging all the topics that are to be included in the release branch\nand make sure it builds and passes all tests after each merge.\n\nThis workflow is a bit cumbersome. NASA uses something like this but\nwith an extra step for multiple peer reviews on every feature/fix for\nsoftware they send to satellites. Or so I've heard anyways.\n\nIf you're thinking of a staging area that should queue all commits\nand lock the repo while testing is in progress, you need to think\nagain, I'm afraid, as that locks up developer time in such huge\namounts that it isn't really worth it. Without the locking, you\nget the problem of trying to automate merges (which may conflict\nand need a manual resolution). Although I guess a merge conflict\ncould result in a delayed push error, so perhaps that could work\ntoo. If you manage to cook up a solution for this, please make a\nsmall writeup on this list how you went about achieving it.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"109506","messageId":"87ljqs7ioz.fsf@rimspace.net","threadId":"18566","inReplyTo":"49CB39E5.5060000@op5.se","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Daniel Pittman","fromEmail":"daniel@rimspace.net","sentAt":"2009-03-26T09:10:52Z","receivedAt":"2009-03-26T09:10:52Z","isPatch":false,"sender":{"key":"daniel@rimspace.net","avatar":"https://gravatar.com/avatar/67f2f02d3521b163f6b7a0dec5943d33fbc5a2a6d9702ab3fef99032e59e692f?d=mp&s=160"},"body":"Andreas Ericsson <ae@op5.se> writes:\n> Daniel Pittman wrote:\n>>\n>> I would like to ensure that my commits are fully bisectable before I\n>> commit them to an upstream repository, at least to the limits of an\n>> automatic tool for testing them.\n>>\n>> 'git bisect run' is similar: it can automatically locate the breaking in\n>> a test suite, for example, but that doesn't help me in the case of three\n>> commits, A (good), B (bad) and C (good, fixing B).\n>>\n>> I would much rather, in this case, use rebase to fix B so that it, too,\n>> builds before I push the changes and pollute a public repository with a\n>> broken changeset — and make bisect that much harder to use in future.\n>\n> You can do that, but it requires manual work too. The trick is to make\n> the release branch immutable on the public repository and use topic\n> branches with per-developer namespaces. The per-developer namespace\n> thing is actually important, as it leaves the freedom to rewind and\n> recreate topics to the developers (which shared branches do not).\n>\n> The manual step comes at merge-time; Someone has to be responsible for\n> merging all the topics that are to be included in the release branch\n> and make sure it builds and passes all tests after each merge.\n\nAh.  You have not quite grasped what I was looking for: I was after a\ntool to help automate that step, rather than a workflow around it.\n\nFor example, the responsible person for that testing could use the\nhypothetical (until someone tells me where to find it):\n\n    git test public..test make test\n\nWhich would then effectively wrap:\n\n    for each revision between public and private:\n         git checkout revision\n         make test\n         # report if that fails, allow fixing the commit or whatever\n         # then 'git test continue' to carry on...\n\nThat turn the process from a manual one to an automated one: it runs\nthat command for every revision until it fails, or until they all pass.\n\nRegards,\n        Daniel\n"},{"id":"109512","messageId":"49CB4ED8.4060205@op5.se","threadId":"18566","inReplyTo":"87ljqs7ioz.fsf@rimspace.net","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-03-26T09:46:00Z","receivedAt":"2009-03-26T09:46:00Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Daniel Pittman wrote:\n> Andreas Ericsson <ae@op5.se> writes:\n>> Daniel Pittman wrote:\n>>> I would like to ensure that my commits are fully bisectable before I\n>>> commit them to an upstream repository, at least to the limits of an\n>>> automatic tool for testing them.\n>>>\n>>> 'git bisect run' is similar: it can automatically locate the breaking in\n>>> a test suite, for example, but that doesn't help me in the case of three\n>>> commits, A (good), B (bad) and C (good, fixing B).\n>>>\n>>> I would much rather, in this case, use rebase to fix B so that it, too,\n>>> builds before I push the changes and pollute a public repository with a\n>>> broken changeset — and make bisect that much harder to use in future.\n>> You can do that, but it requires manual work too. The trick is to make\n>> the release branch immutable on the public repository and use topic\n>> branches with per-developer namespaces. The per-developer namespace\n>> thing is actually important, as it leaves the freedom to rewind and\n>> recreate topics to the developers (which shared branches do not).\n>>\n>> The manual step comes at merge-time; Someone has to be responsible for\n>> merging all the topics that are to be included in the release branch\n>> and make sure it builds and passes all tests after each merge.\n> \n> Ah.  You have not quite grasped what I was looking for: I was after a\n> tool to help automate that step, rather than a workflow around it.\n> \n\nOh right. Sorry, I'm stuck in continuous-integration land where people\ntend to want the server to take care of such things.\n\n> For example, the responsible person for that testing could use the\n> hypothetical (until someone tells me where to find it):\n> \n>     git test public..test make test\n> \n> Which would then effectively wrap:\n> \n>     for each revision between public and private:\n>          git checkout revision\n>          make test\n>          # report if that fails, allow fixing the commit or whatever\n>          # then 'git test continue' to carry on...\n> \n> That turn the process from a manual one to an automated one: it runs\n> that command for every revision until it fails, or until they all pass.\n> \n\nSomething like this?\n--%<--%<--\n#!/bin/sh\n\ngit stash\nrevspec=\"$1\"\nshift\nfor rev in $(git rev-list \"$revspec\"); do\n\tgit checkout $rev\n\t\"$@\" || break\ndone\n--%<--%<--\n\nRun it as such:\n./git-test.sh public..test make test\n\nAdding support for \"continue\" in there shouldn't be hard, since\nyou'd just replace the first rev with the first parent of the\nlast tested revision, although if you want to fix up a commit\nin the middle, you'd have to manually rebase the other ones\nonto the newly created commit (again, not very hard but I'll\nleave both as an exercise to you).\n\nIt doesn't handle merges very nicely, btw, but I guess this\nshould be run prior to merging anyways.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"109514","messageId":"20090326094900.GC14292@coredump.intra.peff.net","threadId":"18566","inReplyTo":"87ljqs7ioz.fsf@rimspace.net","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-26T09:49:00Z","receivedAt":"2009-03-26T09:49:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Mar 26, 2009 at 08:10:52PM +1100, Daniel Pittman wrote:\n\n> For example, the responsible person for that testing could use the\n> hypothetical (until someone tells me where to find it):\n> \n>     git test public..test make test\n> \n> Which would then effectively wrap:\n> \n>     for each revision between public and private:\n>          git checkout revision\n>          make test\n>          # report if that fails, allow fixing the commit or whatever\n>          # then 'git test continue' to carry on...\n> \n> That turn the process from a manual one to an automated one: it runs\n> that command for every revision until it fails, or until they all pass.\n\nI don't think such a script exists. There are continuous integration\nsystems that support git, but I don't know if they actually check every\ncommit.\n\n-Peff\n"},{"id":"109518","messageId":"35F1F494-E496-4126-92CC-79FD241D7BF3@wincent.com","threadId":"18566","inReplyTo":"20090326094900.GC14292@coredump.intra.peff.net","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Wincent Colaiuta","fromEmail":"win@wincent.com","sentAt":"2009-03-26T09:59:59Z","receivedAt":"2009-03-26T09:59:59Z","isPatch":false,"sender":{"key":"greg@hurrell.net","avatar":"https://avatars.githubusercontent.com/u/7074?v=4"},"body":"El 26/3/2009, a las 10:49, Jeff King escribió:\n\n> On Thu, Mar 26, 2009 at 08:10:52PM +1100, Daniel Pittman wrote:\n>\n>> For example, the responsible person for that testing could use the\n>> hypothetical (until someone tells me where to find it):\n>>\n>>    git test public..test make test\n>>\n>> Which would then effectively wrap:\n>>\n>>    for each revision between public and private:\n>>         git checkout revision\n>>         make test\n>>         # report if that fails, allow fixing the commit or whatever\n>>         # then 'git test continue' to carry on...\n>>\n>> That turn the process from a manual one to an automated one: it runs\n>> that command for every revision until it fails, or until they all  \n>> pass.\n>\n> I don't think such a script exists. There are continuous integration\n> systems that support git, but I don't know if they actually check  \n> every\n> commit.\n\nIt shouldn't be too hard to put together a script that does exactly  \nwhat you want.\n\nA couple of years ago I whipped up a simple script (see below) to test  \nall the commits on a topic branch before submitting them upstream.\n\nIt's designed for that specific case, and I'm neither a shell nor a  \nplumbing expert, but it worked for my simple workflow at least.\n\nCheers,\nWincent\n\ndie () {\n   echo \"$1\"\n   exit 1\n}\n\ngit diff --quiet || die \"Unstaged changes; won't start\"\ngit diff --cached --quiet || die \"Staged changes; won't start\"\n\n# parse HEAD (something like \"ref: refs/heads/my_local_branch\") to get  \nremote and merge\nGIT_DIR=$(git rev-parse --git-dir)\nTOPIC=\"$GIT_DIR/HEAD\"\ntest -f \"$TOPIC\" || die \"No HEAD found at '$TOPIC'\"\nBRANCH_REF=$(< \"$TOPIC\")\nif [ \"${BRANCH_REF%%: */*}\" != \"ref\" ]; then\n   die \"expected HEAD '$BRANCH_REF' to be of the form 'ref: .../...'\"\nfi\nBRANCH=${BRANCH_REF##*/}\nREMOTE=$(git config branch.$BRANCH.remote) || die \"failed to get  \nbranch.$BRANCH.remote\"\nMERGE=$(git config branch.$BRANCH.merge) || die \"failed to get branch. \n$BRANCH.merge\"\nMERGE=${MERGE##*/}\n\n# remember which branch we were on prior to starting\ntrap 'git checkout $BRANCH' exit\n\n# will check all commits in topic branch not present in origin\n# (note: may be more appropriate to use \"git merge-base\" here)\necho \"On branch $BRANCH\"\necho \"Commits to be checked:\"\ngit rev-list --pretty=oneline HEAD ^$REMOTE/$MERGE\nfor COMMIT in $(git rev-list HEAD ^$REMOTE/$MERGE); do\n   echo \"Checking $COMMIT\"\n   git checkout $COMMIT\n   make clean\n   make test || die \"Commit $COMMIT is bad (make test failed)\"\ndone\n\necho \"All revisions passed!\"\n"},{"id":"109590","messageId":"8763hviwgw.fsf@rimspace.net","threadId":"18566","inReplyTo":"49CB4ED8.4060205@op5.se","subject":"Re: Test that every revision builds before pushing changes?","fromName":"Daniel Pittman","fromEmail":"daniel@rimspace.net","sentAt":"2009-03-27T01:30:07Z","receivedAt":"2009-03-27T01:30:07Z","isPatch":false,"sender":{"key":"daniel@rimspace.net","avatar":"https://gravatar.com/avatar/67f2f02d3521b163f6b7a0dec5943d33fbc5a2a6d9702ab3fef99032e59e692f?d=mp&s=160"},"body":"Andreas Ericsson <ae@op5.se> writes:\n> Daniel Pittman wrote:\n>> Andreas Ericsson <ae@op5.se> writes:\n>>> Daniel Pittman wrote:\n>>>> I would like to ensure that my commits are fully bisectable before I\n>>>> commit them to an upstream repository, at least to the limits of an\n>>>> automatic tool for testing them.\n\n[...]\n\n>>> The manual step comes at merge-time; Someone has to be responsible for\n>>> merging all the topics that are to be included in the release branch\n>>> and make sure it builds and passes all tests after each merge.\n>>\n>> Ah.  You have not quite grasped what I was looking for: I was after a\n>> tool to help automate that step, rather than a workflow around it.\n>\n> Oh right. Sorry, I'm stuck in continuous-integration land where people\n> tend to want the server to take care of such things.\n\nWe have that also; I am primarily motivated by avoiding trivial breakage\nin the CI server, as well as bisection.\n\n>> For example, the responsible person for that testing could use the\n>> hypothetical (until someone tells me where to find it):\n>>\n>>     git test public..test make test\n\n[...]\n\n> Something like this?\n> --%<--%<--\n> #!/bin/sh\n>\n> git stash\n> revspec=\"$1\"\n> shift\n> for rev in $(git rev-list \"$revspec\"); do\n> \tgit checkout $rev\n> \t\"$@\" || break\n> done\n> --%<--%<--\n>\n> Run it as such:\n> ./git-test.sh public..test make test\n\nThank you, that points me in the right direction, and I can obviously\nseason the rest of it to taste — reverse that revision list, for\nexample.\n\nThank you also to Wincent Colaiuta, who provided a similar script albiet\nwith significantly more detail.\n\n[...]\n\n> It doesn't handle merges very nicely, btw, but I guess this should be\n> run prior to merging anyways.\n\nOnce I have the framework I can quietly work my way through fixing nasty\nissues one way or another.  Thanks.\n\nRegards,\n        Daniel\n"}]}