{"thread":{"id":"25048","subject":"Coping with the pull-before-you-push model","startedAt":"2010-09-09T04:47:53Z","lastAt":"2010-09-15T21:59:54Z","messageCount":14,"participants":["Joshua Jensen","Ævar Arnfjörð Bjarmason","Jon Seymour","Jeff King","Avery Pennarun","Theodore Tso","Eugene Sajine","Ted Ts'o","David Brown"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"150347","messageId":"4C8866F9.1040705@workspacewhiz.com","threadId":"25048","inReplyTo":null,"subject":"Coping with the pull-before-you-push model","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-09T04:47:53Z","receivedAt":"2010-09-09T04:47:53Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  After a deployment of Git on a centralized server at my place of \nbusiness, the largest amount of grumbling has been with the \npull-before-you-push model.  Coming from the file-centric Perforce where \nyou need only have latest of just the files you are submitting, the \npull-before-you-push model has really been a pain in the neck for a \nlarge team.\n\nEven with topic branches being used, merges to master occur frequently.  \nIt can really be a frustrating battle to get your merged branch pushed \nto the central master branch.  In the time it took you to pull, test, \nand push, someone has probably already pushed before you.  To cope with \nthis, people will pull, not bother testing, and immediately push their \nchanges.  Yes, this could result in build instability, but it is \nconsidered better than never being able to make your change live.\n\n(Let's ignore what we should or shouldn't be doing as far as \n'development practices'.  :)  We're solving the problems one step at a \ntime...)\n\nGerrit provides a compelling model where branches are pushed to the code \nreview server in the form refs/for/master, and the given push will \nalways succeed.  Code reviews are performed, someone sets the verified \nbit, and the change is submitted and merged to master by Gerrit itself \nin a queued fashion.  Unfortunately, its general \"requirement\" to squash \nyour branch down to a single commit is, possibly, a showstopper.  If it \ntreated a branch merge as a group of commits that MUST stay together, \nthat would be perfect.\n\nWhat other tools are out there that would let users successfully push \ntheir branch to the server (without having the HEAD master commit), and \nthe push would be automatically merged to the master branch?\n\nIs there another workflow that is successful for your large(-ish) \nenterprise team?\n\nThanks for your insights!\n\nJosh\n"},{"id":"150364","messageId":"AANLkTikY55ZJvSTqyFKLqwABqnJZuODz3yrc7CFvQf0K@mail.gmail.com","threadId":"25048","inReplyTo":"4C8866F9.1040705@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-09-09T13:06:38Z","receivedAt":"2010-09-09T13:06:38Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Thu, Sep 9, 2010 at 04:47, Joshua Jensen <jjensen@workspacewhiz.com> wrote:\n>  After a deployment of Git on a centralized server at my place of business,\n> the largest amount of grumbling has been with the pull-before-you-push\n> model.  Coming from the file-centric Perforce where you need only have\n> latest of just the files you are submitting, the pull-before-you-push model\n> has really been a pain in the neck for a large team.\n>\n> Even with topic branches being used, merges to master occur frequently.  It\n> can really be a frustrating battle to get your merged branch pushed to the\n> central master branch.  In the time it took you to pull, test, and push,\n> someone has probably already pushed before you.  To cope with this, people\n> will pull, not bother testing, and immediately push their changes.  Yes,\n> this could result in build instability, but it is considered better than\n> never being able to make your change live.\n>\n> (Let's ignore what we should or shouldn't be doing as far as 'development\n> practices'.  :)  We're solving the problems one step at a time...)\n\nLet's not ignore that.\n\nPresumably you had exactly the same problem in perforce, i.e. because\nyou only had have the files you were changing checked out in Perforce\nin the time between `hack && pull && test && push` someone else might\nhave already pushed. Thus what you just submitted wasn't guaranteed to\npass tests.\n\nSo is the flow in Git where you don't run the tests again, rebase and\npush and hope for the best any different?\n\n> Gerrit provides a compelling model where branches are pushed to the code\n> review server in the form refs/for/master, and the given push will always\n> succeed.  Code reviews are performed, someone sets the verified bit, and the\n> change is submitted and merged to master by Gerrit itself in a queued\n> fashion.  Unfortunately, its general \"requirement\" to squash your branch\n> down to a single commit is, possibly, a showstopper.  If it treated a branch\n> merge as a group of commits that MUST stay together, that would be perfect.\n\nThis sounds like something that's configurable in Gerrit, or should\nbe.\n\n> [..]\n>\n> Is there another workflow that is successful for your large(-ish) enterprise\n> team?\n\nLinux manages to deal with a huge number of commits, but does so by\nhaving subsystems.\n\nMaybe that's something you can use in your codebase?\n"},{"id":"150367","messageId":"4C88F2A9.2080306@workspacewhiz.com","threadId":"25048","inReplyTo":"AANLkTikY55ZJvSTqyFKLqwABqnJZuODz3yrc7CFvQf0K@mail.gmail.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-09T14:43:53Z","receivedAt":"2010-09-09T14:43:53Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  ----- Original Message -----\nFrom: Ævar Arnfjörð Bjarmason\nDate: 9/9/2010 7:06 AM\n> On Thu, Sep 9, 2010 at 04:47, Joshua Jensen<jjensen@workspacewhiz.com>  wrote:\n>>   After a deployment of Git on a centralized server at my place of business,\n>> the largest amount of grumbling has been with the pull-before-you-push\n>> model.  Coming from the file-centric Perforce where you need only have\n>> latest of just the files you are submitting, the pull-before-you-push model\n>> has really been a pain in the neck for a large team.\n>>\n>> Even with topic branches being used, merges to master occur frequently.  It\n>> can really be a frustrating battle to get your merged branch pushed to the\n>> central master branch.  In the time it took you to pull, test, and push,\n>> someone has probably already pushed before you.  To cope with this, people\n>> will pull, not bother testing, and immediately push their changes.  Yes,\n>> this could result in build instability, but it is considered better than\n>> never being able to make your change live.\n>>\n>> (Let's ignore what we should or shouldn't be doing as far as 'development\n>> practices'.  :)  We're solving the problems one step at a time...)\n> Let's not ignore that.\nFair.\n> Presumably you had exactly the same problem in perforce, i.e. because\n> you only had have the files you were changing checked out in Perforce\n> in the time between `hack&&  pull&&  test&&  push` someone else might\n> have already pushed. Thus what you just submitted wasn't guaranteed to\n> pass tests.\n>\n> So is the flow in Git where you don't run the tests again, rebase and\n> push and hope for the best any different?\nThe end result is the same; submitted code is never really tested \nagainst latest in Perforce either.  The primary difference between the \ntwo is that the Perforce submit is successful the majority of the time \n(odds of someone editing and submitting the same file as you are low), \nand the Git push fails the majority of the time.\n\nDon't get me wrong.  I've given training on why Git's enforced \npull-before-you-push model can be better than what we had before \n(reproducible state, fewer broken builds, etc).  Nevertheless, the issue \nis very frequent, and that's why I am querying others.\n>> Gerrit provides a compelling model where branches are pushed to the code\n>> review server in the form refs/for/master, and the given push will always\n>> succeed.  Code reviews are performed, someone sets the verified bit, and the\n>> change is submitted and merged to master by Gerrit itself in a queued\n>> fashion.  Unfortunately, its general \"requirement\" to squash your branch\n>> down to a single commit is, possibly, a showstopper.  If it treated a branch\n>> merge as a group of commits that MUST stay together, that would be perfect.\n> This sounds like something that's configurable in Gerrit, or should\n> be.\nAgreed, but it appears it is currently a missing feature.  There have \nbeen discussions about it on their mailing list over the past months, \nand a feature request is in their tracker.  I am unsure of their \nprogress or even interest.\n>> [..]\n>>\n>> Is there another workflow that is successful for your large(-ish) enterprise\n>> team?\n> Linux manages to deal with a huge number of commits, but does so by\n> having subsystems.\n>\n> Maybe that's something you can use in your codebase?\nUnless I'm mistaken, though, it seems to work in reverse of what our \nworking model has been for years.\n\nI'm grossly oversimplifying the process, but the Linux model seems to be \nbuilt on hierarchical 'pull requests'.  I can tell a subsystem \nmaintainer I have some changes and then ask that maintainer to pull them \nfrom a certain location.  When that person has time/inclination, the \nchange is pulled, merged in, and then another pull request is sent to \nthe upstream hierarchy.\n\nThis _is_ compelling, but even if it would work within the company I \nwork for, it is such a dramatic shift in workflow that I am certain it \ncould not be done in one fell swoop.\n\nThanks for your insights!\n\nJosh\n"},{"id":"150427","messageId":"AANLkTikdV3W1d7uNokKRRiT4FeznL1uM=Y9SQLDqgAic@mail.gmail.com","threadId":"25048","inReplyTo":"4C88F2A9.2080306@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Jon Seymour","fromEmail":"jon.seymour@gmail.com","sentAt":"2010-09-10T05:35:21Z","receivedAt":"2010-09-10T05:35:21Z","isPatch":false,"sender":{"key":"jon.seymour@gmail.com","avatar":"https://avatars.githubusercontent.com/u/207131?v=4"},"body":"On Fri, Sep 10, 2010 at 12:43 AM, Joshua Jensen\n<jjensen@workspacewhiz.com> wrote:>>\n>> Presumably you had exactly the same problem in perforce, i.e. because\n>> you only had have the files you were changing checked out in Perforce\n>> in the time between `hack&&  pull&&  test&&  push` someone else might\n>> have already pushed. Thus what you just submitted wasn't guaranteed to\n>> pass tests.\n>>\n>> So is the flow in Git where you don't run the tests again, rebase and\n>> push and hope for the best any different?\n>\n> The end result is the same; submitted code is never really tested against\n> latest in Perforce either.  The primary difference between the two is that\n> the Perforce submit is successful the majority of the time (odds of someone\n> editing and submitting the same file as you are low), and the Git push fails\n> the majority of the time.\n>\n> Don't get me wrong.  I've given training on why Git's enforced\n> pull-before-you-push model can be better than what we had before\n> (reproducible state, fewer broken builds, etc).  Nevertheless, the issue is\n> very frequent, and that's why I am querying others.\n\nI am not sure that git does enforce a pull before push model, except\nif you attempt\nto use git without the equivalent of a maintainer - in essence\ndevolving responsibility of maintaining the tip of the shared branch\nto the whole team.\n\nThis is probably the cultural shift that is hardest for enterprises to\naccept - why do I need a _person_ to do this _manual_ work when tools\nlike {insert favourite non-DVCS here} can do this for me? To\nmanagement, this looks like a step-backwards.\n\nIn our case, the team leaders act like the subsystem leads and the\nbuild team acts like the maintainer. Team leads have a more review\noriented focus, where as the build team doesn't understand the code\nthat well and will reject merge conflicts if they are non-trivial.\nTheir focus is just to keep the daily build rhythm going.\n\nIf you don't have a maintainer role, then everyone is obliged to pull,\ntest, pull-again, then push because there is no-one else there to do\nit and no other time to do it. The delays inherent in the test process\ninevitably mean you have to cycle several times until your commit\nwins.\n\nBut with a maintainer role on your project, you don't need to do this.\nEveryone develops on yesterday's baseline and verifies that they don't\nregress anything with respect to that baseline - there is no\nrequirement to certify against the bleeding edge of the integration\nstream - that's the role of the maintainer/build team once the daily\nbuild is done, thereby amortising test execution effort across the\nwhole team.\n\n>\n> I'm grossly oversimplifying the process, but the Linux model seems to be\n> built on hierarchical 'pull requests'.  I can tell a subsystem maintainer I\n> have some changes and then ask that maintainer to pull them from a certain\n> location.  When that person has time/inclination, the change is pulled,\n> merged in, and then another pull request is sent to the upstream hierarchy.\n>\n\nNote that one difference between the maintainer and maintainer-less\nmodel is that the tip of the reference branch is published relatively\nrarely - it is quite stable. This is quite helpful because the\nbaseline doesn't keep shifting every 30 minutes. In the\nmaintainer-less model, you never have stability, nor any opportunity\nto acquire it.\n\n> This _is_ compelling, but even if it would work within the company I work\n> for, it is such a dramatic shift in workflow that I am certain it could not\n> be done in one fell swoo\n>\n\nThe big difference between commercial work and open source projects\nlike git and linux is that the latter have just one constraint -\nquality whereas the former also have budgets and schedules to worry\nabout. Unintegrated crap code has cost the open source project almost\nnothing. Commercial enterprises, on the other hand, pay lots of money\nfor people to develop code whether it is crap or otherwise. The idea\nof leaving expensive code unintegrated causes management paroxysms of\nconcern that are hard to ignore - a tool that blindly integrates crap\ncode automatically is a soothing balm for such people. Hence the\nresistance to tools like git that encourage a maintainer role and,\nimplicit in that, the possibility of review that might result in crap\ncode being exposed for what it is.\n\njon seymour.\n"},{"id":"150437","messageId":"20100910141527.GA6936@sigill.intra.peff.net","threadId":"25048","inReplyTo":"AANLkTikdV3W1d7uNokKRRiT4FeznL1uM=Y9SQLDqgAic@mail.gmail.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-09-10T14:15:27Z","receivedAt":"2010-09-10T14:15:27Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 10, 2010 at 03:35:21PM +1000, Jon Seymour wrote:\n\n> This is probably the cultural shift that is hardest for enterprises to\n> accept - why do I need a _person_ to do this _manual_ work when tools\n> like {insert favourite non-DVCS here} can do this for me? To\n> management, this looks like a step-backwards.\n\nBear in mind that you can still shift to a maintainer model, but keep\nthe maintainer automated. That is, you can queue up \"to-pull\" heads, and\nthen have an automated process pull them one by one and do some basic QA\n(does it merge, does it build, does it pass automated tests, etc). Which\nis not that different from what many shops do in the non-maintainer\nmodel, except that when you break the build, the maintainer process\nnotices _before_ publishing the merged tip, so everybody won't try to\nbuild on your broken crap.\n\nI seem to recall that Gerrit does something like this, but I may be\nmis-remembering. I haven't actually used it for real work.\n\nI still prefer a human maintainer, because they can do things like\nreorder the queue manually (or outright reject flaky topics) to get more\nsensible merges, or do easy but non-trivial merges themselves to avoid\nkicking code back to the developer.\n\n-Peff\n"},{"id":"150660","messageId":"4C8EFBFC.60409@workspacewhiz.com","threadId":"25048","inReplyTo":"AANLkTikdV3W1d7uNokKRRiT4FeznL1uM=Y9SQLDqgAic@mail.gmail.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-14T04:37:16Z","receivedAt":"2010-09-14T04:37:16Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  ----- Original Message -----\nFrom: Jon Seymour\nDate: 9/9/2010 11:35 PM\n>> This _is_ compelling, but even if it would work within the company I work\n>> for, it is such a dramatic shift in workflow that I am certain it could not\n>> be done in one fell swo\n>>\n> The big difference between commercial work and open source projects\n> like git and linux is that the latter have just one constraint -\n> quality whereas the former also have budgets and schedules to worry\n> about. Unintegrated crap code has cost the open source project almost\n> nothing. Commercial enterprises, on the other hand, pay lots of money\n> for people to develop code whether it is crap or otherwise. The idea\n> of leaving expensive code unintegrated causes management paroxysms of\n> concern that are hard to ignore - a tool that blindly integrates crap\n> code automatically is a soothing balm for such people. Hence the\n> resistance to tools like git that encourage a maintainer role and,\n> implicit in that, the possibility of review that might result in crap\n> code being exposed for what it is.\nMy apologies for the late response.  I'm not ignoring this.  I am trying \nto process it in the context of my organization as I establish the \nworkflows we'll be using.  I may have some follow up questions soon.\n\nJosh\n"},{"id":"150661","messageId":"4C8EFE62.7080908@workspacewhiz.com","threadId":"25048","inReplyTo":"20100910141527.GA6936@sigill.intra.peff.net","subject":"Re: Coping with the pull-before-you-push model","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-14T04:47:30Z","receivedAt":"2010-09-14T04:47:30Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  ----- Original Message -----\nFrom: Jeff King\nDate: 9/10/2010 8:15 AM\n> On Fri, Sep 10, 2010 at 03:35:21PM +1000, Jon Seymour wrote:\n>\n>> This is probably the cultural shift that is hardest for enterprises to\n>> accept - why do I need a _person_ to do this _manual_ work when tools\n>> like {insert favourite non-DVCS here} can do this for me? To\n>> management, this looks like a step-backwards.\n> Bear in mind that you can still shift to a maintainer model, but keep\n> the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n> then have an automated process pull them one by one and do some basic QA\n> (does it merge, does it build, does it pass automated tests, etc). Which\n> is not that different from what many shops do in the non-maintainer\n> model, except that when you break the build, the maintainer process\n> notices _before_ publishing the merged tip, so everybody won't try to\n> build on your broken crap.\n>\nDo you know of any existing software that does this?  This may be ideal \nin the short term.\n> I still prefer a human maintainer, because they can do things like\n> reorder the queue manually (or outright reject flaky topics) to get more\n> sensible merges, or do easy but non-trivial merges themselves to avoid\n> kicking code back to the developer.\n>\nI agree with you concerning how valuable a human maintainer is.\n\nJosh\n"},{"id":"150662","messageId":"20100914052451.GA15839@sigill.intra.peff.net","threadId":"25048","inReplyTo":"4C8EFE62.7080908@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-09-14T05:24:51Z","receivedAt":"2010-09-14T05:24:51Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 13, 2010 at 10:47:30PM -0600, Joshua Jensen wrote:\n\n> >Bear in mind that you can still shift to a maintainer model, but keep\n> >the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n> >then have an automated process pull them one by one and do some basic QA\n> >(does it merge, does it build, does it pass automated tests, etc). Which\n> >is not that different from what many shops do in the non-maintainer\n> >model, except that when you break the build, the maintainer process\n> >notices _before_ publishing the merged tip, so everybody won't try to\n> >build on your broken crap.\n> >\n> Do you know of any existing software that does this?  This may be\n> ideal in the short term.\n\nI think that Avery Pennarun's gitbuilder may do what you want:\n\n  http://github.com/apenwarr/gitbuilder/\n\nbut I've never used it.\n\nI seem to recall from one of Shawn's presentations on Gerrit Code Review\nthat it does something like this, too, but I can't seem to find any docs\nabout it in my brief search:\n\n  http://code.google.com/p/gerrit/\n\nIt may be that Gerrit doesn't handle building itself, but that the\nAndroid project is running something alongside it. Shawn may be able to\nsay more.\n\nBasically, what we are talking about is continuous integration, with the\nslight twist that instead of developers pushing commits to a mainline\nbranch which is built and tested, we would build and test their commits\nand then merge them to the mainline branch.\n\nSystems like Hudson that do continuous integration and support git may\nhandle a workflow like this, but I don't know (I've only ever used\nHudson in the everything-goes-to-svn-trunk model).\n\nThere is also a small continuous integration system that lives in the\ncontrib/continuous directory of git.git itself. It's quite old at this\npoint, but I wouldn't be surprised if its descendant is what Gerrit\nuses.\n\n-Peff\n"},{"id":"150664","messageId":"AANLkTik_9q--ZkwXLFwWDWP5CM5jN1ynqtFgezuApp+A@mail.gmail.com","threadId":"25048","inReplyTo":"20100914052451.GA15839@sigill.intra.peff.net","subject":"Re: Coping with the pull-before-you-push model","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-09-14T05:59:54Z","receivedAt":"2010-09-14T05:59:54Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Mon, Sep 13, 2010 at 10:24 PM, Jeff King <peff@peff.net> wrote:\n> On Mon, Sep 13, 2010 at 10:47:30PM -0600, Joshua Jensen wrote:\n>> >Bear in mind that you can still shift to a maintainer model, but keep\n>> >the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n>> >then have an automated process pull them one by one and do some basic QA\n>> >(does it merge, does it build, does it pass automated tests, etc). Which\n>> >is not that different from what many shops do in the non-maintainer\n>> >model, except that when you break the build, the maintainer process\n>> >notices _before_ publishing the merged tip, so everybody won't try to\n>> >build on your broken crap.\n>>\n>> Do you know of any existing software that does this?  This may be\n>> ideal in the short term.\n>\n> I think that Avery Pennarun's gitbuilder may do what you want:\n>\n>  http://github.com/apenwarr/gitbuilder/\n>\n> but I've never used it.\n\ngitbuilder doesn't do this.\n\nHowever, it does let you automatically do user-defined things for each\nbranch, and it stores the result for each one.  So it would be a\npretty short trip from there to auto-merging when a branch is ready.\n\nHave fun,\n\nAvery\n"},{"id":"150669","messageId":"D4360EBB-7891-457E-A6AC-7159CADCAC6C@mit.edu","threadId":"25048","inReplyTo":"4C8EFE62.7080908@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2010-09-14T12:12:13Z","receivedAt":"2010-09-14T12:12:13Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"\nOn Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote:\n\n>>> management, this looks like a step-backwards.\n>> Bear in mind that you can still shift to a maintainer model, but keep\n>> the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n>> then have an automated process pull them one by one and do some basic QA\n>> (does it merge, does it build, does it pass automated tests, etc). Which\n>> is not that different from what many shops do in the non-maintainer\n>> model, except that when you break the build, the maintainer process\n>> notices _before_ publishing the merged tip, so everybody won't try to\n>> build on your broken crap.\n>> \n> Do you know of any existing software that does this?  This may be ideal in the short term.\n\nOur workflow at $WORK involves pushing changes to gerrit to various \"effort branches\", and then once they are approved, we have a \"Mergitator\" script that will attempt to merge the effort branch with the merged master branch, and then attempt to do a build.  If the build succeeds, then the changes will get pushed back to the publically visible merged master branch, and then the Mergitator will move on to the next effort branch that requires merging.   If there is a merge conflict, the Mergitator will refuse the merge, and then give instructions on how to fix up the tree to avoid merge conflicts.\n\nThe Mergitator code hasn't been released, and I suspect the main reason is that there's relatively little code that could be used outside of our environment, and a large amount of code which contains lots of details about our internal build system that would have to be stripped out and generalized before it could be released --- and no one has time to do it.\n\nSo this probably doesn't help you since I suspect you meant to ask the question, \"do you know of any existing publically available software\", but I can tell you that it certainly can be done, and that software exists.  Making it be software which is useful and usable to you would definitely take more work...\n\n-- Ted\n"},{"id":"150694","messageId":"4C8F99FA.3040003@workspacewhiz.com","threadId":"25048","inReplyTo":"D4360EBB-7891-457E-A6AC-7159CADCAC6C@mit.edu","subject":"Re: Coping with the pull-before-you-push model","fromName":"Joshua Jensen","fromEmail":"jjensen@workspacewhiz.com","sentAt":"2010-09-14T15:51:22Z","receivedAt":"2010-09-14T15:51:22Z","isPatch":false,"sender":{"key":"jjensen@workspacewhiz.com","avatar":"https://avatars.githubusercontent.com/u/111687?v=4"},"body":"  ----- Original Message -----\nFrom: Theodore Tso\nDate: 9/14/2010 6:12 AM\n> On Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote:\n>>> Bear in mind that you can still shift to a maintainer model, but keep\n>>> the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n>>> then have an automated process pull them one by one and do some basic QA\n>>> (does it merge, does it build, does it pass automated tests, etc). Which\n>> Do you know of any existing software that does this?  This may be ideal in the short ter\n> Our workflow at $WORK involves pushing changes to gerrit to various \"effort branches\", and then once they are approved, we have a \"Mergitator\" script that will attempt to merge the effort branch with the merged master branch, and then attempt to do a build.  If the build succeeds, then the changes will get pushed back to the publically visible merged master branch, and then the Mergitator will move on to the next effort branch that requires merging.   If there is a merge conflict, the Mergitator will refuse the merge, and then give instructions on how to fix up the tree to avoid merge conflicts.\n>\nHow does the integration with Gerrit work here?  The only thing that \ncomes to mind is that you do something like:\n\ngit push gerrit HEAD:refs/for/merged-master\n\nThen the approvals are done.  Afterward, Gerrit merges to the \nmerged-master branch.\n\nI would suppose an external script is performing regular fetches.  When \nit sees new code, it builds.\n\nNo, this can't be right, but I'll leave my incorrect workflow here.\n> So this probably doesn't help you since I suspect you meant to ask the question, \"do you know of any existing publically available software\", but I can tell you that it certainly can be done, and that software exists.  Making it be software which is useful and usable to you would definitely take more work...\nIt's the branch queueing issue that is my current hang up.  Gerrit's \nmethod is slick, but that won't work outside of JGit.  I'm not opposed \nto JGit; I just haven't touched Java in years.\n\nSo, perhaps, a web interface where the branch owner selects the (pushed \nto central server) branch name that is ready to go.  A merge is \nattempted.  If it succeeds, great.  If it fails, then the merge is reset?\n\nJosh\n"},{"id":"150700","messageId":"AANLkTi=Yg-Pgu7E_hxzPVToO0wDjiTm=NxKPdrgk9JaE@mail.gmail.com","threadId":"25048","inReplyTo":"4C8F99FA.3040003@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Eugene Sajine","fromEmail":"euguess@gmail.com","sentAt":"2010-09-14T16:24:51Z","receivedAt":"2010-09-14T16:24:51Z","isPatch":false,"sender":{"key":"euguess@gmail.com","avatar":null},"body":"On Tue, Sep 14, 2010 at 11:51 AM, Joshua Jensen\n<jjensen@workspacewhiz.com> wrote:\n>  ----- Original Message -----\n> From: Theodore Tso\n> Date: 9/14/2010 6:12 AM\n>>\n>> On Sep 14, 2010, at 12:47 AM, Joshua Jensen wrote:\n>>>>\n>>>> Bear in mind that you can still shift to a maintainer model, but keep\n>>>> the maintainer automated. That is, you can queue up \"to-pull\" heads, and\n>>>> then have an automated process pull them one by one and do some basic QA\n>>>> (does it merge, does it build, does it pass automated tests, etc). Which\n>>>\n>>> Do you know of any existing software that does this?  This may be ideal\n>>> in the short ter\n>>\n>> Our workflow at $WORK involves pushing changes to gerrit to various\n>> \"effort branches\", and then once they are approved, we have a \"Mergitator\"\n>> script that will attempt to merge the effort branch with the merged master\n>> branch, and then attempt to do a build.  If the build succeeds, then the\n>> changes will get pushed back to the publically visible merged master branch,\n>> and then the Mergitator will move on to the next effort branch that requires\n>> merging.   If there is a merge conflict, the Mergitator will refuse the\n>> merge, and then give instructions on how to fix up the tree to avoid merge\n>> conflicts.\n>>\n> How does the integration with Gerrit work here?  The only thing that comes\n> to mind is that you do something like:\n>\n> git push gerrit HEAD:refs/for/merged-master\n>\n> Then the approvals are done.  Afterward, Gerrit merges to the merged-master\n> branch.\n>\n> I would suppose an external script is performing regular fetches.  When it\n> sees new code, it builds.\n>\n> No, this can't be right, but I'll leave my incorrect workflow here.\n>>\n>> So this probably doesn't help you since I suspect you meant to ask the\n>> question, \"do you know of any existing publically available software\", but I\n>> can tell you that it certainly can be done, and that software exists.\n>>  Making it be software which is useful and usable to you would definitely\n>> take more work...\n>\n> It's the branch queueing issue that is my current hang up.  Gerrit's method\n> is slick, but that won't work outside of JGit.  I'm not opposed to JGit; I\n> just haven't touched Java in years.\n>\n> So, perhaps, a web interface where the branch owner selects the (pushed to\n> central server) branch name that is ready to go.  A merge is attempted.  If\n> it succeeds, great.  If it fails, then the merge is reset?\n>\n> Josh\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n\n\nIn our case we have moved from CVS to git and had pretty much same\nproblems.  The only difference was that we are working with java and\ntrying to have every project in separate repository. So we don't have\nmany commits into common codebase. We have a server keeping bare\nmainline repos.\nIn order to integrate the project with others we are using Hudson CI\nwhich performs builds upon each change detected on the server side\n(i.e. mainline repository) and also builds the projects that are\ndependent on the one you just changed. Those dependencies can be\ndescribed inside each hudson job, but we are using Ivy for dependency\nmanagement. If you have good testing infrastructure that is even\nbetter.\n\nSo generally the amount of contributors to one project is pretty\nsmall. Some projects are solo, some have small teams coworking on one\nproject code base.\n\nWe are using two main branch model.\nThe development goes in topic branches. The qa branch is a staging\nbranch for the changes to be accumulated for the release. People are\nusing direct pushes to qa branch to integrate or a maintainer. Hudson\nbuilds this codebase. When all release functionality is done the\nperson responsible for the release is merging qa branch into the\nmaster, which gets released (this must be a fast-forward).\n\nSo, i would say component oriented approach is the key here.\n\nHTH,\nEugene\n"},{"id":"150701","messageId":"20100914164917.GA3730@thunk.org","threadId":"25048","inReplyTo":"4C8F99FA.3040003@workspacewhiz.com","subject":"Re: Coping with the pull-before-you-push model","fromName":"Ted Ts'o","fromEmail":"tytso@mit.edu","sentAt":"2010-09-14T16:49:17Z","receivedAt":"2010-09-14T16:49:17Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Tue, Sep 14, 2010 at 09:51:22AM -0600, Joshua Jensen wrote:\n> How does the integration with Gerrit work here?  The only thing that\n> comes to mind is that you do something like:\n> \n> git push gerrit HEAD:refs/for/merged-master\n\nSo we'll push to something to, say, refs/heads/fs/ext4/for-merged on\nthe gerrit server, and let gerrit do its thing.  After a colleague\napproves all of the patches in the branch, gerrit will release the\ncommits to the branch, and only then will the mergitator script\nattempt to do a trial merge of the effort branch with the\nmerged/master branch.  If the trial merge succeeds, then it will\nattempt to do a trial compile.  If the trial compile succeeds then the\nmerged/master branch will be updated with the commit id of the trial\nmerge, and then the mergitator script will move on to the next effort\nbranch will has been updated.  If the mergitator fails to merge a\nparticular branch, then an e-mail is sent out explanining the cause of\nthe merge failure, so a human can fix things up.  This could be done\nby cherry-picking a commit from merged/master which caused the merge\nconflict, and then fixing up the merge conflict, for example.\n\nA wise developer will do a trial merge on their own *before*\nsubmitting their effort branch to gerrit for code review, but this is\nnot strictly speaking required.\n\n\t\t\t\t\t\t- Ted\n"},{"id":"150760","messageId":"20100915215954.GA20880@huya.qualcomm.com","threadId":"25048","inReplyTo":"20100914052451.GA15839@sigill.intra.peff.net","subject":"Re: Coping with the pull-before-you-push model","fromName":"David Brown","fromEmail":"davidb@codeaurora.org","sentAt":"2010-09-15T21:59:54Z","receivedAt":"2010-09-15T21:59:54Z","isPatch":false,"sender":{"key":"davidb@codeaurora.org","avatar":"https://gravatar.com/avatar/1bacedec21621bd4efa4bc8ecb05c507f0cf0641ecdaa50943c2ed21c8ef22d8?d=mp&s=160"},"body":"On Tue, Sep 14, 2010 at 01:24:51AM -0400, Jeff King wrote:\n\n> I seem to recall from one of Shawn's presentations on Gerrit Code Review\n> that it does something like this, too, but I can't seem to find any docs\n> about it in my brief search:\n> \n>   http://code.google.com/p/gerrit/\n> \n> It may be that Gerrit doesn't handle building itself, but that the\n> Android project is running something alongside it. Shawn may be able to\n> say more.\n> \n> Basically, what we are talking about is continuous integration, with the\n> slight twist that instead of developers pushing commits to a mainline\n> branch which is built and tested, we would build and test their commits\n> and then merge them to the mainline branch.\n\nGerrit doesn't do the build and test, but it isn't all that difficult\nto hook into.  It also allows the results of the build and test to\nrecord their state so that the developer can track what is happening.\n\nThe nice part about it, compared to other CI systems I've seen is that\nit catches the problems before they are merged into master, rather\nthan after.\n\nInternally, we actually do multiple levels of a CI-type thing.  Every\ntime a developer uploads a change, one machine performs a sanity build\non it (with multiple configurations).  These results are visible in\nGerrit, and it keeps people from putting too much effort into\nreviewing code that doesn't even compile.\n\nOnce the code has gotten through two levels of code review, a more\nthroughough build and test system pulls the whole project (from\nnumerous git repos) builds and runs tests.  This takes long enough\nthat it doesn't do individual changes, so failures take work to track\ndown, but it does generally assure that the result of each 'git merge'\nsomewhat works.\n\nThe only real annoying part I've found with the gerrit model is that\nthe tree is filled with lots of merge comments (generally a merge for\nevery real commit).  The other option is to let Gerrit rebase, which\nthen gets annoying when a developer has pulled changes from other\ndevelopers.\n\nIt also has a nice link to a 'git pull' command to pull down an\nindividual change.\n\nDavid\n"}]}