{"thread":{"id":"22632","subject":"Solve continuous integration (pending head / commit queue) problem using git","startedAt":"2010-02-12T16:37:34Z","lastAt":"2010-02-13T22:11:16Z","messageCount":5,"participants":["Jan Koprowski","Avery Pennarun","Daniel Barkalow"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"134355","messageId":"c41cd75d1002120837t20f2a47fi41e8c67245c4284c@mail.gmail.com","threadId":"22632","inReplyTo":null,"subject":"Solve continuous integration (pending head / commit queue) problem using git","fromName":"Jan Koprowski","fromEmail":"jan.koprowski@gmail.com","sentAt":"2010-02-12T16:37:34Z","receivedAt":"2010-02-12T16:37:34Z","isPatch":false,"sender":{"key":"jan.koprowski@gmail.com","avatar":null},"body":"Hi !\n\n This is my first mail on the list so hello everyone :)\n I'am currently write my MA. Part of my thesis is looking for some\nway to stay \"master\" stable.\n First assumption is simple: Control version system make all dirty\njob - programmers can't \"stop\" or \"break\" all procedure and work in\nnatural ways. We can't assume that programmer use some custom hooks on\ntheir side or add some addition parameters to git commands.\n Second assumption is that programmer can't compile source code one\ntheir machine for some reasons. Only way to compile is some other way\n- for example use CruiseControl or Hudson or some CI tools. This isn't\nreally matter in my question now.\n Third assumption: Code cloned from repository is stable = compiling\nwell and pass all tests.\n Forth assumption: compiling is testing are very very very fast.\n\nWhat I meen \"natural way of work\" by programmer. They *clone*\nrepository (or *pull* changes) from some central repository. Then do\nsome stuff with code on their working copy and *push* their changes.\nOk but - their don't know is code working well. And this is a problem.\nI know there is some options: XP pair programming, automated static\ncode analysis, code review and others ... but this is not the point.\nIn my \"configuration\" there are some frequently scheduled build of\nsystem automated by some tool. Tool just get all stuff from repo,\ncompile all stuff, running tests (if compiling successes) and if\nsending e-mail.\nBut SCM should \"somehow\" distinct unstable commits from stable commits\n(after compiling).\n\nNow. My idea. There is some revision tagged as \"stable\". *Clone* and\n*pull* operations is somehow \"overloaded\" from server side and always!\nreturn last revision tagged as stable. After compiling external tool\njust move tag to another revision which pass all tests. Of course\nthere is some additional parameter (for example --last or --unstable)\nwhich can clone fine way of repository.\n\nTwo questions.\n1) Maybe I try to invent the wheel again. Is there any way to take the\neffect without overloading standard git behaviours.\n2) If not how overload git behaviors on git \"server side\" repo?\n\nThanks in advance!\n-- \n><> Jan Koprowski [696775174] GSM\n"},{"id":"134364","messageId":"32541b131002120942w50a29e7cjf2c10820b3286017@mail.gmail.com","threadId":"22632","inReplyTo":"c41cd75d1002120837t20f2a47fi41e8c67245c4284c@mail.gmail.com","subject":"Re: Solve continuous integration (pending head / commit queue) problem using git","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2010-02-12T17:42:44Z","receivedAt":"2010-02-12T17:42:44Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On Fri, Feb 12, 2010 at 11:37 AM, Jan Koprowski wrote:\n> Now. My idea. There is some revision tagged as \"stable\". *Clone* and\n> *pull* operations is somehow \"overloaded\" from server side and always!\n> return last revision tagged as stable. After compiling external tool\n> just move tag to another revision which pass all tests. Of course\n> there is some additional parameter (for example --last or --unstable)\n> which can clone fine way of repository.\n>\n> Two questions.\n> 1) Maybe I try to invent the wheel again. Is there any way to take the\n> effect without overloading standard git behaviours.\n> 2) If not how overload git behaviors on git \"server side\" repo?\n\nIn general, code that lies to you about what's the most revision is\nevil. Sometimes you *do* want to fetch that revision it's lying to\nyou and saying doesn't exist, precisely because you'd like to help fix\nit before integration.\n\nWhat you really want is:\n\n- nobody can push to the \"integration branch\" except the \"integration manager\"\n\n- the \"integration manager\" should be a computer program, so that you\ncan have \"continuous integration\"\n\nThis isn't actually that hard. Give each user their own repository;\nno user can write to any other user's repository. (This is the\ndefault setup on github.com, for example.) Alternatively, just tell\npeople to never, ever push to the master branch by themselves. People\nare easily capable of following rules like that unless they're\nactively trying to screw you.\n\nThen set up something like gitbuilder\n(http://github.com/apenwarr/gitbuilder) (Full disclosure: I wrote it)\nto build *all* the branches from *all* the users. This sounds like it\nwould create exponential work for the build machine, but it doesn't,\nsince most users will have mostly the same commits anyway.\n\nWhen gitbuilder tags a particular commit as having built and passed\nall tests, then it becomes a candidate for merging into the\nintegration branch. Write a little script that goes through candidate\nbranches, checks their gitbuilder status, and if they've passed,\npushes them into the integration branch. The push will only succeed\nif the integration branch can be fast-forwarded to match the branch\nyou're trying to push; if you can't, it'll be rejected, which is what\nyou want, since merging (even conflict-free merging) might break\ntests.\n\nThat mechanism works pretty well at my company, with one exception: we\ndidn't bother with an automatic tool that merges into master. We\nprefer to have a release manager do that.\n\nHave fun,\n\nAvery\n"},{"id":"134369","messageId":"c41cd75d1002121007k4da9a617t161b699a3bca0fa7@mail.gmail.com","threadId":"22632","inReplyTo":"32541b131002120942w50a29e7cjf2c10820b3286017@mail.gmail.com","subject":"Re: Solve continuous integration (pending head / commit queue) problem using git","fromName":"Jan Koprowski","fromEmail":"jan.koprowski@gmail.com","sentAt":"2010-02-12T18:07:38Z","receivedAt":"2010-02-12T18:07:38Z","isPatch":false,"sender":{"key":"jan.koprowski@gmail.com","avatar":null},"body":"On Fri, Feb 12, 2010 at 6:42 PM, Avery Pennarun wrote:\n> On Fri, Feb 12, 2010 at 11:37 AM, Jan Koprowski wrote:\n>> Now. My idea. There is some revision tagged as \"stable\". *Clone* and\n>> *pull* operations is somehow \"overloaded\" from server side and always!\n>> return last revision tagged as stable. After compiling external tool\n>> just move tag to another revision which pass all tests. Of course\n>> there is some additional parameter (for example --last or --unstable)\n>> which can clone fine way of repository.\n>>\n>> Two questions.\n>> 1) Maybe I try to invent the wheel again. Is there any way to take the\n>> effect without overloading standard git behaviours.\n>> 2) If not how overload git behaviors on git \"server side\" repo?\n>\n> In general, code that lies to you about what's the most revision is\n> evil.  Sometimes you *do* want to fetch that revision it's lying to\n> you and saying doesn't exist, precisely because you'd like to help fix\n> it before integration.\n>\n> What you really want is:\n>\n> - nobody can push to the \"integration branch\" except the \"integration manager\"\n>\n> - the \"integration manager\" should be a computer program, so that you\n> can have \"continuous integration\"\n>\n> This isn't actually that hard.  Give each user their own repository;\n> no user can write to any other user's repository.  (This is the\n> default setup on github.com, for example.)  Alternatively, just tell\n> people to never, ever push to the master branch by themselves.  People\n> are easily capable of following rules like that unless they're\n> actively trying to screw you.\n>\n> Then set up something like gitbuilder\n> (http://github.com/apenwarr/gitbuilder) (Full disclosure: I wrote it)\n> to build *all* the branches from *all* the users.  This sounds like it\n> would create exponential work for the build machine, but it doesn't,\n> since most users will have mostly the same commits anyway.\n>\n> When gitbuilder tags a particular commit as having built and passed\n> all tests, then it becomes a candidate for merging into the\n> integration branch.  Write a little script that goes through candidate\n> branches, checks their gitbuilder status, and if they've passed,\n> pushes them into the integration branch.  The push will only succeed\n> if the integration branch can be fast-forwarded to match the branch\n> you're trying to push; if you can't, it'll be rejected, which is what\n> you want, since merging (even conflict-free merging) might break\n> tests.\n>\n> That mechanism works pretty well at my company, with one exception: we\n> didn't bother with an automatic tool that merges into master.  We\n> prefer to have a release manager do that.\n>\n> Have fun,\n>\n> Avery\n>\n\nProbably I don't have a problem (or it is a lateness). Because only\ntagging as stable and making two compile loops: one per management\nalways compiling stable tag and second compiling latest repo... And\nthat is all :D\n\n-- \n><> Jan Koprowski [696775174] GSM\n"},{"id":"134411","messageId":"c41cd75d1002122304i5aab52abt598e6a553defe1cf@mail.gmail.com","threadId":"22632","inReplyTo":"c41cd75d1002121007k4da9a617t161b699a3bca0fa7@mail.gmail.com","subject":"Re: Solve continuous integration (pending head / commit queue) problem using git","fromName":"Jan Koprowski","fromEmail":"jan.koprowski@gmail.com","sentAt":"2010-02-13T07:04:50Z","receivedAt":"2010-02-13T07:04:50Z","isPatch":false,"sender":{"key":"jan.koprowski@gmail.com","avatar":null},"body":"On Fri, Feb 12, 2010 at 7:07 PM, Jan Koprowski wrote:\n> On Fri, Feb 12, 2010 at 6:42 PM, Avery Pennarun wrote:\n>> On Fri, Feb 12, 2010 at 11:37 AM, Jan Koprowski wrote:\n>>> Now. My idea. There is some revision tagged as \"stable\". *Clone* and\n>>> *pull* operations is somehow \"overloaded\" from server side and always!\n>>> return last revision tagged as stable. After compiling external tool\n>>> just move tag to another revision which pass all tests. Of course\n>>> there is some additional parameter (for example --last or --unstable)\n>>> which can clone fine way of repository.\n>>>\n>>> Two questions.\n>>> 1) Maybe I try to invent the wheel again. Is there any way to take the\n>>> effect without overloading standard git behaviours.\n>>> 2) If not how overload git behaviors on git \"server side\" repo?\n>>\n>> In general, code that lies to you about what's the most revision is\n>> evil.  Sometimes you *do* want to fetch that revision it's lying to\n>> you and saying doesn't exist, precisely because you'd like to help fix\n>> it before integration.\n>>\n>> What you really want is:\n>>\n>> - nobody can push to the \"integration branch\" except the \"integration manager\"\n>>\n>> - the \"integration manager\" should be a computer program, so that you\n>> can have \"continuous integration\"\n>>\n>> This isn't actually that hard.  Give each user their own repository;\n>> no user can write to any other user's repository.  (This is the\n>> default setup on github.com, for example.)  Alternatively, just tell\n>> people to never, ever push to the master branch by themselves.  People\n>> are easily capable of following rules like that unless they're\n>> actively trying to screw you.\n>>\n>> Then set up something like gitbuilder\n>> (http://github.com/apenwarr/gitbuilder) (Full disclosure: I wrote it)\n>> to build *all* the branches from *all* the users.  This sounds like it\n>> would create exponential work for the build machine, but it doesn't,\n>> since most users will have mostly the same commits anyway.\n>>\n>> When gitbuilder tags a particular commit as having built and passed\n>> all tests, then it becomes a candidate for merging into the\n>> integration branch.  Write a little script that goes through candidate\n>> branches, checks their gitbuilder status, and if they've passed,\n>> pushes them into the integration branch.  The push will only succeed\n>> if the integration branch can be fast-forwarded to match the branch\n>> you're trying to push; if you can't, it'll be rejected, which is what\n>> you want, since merging (even conflict-free merging) might break\n>> tests.\n>>\n>> That mechanism works pretty well at my company, with one exception: we\n>> didn't bother with an automatic tool that merges into master.  We\n>> prefer to have a release manager do that.\n>>\n>> Have fun,\n>>\n>> Avery\n>>\n>\n> Probably I don't have a problem (or it is a lateness). Because only\n> tagging as stable and making two compile loops: one per management\n> always compiling stable tag and second compiling latest repo... And\n> that is all :D\n>\n> --\n>><> Jan Koprowski [696775174] GSM\n>\n\nHere is a thing :)\nAfter I install Hudson CI and equip them of Git plugin I saw two ways\nsupported by Hudson:\n1) Compile specific branch of code which suggest merging stable\nchanges to this branch\n2) Merging witch \"something\" before building and rollback changes\nafter which suggest some unstable \"branch\" or \"repository\" where all\nprogrammers commit and changes from are \"merged\" before build to the\nstable branch. I don't check details because I choose first option.\n\nSad thing there is no support for tagging in Git plugin.\n\n\n-- \n><> Jan Koprowski [696775174] GSM\n"},{"id":"134499","messageId":"alpine.LNX.2.00.1002131640200.14365@iabervon.org","threadId":"22632","inReplyTo":"32541b131002120942w50a29e7cjf2c10820b3286017@mail.gmail.com","subject":"Re: Solve continuous integration (pending head / commit queue) problem using git","fromName":"Daniel Barkalow","fromEmail":"barkalow@iabervon.org","sentAt":"2010-02-13T22:11:16Z","receivedAt":"2010-02-13T22:11:16Z","isPatch":false,"sender":{"key":"barkalow@iabervon.org","avatar":"https://avatars.githubusercontent.com/u/55364219?v=4"},"body":"On Fri, 12 Feb 2010, Avery Pennarun wrote:\n\n> On Fri, Feb 12, 2010 at 11:37 AM, Jan Koprowski wrote:\n> > Now. My idea. There is some revision tagged as \"stable\". *Clone* and\n> > *pull* operations is somehow \"overloaded\" from server side and always!\n> > return last revision tagged as stable. After compiling external tool\n> > just move tag to another revision which pass all tests. Of course\n> > there is some additional parameter (for example --last or --unstable)\n> > which can clone fine way of repository.\n> >\n> > Two questions.\n> > 1) Maybe I try to invent the wheel again. Is there any way to take the\n> > effect without overloading standard git behaviours.\n> > 2) If not how overload git behaviors on git \"server side\" repo?\n> \n> In general, code that lies to you about what's the most revision is\n> evil. Sometimes you *do* want to fetch that revision it's lying to\n> you and saying doesn't exist, precisely because you'd like to help fix\n> it before integration.\n\nI think a more suitable detail here would be to have the remote system \nrespond to pushes by stating that it's taking your push request under \nadvisement, but cannot give an immediate verdict for that request (and it \nmay want to let you know that it's updated a different ref of its choice \nthat you didn't intentionally request).\n\n$ git push\n f99642a..e70de97 HEAD -> master (proposed, not updated)\n\n$ git log --oneline origin/master\nf99642a Original commit\n\n(wait for external signal, like getting a confirmation email)\n\n$ git fetch\n f99642a..e70de97 maaster -> origin/master\n\n$ git log --oneline origin/master\nf99642a Your commit\n\nI think the only thing that would be needed would be a way for the remote \nserver to report that it's not updating the ref, but it is planning to act \non your request, so that your local git can give a non-error without \nupdating the remote branch inappropriately. (Presumably, the server would \nhave used a pre-update hook to give this response, which would have \nenqueued the request in the CI system; when the CI system likes a change, \nit can push and the hook would detect that it's actually the CI system and \nlet the update happen).\n\n\t-Daniel\n*This .sig left intentionally blank*\n"}]}