{"thread":{"id":"18154","subject":"Help designing work flow","startedAt":"2009-03-05T20:17:36Z","lastAt":"2009-03-26T15:38:28Z","messageCount":8,"participants":["John Dlugosz","Andreas Ericsson","John Tapsell"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"107094","messageId":"450196A1AAAE4B42A00A8B27A59278E709F07D30@EXCHANGE.trad.tradestation.com","threadId":"18154","inReplyTo":null,"subject":"Help designing work flow","fromName":"John Dlugosz","fromEmail":"jdlugosz@tradestation.com","sentAt":"2009-03-05T20:17:36Z","receivedAt":"2009-03-05T20:17:36Z","isPatch":false,"sender":{"key":"jdlugosz@tradestation.com","avatar":null},"body":"I know we (my group) should use \"topic\" branches and apply them back to\nthe dev branch when done.  There is concern that the commit history gets\ntoo full of detailed stuff, especially with several developers, that you\ncan't tell what \"really changed\".  So, our dev branch should appear to\ncontain only commit nodes representing completed assignments; not every\nday's checkpoint and trying to keep one's own stuff on top to avoid\nmerging later.\n\nI guess that's how it is on these Open Source projects where patches are\nsubmitted by email and applied by the maintainer.  We don't see the\ndetails, just the final patch.  But, my situation will be developers\ngathered around an in-house master repo, and everyone should be able to\npush their own changes as assignments are completed.\n\nWhat is the best procedure to achieve that?  Or what are some good\nalternatives with trade-offs?\n\nI see that if a topic branch is merged (disabling FF if applicable), the\nmain line (leftmost parent) will be a history of completed topics.  But,\nwe don't need to keep the detailed side-branches forever, and even if\ngitk and other visualization  tools can be told to just show the main\nline, advanced use such as git log this..that will forever be packed\nwith the micro-details.\n\nSo, unless someone has more input along that line, I'm assuming that we\nwant to apply the completed topic as a single-parent commit.  That is\nthe natural result if preparing patches and then applying them, but is\nthere a simpler, direct way to do that in git?\n\nThe detailed topic branches can be kept around for a while, for the\noriginal author to extend if it needs to be returned to, and to examine\nif the gestalt change in the single commit is too overwhelming to\nunderstand, or to help figure out what might have broken.  But after a\nwhile they can be deleted and then gc will free up the disk space.\n\nAnything else I should look into?\n\n--John\n\n\nTradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.\n  If you received this in error, please contact the sender and delete the material from any computer.\n"},{"id":"107448","messageId":"49B4F5A9.5060304@op5.se","threadId":"18154","inReplyTo":"450196A1AAAE4B42A00A8B27A59278E709F07D30@EXCHANGE.trad.tradestation.com","subject":"Re: Help designing work flow","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-03-09T10:55:37Z","receivedAt":"2009-03-09T10:55:37Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"John Dlugosz wrote:\n> I know we (my group) should use \"topic\" branches and apply them back to\n> the dev branch when done.  There is concern that the commit history gets\n> too full of detailed stuff, especially with several developers, that you\n> can't tell what \"really changed\".  So, our dev branch should appear to\n> contain only commit nodes representing completed assignments; not every\n> day's checkpoint and trying to keep one's own stuff on top to avoid\n> merging later.\n> \n> I guess that's how it is on these Open Source projects where patches are\n> submitted by email and applied by the maintainer.  We don't see the\n> details, just the final patch.  But, my situation will be developers\n> gathered around an in-house master repo, and everyone should be able to\n> push their own changes as assignments are completed.\n> \n> What is the best procedure to achieve that?  Or what are some good\n> alternatives with trade-offs?\n> \n\nUse topic-branches and let someone merge them into master after having\nverified that they work properly.\n\nWe usually commit simple bugfixes directly to master and then have\ndevelopers rebase their changes onto master when they're ready to\nintegrate it. They push their development branches though, and further\nchanges to the topic are done on the topic-branches which can get\nmerged to master (or stable) many times. Once a topic is merged, we\nalways \"git commit --amend\" the merge commit to write a proper\ncommit-message for it, adding a link to our bug- and feature-tracker\nso we get the at-a-glance information quickly and can dig up the\nentire discussion history around the bug/feature later. Each topic\nshould be complete with documentation and test-cases before it's\nmerged.\n\n\n> I see that if a topic branch is merged (disabling FF if applicable), the\n> main line (leftmost parent) will be a history of completed topics.  But,\n> we don't need to keep the detailed side-branches forever, and even if\n> gitk and other visualization  tools can be told to just show the main\n> line, advanced use such as git log this..that will forever be packed\n> with the micro-details.\n> \n\nYou can tell \"git log\" to only show one line of history too, but besides\nthat, micro-details are good. You definitely want to be able to search\nthe micro-details when things go awry (and they will), so you see exactly\nwhy some particular algorithm changed later.\n\n> So, unless someone has more input along that line, I'm assuming that we\n> want to apply the completed topic as a single-parent commit.  That is\n> the natural result if preparing patches and then applying them, but is\n> there a simpler, direct way to do that in git?\n> \n\nYou do not want to do that. We did it for a while, and it was hell when\nwe found out that one of them broke down. The really, really *nice* thing\nabout git is called \"git bisect\". What makes it so awesomely nice is\nthat, instead of looking at a 100k diff-file knowing that somewhere in\nthere a bug was introduced, you get (with good discipline using small\ncommits), a 1-40 line patch with a clear and concise message of why\nthose changes were thought necessary at that time. Applying a topic\nbranch as a single patch would rob you of that functionality, and you\nwill regret it. Trust me on this.\n\n> The detailed topic branches can be kept around for a while, for the\n> original author to extend if it needs to be returned to, and to examine\n> if the gestalt change in the single commit is too overwhelming to\n> understand, or to help figure out what might have broken.  But after a\n> while they can be deleted and then gc will free up the disk space.\n> \n\nBut if they do need to be returned to, you cannot merge them again if\nyou've already applied the topic-patch (ugh), since you'd get conflicts\nif any of the sections touched by the patch have been changed since.\n\nWe use topic-branches quite a lot. When we're done with them we delete\nthe branch-pointers but I wouldn't, ever, dream of re-cooking them as\nmega-patches when applying them to master.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"107451","messageId":"43d8ce650903090444n352f310fs9cd4b8b0184be010@mail.gmail.com","threadId":"18154","inReplyTo":"49B4F5A9.5060304@op5.se","subject":"Re: Help designing work flow","fromName":"John Tapsell","fromEmail":"johnflux@gmail.com","sentAt":"2009-03-09T11:44:47Z","receivedAt":"2009-03-09T11:44:47Z","isPatch":false,"sender":{"key":"johnflux@gmail.com","avatar":"https://gravatar.com/avatar/25f70d4c0f96396b84a2e34bcd9bdc233462c7b4be29b5fdca8266fc53f30b0c?d=mp&s=160"},"body":"2009/3/9 Andreas Ericsson <ae@op5.se>:\n> John Dlugosz wrote:\n>>\n>> I know we (my group) should use \"topic\" branches and apply them back to\n>> the dev branch when done.  There is concern that the commit history gets\n>> too full of detailed stuff, especially with several developers, that you\n>> can't tell what \"really changed\".  So, our dev branch should appear to\n>> contain only commit nodes representing completed assignments; not every\n>> day's checkpoint and trying to keep one's own stuff on top to avoid\n>> merging later.\n>>\n>> I guess that's how it is on these Open Source projects where patches are\n>> submitted by email and applied by the maintainer.  We don't see the\n>> details, just the final patch.  But, my situation will be developers\n>> gathered around an in-house master repo, and everyone should be able to\n>> push their own changes as assignments are completed.\n>>\n>> What is the best procedure to achieve that?  Or what are some good\n>> alternatives with trade-offs?\n>>\n>\n> Use topic-branches and let someone merge them into master after having\n> verified that they work properly.\n>\n> We usually commit simple bugfixes directly to master and then have\n> developers rebase their changes onto master when they're ready to\n\nThe trouble with rebasing is that it then you end up with lots of\npatches that haven't been tested.  You can end up with lots of\nuncompiling commits.\n\nAlthough merging is no better either.  Then you end up with one single\ncommit that tries to merge two trees, making it almost impossible to\ntrack down bugs that resulted from the merge.\n\n> integrate it. They push their development branches though, and further\n> changes to the topic are done on the topic-branches which can get\n> merged to master (or stable) many times. Once a topic is merged, we\n> always \"git commit --amend\" the merge commit to write a proper\n> commit-message for it, adding a link to our bug- and feature-tracker\n> so we get the at-a-glance information quickly and can dig up the\n> entire discussion history around the bug/feature later. Each topic\n> should be complete with documentation and test-cases before it's\n> merged.\n>\n>\n>> I see that if a topic branch is merged (disabling FF if applicable), the\n>> main line (leftmost parent) will be a history of completed topics.  But,\n>> we don't need to keep the detailed side-branches forever, and even if\n>> gitk and other visualization  tools can be told to just show the main\n>> line, advanced use such as git log this..that will forever be packed\n>> with the micro-details.\n>>\n>\n> You can tell \"git log\" to only show one line of history too, but besides\n> that, micro-details are good. You definitely want to be able to search\n> the micro-details when things go awry (and they will), so you see exactly\n> why some particular algorithm changed later.\n>\n>> So, unless someone has more input along that line, I'm assuming that we\n>> want to apply the completed topic as a single-parent commit.  That is\n>> the natural result if preparing patches and then applying them, but is\n>> there a simpler, direct way to do that in git?\n>>\n>\n> You do not want to do that. We did it for a while, and it was hell when\n> we found out that one of them broke down. The really, really *nice* thing\n> about git is called \"git bisect\". What makes it so awesomely nice is\n> that, instead of looking at a 100k diff-file knowing that somewhere in\n> there a bug was introduced, you get (with good discipline using small\n> commits), a 1-40 line patch with a clear and concise message of why\n> those changes were thought necessary at that time. Applying a topic\n> branch as a single patch would rob you of that functionality, and you\n> will regret it. Trust me on this.\n>\n>> The detailed topic branches can be kept around for a while, for the\n>> original author to extend if it needs to be returned to, and to examine\n>> if the gestalt change in the single commit is too overwhelming to\n>> understand, or to help figure out what might have broken.  But after a\n>> while they can be deleted and then gc will free up the disk space.\n>>\n>\n> But if they do need to be returned to, you cannot merge them again if\n> you've already applied the topic-patch (ugh), since you'd get conflicts\n> if any of the sections touched by the patch have been changed since.\n>\n> We use topic-branches quite a lot. When we're done with them we delete\n> the branch-pointers but I wouldn't, ever, dream of re-cooking them as\n> mega-patches when applying them to master.\n>\n> --\n> Andreas Ericsson                   andreas.ericsson@op5.se\n> OP5 AB                             www.op5.se\n> Tel: +46 8-230225                  Fax: +46 8-230231\n>\n> Considering the successes of the wars on alcohol, poverty, drugs and\n> terror, I think we should give some serious thought to declaring war\n> on peace.\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n"},{"id":"107453","messageId":"49B50B3C.50700@op5.se","threadId":"18154","inReplyTo":"43d8ce650903090444n352f310fs9cd4b8b0184be010@mail.gmail.com","subject":"Re: Help designing work flow","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-03-09T12:27:40Z","receivedAt":"2009-03-09T12:27:40Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"John Tapsell wrote:\n> 2009/3/9 Andreas Ericsson <ae@op5.se>:\n>> John Dlugosz wrote:\n>>> I know we (my group) should use \"topic\" branches and apply them back to\n>>> the dev branch when done.  There is concern that the commit history gets\n>>> too full of detailed stuff, especially with several developers, that you\n>>> can't tell what \"really changed\".  So, our dev branch should appear to\n>>> contain only commit nodes representing completed assignments; not every\n>>> day's checkpoint and trying to keep one's own stuff on top to avoid\n>>> merging later.\n>>>\n>>> I guess that's how it is on these Open Source projects where patches are\n>>> submitted by email and applied by the maintainer.  We don't see the\n>>> details, just the final patch.  But, my situation will be developers\n>>> gathered around an in-house master repo, and everyone should be able to\n>>> push their own changes as assignments are completed.\n>>>\n>>> What is the best procedure to achieve that?  Or what are some good\n>>> alternatives with trade-offs?\n>>>\n>> Use topic-branches and let someone merge them into master after having\n>> verified that they work properly.\n>>\n>> We usually commit simple bugfixes directly to master and then have\n>> developers rebase their changes onto master when they're ready to\n> \n> The trouble with rebasing is that it then you end up with lots of\n> patches that haven't been tested.  You can end up with lots of\n> uncompiling commits.\n> \n\nNot really, no. Unit-tests can still run just fine, and integration\ntesting still needs to be done after each feature is completed.\n\n> Although merging is no better either.  Then you end up with one single\n> commit that tries to merge two trees, making it almost impossible to\n> track down bugs that resulted from the merge.\n> \n\nNot really. If bugs are in \"unrelated\" areas (if the topic changed some\nAPI without changing its' other callers, fe), you can backstep between\neach commit on the merged branch, remerge that commit (instead of the\ntip) and then run the tests again. But really, such bugs should have\nbeen detected prior to merging the branch, and in any case \"git bisect\"\nwill find the commit that introduced the bug for you either way.\n\n\nFor next time, please cut away those parts of the email you didn't\nreply to. I had to scroll down to the bottom to make sure you hadn't\nwritten more.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"107455","messageId":"43d8ce650903090622j51b73801gb4be62a7f50029a4@mail.gmail.com","threadId":"18154","inReplyTo":"49B50B3C.50700@op5.se","subject":"Re: Help designing work flow","fromName":"John Tapsell","fromEmail":"johnflux@gmail.com","sentAt":"2009-03-09T13:22:49Z","receivedAt":"2009-03-09T13:22:49Z","isPatch":false,"sender":{"key":"johnflux@gmail.com","avatar":"https://gravatar.com/avatar/25f70d4c0f96396b84a2e34bcd9bdc233462c7b4be29b5fdca8266fc53f30b0c?d=mp&s=160"},"body":"2009/3/9 Andreas Ericsson <ae@op5.se>:\n> John Tapsell wrote:\n\n> Not really. If bugs are in \"unrelated\" areas (if the topic changed some\n> API without changing its' other callers, fe), you can backstep between\n> each commit on the merged branch, remerge that commit (instead of the\n> tip) and then run the tests again.\n\nDoing that manually?  Sounds really complicated, especially if the\nmerge is nontrivial.\n\n> But really, such bugs should have\n> been detected prior to merging the branch, and in any case \"git bisect\"\n> will find the commit that introduced the bug for you either way.\n\nHow will you detect bugs that arise from merging two trees, before you\nmerge them?\n\n> For next time, please cut away those parts of the email you didn't\n> reply to. I had to scroll down to the bottom to make sure you hadn't\n> written more.\n\nI'll try sorry.  Most email clients will hide the quoted text - there\nmight be an option for that in your client?\n\nJohn\n"},{"id":"107456","messageId":"49B51994.1040200@op5.se","threadId":"18154","inReplyTo":"43d8ce650903090622j51b73801gb4be62a7f50029a4@mail.gmail.com","subject":"Re: Help designing work flow","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-03-09T13:28:52Z","receivedAt":"2009-03-09T13:28:52Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"John Tapsell wrote:\n> 2009/3/9 Andreas Ericsson <ae@op5.se>:\n>> John Tapsell wrote:\n> \n>> Not really. If bugs are in \"unrelated\" areas (if the topic changed some\n>> API without changing its' other callers, fe), you can backstep between\n>> each commit on the merged branch, remerge that commit (instead of the\n>> tip) and then run the tests again.\n> \n> Doing that manually?  Sounds really complicated, especially if the\n> merge is nontrivial.\n> \n\nWell, using git bisect should work.\n\n>> But really, such bugs should have\n>> been detected prior to merging the branch, and in any case \"git bisect\"\n>> will find the commit that introduced the bug for you either way.\n> \n> How will you detect bugs that arise from merging two trees, before you\n> merge them?\n> \n\nAssuming you don't start each topic-branch from a completely empty tree\nand that you have *some* tests, you can run those tests at every single\ncommit just as if you had done those on the 'master' branch. API changes\nthat break things should break earlier, unless another branch made those\nchanges.\n\nAside from that, it's quite simple to run tests before publishing the\nresults of the merge, and it's absolutely trivial to undo the merge\nin case tests don't pass.\n\n>> For next time, please cut away those parts of the email you didn't\n>> reply to. I had to scroll down to the bottom to make sure you hadn't\n>> written more.\n> \n> I'll try sorry.  Most email clients will hide the quoted text - there\n> might be an option for that in your client?\n> \n\nI'm sure there is, but then I wouldn't have seen what you were replying\nto, so that's not a very good option. In the space between me sending\nand you replying, I wrote six other emails. Keeping track of them all\nin my head is not something I'm prepared to even consider trying.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"107466","messageId":"450196A1AAAE4B42A00A8B27A59278E70A116188@EXCHANGE.trad.tradestation.com","threadId":"18154","inReplyTo":"49B4F5A9.5060304@op5.se","subject":"RE: Help designing work flow","fromName":"John Dlugosz","fromEmail":"jdlugosz@tradestation.com","sentAt":"2009-03-09T15:36:12Z","receivedAt":"2009-03-09T15:36:12Z","isPatch":false,"sender":{"key":"jdlugosz@tradestation.com","avatar":null},"body":"Thank you very much for your thoughts, especially the specific commands\netc. that I can show the team as reasons.\n\n=== Re: ===\nYou can tell \"git log\" to only show one line of history too, but besides\nthat, micro-details are good. You definitely want to be able to search\nthe micro-details when things go awry (and they will), so you see\nexactly\nwhy some particular algorithm changed later.\n=== end ===\n\nIs there a way that someone (for example, product manager) can view only\nthe main line consisting of topic merges, and not see each one separated\nby many many lines of detail changes?  Especially using the GUI?\n\n--John\n(please excuse the footer; it's not my idea)\n\nTradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.\n  If you received this in error, please contact the sender and delete the material from any computer.\n"},{"id":"109544","messageId":"450196A1AAAE4B42A00A8B27A59278E70A55A05D@EXCHANGE.trad.tradestation.com","threadId":"18154","inReplyTo":"49B4F5A9.5060304@op5.se","subject":"RE: Help designing work flow","fromName":"John Dlugosz","fromEmail":"jdlugosz@tradestation.com","sentAt":"2009-03-26T15:38:28Z","receivedAt":"2009-03-26T15:38:28Z","isPatch":false,"sender":{"key":"jdlugosz@tradestation.com","avatar":null},"body":"> You can tell \"git log\" to only show one line of history too, but\n> besides\n> that, micro-details are good. You definitely want to be able to search\n> the micro-details when things go awry (and they will), so you see\n> exactly\n> why some particular algorithm changed later.\n> \n\nI misread that the first time.  I thought you meant that you can tell\ngit log to follow down the left parents only.\n\nSo, how would you do that?  List the completed merged topic nodes only,\nnot the detailed nodes that make it up?\n\n--John\n\nTradeStation Group, Inc. is a publicly-traded holding company (NASDAQ GS: TRAD) of three operating subsidiaries, TradeStation Securities, Inc. (Member NYSE, FINRA, SIPC and NFA), TradeStation Technologies, Inc., a trading software and subscription company, and TradeStation Europe Limited, a United Kingdom, FSA-authorized introducing brokerage firm. None of these companies provides trading or investment advice, recommendations or endorsements of any kind. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited.\n  If you received this in error, please contact the sender and delete the material from any computer.\n"}]}