{"thread":{"id":"19033","subject":"Git workflow: how to achieve?","startedAt":"2009-04-24T04:58:43Z","lastAt":"2009-04-28T15:59:56Z","messageCount":8,"participants":["George Spelvin","Uwe Kleine-König","Andreas Ericsson","Sitaram Chamarty"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"112157","messageId":"20090424045843.7674.qmail@science.horizon.com","threadId":"19033","inReplyTo":null,"subject":"Git workflow: how to achieve?","fromName":"George Spelvin","fromEmail":"linux@horizon.com","sentAt":"2009-04-24T04:58:43Z","receivedAt":"2009-04-24T04:58:43Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":"Here's something I and my co-workers would like to achieve, but are not\ntoo sure how to arrange.\n\nI want to be committing to a feature branch, but always be compiling\nand testing a merge of that branch and several others.  (Kind of like\nlinux-next.)  I want to be able to switch feature branches easily.\n\nFor example, I may have a background project that I'm working on slowly in\nbetween short-term fixes.  Or I want to be running the latest kernel.org\nkernel while my patches await approval.\n\nIf it's just my own projects, I can just commit in random order and\nstraighten things out later.  Although even that is problematic,\nas I may not remember what line of development a cleanup patch is a\nprerequisite for.  (This is something that darcs is apparently good at.)\n\nBut when I want to be testing something highly volatile like linux-next,\nand ensuring that my work continues to merge with it cleanly, as well\nas helping others with their branches, it becomes a daily pain.\n\nThe best attempt I have so far is to rebase a lot.  But that means that\nI can't do any merging in my development branch lest the rebasing turn\ninto a mess.  And forcing everything to be linear makes changing branches\na pain.  And I can't share half-finished versions with co-workers.\n\nThis is all vaguely quilt-like, although I'd rather not worry about the\norder of patches.  I suppose I'd like git to let me \"commit under\" the\nfinal merge.  When I switch branches, git should reorganize the tree of\nmerges so that the current branch is only one merge from the HEAD.\n\n(Another thing I've wished for that might be related is for a branch\nto have a notion of its origin.  So I can rebase it onto an arbitrary\nplace in the commit tree without having to explicitly specify an origin.)\n\n((Another really simple feature I keep wanting is \"git commit -b\n<newbranch>\".  I should probably try to write a patch...))\n\n\nAnyway, my feature ideas might be unworkable, and in any case, they'll\ntake a long time to implement.  Is there some easier way to achieve more\nor less this effect?\n\nMaybe the planned git-rebase improvements to handle merges better will\nfix this, so I can just commit on top and periodically rebase the changes\nunder the head manually without too much pain?  (git rebase -i -p does\nappear to be working better than I remember.)\n\nH'm... in fact, it might be as easy as replacing \"git pull\" with\ngit rebase -p -i <last merge>^\n(Delete the merge in the editor)\ngit pull <remote>\n\nAnnoying to remember, but not TOO bad.\n"},{"id":"112165","messageId":"20090424073122.GA9603@pengutronix.de","threadId":"19033","inReplyTo":"20090424045843.7674.qmail@science.horizon.com","subject":"Re: Git workflow: how to achieve?","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2009-04-24T07:31:22Z","receivedAt":"2009-04-24T07:31:22Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"On Fri, Apr 24, 2009 at 12:58:43AM -0400, George Spelvin wrote:\n> Here's something I and my co-workers would like to achieve, but are not\n> too sure how to arrange.\n> \n> I want to be committing to a feature branch, but always be compiling\n> and testing a merge of that branch and several others.  (Kind of like\n> linux-next.)  I want to be able to switch feature branches easily.\n> \n> For example, I may have a background project that I'm working on slowly in\n> between short-term fixes.  Or I want to be running the latest kernel.org\n> kernel while my patches await approval.\n> \n> If it's just my own projects, I can just commit in random order and\n> straighten things out later.  Although even that is problematic,\n> as I may not remember what line of development a cleanup patch is a\n> prerequisite for.  (This is something that darcs is apparently good at.)\n> \n> But when I want to be testing something highly volatile like linux-next,\n> and ensuring that my work continues to merge with it cleanly, as well\n> as helping others with their branches, it becomes a daily pain.\n> \n> The best attempt I have so far is to rebase a lot.  But that means that\n> I can't do any merging in my development branch lest the rebasing turn\n> into a mess.  And forcing everything to be linear makes changing branches\n> a pain.  And I can't share half-finished versions with co-workers.\nIf you develop a topic, say gs/fix_bug_12345, optimally you should base\nit on the commit introducing the bug.  Then you can have several\nintegrating branches, say\n\n\tnext/gs/fix_bug_12345\n\tlinus/gs/fix_bug_12345\n\t$company_stable/gs/fix_bug_12345\n\nand in these branches you can regularly merge in the current state of\nyour gs/fix_bug_12345 branch.  You might want to take a look on topgit,\nwhich can automate the propagation for you.\n\nAnd/or you might want to look into the linux-tip scripts by Ingo\nMolnar[1].\n \n> ((Another really simple feature I keep wanting is \"git commit -b\n> <newbranch>\".  I should probably try to write a patch...))\nWell currently this can be done in two commands:\n\n\tgit checkout -b <newbranch>; git commit\n\nAFAIK it even works if you already modified the index.\n\nBest regards\nUwe\n\n[1] git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git tip:.tip\n\n-- \nPengutronix e.K.                              | Uwe Kleine-König            |\nIndustrial Linux Solutions                    | http://www.pengutronix.de/  |\n"},{"id":"112166","messageId":"49F16BA5.9060003@op5.se","threadId":"19033","inReplyTo":"20090424045843.7674.qmail@science.horizon.com","subject":"Re: Git workflow: how to achieve?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-04-24T07:35:01Z","receivedAt":"2009-04-24T07:35:01Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"George Spelvin wrote:\n> Here's something I and my co-workers would like to achieve, but are not\n> too sure how to arrange.\n> \n> I want to be committing to a feature branch, but always be compiling\n> and testing a merge of that branch and several others.\n\nAt $dayjob we do a \"build-on-push\" thing, so every time a developer\nfeels his work is worthy of trying out for others, he ships it to the\nmother-ship repository, where the update hook (should be post-receive,\nbut we haven't bothered changing it since it Just Works as is) kicks\nthe buildbot, triggering builds on all architectures and platforms\nwe support.\n\nAssuming a static set of branches to merge, or a prefix or suffix\nthat marks the branches to merge, it wouldn't be difficult to let it\ntrigger \"merge this and that\" stuff as well.\n\n\n>  I want to be able to switch feature branches easily.\n> \n\nWell, git makes that absolutely trivial. No other vcs has as cheap\nbranching as git does.\n\n> For example, I may have a background project that I'm working on slowly in\n> between short-term fixes.  Or I want to be running the latest kernel.org\n> kernel while my patches await approval.\n> \n> If it's just my own projects, I can just commit in random order and\n> straighten things out later.  Although even that is problematic,\n> as I may not remember what line of development a cleanup patch is a\n> prerequisite for.  (This is something that darcs is apparently good at.)\n> \n\nSounds like you'd benefit somewhat from using TopGit or some other\npatch-queue management tool. I haven't used them myself, but I believe\nthis is the kind of problem they set out to solve.\n\n> But when I want to be testing something highly volatile like linux-next,\n> and ensuring that my work continues to merge with it cleanly, as well\n> as helping others with their branches, it becomes a daily pain.\n> \n\nWhy? Just merge it to make sure it merges, and then throw away the result.\nThere's a post made by Linus somewhere on how he would like people to do\nmerges (ie, which to keep and which to throw away).\n\n> The best attempt I have so far is to rebase a lot.  But that means that\n> I can't do any merging in my development branch lest the rebasing turn\n> into a mess.\n\nMerging other branches into your development branch and keeping the result\nis often a bad idea, since you can't later merge the development branch\nwithout also merging everything that *you* merged. This makes it nearly\nimpossible to pick your feature into random spots in the DAG. If you ever\nwork anywhere but on the bleeding edge, you'll find merging the \"main dev\nbranch\" is a *really* bad idea.\n\n>  And forcing everything to be linear makes changing branches\n> a pain.\n\nEverything needn't be linear, but if you can manage to keep each topic\nlinear that makes your series a lot easier to review.\n\n>  And I can't share half-finished versions with co-workers.\n> \n\nSo finish it before you push. Anyone fetching from your local repository\nshould be aware that not all branches point to something sensible. If\nthey don't, they'll be sure to ask you so you can bring out the LART.\n\n> This is all vaguely quilt-like, although I'd rather not worry about the\n> order of patches.  I suppose I'd like git to let me \"commit under\" the\n> final merge.  When I switch branches, git should reorganize the tree of\n> merges so that the current branch is only one merge from the HEAD.\n> \n\nUmm... Use a separate branch to do the merge, and enable rerere caching\nso your recorded resolutions get reused when you do the merge \"for real\".\n\n> (Another thing I've wished for that might be related is for a branch\n> to have a notion of its origin.  So I can rebase it onto an arbitrary\n> place in the commit tree without having to explicitly specify an origin.)\n> \n\nThis is a bad idea.\n\n> ((Another really simple feature I keep wanting is \"git commit -b\n> <newbranch>\".  I should probably try to write a patch...))\n> \n\nThis is a good idea (assuming you mean \"commit this state to that branch\",\nand it should be fairly trivial to implement using something like this:\n\n    git stash && git checkout -b newbranch && git stash pop && git commit\n\n> \n> Anyway, my feature ideas might be unworkable, and in any case, they'll\n> take a long time to implement.  Is there some easier way to achieve more\n> or less this effect?\n> \n\nI'm not sure what effect you're after. If you can write down, with very\n*little* detail, what it is you want to achieve, it will be a lot easier\nto help you than if you ask about small steps in a pre-thought-out solution\nthat may not even be right from the beginning.\n\n> Maybe the planned git-rebase improvements to handle merges better will\n> fix this, so I can just commit on top and periodically rebase the changes\n> under the head manually without too much pain?  (git rebase -i -p does\n> appear to be working better than I remember.)\n> \n\nWhat planned rebase improvements are you talking about?\n\n> H'm... in fact, it might be as easy as replacing \"git pull\" with\n> git rebase -p -i <last merge>^\n> (Delete the merge in the editor)\n> git pull <remote>\n> \n> Annoying to remember, but not TOO bad.\n\nUse an alias, or a wrapper script.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"112472","messageId":"20090427225900.29793.qmail@science.horizon.com","threadId":"19033","inReplyTo":"49F16BA5.9060003@op5.se","subject":"Re: Git workflow: how to achieve?","fromName":"George Spelvin","fromEmail":"linux@horizon.com","sentAt":"2009-04-27T22:59:00Z","receivedAt":"2009-04-27T22:59:00Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":"Andreas Ericsson <ae@op5.se> wrote:\n>>  I want to be able to switch feature branches easily.\n> \n> Well, git makes that absolutely trivial. No other vcs has as cheap\n> branching as git does.\n\nYes, I'm quite aware, but I mean while keeping the same merged result.\nThat is, I'm compiling and testing base+feature_a+feature_b+feature_c,\nbut I want to be able to change which feature I'm committing to.\n\nThis comes up if I arrange some merges so that the \"current branch\" is\none below the top merge:\n\no--o--o--o--o--o--o <-- HEAD\n           /  /  /\n    ...c--c  /  /\n      ...b--b  /\n        ...a--a\n\nAnd then want to switch to branch c.\n\n> Sounds like you'd benefit somewhat from using TopGit or some other\n> patch-queue management tool. I haven't used them myself, but I believe\n> this is the kind of problem they set out to solve.\n\nThanks, I didn't know about TopGit; I'm looking at it now.\n\n>> But when I want to be testing something highly volatile like linux-next,\n>> and ensuring that my work continues to merge with it cleanly, as well\n>> as helping others with their branches, it becomes a daily pain.\n> \n> Why? Just merge it to make sure it merges, and then throw away the result.\n> There's a post made by Linus somewhere on how he would like people to do\n> merges (ie, which to keep and which to throw away).\n\nI don't want to just merge it, I want to *compile it* and *run it*.\nBecause the whole point of using linux-next is to test it.\nAnd when it breaks, I want access to the relevant source code.\n\n>> The best attempt I have so far is to rebase a lot.  But that means that\n>> I can't do any merging in my development branch lest the rebasing turn\n>> into a mess.\n> \n> Merging other branches into your development branch and keeping the result\n> is often a bad idea, since you can't later merge the development branch\n> without also merging everything that *you* merged. This makes it nearly\n> impossible to pick your feature into random spots in the DAG. If you ever\n> work anywhere but on the bleeding edge, you'll find merging the \"main dev\n> branch\" is a *really* bad idea.\n\nI didn't mean merging main into my dev branch, but merging within my\ndev branch.  As in, \"that refactorization of the foo_* helpers is a\nbetter base for development, so I'll merge that into my bar branch\", and\nthen test-merge THAT into main.\n\n>>  And forcing everything to be linear makes changing branches\n>> a pain.\n> \n> Everything needn't be linear, but if you can manage to keep each topic\n> linear that makes your series a lot easier to review.\n\nYes, I know.  I generally try, but it's occasionally nice to have sub-branches\nwithin a feature branch.\n\n>>  And I can't share half-finished versions with co-workers.\n> \n> So finish it before you push. Anyone fetching from your local repository\n> should be aware that not all branches point to something sensible. If\n> they don't, they'll be sure to ask you so you can bring out the LART.\n> \n>> This is all vaguely quilt-like, although I'd rather not worry about the\n>> order of patches.  I suppose I'd like git to let me \"commit under\" the\n>> final merge.  When I switch branches, git should reorganize the tree of\n>> merges so that the current branch is only one merge from the HEAD.\n> \n> Umm... Use a separate branch to do the merge, and enable rerere caching\n> so your recorded resolutions get reused when you do the merge \"for real\".\n\n>> (Another thing I've wished for that might be related is for a branch\n>> to have a notion of its origin.  So I can rebase it onto an arbitrary\n>> place in the commit tree without having to explicitly specify an origin.)\n> \n> This is a bad idea.\n\nCan you explain why?  When I talk about a \"feature branch\", I mean a chain\nof commits that introduces a feature.  Why would it be bad for a reference\nto have an optional \"origin\" attribute that indicates a start position?\n\n>> ((Another really simple feature I keep wanting is \"git commit -b\n>> <newbranch>\".  I should probably try to write a patch...))\n> \n> This is a good idea (assuming you mean \"commit this state to that branch\",\n> and it should be fairly trivial to implement using something like this:\n> \n>     git stash && git checkout -b newbranch && git stash pop && git commit\n\nActually, just \ngit checkout -b newbranch; git commit \"$@\"\ndoes the job quite nicely, but \n\n>> Anyway, my feature ideas might be unworkable, and in any case, they'll\n>> take a long time to implement.  Is there some easier way to achieve more\n>> or less this effect?\n> \n> I'm not sure what effect you're after. If you can write down, with very\n> *little* detail, what it is you want to achieve, it will be a lot easier\n> to help you than if you ask about small steps in a pre-thought-out solution\n> that may not even be right from the beginning.\n\nWe may have a language problem.  \"very little detail\" means almost no detail,\nan absence of details.  Did you mean \"every little detail\"?\n\nIt actually came to mind recently during $DAY_JOB work, but let me give a\nconcrete example based on the Linux kernel:\n\nI am running a customized Linux kernel.  On top of Linus's base,\nI have local patches to enable 64-bit DMA on the SB600 SATA controller,\nsome local patches to make the RAID (md) code print out the locations\nof mismatches when verifying, the EDAC quilt series, a merge of the\nLinuxPPS code, a number of local patches to the LinuxPPS code (that\nI'm discussing with Rodolfo Giometti), and some revisions to the serial\ninterrupt handler (that I'm discussing with Alan Cox).\n\nThere's also the beginning of an ethernet driver that I'm trying to\nwrite, but it's going slow.\n\nEvery week or two, I rebase all that on top of Linus's latest.  Plain\nrebase doesn't like the LinuxPPS merge, so I've been doing it manually\nin two parts.  Although rebase -p is apprently working much better than\nI remember.\n\nI would also like to be able to revise each of those features\nindependently, based on feedback from the relevant upstream maintainer.\nI also don't know in what order they'll be acecpted upstream (if ever).\nRegardless of that, I want to my running kernels to have all of the\npatches applied.\n\nCurrently, some patches get deeply buried in the stack and I have to\ndo a lot of deep rebasing.\n\n>> Maybe the planned git-rebase improvements to handle merges better will\n>> fix this, so I can just commit on top and periodically rebase the changes\n>> under the head manually without too much pain?  (git rebase -i -p does\n>> appear to be working better than I remember.)\n> \n> What planned rebase improvements are you talking about?\n\nI was referring to Dscho's \"Heads up: major rebase -i -p rework coming up\"\nhttp://marc.info/?l=git&m=123282871626215\n\n>> H'm... in fact, it might be as easy as replacing \"git pull\" with\n>> git rebase -p -i <last merge>^\n>> (Delete the merge in the editor)\n>> git pull <remote>\n>> \n>> Annoying to remember, but not TOO bad.\n> \n> Use an alias, or a wrapper script.\n\nOnce I've done it manually a few times to verify there are no surprises.\n"},{"id":"112495","messageId":"49F6AF12.70906@op5.se","threadId":"19033","inReplyTo":"20090427225900.29793.qmail@science.horizon.com","subject":"Re: Git workflow: how to achieve?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-04-28T07:24:02Z","receivedAt":"2009-04-28T07:24:02Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"George Spelvin wrote:\n> Andreas Ericsson <ae@op5.se> wrote:\n>>>  I want to be able to switch feature branches easily.\n>> Well, git makes that absolutely trivial. No other vcs has as cheap\n>> branching as git does.\n> \n> Yes, I'm quite aware, but I mean while keeping the same merged result.\n> That is, I'm compiling and testing base+feature_a+feature_b+feature_c,\n> but I want to be able to change which feature I'm committing to.\n> \n> This comes up if I arrange some merges so that the \"current branch\" is\n> one below the top merge:\n> \n> o--o--o--o--o--o--o <-- HEAD\n>            /  /  /\n>     ...c--c  /  /\n>       ...b--b  /\n>         ...a--a\n> \n> And then want to switch to branch c.\n> \n\nSorry, I still don't see a problem here. If you want to commit to C,\njust checkout C and commit. What am I missing?\n\nI know some kernel-lieutenants use a magic branch-prefix to mark which\nbranches to merge and compile-test, and then use a homegrown script\nto actually merge all those branches.\n\n> \n>>> But when I want to be testing something highly volatile like linux-next,\n>>> and ensuring that my work continues to merge with it cleanly, as well\n>>> as helping others with their branches, it becomes a daily pain.\n>> Why? Just merge it to make sure it merges, and then throw away the result.\n>> There's a post made by Linus somewhere on how he would like people to do\n>> merges (ie, which to keep and which to throw away).\n> \n> I don't want to just merge it, I want to *compile it* and *run it*.\n> Because the whole point of using linux-next is to test it.\n> And when it breaks, I want access to the relevant source code.\n> \n\nOk. Sounds like you'd need some CI tool with special smarts to try merging\nall relevant branches and bailing out on conflicts or something, unless you\nintend to compile it locally and run it from there.\n\n> \n>>>  And I can't share half-finished versions with co-workers.\n>> So finish it before you push. Anyone fetching from your local repository\n>> should be aware that not all branches point to something sensible. If\n>> they don't, they'll be sure to ask you so you can bring out the LART.\n>>\n>>> This is all vaguely quilt-like, although I'd rather not worry about the\n>>> order of patches.  I suppose I'd like git to let me \"commit under\" the\n>>> final merge.  When I switch branches, git should reorganize the tree of\n>>> merges so that the current branch is only one merge from the HEAD.\n>> Umm... Use a separate branch to do the merge, and enable rerere caching\n>> so your recorded resolutions get reused when you do the merge \"for real\".\n> \n>>> (Another thing I've wished for that might be related is for a branch\n>>> to have a notion of its origin.  So I can rebase it onto an arbitrary\n>>> place in the commit tree without having to explicitly specify an origin.)\n>> This is a bad idea.\n> \n> Can you explain why?  When I talk about a \"feature branch\", I mean a chain\n> of commits that introduces a feature.  Why would it be bad for a reference\n> to have an optional \"origin\" attribute that indicates a start position?\n> \n\nBecause they already have a reference to where they started. It's called\nthe merge-base and it differs depending on which branch you want to merge\nit into. If you have 'maint', 'next' and 'master', the \"point of origin\"\nfor \"next\" is \"maint\" from the point of view of \"maint\", but it's \"master\"\nfrom the point of view of \"master\". Marking it as \"master\" would mean you\ncouldn't use that mark when merging it into maint, so you'd have to either\nmark it as a sub-branch of all previous branches, or do what's done today.\n\nIOW, the mark would help one corner-case but would make all other cases\nmore complicated. A bad idea indeed.\n\n>>> ((Another really simple feature I keep wanting is \"git commit -b\n>>> <newbranch>\".  I should probably try to write a patch...))\n>> This is a good idea (assuming you mean \"commit this state to that branch\",\n>> and it should be fairly trivial to implement using something like this:\n>>\n>>     git stash && git checkout -b newbranch && git stash pop && git commit\n> \n> Actually, just \n> git checkout -b newbranch; git commit \"$@\"\n> does the job quite nicely, but \n> \n>>> Anyway, my feature ideas might be unworkable, and in any case, they'll\n>>> take a long time to implement.  Is there some easier way to achieve more\n>>> or less this effect?\n>> I'm not sure what effect you're after. If you can write down, with very\n>> *little* detail, what it is you want to achieve, it will be a lot easier\n>> to help you than if you ask about small steps in a pre-thought-out solution\n>> that may not even be right from the beginning.\n> \n> We may have a language problem.  \"very little detail\" means almost no detail,\n> an absence of details.  Did you mean \"every little detail\"?\n> \n\nNo, I mean with very little detail. Usually when trying to answer a question\nand I can't make sense of the question itself it's because the person asking\nhas already started down some road and entered into a frame of mind which I\ncannot share. That mindset may not be correct to solve the problem, so removing\ndetail usually helps.\n\n\"I'd like to automatically merge a bunch of branches and compile-test them\non every commit. How do I go about doing that?\" is a much more open question\nthat invites to giving a lot more correct answers. Since you haven't asked\nsuch a high-level question yet, I'm not sure what your need is, and therefore\nI cannot help you find a suitable solution.\n\n> It actually came to mind recently during $DAY_JOB work, but let me give a\n> concrete example based on the Linux kernel:\n> \n> I am running a customized Linux kernel.  On top of Linus's base,\n> I have local patches to enable 64-bit DMA on the SB600 SATA controller,\n> some local patches to make the RAID (md) code print out the locations\n> of mismatches when verifying, the EDAC quilt series, a merge of the\n> LinuxPPS code, a number of local patches to the LinuxPPS code (that\n> I'm discussing with Rodolfo Giometti), and some revisions to the serial\n> interrupt handler (that I'm discussing with Alan Cox).\n> \n> There's also the beginning of an ethernet driver that I'm trying to\n> write, but it's going slow.\n> \n> Every week or two, I rebase all that on top of Linus's latest.  Plain\n> rebase doesn't like the LinuxPPS merge, so I've been doing it manually\n> in two parts.  Although rebase -p is apprently working much better than\n> I remember.\n> \n\nThis step is rather unnecessary unless there are changes to API's you\ndepend on. Doing a single rebase once you're done with the patch-series\nwould be far better and would, I expect, remove quite a lot of the\ncomplexity you're experiencing.\n\n> I would also like to be able to revise each of those features\n> independently, based on feedback from the relevant upstream maintainer.\n> I also don't know in what order they'll be acecpted upstream (if ever).\n> Regardless of that, I want to my running kernels to have all of the\n> patches applied.\n> \n> Currently, some patches get deeply buried in the stack and I have to\n> do a lot of deep rebasing.\n> \n\nI'm not sure what you mean by \"deep rebasing\" or \"stack\", unless you've\nstarted using topgit already (which, I believe, does patch-stacks).\n\nNo patch should ever get \"deeply buried\" unless you do really, really\nweird things with your topic-branches though. They *should* remain the\nsame each and every time.\n\nBut the \"all patches in my running kernel\" problem could pretty easily\nbe solved by simply merging all your features into one branch and then\nperiodically merge Linus' latest into that branch whenever you want to\nupdate your kernel. When the merge conflicts, you'll have to figure out\nwhich of your features cause it (shouldn't be a problem unless they\nconflict with themselves) and then rebase *that* feature onto the new\nupstream (or something).\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nRegister now for Nordic Meet on Nagios, June 3-4 in Stockholm\n http://nordicmeetonnagios.op5.org/\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"112519","messageId":"20090428140248.31237.qmail@science.horizon.com","threadId":"19033","inReplyTo":"49F6AF12.70906@op5.se","subject":"Re: Git workflow: how to achieve?","fromName":"George Spelvin","fromEmail":"linux@horizon.com","sentAt":"2009-04-28T14:02:48Z","receivedAt":"2009-04-28T14:02:48Z","isPatch":false,"sender":{"key":"linux@horizon.com","avatar":null},"body":">From exon@op5.com Tue Apr 28 07:24:19 2009\nSender: Andreas Ericsson <exon@op5.com>\nDate: Tue, 28 Apr 2009 09:24:02 +0200\nFrom: Andreas Ericsson <ae@op5.se>\nUser-Agent: Thunderbird 2.0.0.21 (X11/20090320)\nMIME-Version: 1.0\nTo: George Spelvin <linux@horizon.com>\nCC: git@vger.kernel.org\nSubject: Re: Git workflow: how to achieve?\nReferences: <20090427225900.29793.qmail@science.horizon.com>\nIn-Reply-To: <20090427225900.29793.qmail@science.horizon.com>\nContent-Type: text/plain; charset=ISO-8859-15; format=flowed\nContent-Transfer-Encoding: 7bit\n\nAndreas Ericsson <ae@op5.se> wrote:\n> George Spelvin wrote:\n>> Andreas Ericsson <ae@op5.se> wrote:\n>>>>  I want to be able to switch feature branches easily.\n>>> Well, git makes that absolutely trivial. No other vcs has as cheap\n>>> branching as git does.\n>> \n>> Yes, I'm quite aware, but I mean while keeping the same merged result.\n>> That is, I'm compiling and testing base+feature_a+feature_b+feature_c,\n>> but I want to be able to change which feature I'm committing to.\n>> \n>> This comes up if I arrange some merges so that the \"current branch\" is\n>> one below the top merge:\n>> \n>> o--o--o--o--o--o--o <-- HEAD\n>>            /  /  /\n>>     ...c--c  /  /\n>>       ...b--b  /\n>>         ...a--a\n>> \n>> And then want to switch to branch c.\n\n> Sorry, I still don't see a problem here. If you want to commit to C,\n> just checkout C and commit. What am I missing?\n\nIf a or b touched one of the files that I edited (even in very different\nplaces), the checkout will fail.  And even if it succeeds, I have to\nre-do the merges.\n\nI expect it would be easier to commit on top of the pile, test, and\nthen rebase the last few patches to push the new changes under the\nmerges.\n\nI know some kernel-lieutenants use a magic branch-prefix to mark which\nbranches to merge and compile-test, and then use a homegrown script\nto actually merge all those branches.\n\n>>>> But when I want to be testing something highly volatile like linux-next,\n>>>> and ensuring that my work continues to merge with it cleanly, as well\n>>>> as helping others with their branches, it becomes a daily pain.\n>>> Why? Just merge it to make sure it merges, and then throw away the result.\n>>> There's a post made by Linus somewhere on how he would like people to do\n>>> merges (ie, which to keep and which to throw away).\n>> \n>> I don't want to just merge it, I want to *compile it* and *run it*.\n>> Because the whole point of using linux-next is to test it.\n>> And when it breaks, I want access to the relevant source code.\n> \n> Ok. Sounds like you'd need some CI tool with special smarts to try merging\n> all relevant branches and bailing out on conflicts or something, unless you\n> intend to compile it locally and run it from there.\n\n> Because they already have a reference to where they started. It's called\n> the merge-base and it differs depending on which branch you want to merge\n> it into. If you have 'maint', 'next' and 'master', the \"point of origin\"\n> for \"next\" is \"maint\" from the point of view of \"maint\", but it's \"master\"\n> from the point of view of \"master\". Marking it as \"master\" would mean you\n> couldn't use that mark when merging it into maint, so you'd have to either\n> mark it as a sub-branch of all previous branches, or do what's done today.\n> \n> IOW, the mark would help one corner-case but would make all other cases\n> more complicated. A bad idea indeed.\n\nEr... obviously, merges have to work the way they do.  This is all for\nrebases (cherry-picks).  And if the merge base is more recent than\nthe branch base, that should take precendence.\n\nBasically, \"git-rebase upstream branch\" current cherry-picks the commits\nin upstream..branch, a.k.a branch ^upstream.  I'd like it to be\nbranch ^branch.base ^upstream.\n\n>> We may have a language problem.  \"very little detail\" means almost no detail,\n>> an absence of details.  Did you mean \"every little detail\"?\n> \n> No, I mean with very little detail. Usually when trying to answer a\n> question and I can't make sense of the question itself it's because\n> the person asking has already started down some road and entered into\n> a frame of mind which I cannot share. That mindset may not be correct\n> to solve the problem, so removing detail usually helps.\n\nAh, sorry, you wanted more abstract, not more concrete.\n\n> \"I'd like to automatically merge a bunch of branches and compile-test them\n> on every commit. How do I go about doing that?\" is a much more open question\n> that invites to giving a lot more correct answers. Since you haven't asked\n> such a high-level question yet, I'm not sure what your need is, and therefore\n> I cannot help you find a suitable solution.\n\nOkay, \"I'd like to commit to somewhere further back in the history rather\nthan HEAD, and have the HEAD automagically rebased\".  How do I do that?\n\n>> It actually came to mind recently during $DAY_JOB work, but let me give a\n>> concrete example based on the Linux kernel:\n>> \n>> I am running a customized Linux kernel.  On top of Linus's base,\n>> I have local patches to enable 64-bit DMA on the SB600 SATA controller,\n>> some local patches to make the RAID (md) code print out the locations\n>> of mismatches when verifying, the EDAC quilt series, a merge of the\n>> LinuxPPS code, a number of local patches to the LinuxPPS code (that\n>> I'm discussing with Rodolfo Giometti), and some revisions to the serial\n>> interrupt handler (that I'm discussing with Alan Cox).\n>> \n>> There's also the beginning of an ethernet driver that I'm trying to\n>> write, but it's going slow.\n>> \n>> Every week or two, I rebase all that on top of Linus's latest.  Plain\n>> rebase doesn't like the LinuxPPS merge, so I've been doing it manually\n>> in two parts.  Although rebase -p is apprently working much better than\n>> I remember.\n> \n> This step is rather unnecessary unless there are changes to API's you\n> depend on. Doing a single rebase once you're done with the patch-series\n> would be far better and would, I expect, remove quite a lot of the\n> complexity you're experiencing.\n\n*Frustration*  I'm somehow having trouble communicating.\n\nI need to rebase it so I have a merged source tree that I can compile\nand test.  How else can I test Linus's latest plus my local changes?\nWhat alternative are you suggesting?\n\nI want, AT ALL TIMES, to be running the kernel consisting of\nLinus's latest plus my various local hacks.  I want to be able to\nfreely update any of the components that make up the result,\nand have the sum automatically recomputed for me.\n\n>> Currently, some patches get deeply buried in the stack and I have to\n>> do a lot of deep rebasing.\n> \n> I'm not sure what you mean by \"deep rebasing\" or \"stack\", unless you've\n> started using topgit already (which, I believe, does patch-stacks).\n> \n> No patch should ever get \"deeply buried\" unless you do really, really\n> weird things with your topic-branches though. They *should* remain the\n> same each and every time.\n\nI mean that the patch I want to edit is 20 or more patches back\nin history (I'm running 2.6.30-rc3-00063-gd3de9aa at the moment),\nso amending it involves a considerable amount of rebasing.\n\n(Because I'm currently organized as a linear list of local\npatches on top of upstream.  I'd prefer separate feature\nbranches plus merges, but that's what I'm asking how to make\nwork efficiently.)\n"},{"id":"112523","messageId":"49F719FA.8050700@op5.se","threadId":"19033","inReplyTo":"20090428140248.31237.qmail@science.horizon.com","subject":"Re: Git workflow: how to achieve?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2009-04-28T15:00:10Z","receivedAt":"2009-04-28T15:00:10Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"George Spelvin wrote:\n>>> We may have a language problem.  \"very little detail\" means almost no detail,\n>>> an absence of details.  Did you mean \"every little detail\"?\n>> No, I mean with very little detail. Usually when trying to answer a\n>> question and I can't make sense of the question itself it's because\n>> the person asking has already started down some road and entered into\n>> a frame of mind which I cannot share. That mindset may not be correct\n>> to solve the problem, so removing detail usually helps.\n> \n> Ah, sorry, you wanted more abstract, not more concrete.\n> \n>> \"I'd like to automatically merge a bunch of branches and compile-test them\n>> on every commit. How do I go about doing that?\" is a much more open question\n>> that invites to giving a lot more correct answers. Since you haven't asked\n>> such a high-level question yet, I'm not sure what your need is, and therefore\n>> I cannot help you find a suitable solution.\n> \n> Okay, \"I'd like to commit to somewhere further back in the history rather\n> than HEAD, and have the HEAD automagically rebased\".  How do I do that?\n> \n\nNow we're talking.\n\nIf I were you, I'd do (assuming 'master' is upstream's head):\n\ngit checkout -b private/test master\ngit merge $all_feature_branches\n# some sort of failure detected\ngit checkout $failed_topic\ngit rebase -i -p $(git merge-base HEAD master)\n# select 'edit' for the commits you want to edit\ngit branch -D private/test\n# goto 1\n\n\nThere may be even simpler workflows. I do not know.\n\n>>> It actually came to mind recently during $DAY_JOB work, but let me give a\n>>> concrete example based on the Linux kernel:\n>>>\n>>> I am running a customized Linux kernel.  On top of Linus's base,\n>>> I have local patches to enable 64-bit DMA on the SB600 SATA controller,\n>>> some local patches to make the RAID (md) code print out the locations\n>>> of mismatches when verifying, the EDAC quilt series, a merge of the\n>>> LinuxPPS code, a number of local patches to the LinuxPPS code (that\n>>> I'm discussing with Rodolfo Giometti), and some revisions to the serial\n>>> interrupt handler (that I'm discussing with Alan Cox).\n>>>\n>>> There's also the beginning of an ethernet driver that I'm trying to\n>>> write, but it's going slow.\n>>>\n>>> Every week or two, I rebase all that on top of Linus's latest.  Plain\n>>> rebase doesn't like the LinuxPPS merge, so I've been doing it manually\n>>> in two parts.  Although rebase -p is apprently working much better than\n>>> I remember.\n>> This step is rather unnecessary unless there are changes to API's you\n>> depend on. Doing a single rebase once you're done with the patch-series\n>> would be far better and would, I expect, remove quite a lot of the\n>> complexity you're experiencing.\n> \n> *Frustration*  I'm somehow having trouble communicating.\n> \n> I need to rebase it so I have a merged source tree that I can compile\n> and test.  How else can I test Linus's latest plus my local changes?\n> What alternative are you suggesting?\n> \n> I want, AT ALL TIMES, to be running the kernel consisting of\n> Linus's latest plus my various local hacks.  I want to be able to\n> freely update any of the components that make up the result,\n> and have the sum automatically recomputed for me.\n> \n\nJust merge them in. You're going through an obscene amount of trouble\njust to keep everything linear when there's no need for you to do so.\n\n  git checkout private/test\n  git merge master\n\nto merge latest upstream stuff. If you get conflicts, you may have to\namend your patch-series with one or more extra commits to resolve those\nconflicts, but there's no need to rebase them again if there are zero\nconflicts (and all tests pass, ofcourse; Not all post-merge code\nconflicts are caused by the actual merge algorithm).\n\nIf there are no new breakages or no new conflicts, there's no need to\nrebase your topics again at this point.\n\n>>> Currently, some patches get deeply buried in the stack and I have to\n>>> do a lot of deep rebasing.\n>> I'm not sure what you mean by \"deep rebasing\" or \"stack\", unless you've\n>> started using topgit already (which, I believe, does patch-stacks).\n>>\n>> No patch should ever get \"deeply buried\" unless you do really, really\n>> weird things with your topic-branches though. They *should* remain the\n>> same each and every time.\n> \n> I mean that the patch I want to edit is 20 or more patches back\n> in history (I'm running 2.6.30-rc3-00063-gd3de9aa at the moment),\n> so amending it involves a considerable amount of rebasing.\n> \n> (Because I'm currently organized as a linear list of local\n> patches on top of upstream.  I'd prefer separate feature\n> branches plus merges, but that's what I'm asking how to make\n> work efficiently.)\n\nI think I answered this up above, but I'm superbly tired right now so\nit's entirely possible I misunderstood one thing or another.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n\nRegister now for Nordic Meet on Nagios, June 3-4 in Stockholm\n http://nordicmeetonnagios.op5.org/\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"112540","messageId":"slrngve9vs.is7.sitaramc@sitaramc.homelinux.net","threadId":"19033","inReplyTo":"20090428140248.31237.qmail@science.horizon.com","subject":"Re: Git workflow: how to achieve?","fromName":"Sitaram Chamarty","fromEmail":"sitaramc@gmail.com","sentAt":"2009-04-28T15:59:56Z","receivedAt":"2009-04-28T15:59:56Z","isPatch":false,"sender":{"key":"sitaramc@gmail.com","avatar":"https://avatars.githubusercontent.com/u/43316?v=4"},"body":"On 2009-04-28, George Spelvin <linux@horizon.com> wrote:\n\n> I want, AT ALL TIMES, to be running the kernel consisting of\n> Linus's latest plus my various local hacks.  I want to be able to\n> freely update any of the components that make up the result,\n> and have the sum automatically recomputed for me.\n\n[and]\n\n> (Because I'm currently organized as a linear list of local\n> patches on top of upstream.  I'd prefer separate feature\n> branches plus merges, but that's what I'm asking how to make\n> work efficiently.)\n\nI'll freely admit I'm focusing on just those two paras out\nof a fairly long email, but topgit does precisely this.\nI've started to use it, and it seems to do the job well.\nWhen a feature branch gets updated, you just check out what\nI will call \"t/all\" and do a \"tg update\" and it picks up the\nnew stuff.\n\nSo the everyday stuff works.\n\nThe only sort-of-open question, right now, is how do you\nremove a dependency when one of your \"local hacks\" grows up\nand goes off to college.  Er... I mean gets accepted\nupstream :-)\n\nMy guess is that as long as all these local hacks are indeed\nlocal, the branch that represents the \"sum total\" can be\nblown away and rebuilt when that happens; you'd just list\nall but the recently graduated feature as dependencies.\n\nRegards,\n\nSitaram\n"}]}