{"thread":{"id":"35085","subject":"A workflow for local patch maintenance","startedAt":"2013-10-08T18:12:22Z","lastAt":"2013-10-11T15:30:53Z","messageCount":8,"participants":["Tony Finch","Jeff King","Jonathan Nieder","Stephen Bash"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"228618","messageId":"alpine.LSU.2.00.1310081906250.5715@hermes-2.csi.cam.ac.uk","threadId":"35085","inReplyTo":null,"subject":"A workflow for local patch maintenance","fromName":"Tony Finch","fromEmail":"dot@dotat.at","sentAt":"2013-10-08T18:12:22Z","receivedAt":"2013-10-08T18:12:22Z","isPatch":false,"sender":{"key":"dot@dotat.at","avatar":"https://avatars.githubusercontent.com/u/68429?v=4"},"body":"This is a copy of an article I published at\nhttp://fanf.livejournal.com/128282.html\nI'm sending a copy here because I'm interested to know what other ways\nthere might be of handling this situation.\n\n--\n\nWe often need to patch the software that we run in order to fix bugs\nquickly rather than wait for an official release, or to add functionality\nthat we need. In many cases we have to maintain a locally-developed patch\nfor a significant length of time, across multiple upstream releases,\neither because it is not yet ready for incorporation into a stable\nupstream version, or because it is too specific to our setup so will not\nbe suitable for passing upstream without significant extra work.\n\nI have been experimenting with a git workflow in which I have a feature\nbranch per patch. (Usually there is only one patch for each change we\nmake.) To move them on to a new feature release, I tag the feature branch\nheads (to preserve history), rebase them onto the new release version, and\noctopus merge them to create a new deployment version. This is rather\nunsatisfactory, because there is a lot of tedious per-branch work, and I\nwould prefer to have branches recording the development of our patches\nrather than a series of tags.\n\nHere is a git workflow suggested by Ian Jackson which I am trying out\ninstead. I don't yet have much experience with it; I am writing it down\nnow as a form of documentation.\n\nThere are three branches:\n\nupstream, which is where public releases live\nworking, which is where development happens\ndeployment, which is what we run\n\nWhich branch corresponds to upstream may change over time, for instance\nwhen we move from one stable version to the next one.\n\nThe working branch exists on the developer's workstation and is not\nnormally published. There might be multiple working branches for\nwork-in-progress. They get rebased a lot.\n\nStarting from an upstream version, a working branch will have a number of\nmature patches. The developer works on top of these in\ncommit-early-commit-often mode, without worrying about order of changes or\ncleanliness. Every so often we use git rebase --interactive to tidy up the\npatch set. Often we'll use the \"squash\" command to combine new commits\nwith the mature patches that they amend. Sometimes it will be rebased onto\na new upstream version.\n\nWhen the working branch is ready, we use the commands below to update the\ndeployment branch. The aim is to make it look like updates from the\nworking branch are repeatedly merged into the deployment branch. This is\nso that we can push updated versions of the patch set to a server without\nhaving to use --force, and pulling updates into a checked out version is\njust a fast-forward. However this isn't a normal merge since the tree at\nthe head of deployment always matches the most recent good version of\nworking. (This is similar to what stg publish does.) Diagramatically,\n\n     |\n    1.1\n     | \\\n     |  `A---B-- 1.1-patched\n     |    \\       |\n     |     \\      |\n     |      `C-- 1.1-revised\n     |            |\n    2.0           |\n     | \\          |\n     |  `-C--D-- 2.0-patched\n     |            |\n    3.1           |\n     | \\          |\n     |  `-C--E-- 3.1-patched\n     |            |\n  upstream        |\n              deployment\n\nThe horizontal-ish lines are different rebased versions of the patch set.\nLetters represent patches and numbers represent version tags. The tags on\nthe deployment branch are for the install scripts so I probably won't need\none on every update.\n\nIdeally we would be able to do this with the following commands:\n\n    $ git checkout deployment\n    $ git merge -s theirs working\n\nHowever there is an \"ours\" merge strategy but not a \"theirs\" merge\nstrategy. Johannes Sixt described how to simulate git merge -s theirs in a\npost to the git mailing list in 2010.\nhttp://article.gmane.org/gmane.comp.version-control.git/163631\nSo the commands are:\n\n    $ git checkout deployment\n    $ git merge --no-commit -s ours working\n    $ git read-tree -m -u working\n    $ git commit -m \"Update to $(git describe working)\"\n\nMark Wooding suggested the following more plumbing-based version, which\nunlike the above does not involve switching to the deployment branch.\n\n    $ d=$(git rev-parse deployment)\n    $ w=$(git rev-parse working)\n    $ c=$(echo \"Update to $(git describe working)\" |\n          git commit-tree -p $d -p $w working^{tree})\n    $ git update-ref deployment $c $d\n    $ unset c d w\n\nTony.\n-- \nf.anthony.n.finch  <dot@dotat.at>  http://dotat.at/\nForties, Cromarty: East, veering southeast, 4 or 5, occasionally 6 at first.\nRough, becoming slight or moderate. Showers, rain at first. Moderate or good,\noccasionally poor at first.\n"},{"id":"228679","messageId":"20131010013343.GB14429@sigill.intra.peff.net","threadId":"35085","inReplyTo":"alpine.LSU.2.00.1310081906250.5715@hermes-2.csi.cam.ac.uk","subject":"Re: A workflow for local patch maintenance","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-10-10T01:33:43Z","receivedAt":"2013-10-10T01:33:43Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 08, 2013 at 07:12:22PM +0100, Tony Finch wrote:\n\n> We often need to patch the software that we run in order to fix bugs\n> quickly rather than wait for an official release, or to add functionality\n> that we need. In many cases we have to maintain a locally-developed patch\n> for a significant length of time, across multiple upstream releases,\n> either because it is not yet ready for incorporation into a stable\n> upstream version, or because it is too specific to our setup so will not\n> be suitable for passing upstream without significant extra work.\n\nDo you need to keep the modifications you make on top of upstream as a\nnice, clean series of rebased patches? If not, then you can avoid the\nrepeated rebasing, and just use a more traditional topic-branch\nworkflow. Treat modifications from upstream as just another topic.\n\nFor example, start with some version (let's say 1.0) of the upstream\nsoftware as your \"master\" branch. If it's kept in git, build on\nupstream's git history.  If all you get are tarballs, create an\n\"upstream\" branch with v1.0, and fork \"master\" from it.\n\nBuild on master as you would if it were your own. Fork topic branches,\ndevelop the topics, test them, and then merge them back to \"master\" when\nthey're ready (or do development straight on master, or whatever\nworkflow you're accustomed to).\n\nWhen v1.1 of the upstream software comes out, create a \"merge-upstream\"\ntopic branch from the tip of your \"master\". If upstream is in git, just\n\"git merge v1.1\" from upstream. If not, then checkout your pristine\n\"upstream\" branch (which should still be sitting at the v1.0 commit),\nand build a v1.1 commit on top of it. And then \"git merge upstream\" to\npick up the new changes.\n\nTest your merge-upstream topic in isolation, and when you think it's\nready merge it into master and deploy.\n\nThe most difficult part is the merge of upstream into the topic branch.\nBut git's 3-way merge tends to do a pretty good job (e.g., if you\ncontributed your patches upstream, then there should be no conflict).\nYou can also break up the work by keeping the \"merge\" topic running for\na long time, and merging as often as possible from upstream. That breaks\nthe conflict resolution into smaller chunks, and lets you do it closer\nto when the conflicting patches were actually made, when they are\nhopefully closer in your mind. And you don't have to worry about having\na broken intermediate result, because you're not deploying it; you're\njust keeping the topic up to date until you're ready to test it.\n\nYou can also try git-imerge, which can make big merges a little more\nmanageable (though it can also make them harder sometimes...):\n\n  https://github.com/mhagger/git-imerge\n\nThe history for such a repository might look like:\n\n       o--o--B--o--o--C  <-- upstream branch\n      /       \\        \\\n     o--o---o--o--o--o--D  <-- upstream-merge branch\n    /      /        /    \\\n   A--o---E--o--o--F--o---G <-- master branch\n    \\    / \\      /\n     o--o   o----o  <-- topic branches\n\nwhere:\n\n  - A is the v1.0 commit you start at\n\n  - B and C are milestones where you merged upstream into your\n    upstream-merge topic branch. These could be releases (like v1.1), or\n    they could just be random spots where you felt like merging to keep\n    things up to date. It depends how you want to break up the conflict\n    resolution\n\n  - D is a state of the upstream-merge branch that you test to make sure\n    the merge happened OK\n\n  - E and F are merges of regular topic branches (i.e., the patches you\n    are working on locally). Note that we also merge those up to the\n    upstream-merge branch, so that we can resolve early any conflicts\n    between what's happening on master and what's happening upstream.\n\n  - G is the merge of D into the master branch, after we have decided\n    it's good to deploy\n\nThis all assumes that \"master\" is your known-good state that you deploy\nor ship. If you prefer to have a \"deploy\" or \"maint\" branch for\nhotfixes, you can do that too.\n\nHope that helps,\n-Peff\n"},{"id":"228702","messageId":"alpine.LSU.2.00.1310100927270.3100@hermes-2.csi.cam.ac.uk","threadId":"35085","inReplyTo":"20131010013343.GB14429@sigill.intra.peff.net","subject":"Re: A workflow for local patch maintenance","fromName":"Tony Finch","fromEmail":"dot@dotat.at","sentAt":"2013-10-10T16:53:57Z","receivedAt":"2013-10-10T16:53:57Z","isPatch":false,"sender":{"key":"dot@dotat.at","avatar":"https://avatars.githubusercontent.com/u/68429?v=4"},"body":"Jeff King <peff@peff.net> wrote:\n>\n> Do you need to keep the modifications you make on top of upstream as a\n> nice, clean series of rebased patches? If not, then you can avoid the\n> repeated rebasing, and just use a more traditional topic-branch\n> workflow. Treat modifications from upstream as just another topic.\n\nThanks for the suggestion!\n\nOur aim is to get as many patches into the upstream version as we can,\nwhich is why my starting point is a clean rebased patch series. I am also\nthinking that this will help me to know when a patch can be dropped from\nthe series because upstream have incorporated something like it. If\nupstream works like git upstream (incorporating patches verbatim after\nthey pass review) then git can handle this automatically, but if the patch\ngets re-worked it might be easier for me to drop it when rebasing rather\nthan resolve conflicts. I'm also thinking that for packages which we\nupdate relatively infrequently, having a clean patch series makes it\neasier to review whether they are all still necessary when updating. But\nperhaps I am too wedded to manual patch management...\n\nTony.\n-- \nf.anthony.n.finch  <dot@dotat.at>  http://dotat.at/\nForties, Cromarty: East, veering southeast, 4 or 5, occasionally 6 at first.\nRough, becoming slight or moderate. Showers, rain at first. Moderate or good,\noccasionally poor at first.\n"},{"id":"228703","messageId":"20131010173628.GB24782@sigill.intra.peff.net","threadId":"35085","inReplyTo":"alpine.LSU.2.00.1310100927270.3100@hermes-2.csi.cam.ac.uk","subject":"Re: A workflow for local patch maintenance","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-10-10T17:36:28Z","receivedAt":"2013-10-10T17:36:28Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Oct 10, 2013 at 05:53:57PM +0100, Tony Finch wrote:\n\n> Our aim is to get as many patches into the upstream version as we can,\n> which is why my starting point is a clean rebased patch series. I am also\n> thinking that this will help me to know when a patch can be dropped from\n> the series because upstream have incorporated something like it. If\n> upstream works like git upstream (incorporating patches verbatim after\n> they pass review) then git can handle this automatically, but if the patch\n> gets re-worked it might be easier for me to drop it when rebasing rather\n> than resolve conflicts. I'm also thinking that for packages which we\n> update relatively infrequently, having a clean patch series makes it\n> easier to review whether they are all still necessary when updating. But\n> perhaps I am too wedded to manual patch management...\n\nI am in a similar situation to you. At GitHub, we run more-or-less stock\ngit on our backend, but often make bug-fixes or enhancements that are\nintended for upstream, but which we want to start using before the next\nrelease.\n\nWe used to keep the patches as series, and rebase them on newer versions\nof git from time to time. These days we use the workflow I described\nearlier. The specific things we wanted to fix were:\n\n  1. It was a giant pain to work on or modify a patch series.\n\n  2. It did not scale well beyond one person handling the patches\n     and rebasing. Now people more or less work on our forked repository\n     as they would normally, and don't have to care; merging from\n     upstream is just another feature (that happens to bring in a ton of\n     commits :) ).\n\n  3. The pain in doing the big rebase-test-deploy cycle meant that we\n     often delayed it, keeping us several versions behind upstream.\n     This is bad not only for the end product (you aren't getting other\n     bugfixes from upstream as quickly), but also because the longer you\n     wait to rebase or merge, the more painful it generally is.\n\nThat being said, there are some new downsides, as you noted:\n\n  1. Resolving conflicts between your version and the reworked upstream\n     version can be a pain.\n\n  2. If your local development does not happen in a clean series, it can\n     be hard to create a clean series for upstream, and/or revert in\n     favor of upstream when necessary.\n\nI don't have silver bullets for either, unfortunately. To mitigate\nproblem 1, I will sometimes revert a local topic before doing the\nupstream merge, if I know it has been reworked. You can do this right\nbefore merging. Or, as soon as you see that upstream is taking a\nreworked version, you can revert what you have locally and apply the\nupstream fix. This latter has the advantage of doing it much closer to\nthe actual development time, so handling any irregularities is easier.\nBut it is not always a possibility if upstream's reworking involved\nbuilding on other changes that you do not want to grab. :)\n\nFor problem 2, it helps if you can do development with topic branches as\ngit.git does, with aggressive rebasing while a topic is in development,\nand then merging to master once it is mature.  Then at least your \"git\nlog --first-parent\" view of \"master\" shows you which topics you made,\nand the topics themselves are relatively clean. Sometimes you end up\nneeding to make changes to a topic after it is mature, and the ordering\nis not exactly what you would send upstream (e.g., for a pure patch\nseries going to upstream, you would not fix the bug on top, you would\nsquash the fix into an earlier commit). I mostly just handle this\nmanually, and it doesn't come up too often (and in many cases, by the\ntime you have the \"bugfix on top\" locally, upstream has also already\napplied the original patches, and they want it as a bugfix on top, too).\n\nWe can no longer easily say \"this is stock git, with patches X-Y-Z on\ntop\" (we can get the set of commits we have that git does not have, of\ncourse, but there's no easy way to say \"these ones aren't relevant\nanymore\"). But we've found it's not all _that_ important. With a\nrebasing strategy, you really want to know so that you don't\naccidentally drop a patch that is necessary. But with a merge strategy,\nyou cannot accidentally drop a patch (you might botch the conflict\nresolution, of course, but that is a bit harder).\n\n-Peff\n"},{"id":"228707","messageId":"20131010191827.GO9464@google.com","threadId":"35085","inReplyTo":"20131010173628.GB24782@sigill.intra.peff.net","subject":"Re: A workflow for local patch maintenance","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2013-10-10T19:18:27Z","receivedAt":"2013-10-10T19:18:27Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Jeff King wrote:\n\n>   3. The pain in doing the big rebase-test-deploy cycle meant that we\n>      often delayed it, keeping us several versions behind upstream.\n>      This is bad not only for the end product (you aren't getting other\n>      bugfixes from upstream as quickly), but also because the longer you\n>      wait to rebase or merge, the more painful it generally is.\n>\n> That being said, there are some new downsides, as you noted:\n>\n>   1. Resolving conflicts between your version and the reworked upstream\n>      version can be a pain.\n>\n>   2. If your local development does not happen in a clean series, it can\n>      be hard to create a clean series for upstream, and/or revert in\n>      favor of upstream when necessary.\n\nThat suggests a possible hybrid approach: use a normal merge-heavy\nworkflow day to day, but occasionally clean up, for example by\nrebasing against upstream.\n\nThat doesn't address the question of \"how do I preserve old versions\nof my patchset after a rebase\", though.\n\nThe msysgit project uses a script called merging-rebase.sh[1] to\nkeep their patches current on top of the shifting target of git's\n\"next\".  It's similar to your \"merge -s theirs\" approach.  It has some\nproblems (once you get past the current version of the patch stack,\nhistory mining is complicated by all the old versions of the patch\nstack) but for their day-to-day development it works ok.\n\nThere is an interesting approach that involves only merging and never\nrebasing, while still being able to create a presentable patch series\nwhen you're done.  The idea is to keep each patch meant for upstream\nconsumption in a separate (specially named) branch, with tracked files\nlike \".topmsg\" containing its commit message, dependencies, and other\nmetadata.  There is a tool called 'tg' (TopGit) for working with this\nkind of repo[2].  The Hurd uses it for their binutils and glibc\npatches.\n\nAnother tool for maintaining a public patch stack, this time using a\n\"quilt\"-style workflow instead of aggressively using native git\ncommands, is guilt[3], used for example to maintain the ext4 patch\nqueue.\n\nIn practice I tend to find all these too formal, and just keep one\nbranch that moves forward and is never rebased and a separate branch\nthat is constantly rebased with commits explaining all my changes to\nthe upstream code.  E.g., see [4].  This probably only works when the\npatch stack is not very large.\n\nJonathan\n\n[1] https://github.com/msysgit/msysgit/blob/master/share/msysGit/merging-rebase.sh\n[2] https://github.com/greenrd/topgit#readme\n[3] http://repo.or.cz/w/guilt.git\n[4] git://repo.or.cz/xz/debian.git\n\n    Here the constantly-rebased branch is not even published, since\n    it is easy to re-create by applying the patches.\n\n    The constantly-advancing branch is \"master\", which consists of\n    patched upstream source + extra metadata in the debian/\n    subdirectory.\n\n    The constantly-rebased branch can be revived by applying the\n    patches from debian/diff/ to the \"upstream\" branch.\n"},{"id":"228737","messageId":"247350414.2015225.1381497748024.JavaMail.root@genarts.com","threadId":"35085","inReplyTo":"20131010173628.GB24782@sigill.intra.peff.net","subject":"Re: A workflow for local patch maintenance","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2013-10-11T13:22:28Z","receivedAt":"2013-10-11T13:22:28Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"----- Original Message -----\n> From: \"Jeff King\" <peff@peff.net>\n> Sent: Thursday, October 10, 2013 1:36:28 PM\n> Subject: Re: A workflow for local patch maintenance\n> \n> ... snip ...\n>\n> That being said, there are some new downsides, as you noted:\n> \n>   1. Resolving conflicts between your version and the reworked\n>   upstream version can be a pain.\n> \n> ... snip ...\n> \n> To mitigate problem 1, I will sometimes revert a local topic before\n> doing the upstream merge, if I know it has been reworked.\n\nPeff (slightly off topic) - A coworker of mine actually ran into this\nproblem earlier this week.  Is there recommended way to revert a merged\ntopic branch?  I assume it's essentially reverted each commit introduced\nby the branch, but is there a convenient invocation of revert? (easy to \nremember and hard to screw up)\n\nThanks,\nStephen\n"},{"id":"228739","messageId":"20131011151614.GA29226@sigill.intra.peff.net","threadId":"35085","inReplyTo":"247350414.2015225.1381497748024.JavaMail.root@genarts.com","subject":"Re: A workflow for local patch maintenance","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-10-11T15:16:14Z","receivedAt":"2013-10-11T15:16:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Oct 11, 2013 at 09:22:28AM -0400, Stephen Bash wrote:\n\n> > To mitigate problem 1, I will sometimes revert a local topic before\n> > doing the upstream merge, if I know it has been reworked.\n> \n> Peff (slightly off topic) - A coworker of mine actually ran into this\n> problem earlier this week.  Is there recommended way to revert a merged\n> topic branch?  I assume it's essentially reverted each commit introduced\n> by the branch, but is there a convenient invocation of revert? (easy to \n> remember and hard to screw up)\n\nIf you merged the whole topic in at once, then you can use \"git revert\n-m 1 $merge_commit\" to undo the merge. If it came in individual pieces,\nthen you have to revert each one individually (though if it was a series\nof merges, you can in theory revert each merge in reverse order).\n\n-Peff\n"},{"id":"228740","messageId":"636938755.2020935.1381505453539.JavaMail.root@genarts.com","threadId":"35085","inReplyTo":"20131011151614.GA29226@sigill.intra.peff.net","subject":"Re: A workflow for local patch maintenance","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2013-10-11T15:30:53Z","receivedAt":"2013-10-11T15:30:53Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"----- Original Message -----\n> From: \"Jeff King\" <peff@peff.net>\n> To: \"Stephen Bash\" <bash@genarts.com>\n> Cc: git@vger.kernel.org, \"Tony Finch\" <dot@dotat.at>\n> Sent: Friday, October 11, 2013 11:16:14 AM\n> Subject: Re: A workflow for local patch maintenance\n> \n> On Fri, Oct 11, 2013 at 09:22:28AM -0400, Stephen Bash wrote:\n> \n> > > To mitigate problem 1, I will sometimes revert a local topic\n> > > before doing the upstream merge, if I know it has been reworked.\n> > \n> > Peff (slightly off topic) - A coworker of mine actually ran into\n> > this problem earlier this week.  Is there recommended way to revert\n> > a merged topic branch?  I assume it's essentially reverted each\n> > commit introduced by the branch, but is there a convenient\n> > invocation of revert?  (easy to remember and hard to screw up)\n> \n> If you merged the whole topic in at once, then you can use \"git revert\n> -m 1 $merge_commit\" to undo the merge. If it came in individual\n> pieces, then you have to revert each one individually (though if it\n> was a series of merges, you can in theory revert each merge in reverse\n> order).\n\nThanks for the pointer.  That got me to the right place on the revert\nmanpage, and there I found the link to howto/revert-a-faulty-merge.txt\nwhich was extremely helpful.\n\nThanks!\nStephen\n"}]}