{"thread":{"id":"1899","subject":"[Core GIT] Long-term cherrypicking","startedAt":"2005-09-21T16:40:15Z","lastAt":"2005-09-22T10:17:16Z","messageCount":8,"participants":["Petr Baudis","Junio C Hamano","Greg KH","Catalin Marinas"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"9055","messageId":"20050921164015.GC21971@pasky.or.cz","threadId":"1899","inReplyTo":null,"subject":"[Core GIT] Long-term cherrypicking","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2005-09-21T16:40:15Z","receivedAt":"2005-09-21T16:40:15Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hello,\n\n  I've been working out some workflow possibilities with Karsten Keil at\nthe SuSE Labs conference, and I'd like to see your opinions. His\nposition does not seem unique at all, so I'm a bit surprised that noone\nsolved this before, or at least I didn't find any documentation about\nthis in the tutorial and howtos (or perhaps I missed it?).\n\n  His situation is that he has some patches in the ISDN subsystem, a\npublic repository, but sends the patches over e-mail to Linus. So he is\nsomething between the subsystem maintainer and individual developer in\nthe categories listed out in the tutorial. What should be his merging\nstrategy?\n\n  Since he has public repository, he cannot rebase, but he needs to\nbe cherrypicking. The trouble is, git-cherry really does not work\n\"through\" merges now - it stops at his last merge with Linus. The\nobvious solution would be something like this patch:\n\n\ndiff --git a/git-cherry.sh b/git-cherry.sh\n--- a/git-cherry.sh\n+++ b/git-cherry.sh\n@@ -5,18 +5,22 @@\n \n . git-sh-setup || die \"Not a git archive.\"\n \n-usage=\"usage: $0 \"'[-v] <upstream> [<head>]\n+usage=\"usage: $0 \"'[-v] <upstream> [<head>] [<base>]\n \n              __*__*__*__*__> <upstream>\n-            /\n-  fork-point\n+            /         \\\n+      <base>           \\ fork-point\n             \\__+__+__+__+__+__+__+__> <head>\n \n Each commit between the fork-point and <head> is examined, and\n-compared against the change each commit between the fork-point and\n+compared against the change each commit between the <base> and\n <upstream> introduces.  If the change seems to be in the upstream,\n it is shown on the standard output with prefix \"+\".  Otherwise\n it is shown with prefix \"-\".\n+\n+If no <base> is specified, it is assumed to be the same as fork-point.\n+This is the case if you run git-cherry only before you merge the\n+upstream.\n '\n \n case \"$1\" in -v) verbose=t; shift ;; esac \n@@ -35,12 +39,16 @@ case \"$#\" in\n 2) upstream=`git-rev-parse --verify \"$1\"` &&\n    ours=`git-rev-parse --verify \"$2\"` || exit\n    ;;\n+3) upstream=`git-rev-parse --verify \"$1\"` &&\n+   ours=`git-rev-parse --verify \"$2\"` &&\n+   base=`git-rev-parse --verify \"$3\"` || exit\n *) echo >&2 \"$usage\"; exit 1 ;;\n esac\n \n # Note that these list commits in reverse order;\n # not that the order in inup matters...\n-inup=`git-rev-list ^$ours $upstream` &&\n+[ -z \"$base\" ] && base=$ours\n+inup=`git-rev-list ^$base $upstream` &&\n ours=`git-rev-list $ours ^$upstream` || exit\n \n tmp=.cherry-tmp$$\n\n\n  Then you would invoke it like\n\n\tgit cherry origin master forkpoint\n\nwhere forkpoint is a tag pointing at the last known point in Linus'\nbranch when he had all my (well, Karsten's) patches integrated in his\ntree. I tested it briefly and it seems to work - I suppose that I will\nleave more testing to him. :-) If GIT wants to be even cooler, it could\nremember the last forkpoint and automatically advance it.\n\n  Opinions?\n\n\nPS: Perex hints over my shoulder that the ALSA guys are just manually\nrecording which patches were merged and which weren't. It's a shame that\nGIT has no UI to do that for them if it can. Anyway, I'll be thinking\nabout if/how to implement the cherrypicking workflow support for Cogito.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nVI has two modes: the one in which it beeps and the one in which\nit doesn't.\n"},{"id":"9070","messageId":"7vzmq6p2an.fsf@assigned-by-dhcp.cox.net","threadId":"1899","inReplyTo":"20050921164015.GC21971@pasky.or.cz","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-09-21T20:57:04Z","receivedAt":"2005-09-21T20:57:04Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Let me see if I understand what you are trying to solve.\n\nS is subsystem line of development, L is Linus line of\ndevelopment, M is a merge from L into S.\n\nA' and B' are resulting commits from e-mail patch, that contain\nchanges introduced to S by A and B commits, respectively.\n\n S:     A - B - C - M - D - ...\n      /           /      \n L:  O- X - A'- Y - Z - ...\n\nAfter sending off patches A and B, L accepts only A and creates\ncommit A'.  After a while, S pulls from L and merges at M [*1*].\nIn the meantime, there are other changes on both ends (C and Y).\nAlso note that I am not drawing commits in S that are not to\nbe sent to L via e-mail nor pull request -- there may be other\ncommits on lines between B and C, C and M, etc.\n\nM contains changes B and C which are lacking from L.  On top of\nthat, a commit D is made and S wants to ask L to incorporate\nchanges in S but not in L (B, C and D), but finding B, C and D\nis cumbersome [*2*].  You solved this by 'git cherry' to go\nbeyond M, which is a valid workaround, but then the problem\nbecomes how to find where to stop.\n\nI am assuming the reason this is a cherry-pick problem is\nbecause S does not necessarily send everything out, and whenever\nS does 'format-patch L' to generate a list of patches, only a\nsubset of them is actually sent out.  If that is the case, how\nabout maintaining another branch T on the S side, which is\nprivate to S?\n\nT starts out as the same commit as the head of L.  Whenever S\nhas merge-worthy commits, run 'git-cherry-pick' to copy them\nover to T, and 'format-patch L' is run in T, not in S, to\ngenerate patches to be sent out.\n\nT being a private tree allows you to rebase every once in a\nwhile to L [*3*], so that already applied patches are dropped\nfrom it.\n\n\n[Footnote]\n\n*1* I am assuming that merge M would not be made any more\ndifficult by having A and A' on both sides.  Is it a problem in\npractice?\n\n*2* Finding B for resend may be debatable -- it is more of an\ninter developer communication issue.\n\n*3* For the above workflow to work, I think the current 'git\nrebase' needs to be improved much in two aspects.  (1) The\ncurrent 'cherry' uses 'git-patch-id' which shares the same\nstrictness with 'git-apply' and would not tolerate much fuzz\nbetween what you send out and what got actually applied. (2)\n'rebase' keeps going when a merge is not cleanly done, noting\nthem to sort them out by hand at the end.  This loses the order\nof commits in the tree being rebased and very unwieldy.  I have\na plan to fix this in the TODO list for some time.\n"},{"id":"9077","messageId":"20050921221931.GC10575@pasky.or.cz","threadId":"1899","inReplyTo":"7vzmq6p2an.fsf@assigned-by-dhcp.cox.net","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2005-09-21T22:19:31Z","receivedAt":"2005-09-21T22:19:31Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Sep 21, 2005 at 10:57:04PM CEST, I got a letter\nwhere Junio C Hamano <junkio@cox.net> told me that...\n> You solved this by 'git cherry' to go\n> beyond M, which is a valid workaround, but then the problem\n> becomes how to find where to stop.\n\nWhen forking your branch, record that in a tag. When running git-cherry\nwith the tag as base, take the earliest cherrypick found and find the\nLCA of the two corresponding commits. Reset the tag to that.\n\n> I am assuming the reason this is a cherry-pick problem is\n> because S does not necessarily send everything out, and whenever\n> S does 'format-patch L' to generate a list of patches, only a\n> subset of them is actually sent out.  If that is the case, how\n> about maintaining another branch T on the S side, which is\n> private to S?\n\nThat is not the case, this is a cherry-pick problem since L dropped some\nof the patches (if the git's definition of \"cherry-pick problem\" isn't\nsomething different to what I'd expect)...\n\n> T starts out as the same commit as the head of L.  Whenever S\n> has merge-worthy commits, run 'git-cherry-pick' to copy them\n> over to T, and 'format-patch L' is run in T, not in S, to\n> generate patches to be sent out.\n> \n> T being a private tree allows you to rebase every once in a\n> while to L [*3*], so that already applied patches are dropped\n> from it.\n\n...nevertheless your solution still seems to apply and sounds good to\nme, thanks. I wonder why this didn't occur to me. :-) Perhaps it could\nbe noted in the tutorial?\n\nThe disadvantage is that this seems as a lot of work. You have to\nexplicitly cherry-pick each commit, and the fact that you have a\nfloating (continuously rebased) tree means that you can conveniently\nkeep it only on just one machine. [*1*]\n\nThe advantage is that it is easy to \"forget\" commits which were dropped\naltogether or applied modified. You could put together some filter and\ngrep -v, but that's ugly and doesn't scale.\n\nWell, I guess that if you are going to do this all, it really gets\nvisible that this rebasing stuff is really quite an ugly hack and that\nyou are probably much better off with something like StGIT. ;-)\n\n> [Footnote]\n> \n> *1* I am assuming that merge M would not be made any more\n> difficult by having A and A' on both sides.  Is it a problem in\n> practice?\n\nThat's something Karsten Keil would be more qualified to reply to, but\nunfortunately you removed him from the Cc list. ;-)\n\n\n*1* I find the fast-forward restriction more and more annoying in this\ncontext, actually, and I'm seriously considering dropping it from Cogito\naltogether. GIT seems to care a lot about it, but Cogito doesn't so far,\nit would only have problems fast-forwarding your master to catch up with\nthe branch you are merging, trying to do a tree merge instead. But\n\n\tif old_ref == master then master = new_ref\n\nwould solve that. Can it ever really hurt?\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nVI has two modes: the one in which it beeps and the one in which\nit doesn't.\n"},{"id":"9079","messageId":"7voe6mngsk.fsf@assigned-by-dhcp.cox.net","threadId":"1899","inReplyTo":"20050921221931.GC10575@pasky.or.cz","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2005-09-21T23:26:51Z","receivedAt":"2005-09-21T23:26:51Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis <pasky@suse.cz> writes:\n\n> Well, I guess that if you are going to do this all, it really gets\n> visible that this rebasing stuff is really quite an ugly hack and that\n> you are probably much better off with something like StGIT. ;-)\n\nYes.\n\n> *1* I find the fast-forward restriction more and more annoying in this\n> context, actually, and I'm seriously considering dropping it from Cogito\n> altogether.\n\nMy \"pu\" is always floating and I have this in my\n.git/remotes/ko:\n\n    URL: master.kernel.org:/pub/scm/git/git.git/\n    Push: master maint rc +pu\n"},{"id":"9088","messageId":"20050922083142.GA6866@kroah.com","threadId":"1899","inReplyTo":"20050921164015.GC21971@pasky.or.cz","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Greg KH","fromEmail":"greg@kroah.com","sentAt":"2005-09-22T08:31:42Z","receivedAt":"2005-09-22T08:31:42Z","isPatch":false,"sender":{"key":"greg@kroah.com","avatar":"https://gravatar.com/avatar/5bb5aa0cc2e01c00ec899d11130c07796bc186e465bae57bc34873b13b72c7c8?d=mp&s=160"},"body":"On Wed, Sep 21, 2005 at 06:40:15PM +0200, Petr Baudis wrote:\n>   Hello,\n> \n>   I've been working out some workflow possibilities with Karsten Keil at\n> the SuSE Labs conference, and I'd like to see your opinions. His\n> position does not seem unique at all, so I'm a bit surprised that noone\n> solved this before, or at least I didn't find any documentation about\n> this in the tutorial and howtos (or perhaps I missed it?).\n> \n>   His situation is that he has some patches in the ISDN subsystem, a\n> public repository, but sends the patches over e-mail to Linus. So he is\n> something between the subsystem maintainer and individual developer in\n> the categories listed out in the tutorial. What should be his merging\n> strategy?\n\nNot to use git for this.\n\nSeriously, that's what I have switched to doing, and it's so much\neasier.  I use quilt to manage patches from the community, and Andrew\npulls them into to the -mm releases, and all users can test them.  I\nkeep them up to date with the git snapshots, which handles the different\nmerge and fuzz issues very well.\n\nThen, when it's time to merge with Linus, I pick and choose the patches\nthat I want to send off, create a git tree, add them to the tree, and\nsend them off.\n\nIt's ended up saving me a lot of time (I used to do what Karsten is\ntrying to do with bitkeeper, and it had the same issues that he is\nrunning into) and would recommend this situation for anyone who wants to\nkeep patches from being merged immediatly (like he is trying to do.)\n\nthanks,\n\ngreg k-h\n"},{"id":"9094","messageId":"tnxpsr12zu0.fsf@arm.com","threadId":"1899","inReplyTo":"20050922083142.GA6866@kroah.com","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T09:53:27Z","receivedAt":"2005-09-22T09:53:27Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"Greg KH <greg@kroah.com> wrote:\n> On Wed, Sep 21, 2005 at 06:40:15PM +0200, Petr Baudis wrote:\n>>   His situation is that he has some patches in the ISDN subsystem, a\n>> public repository, but sends the patches over e-mail to Linus. So he is\n>> something between the subsystem maintainer and individual developer in\n>> the categories listed out in the tutorial. What should be his merging\n>> strategy?\n>\n> Not to use git for this.\n>\n> Seriously, that's what I have switched to doing, and it's so much\n> easier.  I use quilt to manage patches from the community, and Andrew\n> pulls them into to the -mm releases, and all users can test them.  I\n> keep them up to date with the git snapshots, which handles the different\n> merge and fuzz issues very well.\n\nYou might want to try StGIT as well, unless you have a huge number of\npatches and rebasing them takes too much time. It follows the quilt\nideas but it does a three-way merge instead of patch+fuzz. This allows\nit to detect patches accepted by Linus and also see whether they were\nnot fully applied or were modified. You can run 'stg export' to\ngenerate a patch series to send to Andrew.\n\n> Then, when it's time to merge with Linus, I pick and choose the patches\n> that I want to send off, create a git tree, add them to the tree, and\n> send them off.\n\nWith StGIT, you can either use 'stg mail <patches...>' or re-organise\nthe stack with push/pop so that you only keep the patches to be merged\nby Linus and tell him the HEAD value to merge. Even if you modify the\nstack afterwards, the HEAD value still represents the stack state at\nthat moment (unless you run 'git prune' and the value is not saved in\na file under refs/heads/).\n\nStGIT also support cherry-picking via 'stg import --commit=...' if you\nwant to create a separate branch.\n\nBut what I understand from Petr's e-mail is that re-basing the patches\nis not acceptable for a repository which is public. One option would\nbe to keep a private tree where the patches are re-based and a public\none which periodically pulls from Linus' tree and your private\none. Not sure how complicated the commit graph would look.\n\n> It's ended up saving me a lot of time (I used to do what Karsten is\n> trying to do with bitkeeper, and it had the same issues that he is\n> running into) and would recommend this situation for anyone who wants to\n> keep patches from being merged immediatly (like he is trying to do.)\n\nI only maintain a small set of patches (< 20) and send some of them\nupstream via RMK. I would recommend StGIT :-) as an alternative to\nquilt (I don't say a better one but some people might like it).\n\n-- \nCatalin\n"},{"id":"9096","messageId":"20050922095845.GE21019@pasky.or.cz","threadId":"1899","inReplyTo":"tnxpsr12zu0.fsf@arm.com","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2005-09-22T09:58:45Z","receivedAt":"2005-09-22T09:58:45Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Sep 22, 2005 at 11:53:27AM CEST, I got a letter\nwhere Catalin Marinas <catalin.marinas@gmail.com> told me that...\n> But what I understand from Petr's e-mail is that re-basing the patches\n> is not acceptable for a repository which is public. One option would\n> be to keep a private tree where the patches are re-based and a public\n> one which periodically pulls from Linus' tree and your private\n> one. Not sure how complicated the commit graph would look.\n\nIt would be pretty ugly since the same patches would re-appear in the\ncommit graph in another incarnations many times.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nVI has two modes: the one in which it beeps and the one in which\nit doesn't.\n"},{"id":"9099","messageId":"tnxll1p2yqb.fsf@arm.com","threadId":"1899","inReplyTo":"20050922095845.GE21019@pasky.or.cz","subject":"Re: [Core GIT] Long-term cherrypicking","fromName":"Catalin Marinas","fromEmail":"catalin.marinas@gmail.com","sentAt":"2005-09-22T10:17:16Z","receivedAt":"2005-09-22T10:17:16Z","isPatch":false,"sender":{"key":"catalin.marinas@gmail.com","avatar":null},"body":"Petr Baudis <pasky@suse.cz> wrote:\n> Dear diary, on Thu, Sep 22, 2005 at 11:53:27AM CEST, I got a letter\n> where Catalin Marinas <catalin.marinas@gmail.com> told me that...\n>> But what I understand from Petr's e-mail is that re-basing the patches\n>> is not acceptable for a repository which is public. One option would\n>> be to keep a private tree where the patches are re-based and a public\n>> one which periodically pulls from Linus' tree and your private\n>> one. Not sure how complicated the commit graph would look.\n>\n> It would be pretty ugly since the same patches would re-appear in the\n> commit graph in another incarnations many times.\n\nTrue. And re-basing patches in the public branch would generate the\nsame complicated graph in people's repositories which were pulling\nfrom it. If this is not desirable, StGIT on the public branch wouldn't\nhelp much (and neither quilt since quilt patches are not visible via\nGIT anyway).\n\nAs Junio said, using 'git cherry-pick' (or 'stg import --commit') on a\nprivate branch would help organising the patches to be sent\nupstream. One problem with this approach is that the conflicts\ngenerated by merging with Linus in the public branch (and already\nfixed there) would need to be re-solved in the private one after\ncherry-picking.\n\n-- \nCatalin\n"}]}