{"thread":{"id":"27983","subject":"Branch dependencies","startedAt":"2011-08-01T12:19:46Z","lastAt":"2011-08-04T17:38:39Z","messageCount":8,"participants":["martin f krafft","Bert Wesarg","Nicolas Sebrecht"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"172537","messageId":"20110801121946.GA575@fishbowl.rw.madduck.net","threadId":"27983","inReplyTo":null,"subject":"Branch dependencies","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2011-08-01T12:19:46Z","receivedAt":"2011-08-01T12:19:46Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"Dear list,\n\nWe are trying to approach a functionality I call \"branch\ndependencies\". Essentially, the idea is rooted in distro\ndevelopment, but probably applies to normal development too.\n\nFor instance, you might have a feature branch off upstream. Once\nupstream advances, you should merge upstream into your feature\nbranch (or rebase your feature branch) to ensure that you are\nworking on the right assumptions.\n\nAnother case might be a feature you want to write, which depends on\ntwo (or more) feature branches of other people, e.g. say you need\na new configuration option introduced in feature branch\n\"conf-option\", and you also base your work on the \"speedup\" branch,\nbecause otherwise the software is too slow for your new feature. If\none or both feature branches advance, you should merge, as before.\n\nIt would be useful if Git could help you keep track of what needs to\nbe merged, especially as the number of feature branches and\ndependencies increases. TopGit was Petr's answer to this challenge,\nand it works fine, albeit it's a bit too complex and we find it\nscaring new contributors, rather than making their lives easier.\n\nTherefore I am investigating ways in which to simplify/improve\nTopGit. In doing so, I discovered that you guys made a lot of\nprogress in Git since the last time I had time to really dive into\nyour tool.\n\nIf you would permit me, then please let me ask if you can think of\nGit functionality that could be useful in achieving what we're\ntrying to do.\n\nFor instance, there is git-branch --set-upstream, which could be\nuseful, but it only seems to support one \"dependency\" (which is\nusually the remote ref being tracked by a branch).\n\nOne challenge seems to be that a branching point has no information\nabout which of the children continues as mainline — this information\nis only available in a project's workflow policy. For instance:\n\n  o--o--o--●    upstream\n      \\\n       o--●     feature\n\nBut his is actually just the same as\n\n      ,o--●     feature\n  o--o\n      `o--●     upstream\n\nand Git has no way to find out whether it is now \"feature\" that\nneeds a merge of \"upstream\" or vice versa.\n\nIt is thus necessary somehow to store the (project-specific)\ndependency information, to be able to (automatically) determine that\n\"feature\" needs an update in the above.\n\nTopGit does this using a file in the worktree, but many of us find\nthis suboptimal.\n\nI have had the following alternative ideas:\n\n  1. a separate DAG, like Git notes. The problem is that this\n     requires additional refspecs to be set up for merges and\n     fetches;\n\n  2. like (1.), but a ref in refs/heads/* (like pristine-tar). This\n     could be considered ugly as it exposes too much implementation\n     detail;\n\n  3. information stored in the Git commit messages. Again, too much\n     implementation detail exposed and ugly;\n\n  4. additional Git commit headers — this is not supported at the\n     moment (cf. commit generation discussion);\n\n  5. orphan parent nodes to certain commits, in which these data can\n     be stored, e.g.\n\n       o--o--o--●\n            /\n           o\n\n     To fetch these data, one would walk up the DAG until one finds\n     a multi-parent commit with a parent having a specific format\n     (somewhat brittle…)\n\n     To me, this is the least offensive, but it does expose\n     implementation details in the commit history.\n\nHow else could I store the dependency information, keeping in mind\nthat I might have more than one dependency?\n\nAnd the original question remains: given such dependency\ninformation, which Git tools could I harness for the purpose, trying\nto reduce the amount of additional code needed?\n\nThanks,\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \nit is better to have loft and lost\nthan to never have loft at all.\n                                                       -- groucho marx\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"172655","messageId":"CAKPyHN0kAJ-MVsrXam5NjsOYkta4nsSrZUvKoMSi-FeRUSuLEw@mail.gmail.com","threadId":"27983","inReplyTo":"20110801121946.GA575@fishbowl.rw.madduck.net","subject":"Re: Branch dependencies","fromName":"Bert Wesarg","fromEmail":"bert.wesarg@googlemail.com","sentAt":"2011-08-02T13:06:40Z","receivedAt":"2011-08-02T13:06:40Z","isPatch":false,"sender":{"key":"bert.wesarg@googlemail.com","avatar":"https://avatars.githubusercontent.com/u/111934?v=4"},"body":"Hi,\n\nOn Mon, Aug 1, 2011 at 14:19, martin f krafft <madduck@madduck.net> wrote:\n> Dear list,\n\nwhile I appreciate, that you dig this topic up. I think you are trying\nto solve the wrong problem first. My main problem with the TopGit\napproach is, that you can't freely change the dependencies of a topic.\nThis may be not the most common case in distro development. But in my\neyes more problematic than maintaining the meta data.\n\nPlease note, that I'm more than aware the the TopGit approach for\nhandling the meta data is awful and we need a new way here. My\npersonal impression is, that the git notes is the best we can have.\n\nFor my first mentioned problem, I think a new 'system' needs to be\n'rebase' based, not merge based like TopGit.\n\nBert\n"},{"id":"172689","messageId":"20110802190806.GA16674@fishbowl.rw.madduck.net","threadId":"27983","inReplyTo":"CAKPyHN0kAJ-MVsrXam5NjsOYkta4nsSrZUvKoMSi-FeRUSuLEw@mail.gmail.com","subject":"Re: Branch dependencies","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2011-08-02T19:08:06Z","receivedAt":"2011-08-02T19:08:06Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:\n> while I appreciate, that you dig this topic up. I think you are trying\n> to solve the wrong problem first. My main problem with the TopGit\n> approach is, that you can't freely change the dependencies of a topic.\n> This may be not the most common case in distro development. But in my\n> eyes more problematic than maintaining the meta data.\n\nHello Bert, thank you for taking the time to respond!\n\nCould you please try to illuminate me a bit on a use-case of\nchanging dependencies? I am aware that TopGit has had a problem with\nchanging dependencies due to renamed branches, and I think I have\na solution to that (encode the dependent ref, not the branch head),\nbut I cannot come up with a use case for freely changing\ndependencies just like that.\n\n> For my first mentioned problem, I think a new 'system' needs to be\n> 'rebase' based, not merge based like TopGit.\n\nThe problem with rebasing is that you cannot publish the branches.\n\nHowever, maybe I am simply not seeing the light here. Do you have\nsome further ideas about what this would be like? Please keep in\nmind that what I seek is not just a way to bring feature branches\nup-to-date with upstream, but also to have those branches be shared\namong developers.\n\nThanks,\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \n\"gott ist tot! und wir haben ihn getötet.\"\n                                                 - friedrich nietzsche\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"172720","messageId":"CAKPyHN0EsXMKQ2g7ONaO4yw2ioPbMhg8XCsmB20je=O1DDeE5Q@mail.gmail.com","threadId":"27983","inReplyTo":"20110802190806.GA16674@fishbowl.rw.madduck.net","subject":"Re: Branch dependencies","fromName":"Bert Wesarg","fromEmail":"bert.wesarg@googlemail.com","sentAt":"2011-08-02T23:25:42Z","receivedAt":"2011-08-02T23:25:42Z","isPatch":false,"sender":{"key":"bert.wesarg@googlemail.com","avatar":"https://avatars.githubusercontent.com/u/111934?v=4"},"body":"Hi,\n\nOn Tue, Aug 2, 2011 at 21:08, martin f krafft <madduck@madduck.net> wrote:\n> also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:\n>> while I appreciate, that you dig this topic up. I think you are trying\n>> to solve the wrong problem first. My main problem with the TopGit\n>> approach is, that you can't freely change the dependencies of a topic.\n>> This may be not the most common case in distro development. But in my\n>> eyes more problematic than maintaining the meta data.\n>\n> Hello Bert, thank you for taking the time to respond!\n>\n> Could you please try to illuminate me a bit on a use-case of\n> changing dependencies? I am aware that TopGit has had a problem with\n> changing dependencies due to renamed branches, and I think I have\n> a solution to that (encode the dependent ref, not the branch head),\n> but I cannot come up with a use case for freely changing\n> dependencies just like that.\n\nNot each feature branch may end up in master. And there does not need\nto be one feature branch which depends on all other features. For the\nlatter you have probably an empty feature branch which just depends on\nall features. I call this branch mostly 'tip'. Removing a feature\nbranch from the tips dependency list only with merges can't be done\nright now, and the proposed solutions never reached a usable state.\n\nMy second usecase is to convert a big quilt patch series into TopGit.\nSuch big Quilt patches have mostly an artificial dependency to its\npredecessors. Removing these artifical dependencies makes it necessary\nto remove dependencies from patches.\n\n>\n>> For my first mentioned problem, I think a new 'system' needs to be\n>> 'rebase' based, not merge based like TopGit.\n>\n> The problem with rebasing is that you cannot publish the branches.\n\nThat doesn't hold you back to publish them. But the other side need to\nknow how to deal with them.\n\nSaying that doesn't mean I know a good way to deal with them. I mostly\nend up using plumbing commands to deal with this.\n\n>\n> However, maybe I am simply not seeing the light here. Do you have\n> some further ideas about what this would be like? Please keep in\n> mind that what I seek is not just a way to bring feature branches\n> up-to-date with upstream, but also to have those branches be shared\n> among developers.\n\nI think that having the TopGit philosophy of one feature branch is one\npatch, you can handle an rebased upstream. Thinking of a feature\nbranch as a series of patches makes this way harder. But I would like\nto have this philosophy.\n\nBert\n\n>\n> Thanks,\n>\n"},{"id":"172740","messageId":"20110803095103.GA27996@fishbowl.rw.madduck.net","threadId":"27983","inReplyTo":"CAKPyHN0EsXMKQ2g7ONaO4yw2ioPbMhg8XCsmB20je=O1DDeE5Q@mail.gmail.com","subject":"changing the set of dependencies (was: Branch dependencies)","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2011-08-03T09:51:03Z","receivedAt":"2011-08-03T09:51:03Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:\n> Not each feature branch may end up in master. And there does not need\n> to be one feature branch which depends on all other features. For the\n> latter you have probably an empty feature branch which just depends on\n> all features. I call this branch mostly 'tip'. Removing a feature\n> branch from the tips dependency list only with merges can't be done\n> right now, and the proposed solutions never reached a usable state.\n> \n> My second usecase is to convert a big quilt patch series into TopGit.\n> Such big Quilt patches have mostly an artificial dependency to its\n> predecessors. Removing these artifical dependencies makes it necessary\n> to remove dependencies from patches.\n\nDear Bert, thank you for your reply.\n\nOkay, understood. Yes, I agree entirely, the set of dependencies\nneeds to be mutable. And they must not be invalidated by branch\nrenames. Therefore, we really need some sort of other way to\nidentify branches.\n\nIs there a way other than lamenting that refs did not get UUIDs\nassigned to them from the early days onwards? ;)\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \n\"if one cannot enjoy reading a book over and over again,\n there is no use in reading it at all.\"\n                                                        -- oscar wilde\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"172741","messageId":"20110803100156.GB27996@fishbowl.rw.madduck.net","threadId":"27983","inReplyTo":"CAKPyHN0EsXMKQ2g7ONaO4yw2ioPbMhg8XCsmB20je=O1DDeE5Q@mail.gmail.com","subject":"TopGit with rebased branches, problems with publishing (was: Branch dependencies)","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2011-08-03T10:01:56Z","receivedAt":"2011-08-03T10:01:56Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:\n> >> For my first mentioned problem, I think a new 'system' needs to be\n> >> 'rebase' based, not merge based like TopGit.\n> >\n> > The problem with rebasing is that you cannot publish the branches.\n> \n> That doesn't hold you back to publish them. But the other side need to\n> know how to deal with them.\n> \n> Saying that doesn't mean I know a good way to deal with them. I mostly\n> end up using plumbing commands to deal with this.\n\nLet me address this in terms of patch queue branches. Obviously, you\nare talking topic branches, but wrt rebase-publish, the two are\nidentical. Patch queue branches are essentially quilt imports, e.g.\none-commit-one-patch, usually edited with git rebase -i (which some\npeople consider to be the better quilt).\n\nThe problem that led me away from patch queue branches is that\n— albeit unlikely in our context (distro packaging) — it is possible\nthat you and I both rewrite the patch queue at the same time. The\nfirst one to push will then have his changes overwritten by the\nsecond push — in order to push the rebased ref,\nreceive.denyNonFastForwards needs to be off.\n\nUnfortunately, I can't see a remedy to this, apart from tagging each\nand every patch queue branch, but that does not solve the problem of\nhaving to merge changes made simultaneously. It could go something\nlike this:\n\n  1. ensure that noone has pushed an updated ref since you last\n     fetched.\n\n  2. if a new ref is found, download it and rebase your own patch\n     queue on top of it, consolidating the changes.\n\n  3. tag and push your new ref.\n\nBut there's a race condition in this, and so we're back to square\none.\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \nuʍop ǝpısdn sı ɹoʇıuoɯ ɹnoʎ\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"172742","messageId":"20110803101022.GC27996@fishbowl.rw.madduck.net","threadId":"27983","inReplyTo":"CAKPyHN0EsXMKQ2g7ONaO4yw2ioPbMhg8XCsmB20je=O1DDeE5Q@mail.gmail.com","subject":"Tracking topic branches with rebases vs. merges (was: Branch dependencies)","fromName":"martin f krafft","fromEmail":"madduck@madduck.net","sentAt":"2011-08-03T10:10:22Z","receivedAt":"2011-08-03T10:10:22Z","isPatch":false,"sender":{"key":"madduck@madduck.net","avatar":null},"body":"also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.03.0125 +0200]:\n> I think that having the TopGit philosophy of one feature branch is\n> one patch, you can handle an rebased upstream. Thinking of\n> a feature branch as a series of patches makes this way harder. But\n> I would like to have this philosophy.\n\nLet me just make sure I understand you right: you do not like the\nfollowing branch management strategy:\n\n       o--o--o--+--o--o--+--o-.\n      /        /        /      \\\n  o--o--o--o--o--o--o--o--o--o--o--●\n\nand you would prefer if the topic branch was rebased all along (like\nwhat git.git advocates), and then transferred to mainline with\ngit-format-patch/git-send-e-mail/git-am (or ff-merged).\n\nI am torn on this issue. On the one hand, I do not find the above to\nbe so problematic, especially not if the downstream merges happen\nonly infrequently (undo merges that do not produce conflicts, as\nadvocated by gitworkflows(7)).\n\nOn the other hand, I completely agree with you that rebasing is much\nnicer, and it certainly works without publishing the branch. In\ngit.git, people seem to maintain their own branches and send patch\nsets to the mailing list for review.\n\nBut how could you and I truly cooperate on a feature branch without\nusing merges, and without establishing a lock-unlock protocol to\nensure only sequential updates of the refs?\n\nThanks,\n\n-- \nmartin | http://madduck.net/ | http://two.sentenc.es/\n \n\"alles gackert, aber wer will noch still\n auf dem nest sitzen und eier zu brüten?\"\n                                      -- friedrich wilhelm nietzsche\n \nspamtraps: madduck.bogus@madduck.net\n"},{"id":"172937","messageId":"20110804173838.GA10298@vidovic.ultras.lan","threadId":"27983","inReplyTo":"20110802190806.GA16674@fishbowl.rw.madduck.net","subject":"Re: Branch dependencies","fromName":"Nicolas Sebrecht","fromEmail":"nicolas.s.dev@gmx.fr","sentAt":"2011-08-04T17:38:39Z","receivedAt":"2011-08-04T17:38:39Z","isPatch":false,"sender":{"key":"nicolas.s.dev@gmx.fr","avatar":null},"body":"The 02/08/11, martin f krafft wrote:\n> also sprach Bert Wesarg <bert.wesarg@googlemail.com> [2011.08.02.1506 +0200]:\n\n> > For my first mentioned problem, I think a new 'system' needs to be\n> > 'rebase' based, not merge based like TopGit.\n> \n> The problem with rebasing is that you cannot publish the branches.\n\nBut you may make public the way to set up the builded branch. Say you\nshare A and B branches and you build M branch on top of them. You are\nable to build M as long as others know M rely on A and B.\n\nIn a situation like\n\n      a--+       (A)\n         M--y--z (M)\n      b--+       (B)\n\nor\n\n   a--+          (A)\n      b--+       (B)\n         M--y--z (M)\n\nM is the builded branch. Commits y and z should not be public, IMHO.\n\n-- \nNicolas Sebrecht\n"}]}