{"thread":{"id":"18975","subject":"top-git, idea .topbase replacing refs/top-bases, implementation issues and questions..","startedAt":"2009-04-20T20:50:23Z","lastAt":"2009-04-23T23:11:24Z","messageCount":3,"participants":["Marc Weber","Uwe Kleine-König"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"111780","messageId":"20090420205023.GB2005@gmx.de","threadId":"18975","inReplyTo":null,"subject":"top-git, idea .topbase replacing refs/top-bases, implementation issues and questions..","fromName":"Marc Weber","fromEmail":"marco-oweber@gmx.de","sentAt":"2009-04-20T20:50:23Z","receivedAt":"2009-04-20T20:50:23Z","isPatch":false,"sender":{"key":"marco-oweber@gmx.de","avatar":null},"body":"introduction:\n\ntopgit (tg) is a tool keeping track of patches on top of branches so that you\ncan work on those patches before comitting them upstream.\n\nSo you can do\n  tg create dep1 dep2\nto make your new topic branch depend on the two branches dep1, dep2.\n\ntg first merges dep1 dep2 into a base. On top of this base you start\ncomitting your changes. when following upstream a new base is created by\nmerging the old base with dep1, then dep2 again. The new base is merged\ninto your topic branch.\n\nThis requires keeping track of the base. That has been done by adding an\nextra references to .git/refs/top-bases/TG_NAME.\nproblems: You have to push the branch and the base to the remote\nlocation. If you clone a top git related branch you have to get the\nbases as well (tg remote --populate does this for you).\n\nUwe Kleine-König (current maintainer?) has proposed no longer tracking\nthe topgit branch base using a ref but adding antoher .top* file .topbase\ncontaining the hash of the base. Neither pushing nor pulling causes\ntrouble because the base is always contained in the topic branch. So no\nneed to extra push/ pull contents of that branch. Another benefit of\nthis change is that you can easily make a top git branch out of an\nordinary branch by adding those 3 files (.topbase, .topmsg, .topdeps).\n\nI don't see any problems implementing this except one. I'm not sure in\nwhich way this has been adressed in the past by topgit:\n\n\nHave a look at the following illustration:\nhttp://mawercer.de/~marc/tg.svg (32kb).\n\nImagine two independant clones of a topgit branch (red, green).\nNow users run tg update at slightly different points in time. So both\nwill get a different new base (B2 and B3) of the clone topgit branch.\nNow red merges the topic branch (merge 1). Then a third user (blue)\nmerges that merge (merge 2). If a merge strategy \"ours\" is used the way\nits done for the other .top* files the blue user's base won't know about\nB3.\n\nFirst merge we have a conflict:\n\nIf B3 and B2 are different a merge conflict occurs:\n.topbase:\n<<<<\nhash1\n====\nhash2\n>>>>\n\nNow the two bases should be merged to a new base (Bnew).\nThat new base should be written to .topbase and everything is fine\nagain?\n\nIs there a way to specify such a merge action:\nextract both hashes (easy)\nmerge them and put the resulting hash back into .topbase.\nOf course this merge action must handle merge conflicts in a way which\nis similar to what tg already does. tg starts a subshell and asking the\nuser to fix the conflicts.\n\nEven if there is a nice way to implement this merge action this doesn't\nseem to solve all trouble?\nThe Bnew base is neither part of the red nor the green topic branch.\nThus it can't be pushed somewhere by pushing red or green only\ncontaining the ref in .topbase?\n\nSo in the end:\nHave I missed a point ?\n\nSo are there more  advantages having a self contained top git branch\ncompared to beeing able to merge bases?\n\nMaybe merging bases doesn't occur that often in practise?\n\nSo if we don't want to give up beeing able to merge bases I \ntill want to see a git push branch command also pushing the base.\nCurrently you can do $git push which will push *all* bases and branches.\n\nProbably I've missed some small details while explaining this idea and\ntalking about its issues. So please do ask.\n\nSincerly\nMarc Weber\n"},{"id":"111859","messageId":"20090421090413.GA5718@pengutronix.de","threadId":"18975","inReplyTo":"20090420205023.GB2005@gmx.de","subject":"Re: top-git, idea .topbase replacing refs/top-bases, implementation issues and questions..","fromName":"Uwe Kleine-König","fromEmail":"u.kleine-koenig@pengutronix.de","sentAt":"2009-04-21T09:04:13Z","receivedAt":"2009-04-21T09:04:13Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hi Marc,\n\nI supplement and correct your description a bit, not to be pedantic but\nto give the other readers a better overview.\n\nOn Mon, Apr 20, 2009 at 10:50:23PM +0200, Marc Weber wrote:\n> introduction:\n> \n> topgit (tg) is a tool keeping track of patches on top of branches so that you\n> can work on those patches before comitting them upstream.\nAnd IMHO more importantly you can update your branches after feedback\nfrom upstream without rebasing.\n\n> So you can do\n>   tg create dep1 dep2\nThis should read\n\ttg create TG_NAME dep1 dep2\n\n> to make your new topic branch depend on the two branches dep1, dep2.\n> \n> tg first merges dep1 dep2 into a base. On top of this base you start\n> comitting your changes. when following upstream a new base is created by\n> merging the old base with dep1, then dep2 again. The new base is merged\n> into your topic branch.\nThe patch represented by TG_NAME is always the diff between the base and\nthe topic branch.\n \n> This requires keeping track of the base. That has been done by adding an\n> extra references to .git/refs/top-bases/TG_NAME.\n> problems: You have to push the branch and the base to the remote\n> location. If you clone a top git related branch you have to get the\n> bases as well (tg remote --populate does this for you).\ntg remote adds some config to your repo such that a later git fetch gets\nthe top-bases, too.\n\n> Uwe Kleine-König (current maintainer?\nyes\n\n>                                      ) has proposed no longer tracking\n> the topgit branch base using a ref but adding antoher .top* file .topbase\n> containing the hash of the base.\nI think for now using both is a good start.  There are too many places\nin topgit that depend on the topbase branch.\n\n>                                  Neither pushing nor pulling causes\n> trouble because the base is always contained in the topic branch. So no\n> need to extra push/ pull contents of that branch. Another benefit of\n> this change is that you can easily make a top git branch out of an\n> ordinary branch by adding those 3 files (.topbase, .topmsg, .topdeps).\nhhm, not adding .topbase but another (git) branch isn't much harder, is\nit?\n \n> I don't see any problems implementing this except one. I'm not sure in\n> which way this has been adressed in the past by topgit:\n> \n> \n> Have a look at the following illustration:\n> http://mawercer.de/~marc/tg.svg (32kb).\nEither I don't interpret your image correctly or it's not exact.\nB1 is a base?  Where is the corresponding topic-branch that both cloners\nshould have in common?  Further it's not clear to me if merge1 and\nmerge2 are bases or topic branches.\n\nLet me try an ascii-history with slightly different names ...\n\n> Imagine two independant clones of a topgit branch (red, green).\nThis is the state cloned by red and green:\n\n\t                            T.origin\n\t                           /\n\tdep1.1 --- dep1.2--B.origin\n\t                  /\n\tdep2.1 --- dep2.2-\n\nwhere B.something is a topbase, T.thesamesomething is the corresponding\ntopic branch.\n\n> Now users run tg update at slightly different points in time. So both\n> will get a different new base (B2 and B3) of the clone topgit branch.\nThen it looks as follows (simplified to only a single dependency dep1\nand assuming they get different commits of dep1, not only different\ntimes, as suggested by your image).\n\n\t                      ,------------------------T.green\n\t                     /                          /\n\t             ,-------------------------------B.green\n\t            /      /                        /\n\t           /   T.origin----T.red           /\n\t          /   /           /               /\n\t        B.origin-------B.red             /\n\t      /               /                 /\n\tdep1.2----------dep1.3----------------dep1.4\n\nso my B.red is your B2, my B.green is your B3.\n\n> Now red merges the topic branch (merge 1).\nThe image suggests red merges green's work.\n\nOK, I fail to make a picture for that, anyhow then we have:\n\n\tB.red2 = B.red + B.green\n\tT.red2 = T.red + T.green + B.red2\n\nwhere + stands for a merge[1][2].\n\nAnd obviously T.red2:.topbase has to contain $(git rev-parse B.red2)\n\n>                                            Then a third user (blue)\n> merges that merge (merge 2).\nOK, blue having T.origin merges T.red2, then we get:\n\n\tB.blue = B.origin + B.red2\n\tT.blue = T.origin + T.red2 + B.blue\n\n>                              If a merge strategy \"ours\" is used the way\n> its done for the other .top* files the blue user's base won't know about\n> B3.\n\"ours\" is wrong for .topbase.  You have to merge to get the right\ncontent.\n\n> First merge we have a conflict:\n> \n> If B3 and B2 are different a merge conflict occurs:\n> .topbase:\n> <<<<\n> hash1\n> ====\n> hash2\n> >>>>\nI think it's not helpful to thing about merge conflicts, you simply need\nto handle .topbase explicitly.\n\n> Now the two bases should be merged to a new base (Bnew).\n> That new base should be written to .topbase and everything is fine\n> again?\n> \n> Is there a way to specify such a merge action:\n> extract both hashes (easy)\n> merge them and put the resulting hash back into .topbase.\n> Of course this merge action must handle merge conflicts in a way which\n> is similar to what tg already does. tg starts a subshell and asking the\n> user to fix the conflicts.\nActually I don't like this subshelling.  git does it better, it saves\nthe state somewhere in .git and exits.  I'm sure that people who's shell\nis != /bin/sh don't like that we spawn a sh.\n \n> Even if there is a nice way to implement this merge action this doesn't\n> seem to solve all trouble?\nI don't see any trouble, but maybe it's me who missed something.\n\n> The Bnew base is neither part of the red nor the green topic branch.\n> Thus it can't be pushed somewhere by pushing red or green only\n> containing the ref in .topbase?\nI fail to understand that.  only blue has Bnew, so of course neither red\nnor green can push it. ???\n\n> So in the end:\n> Have I missed a point ?\n> \n> So are there more  advantages having a self contained top git branch\n> compared to beeing able to merge bases?\nYou can have both.\n\n> Maybe merging bases doesn't occur that often in practise?\nyou need it whenever you merge a remote.  There is no way around it.\n\n> So if we don't want to give up beeing able to merge bases I \n> till want to see a git push branch command also pushing the base.\n> Currently you can do $git push which will push *all* bases and branches.\nAlternatively you can push the branch+base explicitly to have more\ncontrol.  OK, not very comfortable, though you could wrap that in a tg\npush command.\n\nBest regards\nUwe\n\n[1] I have not thought deeply about which is the best of the various\npossibilies to update T when merging, here are some of them:\n\n\t1) T.mynew = (T.myold + T.tomerge) + B.mynew\n\t2) T.mynew = T.myold + (T.tomerge + B.mynew)\n\t3) T.mynew = (T.myold + B.mynew) + T.tomerge\n\t4) T.mynew = T.myold + (B.mynew + T.tomerge)\n\t5) T.mynew = T.myold + T.tomerge\n\n5) is out because then T.mynew doesn't contain B.mynew, but it only\nmisses a single merge, namely B.myold + B.tomerge.  Both bases as such\nare already included in T.myold + T.tomerge.  So this is only a \"would\nbe nice\".\nAd hoc I'd choose 3) but this needs some more thinking.\nJust from reading the code, I'd say topgit currently does 4)\n\n[2] and note, that + has to behave differently in different situations\nfor the .top* files.  The following merges can happen\n\n\ta) merge a base into a topic\n\tb) merge two bases\n\tc) merge two topics\n\nand depending on this the following low-level merge drivers should be\nused for .topmsg and .topdeps:\n\n\ta): ours\n\tb): don't care (delete?)\n\tc): text\n\nIt would be nice if the topbases didn't contain any .top* files, this\nonly has some overhead if a patch has single (tgish) dependency because\nthen you cannot just use this branch as base.\n\n-- \nPengutronix e.K.                              | Uwe Kleine-König            |\nIndustrial Linux Solutions                    | http://www.pengutronix.de/  |\n"},{"id":"112144","messageId":"20090423231124.GC17647@gmx.de","threadId":"18975","inReplyTo":"20090421090413.GA5718@pengutronix.de","subject":"tg-update rewrite proposal","fromName":"Marc Weber","fromEmail":"marco-oweber@gmx.de","sentAt":"2009-04-23T23:11:24Z","receivedAt":"2009-04-23T23:11:24Z","isPatch":false,"sender":{"key":"marco-oweber@gmx.de","avatar":null},"body":"Hi Uwe,\n\nI came up with this proposal for a rewrite for tg-update.sh now:\nI hope this fits your ideas about tg update as well..\n\nNote that second \"for each dependency [1]\" below allowing handling updates\nagainst multiple remote repos.\n\nI hope that its easy to understand this short documentation draft.\n\nSincerly\nMarc Weber\n\n# documentation:\n\n# Uwe Klein Koenig suggests keeping updating with remote branches\n# which corresponds to \"git pull\" in default git separate from\n# updating branches with deps (normal branhces and/or tg branches)\n# \n# Because of [3] my implementation of tg update starts at the bottom going up\n# first updating with remotes then with deps by default.\n# However you can choose to only update with remotes\n# or with deps only using --remotes-only or --local-only\n# I hope this satisfies everyone.\n\n# Providing a subshellless update experience can only be done by saving\n# some state the way git-rebase does. STATE_DIR will be used to save this state\n#\n# What's saved in STATE_DIR ?\n#\n# $STATE_DIR/$BRANCHES_VISITING_ORDER is a list of dependency names [1] which is generated\n# by traversing the dependency tree of branches form bottom up\n#\n#  Example:\n#     A    B  C   D\n#      \\  /    \\ /\n#       t/1   t/2\n#         \\   /\n#       t/working\n#\n#  will result in\n#\n#     A\n#     B\n#     t/1\n#     C\n#     D\n#     t/2\n#     t/working\n\n# Then the first line is picked and the specified branch is updated.\n# updating is done in this order [3]:\n#\n#   first:\n#   if not --local-only\n#      foreach remote location:\n#        - merge base with remote tg branch (thereby also including the remote base)\n#          update the base ref [2]\n#        - update tg [*]\n#    \n#   then:\n#   if not --remote-only\n#      for each dependency [1]\n#         - merge base with dep [2]\n#      - update tg [*]\n#    \n#     [*]: merge new base with topgit branch updating the topgit ref.\n#          After this step the base is always contained in the current branch\n#\n# Similar to $STATE_DIR/BRANCHES_VISITING_ORDER a file $STATE_DIR/$BRANCH_UPDATE_TASKS is\n# created containing the subtasks of a branch update.\n# \n# Example file with additional comment lines, two remote locations:\n#\n#   # first:\n#   merge_base_with: $REMOTE_HASH\n#   update_tg\n#   merge_base_with: $REMOTE_HASH2\n#   update_tg\n#\n#   # then:\n#   merge_base_with: $DEP1_HASH\n#   update_tg\n#   merge_base_with: $DEP2_HASH\n#   update_tg\n#   merge_base_with: $DEP3_HASH\n#   update_tg\n\n\n\n# So the overall implementation is easy and looks like this:\n#\n# if !exists $STATE_DIR/BRANCHES_VISITING_ORDER\n#   then create file, goto else\n#   else pop_or_continue\n#\n# pop_or_continue:\n# if !exists $STATE_DIR/$BRANCH_UPDATE_TASKS\n#   then create file, goto else\n#   else pop_or_continue_subtask\n#\n#  pop_or_continue_subtask:\n#  ...\n\n# [1] note for users who knew the old code: not including remotes\n#\n# [2] This means creating a commit to the topic branch changing the .topbase file only\n#\n# [3] I think the following statements make sense:\n#     *) You want to update against remote first. Or you want to have a way to do so.\n#        Why? The remote may already have done some of the dirty merge work for you.\n#        If you resolving the same conflicts first by updating with deps and then merge\n#        with remote -> hell!\n#     *) You don't want to run tg update multiple times. Thus the update strategy has\n#        to start at the bottom and go up to the current branch\n\n\n\n# state dir and filenames:\nSTATE_DIR=.git/tg-update-in-progress \nBRANCHES_VISITING_ORDER=branches-visiting-order\nBRANCH_UPDATE_TASKS=branch-update-tasks\n\n## Parse options\n\nif [ -n \"$1\" ]; then\n        echo \"Usage: tg [--local-only] [--remote-only] update\" >&2\n        exit 1\nfi\n"}]}