{"thread":{"id":"25397","subject":"[TopGit PATCH] t/depend-add-using-export","startedAt":"2010-10-09T01:43:18Z","lastAt":"2010-10-09T12:21:11Z","messageCount":6,"participants":["Olaf Dabrunz","Bert Wesarg"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"153044","messageId":"1286588598-4005-1-git-send-email-odabrunz@gmx.net","threadId":"25397","inReplyTo":null,"subject":"[TopGit PATCH] t/depend-add-using-export","fromName":"Olaf Dabrunz","fromEmail":"odabrunz@gmx.net","sentAt":"2010-10-09T01:43:18Z","receivedAt":"2010-10-09T01:43:18Z","isPatch":true,"sender":{"key":"odabrunz@gmx.net","avatar":null},"body":"When dependencies can be removed as well as added, tg depend add\nneeds to make sure that the new dependency can bring in changes\nfrom a branch that has previously been removed as a dependency\nfrom the current TopGit branch.\n\nThis implementation uses an exported branch set up by tg export,\nand merges the new dependency into the commit that corresponds to\nthe current base. Using the exported branch in the merge has the\nadvantage that removed dependencies do not appear as parents, and\nthe merge base selected by git merge does not include changes from\na removed dependency. As a result, these changes can be merged in\nagain if the new dependency brings in these changes.\n\nThe tree of the merge commit is then used to create the next\ncommit on the TopGit base branch.\n\nUwe Kleine-König had the idea to use tg export here.\n\nSigned-off-by: Olaf Dabrunz \n\n---\n\nThis is an experimental implementation. It did not get much testing, mainly\nbecause testing ran into a fatal \"tg export\" bug with my test repository. \"tg\nexport\" could be convinced to continue past the error with \"bash -x\", and that\nis how this patch was tested.\n\nThis is also my first feature patch for TopGit, so I expect people to have\ncomments.\n\nKnown issues:\n - .top* files in the base should probably be excluded with pretty_tree\n - handle tg export producing zero commits for a dependency\n\n\n tg-depend.sh | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 80 insertions(+), 0 deletions(-)\n\ndiff --git a/tg-depend.sh b/tg-depend.sh\nindex 474ccda..4d1572d 100644\n--- a/tg-depend.sh\n+++ b/tg-depend.sh\n@@ -60,6 +60,86 @@ depend_add()\n \tgrep -F -x -e \"$name\" \"$root_dir/.topdeps\" >/dev/null &&\n \t\tdie \"tg: $current_name already depends on $name\"\n \n+\ttmp_export=\"_tg_tmp/tg_depend\"\n+\ttrap 'git branch -D \"$tmp_export\" 2>/dev/null || true' EXIT\n+\n+\t# Create a separate history that does not contain parent pointers to\n+\t# removed dependencies. This makes sure that these dependencies can\n+\t# come in again if any of them were merged into the new dependency.\n+\t# (I.e., not having any parents leading to removed deps makes sure that\n+\t# \"git merge new_dep\" selects a merge base that does not already\n+\t# include the removed dependencies.)\n+\n+\t# tg export sets up a branch with the following properties:\n+\t# - For each of the exported TopGit branch and it's recursive TopGit\n+\t# dependencies, zero, one or two commits are created. If there is\n+\t# more than one dependency, one commit is created that contains the\n+\t# tree from the tip of the base branch (i.e., the merge of all\n+\t# current dependencies). If the TopGit patch branch is non-empty,\n+\t# another commit contains the collapsed patch.\n+\t# - Only current dependencies from .topdeps are considered; removed\n+\t# dependencies are not part of the history of the branch.\n+\t# - The .top* files are removed.\n+\t#\n+\t# tg export bails if the TopGit branch is not up-to-date.\n+\t$tg export \"$tmp_export\" || exit\n+\n+\t# Now checkout the commit representing the base of the current branch\n+\t# in the export, and use that to merge in the new dep, resulting in the\n+\t# new base tree.\n+\t# FIXME: also handle the zero commits case -- does TopGit actually make\n+\t# that case possible?\n+\tif branch_empty \"$current_name\"; then\n+\t\t# There is no patch commit. Use tmp_export directly.\n+\t\tgit checkout -q \"$tmp_export\"\n+\telse\n+\t\t# Set $tmp_export to it's parent commit, which has the merged\n+\t\t# deps without the current patch on top, and check that out.\n+\t\t# FIXME: keep the tree of the HEAD commit with the collapsed\n+\t\t# patch / new dep merge resolution and use it in the commit on\n+\t\t# the patch branch, instead of running tg update? -- this would\n+\t\t# re-use conflict resolutions (and other changes) -- but would\n+\t\t# git rerere not kick in anyway?\n+\t\tgit update-ref \"refs/heads/$tmp_export\" \"$tmp_export^\"\n+\t\tgit checkout -q \"$tmp_export\"\n+\tfi\n+\n+\t# merge the new dependency into the temp export\n+\tif ! git merge \"$name\"; then\n+\t\t# open up a shell to work on the conflict\n+\t\tinfo \"You are in a subshell. If you abort the merge,\"\n+\t\tinfo \"use \\`exit 1\\` to abort adding the dependency.\"\n+\t\tif ! sh -i >\"$root_dir/.topdeps\"\n \tgit add -f \"$root_dir/.topdeps\"\n \tgit commit -m\"New TopGit dependency: $name\"\n-- \ntg: (9404aa1..) t/depend-add-using-export (depends on: master)\n"},{"id":"153045","messageId":"AANLkTik6R89vPhwKyWOheHPkxQ9JPpmrQVmXon8Vn4+A@mail.gmail.com","threadId":"25397","inReplyTo":"1286588598-4005-1-git-send-email-odabrunz@gmx.net","subject":"Re: [TopGit PATCH] t/depend-add-using-export","fromName":"Bert Wesarg","fromEmail":"bert.wesarg@googlemail.com","sentAt":"2010-10-09T07:45:06Z","receivedAt":"2010-10-09T07:45:06Z","isPatch":true,"sender":{"key":"bert.wesarg@googlemail.com","avatar":"https://avatars.githubusercontent.com/u/111934?v=4"},"body":"On Sat, Oct 9, 2010 at 03:43, Olaf Dabrunz wrote:\n> When dependencies can be removed as well as added, tg depend add\n> needs to make sure that the new dependency can bring in changes\n> from a branch that has previously been removed as a dependency\n> from the current TopGit branch.\n>\n> This implementation uses an exported branch set up by tg export,\n> and merges the new dependency into the commit that corresponds to\n> the current base. Using the exported branch in the merge has the\n> advantage that removed dependencies do not appear as parents, and\n> the merge base selected by git merge does not include changes from\n> a removed dependency. As a result, these changes can be merged in\n> again if the new dependency brings in these changes.\n>\n> The tree of the merge commit is then used to create the next\n> commit on the TopGit base branch.\n\nI'm really not an expert, but history comes from the commits not from\nthe trees. Just using an hand made tree and commit this to the base\ndoesn't change anything, because somewhere in the parent commits is\nstill the information, that we merged in a branch that we just tried\nto remove (i'm speaking for tg depend remove). But maybe I don't\nunderstand what this patch changes. The only idea is, that we don't\nuse merge commits for the base any more! Is this what this patch does?\n\nBert\n"},{"id":"153052","messageId":"20101009105456.GA7328@santana.dyndns.org","threadId":"25397","inReplyTo":"AANLkTik6R89vPhwKyWOheHPkxQ9JPpmrQVmXon8Vn4+A@mail.gmail.com","subject":"Re: [TopGit PATCH] t/depend-add-using-export","fromName":"Olaf Dabrunz","fromEmail":"olaf.dabrunz@gmx.net","sentAt":"2010-10-09T10:54:56Z","receivedAt":"2010-10-09T10:54:56Z","isPatch":true,"sender":{"key":"olaf.dabrunz@gmx.net","avatar":null},"body":"On 09-Oct-10, Bert Wesarg wrote:\n> On Sat, Oct 9, 2010 at 03:43, Olaf Dabrunz wrote:\n> > When dependencies can be removed as well as added, tg depend add\n> > needs to make sure that the new dependency can bring in changes\n> > from a branch that has previously been removed as a dependency\n> > from the current TopGit branch.\n> >\n> > This implementation uses an exported branch set up by tg export,\n> > and merges the new dependency into the commit that corresponds to\n> > the current base. Using the exported branch in the merge has the\n> > advantage that removed dependencies do not appear as parents, and\n> > the merge base selected by git merge does not include changes from\n> > a removed dependency. As a result, these changes can be merged in\n> > again if the new dependency brings in these changes.\n> >\n> > The tree of the merge commit is then used to create the next\n> > commit on the TopGit base branch.\n> \n> I'm really not an expert, but history comes from the commits not from\n> the trees. Just using an hand made tree and commit this to the base\n> doesn't change anything, because somewhere in the parent commits is\n> still the information, that we merged in a branch that we just tried\n> to remove (i'm speaking for tg depend remove). But maybe I don't\n\nWhen the hand made tree is committed (as a hand made merge) on top of\nthe base, we have a correct merge on top of the base.\n\nThis actually should change everything. After such a \"tg depend add\", we\nhave either brought in some removed dependency as well, or we did not.\nIf we didn't, the removed deps in the base's history do not matter for\nsubsequent merges done by \"tg update\". But if we did bring in a\npreviously removed dep, the removed deps in the base's history _would_\nmatter. But since we already made a successful merge on top of the\nbase's history with a correct tree, and this merge also brings in the\nremoved dep through the history from the new dep's side, subsequent \"tg\nupdate\" merges will use this hand made merge as the merge base, which is\nfine for future merges.\n\n> understand what this patch changes. The only idea is, that we don't\n> use merge commits for the base any more! Is this what this patch does?\n\nMaybe I addressed this above already. But just to clarify, I left the\nfollowing description of the patch in this mail, because it describes\nthe patch with different words.\n\nWhat the patch does is the following: it still creates a merge commit on\nthe base to bring in a new dependency, but this merge commit is crafted\nto contain the \"correctly\" merged tree. So there are three steps:\n\n - use tg export to create a completely separate history (the \"temp\n export branch\") which _does not_ reference removed dependencies as\n parents\n - use git merge to merge in the new dependency on top of this\n temp export branch -- as none of the removed dependencies are\n referenced by the temp export branch, this merge is not influenced\n by them and will do the right thing\n - then take the tree (!) from that merge and hand-craft a new merge\n commit with that tree and use the old base and the new dependency\n as parents of this hand-crafted merge commit\n\nOlaf\n\n-- \nOlaf Dabrunz (Olaf.Dabrunz gmx.net)\n"},{"id":"153054","messageId":"AANLkTi=Xhd8DY+f1ZYR6dEhHvC-=50iHLnhdv8GTacNX@mail.gmail.com","threadId":"25397","inReplyTo":"20101009105456.GA7328@santana.dyndns.org","subject":"Re: [TopGit PATCH] t/depend-add-using-export","fromName":"Bert Wesarg","fromEmail":"bert.wesarg@googlemail.com","sentAt":"2010-10-09T11:27:44Z","receivedAt":"2010-10-09T11:27:44Z","isPatch":true,"sender":{"key":"bert.wesarg@googlemail.com","avatar":"https://avatars.githubusercontent.com/u/111934?v=4"},"body":"On Sat, Oct 9, 2010 at 12:54, Olaf Dabrunz wrote:\n> When the hand made tree is committed (as a hand made merge) on top of\n> the base, we have a correct merge on top of the base.\n>\n> This actually should change everything. After such a \"tg depend add\", we\n> have either brought in some removed dependency as well, or we did not.\n> If we didn't, the removed deps in the base's history do not matter for\n> subsequent merges done by \"tg update\". But if we did bring in a\n> previously removed dep, the removed deps in the base's history _would_\n> matter. But since we already made a successful merge on top of the\n> base's history with a correct tree, and this merge also brings in the\n> removed dep through the history from the new dep's side, subsequent \"tg\n> update\" merges will use this hand made merge as the merge base, which is\n> fine for future merges.\n\nWhen I understand this correctly, this hand made merge commit on base,\nhas also the just removed dep as parent. But the tree does not include\nany code from that dep. So git merge-base would select this commit as\nthe merge base.\n\nRight?\n\nBert\n\n>\n> Olaf\n>\n> --\n> Olaf Dabrunz (Olaf.Dabrunz gmx.net)\n>\n>\n"},{"id":"153056","messageId":"AANLkTik=tdfYHEKKLOMKBSm55mXptROKGis5gmusGFPK@mail.gmail.com","threadId":"25397","inReplyTo":"20101009121003.GA19991@santana.dyndns.org","subject":"Re: [TopGit PATCH] t/depend-add-using-export","fromName":"Bert Wesarg","fromEmail":"bert.wesarg@googlemail.com","sentAt":"2010-10-09T12:21:06Z","receivedAt":"2010-10-09T12:21:06Z","isPatch":true,"sender":{"key":"bert.wesarg@googlemail.com","avatar":"https://avatars.githubusercontent.com/u/111934?v=4"},"body":"[ Just bringing back all Cc's ]\n\nOn Sat, Oct 9, 2010 at 14:10, Olaf Dabrunz wrote:\n> On 09-Oct-10, Bert Wesarg wrote:\n>> On Sat, Oct 9, 2010 at 12:54, Olaf Dabrunz wrote:\n>> > [...]\n>>\n>> When I understand this correctly, this hand made merge commit on base,\n>> has also the just removed dep as parent. But the tree does not include\n>\n> The hand made merge commit on base only has a removed dep as a parent if\n> that removed dep is brought in again directly, as the new dep that is\n> added here. The only parents of the hand made merge commit are: the tip\n> of the current topgit branch's base and the new dependency.\n>\n>    hand made merge commit on base =                        \\\n>        git-commit-tree $tree_from_merge_on_export          \\\n>                        -p $topgit_base_branch -p $new_dep\n>\n> In general, the new dep may already have merged in a dep that we\n> previously removed from our topgit branch. So the previously removed dep\n> is brought in indirectly by merging a branch that contains it.\n>\n> This patch tries to cover both cases. The latter case and requirements\n> for tg depend add to cover it were discussed by you and Uwe in this\n> thread:\n>\n> http://lists-archives.org/git/688698-add-list-and-rm-sub-commands-to-tg-depend.html\n>\n>> any code from that dep. So git merge-base would select this commit as\n>\n> If the new dep _is_ a previously removed dep, the code of the new dep is\n> contained in the hand made merge as well.\n>\n> If the new dep _brings in_ a previously removed dep, it depends on how\n> the previously removed dep was merged into the new dep we are trying to\n> merge now. If our new dep merged in the previously removed dep but used\n> the \"ours\" merge strategy to supersede that dep's contents, it does not\n> bring in any code from that dep but only brings in that dep as part of\n> it's history.\n>\n>> the merge base.\n>\n> Yes, as the previously removed dep is now -- directly or indirectly --\n> part of the history of the hand made merge on base.\n>\n> I should really draw graphs. But here is the mail anyway (release\n> early...).\n>\n> Olaf\n>\n> --\n> Olaf Dabrunz (Olaf dabrunz.com)\n>\n>\n"},{"id":"153057","messageId":"20101009122111.GA29010@santana.dyndns.org","threadId":"25397","inReplyTo":"AANLkTi=Xhd8DY+f1ZYR6dEhHvC-=50iHLnhdv8GTacNX@mail.gmail.com","subject":"Re: [TopGit PATCH] t/depend-add-using-export","fromName":"Olaf Dabrunz","fromEmail":"olaf@dabrunz.com","sentAt":"2010-10-09T12:21:11Z","receivedAt":"2010-10-09T12:21:11Z","isPatch":true,"sender":{"key":"olaf@dabrunz.com","avatar":null},"body":"On 09-Oct-10, Bert Wesarg wrote:\n> On Sat, Oct 9, 2010 at 12:54, Olaf Dabrunz wrote:\n> > [...]\n> \n> When I understand this correctly, this hand made merge commit on base,\n> has also the just removed dep as parent. But the tree does not include\n\nThe hand made merge commit on base only has a removed dep as a parent if\nthat removed dep is brought in again directly, as the new dep that is\nadded here. The only parents of the hand made merge commit are: the tip\nof the current topgit branch's base and the new dependency.\n\n hand made merge commit on base = \\\n git-commit-tree $tree_from_merge_on_export \\\n -p $topgit_base_branch -p $new_dep\n\nIn general, the new dep may already have merged in a dep that we\npreviously removed from our topgit branch. So the previously removed dep\nis brought in indirectly by merging a branch that contains it.\n\nThis patch tries to cover both cases. The latter case and requirements\nfor tg depend add to cover it were discussed by you and Uwe in this\nthread:\n\nhttp://lists-archives.org/git/688698-add-list-and-rm-sub-commands-to-tg-depend.html\n\n> any code from that dep. So git merge-base would select this commit as\n\nIf the new dep _is_ a previously removed dep, the code of the new dep is\ncontained in the hand made merge as well.\n\nIf the new dep _brings in_ a previously removed dep, it depends on how\nthe previously removed dep was merged into the new dep we are trying to\nmerge now. If our new dep merged in the previously removed dep but used\nthe \"ours\" merge strategy to supersede that dep's contents, it does not\nbring in any code from that dep but only brings in that dep as part of\nit's history.\n\n> the merge base.\n\nYes, as the previously removed dep is now -- directly or indirectly --\npart of the history of the hand made merge on base.\n\nI should really draw graphs. But here is the mail anyway (release\nearly...).\n\nOlaf\n\n-- \nOlaf Dabrunz (Olaf dabrunz.com)\n"}]}