{"thread":{"id":"3093","subject":"Joining Repositories","startedAt":"2006-01-18T12:25:59Z","lastAt":"2006-01-19T18:24:39Z","messageCount":25,"participants":["Mathias Waack","Petr Baudis","Ryan Anderson","Andreas Ericsson","Linus Torvalds","H. Peter Anvin","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"14805","messageId":"200601181325.59832.Mathias.Waack@rantzau.de","threadId":"3093","inReplyTo":null,"subject":"Joining Repositories","fromName":"Mathias Waack","fromEmail":"mathias.waack@rantzau.de","sentAt":"2006-01-18T12:25:59Z","receivedAt":"2006-01-18T12:25:59Z","isPatch":false,"sender":{"key":"mathias.waack@rantzau.de","avatar":null},"body":"Hello everybody, \n\nwe're using git with cogito as a frontend. For some reasons I forgot we have \nsome repositories which belong to the same project. To simplify the whole \nthing I would like to join these repositories. It mainly means to move some \ndirectories. So lets say I have: \n\n\t/r1/.git\n\t/r2/.git\n\nand what I would like to have is\n\n\t/r/.git\n\t   r1\n           r2\n\nOf course the history should remain (otherwise it would be to easy). \n\nHow should I do this?\n\nThanks for your answers\n\n\tMathias\n"},{"id":"14806","messageId":"20060118125158.GN28365@pasky.or.cz","threadId":"3093","inReplyTo":"200601181325.59832.Mathias.Waack@rantzau.de","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-18T12:51:58Z","receivedAt":"2006-01-18T12:51:58Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hello,\n\nDear diary, on Wed, Jan 18, 2006 at 01:25:59PM CET, I got a letter\nwhere Mathias Waack <Mathias.Waack@rantzau.de> said that...\n> we're using git with cogito as a frontend. For some reasons I forgot we have \n> some repositories which belong to the same project. To simplify the whole \n> thing I would like to join these repositories. It mainly means to move some \n> directories. So lets say I have: \n> \n> \t/r1/.git\n> \t/r2/.git\n> \n> and what I would like to have is\n> \n> \t/r/.git\n> \t   r1\n>            r2\n> \n> Of course the history should remain (otherwise it would be to easy). \n> \n> How should I do this?\n\n  The crucial question is what the joined repository's directory\nstructure should be. If from r1/a, r2/b you want r/a, r/b then it's\neasy:\n\n\tcd r1\n\tcg-branch-add r2 ../r2/.git\n\tcg-fetch r2\n\tcg-merge -j r2\n\n  But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\nto rewrite the history of each (and then do the above). One rough\nand untested sketch...\n\n\tmkdir r1-rewritten\n\tcd r1-rewritten\n\tcp -a ../r1/.git .\n\tfor commit in $(git-rev-list --topo-order HEAD | tac); do\n\t\tgit-read-tree --prefix=r1/ $commit\n\t\t# here, setup the AUTHOR/COMMITTER variables\n\t\t# appropriately (you can look at cg-commit's\n\t\t# $copy_commit handler)\n\t\tgit-cat-file commit \"$commit\" | sed -e '1,/^$/d' | \\\n\t\t\tgit-commit-tree $(git-write-tree) \\\n\t\t\t\t$(for parent in $(cg-object-id -p $commit); do\n\t\t\t\t\tcat commitmap/$parent\n\t\t\t\t  done) >commitmap/$commit\n\t\tlast_commit=$commit\n\tdone\n\tgit-update-ref HEAD $(cat commitmap/$last_commit)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14807","messageId":"20060118125857.GO28365@pasky.or.cz","threadId":"3093","inReplyTo":"20060118125158.GN28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-18T12:58:57Z","receivedAt":"2006-01-18T12:58:57Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Jan 18, 2006 at 01:51:58PM CET, I got a letter\nwhere Petr Baudis <pasky@suse.cz> said that...\n>   But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\n> to rewrite the history of each (and then do the above). One rough\n> and untested sketch...\n> \n> \tmkdir r1-rewritten\n> \tcd r1-rewritten\n> \tcp -a ../r1/.git .\n\n\tmkdir commitmap\n\n> \tfor commit in $(git-rev-list --topo-order HEAD | tac); do\n> \t\tgit-read-tree --prefix=r1/ $commit\n> \t\t# here, setup the AUTHOR/COMMITTER variables\n> \t\t# appropriately (you can look at cg-commit's\n> \t\t# $copy_commit handler)\n> \t\tgit-cat-file commit \"$commit\" | sed -e '1,/^$/d' | \\\n> \t\t\tgit-commit-tree $(git-write-tree) \\\n> \t\t\t\t$(for parent in $(cg-object-id -p $commit); do\n\n\t\t\t\t\techo -p\n\n> \t\t\t\t\tcat commitmap/$parent\n> \t\t\t\t  done) >commitmap/$commit\n> \t\tlast_commit=$commit\n> \tdone\n> \tgit-update-ref HEAD $(cat commitmap/$last_commit)\n\n\trm -rf commitmap\n\tcg-reset\n\n  And at this point, git-prune will have plenty of work to do (many of\nyour tree and commit objects will be outdated - you just rewrote them).\n\n  It's quite beautiful that you can do this all without checkouting a\nsingle thing - you just load the tree to the index, prepend \"r1/\" to all\nthe paths, and commit it back.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14811","messageId":"200601181501.38791.Mathias.Waack@rantzau.de","threadId":"3093","inReplyTo":"20060118125857.GO28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Mathias Waack","fromEmail":"mathias.waack@rantzau.de","sentAt":"2006-01-18T14:01:38Z","receivedAt":"2006-01-18T14:01:38Z","isPatch":false,"sender":{"key":"mathias.waack@rantzau.de","avatar":null},"body":"Hello Petr, \n\nthanks for your quick answer, but...\n\nOn Wednesday 18 January 2006 01:58 pm, Petr Baudis wrote:\n> Dear diary, on Wed, Jan 18, 2006 at 01:51:58PM CET, I got a letter\n> where Petr Baudis <pasky@suse.cz> said that...\n>\n> >   But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\n> > to rewrite the history of each (and then do the above). One rough\n> > and untested sketch...\n> >\n> > \tmkdir r1-rewritten\n> > \tcd r1-rewritten\n> > \tcp -a ../r1/.git .\n>\n> \tmkdir commitmap\n>\n> > \tfor commit in $(git-rev-list --topo-order HEAD | tac); do\n> > \t\tgit-read-tree --prefix=r1/ $commit\n\n...git-read-tree doesn't know the parameter --prefix (just downloaded and \ntried it on 0.99.9i). What version shall I use?\n\nMathias\n"},{"id":"14813","messageId":"20060118140917.GA15438@mythryan2.michonline.com","threadId":"3093","inReplyTo":"20060118125158.GN28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-01-18T14:09:17Z","receivedAt":"2006-01-18T14:09:17Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"On Wed, Jan 18, 2006 at 01:51:58PM +0100, Petr Baudis wrote:\n>   Hello,\n> \n> Dear diary, on Wed, Jan 18, 2006 at 01:25:59PM CET, I got a letter\n> where Mathias Waack <Mathias.Waack@rantzau.de> said that...\n> > we're using git with cogito as a frontend. For some reasons I forgot we have \n> > some repositories which belong to the same project. To simplify the whole \n> > thing I would like to join these repositories. It mainly means to move some \n> > directories. So lets say I have: \n> > \n> > \t/r1/.git\n> > \t/r2/.git\n> > \n> > and what I would like to have is\n> > \n> > \t/r/.git\n> > \t   r1\n> >            r2\n> > \n> > Of course the history should remain (otherwise it would be to easy). \n> > \n> > How should I do this?\n> \n>   The crucial question is what the joined repository's directory\n> structure should be. If from r1/a, r2/b you want r/a, r/b then it's\n> easy:\n> \n> \tcd r1\n> \tcg-branch-add r2 ../r2/.git\n> \tcg-fetch r2\n> \tcg-merge -j r2\n> \n>   But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\n> to rewrite the history of each (and then do the above). One rough\n> and untested sketch...\n\n... Why would that be the optimal method?\n\nAssuming both repositories are clean, no extraneous files, and without\ntesting, of course:\n\nIn r1:\n\tmkdir r1\n\t# move everything into the subdirectory called r1.\n\tgit mv $(ls -a | grep -v -e ^.git$ -e ^r1$) r1/\n\tgit commit -a \"Restructure directory\"\n\nIn r2:\n\tmkdir r2\n\t# move all files into r2/\n\tgit mv $(ls -a | grep -v -e ^.git$ -e ^r2$) r2/\n\tgit commit -a \"Restructure directory\"\n\n\tgit fetch ../r1/\n\tGIT_INDEX_FILE=.git/tmp-index git-read-tree FETCH_HEAD\n\tGIT_INDEX_FILE=.git/tmp-index git-checkout-cache -a -u\n\tgit-update-cache --add -- $(GIT_INDEX_FILE=.git/tmp-index git-ls-files)\n\tcp .git/FETCH_HEAD .git/MERGE_HEAD\n\tgit commit\n\nNo history rewritten, merging with the old repositories should, at least\ntheoretically, work, etc.\n\n(This is just a restatement of Linus's \"Coolest merge ever\" / union\nmerge)\n\n-- \n\nRyan Anderson\n  sometimes Pug Majere\n"},{"id":"14814","messageId":"20060118141442.GP28365@pasky.or.cz","threadId":"3093","inReplyTo":"200601181501.38791.Mathias.Waack@rantzau.de","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-18T14:14:43Z","receivedAt":"2006-01-18T14:14:43Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hello,\n\nDear diary, on Wed, Jan 18, 2006 at 03:01:38PM CET, I got a letter\nwhere Mathias Waack <Mathias.Waack@rantzau.de> said that...\n> On Wednesday 18 January 2006 01:58 pm, Petr Baudis wrote:\n> > Dear diary, on Wed, Jan 18, 2006 at 01:51:58PM CET, I got a letter\n> > where Petr Baudis <pasky@suse.cz> said that...\n> >\n> > >   But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\n> > > to rewrite the history of each (and then do the above). One rough\n> > > and untested sketch...\n> > >\n> > > \tmkdir r1-rewritten\n> > > \tcd r1-rewritten\n> > > \tcp -a ../r1/.git .\n> >\n> > \tmkdir commitmap\n> >\n> > > \tfor commit in $(git-rev-list --topo-order HEAD | tac); do\n> > > \t\tgit-read-tree --prefix=r1/ $commit\n> \n> ...git-read-tree doesn't know the parameter --prefix (just downloaded and \n> tried it on 0.99.9i). What version shall I use?\n\n  oops, it seems this is only in the latest pu branch of git. If you are\nnot brave enough for that, you will need to use the prefix facility of\ncheckout-index instead, and it'll take much longer:\n\n\tgit-read-tree $commit\n\tgit-checkout-index -a --prefix=r1/\n\trm .git/index\n\tcg-add -r r1/\n\trm -rf r1\n\t.. then proceed with git-cat-file etc ..\n\n  Note that I'm not sure when which feature was introduced. Your best\nbet is to just use the latest stable GIT/Cogito versions.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14815","messageId":"43CE4D5B.10604@op5.se","threadId":"3093","inReplyTo":"200601181501.38791.Mathias.Waack@rantzau.de","subject":"Re: Joining Repositories","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2006-01-18T14:14:51Z","receivedAt":"2006-01-18T14:14:51Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"Mathias Waack wrote:\n> Hello Petr, \n> \n> thanks for your quick answer, but...\n> \n> On Wednesday 18 January 2006 01:58 pm, Petr Baudis wrote:\n> \n>>Dear diary, on Wed, Jan 18, 2006 at 01:51:58PM CET, I got a letter\n>>where Petr Baudis <pasky@suse.cz> said that...\n>>\n>>\n>>>  But if you want r1/a, r2/b to turn to r/r1/a, r/r2/b, you will have\n>>>to rewrite the history of each (and then do the above). One rough\n>>>and untested sketch...\n>>>\n>>>\tmkdir r1-rewritten\n>>>\tcd r1-rewritten\n>>>\tcp -a ../r1/.git .\n>>\n>>\tmkdir commitmap\n>>\n>>\n>>>\tfor commit in $(git-rev-list --topo-order HEAD | tac); do\n>>>\t\tgit-read-tree --prefix=r1/ $commit\n> \n> \n> ...git-read-tree doesn't know the parameter --prefix (just downloaded and \n> tried it on 0.99.9i). What version shall I use?\n> \n\n\nIf you're using 0.99.9i you're missing out on the 584 commits that has \nbeen committed since then. 1.1.3 was released a couple of days ago. I \nsuggest you upgrade.\n\n-- \nAndreas Ericsson                   andreas.ericsson@op5.se\nOP5 AB                             www.op5.se\nTel: +46 8-230225                  Fax: +46 8-230231\n"},{"id":"14816","messageId":"20060118142139.GQ28365@pasky.or.cz","threadId":"3093","inReplyTo":"20060118140917.GA15438@mythryan2.michonline.com","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-18T14:21:39Z","receivedAt":"2006-01-18T14:21:39Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Jan 18, 2006 at 03:09:17PM CET, I got a letter\nwhere Ryan Anderson <ryan@michonline.com> said that...\n> ... Why would that be the optimal method?\n\nOw. I suppose that's what you call \"blinded by invalid implicit\nassumptions\". Obviously, if you are ok with such a grand move, this is\nthe best.\n\nThis also reminds me that I should finally merge the cg-mv patch.\n\n> \tgit fetch ../r1/\n> \tGIT_INDEX_FILE=.git/tmp-index git-read-tree FETCH_HEAD\n> \tGIT_INDEX_FILE=.git/tmp-index git-checkout-cache -a -u\n> \tgit-update-cache --add -- $(GIT_INDEX_FILE=.git/tmp-index git-ls-files)\n> \tcp .git/FETCH_HEAD .git/MERGE_HEAD\n> \tgit commit\n\nBut if you want to have an idea what's actually going on on the high\nlevel, perhaps the\n\n\tcd r2\n\tcg-branch-add r1 ../r1/.git\n\tcg-fetch r1\n\tcg-merge -j r1\n\nmight be easier. ;-)\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14817","messageId":"43CE5315.8060008@michonline.com","threadId":"3093","inReplyTo":"20060118142139.GQ28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-01-18T14:39:17Z","receivedAt":"2006-01-18T14:39:17Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Petr Baudis wrote:\n\n>Dear diary, on Wed, Jan 18, 2006 at 03:09:17PM CET, I got a letter\n>where Ryan Anderson <ryan@michonline.com> said that...\n>  \n>\n>>... Why would that be the optimal method?\n>>\n>Ow. I suppose that's what you call \"blinded by invalid implicit\n>assumptions\". Obviously, if you are ok with such a grand move, this is\n>the best.\n>  \n>\nSorry, that came out a bit harsher than I had intended.\n\n>This also reminds me that I should finally merge the cg-mv patch.\n>\n>>\tgit fetch ../r1/\n>>\tGIT_INDEX_FILE=.git/tmp-index git-read-tree FETCH_HEAD\n>>\tGIT_INDEX_FILE=.git/tmp-index git-checkout-cache -a -u\n>>\tgit-update-cache --add -- $(GIT_INDEX_FILE=.git/tmp-index git-ls-files)\n>>\tcp .git/FETCH_HEAD .git/MERGE_HEAD\n>>\tgit commit\n>>    \n>>\n>\n>But if you want to have an idea what's actually going on on the high\n>level, perhaps the\n>\n>\tcd r2\n>\tcg-branch-add r1 ../r1/.git\n>\tcg-fetch r1\n>\tcg-merge -j r1\n>\n>might be easier. ;-)\n>\nAh, yes, cg-merge seems like it might be somewhat simpler here.\n\n\n-- \n\nRyan Anderson\n  sometimes Pug Majere\n\n"},{"id":"14821","messageId":"Pine.LNX.4.64.0601180813170.3240@g5.osdl.org","threadId":"3093","inReplyTo":"20060118140917.GA15438@mythryan2.michonline.com","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T16:23:03Z","receivedAt":"2006-01-18T16:23:03Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Ryan Anderson wrote:\n>\n> Assuming both repositories are clean, no extraneous files, and without\n> testing, of course:\n> \n> In r1:\n> \tmkdir r1\n> \t# move everything into the subdirectory called r1.\n> \tgit mv $(ls -a | grep -v -e ^.git$ -e ^r1$) r1/\n> \tgit commit -a \"Restructure directory\"\n> \n> In r2:\n> \tmkdir r2\n> \t# move all files into r2/\n> \tgit mv $(ls -a | grep -v -e ^.git$ -e ^r2$) r2/\n> \tgit commit -a \"Restructure directory\"\n> \n> \tgit fetch ../r1/\n> \tGIT_INDEX_FILE=.git/tmp-index git-read-tree FETCH_HEAD\n> \tGIT_INDEX_FILE=.git/tmp-index git-checkout-cache -a -u\n> \tgit-update-cache --add -- $(GIT_INDEX_FILE=.git/tmp-index git-ls-files)\n> \tcp .git/FETCH_HEAD .git/MERGE_HEAD\n> \tgit commit\n> \n> No history rewritten,\n\nRight.\n\n> merging with the old repositories should, at least theoretically, work, \n> etc.\n\nNo. This - and the history rewriting - both have a fundamental problem: it \nbecomes totally impossible to merge back any changes of the subprojects, \nat least automatically. The renaming just ends up making pretty much any \nmerge a manual affair (git can _help_, but it's going to be a matter of \nluck if it works or not - and it usually won't work very well, because \nyou'll probably end up having files that have the same name in the new and \nthe old repo, and eventually you'll just have tons of confusion).\n\nSo it works only if you want to _really_ join the projects (like in the \n\"git tools\" case), and never touch the old one again but it sucks \notherwise.\n\nIn the \"gitk\" case, we could actually continue to merge stuff after a \njoin, but that was largaly because there was no renaming. That's a very \nspecial case, and it also became impossible to merge back.\n\nNow, let's see what Junio comes up with, but in the meantime, I'd almost \nsuggest just something like\n\n\tgit clone oldrepo r1\n\tcd r1\n\tgit checkout -b join-branch\n\tcd ..\n\tgit add r1/.git/refs/heads/join-branch\n\tgit commit -m \"Join repo 'oldrepo' at 'r1'\"\n\nwhich should actually work except for the fact that we don't like the \n\".git\" component even as part of a sub-component (ie small hack to git \nrequired to make it not ignore that path).\n\n\t\tLinus\n"},{"id":"14822","messageId":"43CE6F15.2050801@zytor.com","threadId":"3093","inReplyTo":"200601181325.59832.Mathias.Waack@rantzau.de","subject":"Re: Joining Repositories","fromName":"H. Peter Anvin","fromEmail":"hpa@zytor.com","sentAt":"2006-01-18T16:38:45Z","receivedAt":"2006-01-18T16:38:45Z","isPatch":false,"sender":{"key":"hpa@zytor.com","avatar":null},"body":"Mathias Waack wrote:\n> Hello everybody, \n> \n> we're using git with cogito as a frontend. For some reasons I forgot we have \n> some repositories which belong to the same project. To simplify the whole \n> thing I would like to join these repositories. It mainly means to move some \n> directories. So lets say I have: \n> \n> \t/r1/.git\n> \t/r2/.git\n> \n> and what I would like to have is\n> \n> \t/r/.git\n> \t   r1\n>            r2\n> \n> Of course the history should remain (otherwise it would be to easy). \n> \n> How should I do this?\n> \n\nRename the files in each repository independently (with a checkin), then \nmerge the repositories.\n\nGit *should* track across the rename, as long as it's \"obvious\".\n\n\t-hpa\n"},{"id":"14825","messageId":"20060118170536.GS28365@pasky.or.cz","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601180813170.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-18T17:05:37Z","receivedAt":"2006-01-18T17:05:37Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Wed, Jan 18, 2006 at 05:23:03PM CET, I got a letter\nwhere Linus Torvalds <torvalds@osdl.org> said that...\n> On Wed, 18 Jan 2006, Ryan Anderson wrote:\n> > merging with the old repositories should, at least theoretically, work, \n> > etc.\n> \n> No. This - and the history rewriting - both have a fundamental problem: it \n> becomes totally impossible to merge back any changes of the subprojects, \n> at least automatically. The renaming just ends up making pretty much any \n> merge a manual affair (git can _help_, but it's going to be a matter of \n> luck if it works or not - and it usually won't work very well, because \n> you'll probably end up having files that have the same name in the new and \n> the old repo, and eventually you'll just have tons of confusion).\n\nWell, the recursive merge strategy is at least advertised to be able to\nmerge across renames (as long as they are autodetected). Not that I\nwould have any practical experiences with it and I don't know how smart\nis it done (if it just runs rename detection between the merge base and\ncurrent head, yes, that might not give very good results in this case).\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14829","messageId":"7vbqy9xx2r.fsf@assigned-by-dhcp.cox.net","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601180813170.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-18T17:30:52Z","receivedAt":"2006-01-18T17:30:52Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> On Wed, 18 Jan 2006, Ryan Anderson wrote:\n>>\n>> Assuming both repositories are clean, no extraneous files, and without\n>> testing, of course:\n>>\n>> ... \"coolest merge ever\" recipe here ... \n>> \n>> No history rewritten,\n>\n> Right.\n\n>> merging with the old repositories should, at least theoretically, work, \n>> etc.\n>\n> No. This - and the history rewriting - both have a fundamental problem: it \n> becomes totally impossible to merge back any changes of the subprojects, \n> at least automatically.\n\nRight.\n\n> In the \"gitk\" case, we could actually continue to merge stuff after a \n> join, but that was largaly because there was no renaming. That's a very \n> special case, and it also became impossible to merge back.\n\nCorrect.\n\n> Now, let's see what Junio comes up with,...\n\nWell, since the reason the original RFI came was \"somehow we\nforgot these things are together but ended up having to and need\na way to rectify the situation\", in this particular case,\nmerging back the changes to subprojects (which is very awkward\nif not impossible after a \"coolest merge ever\") is a non issue.\n\nTracking history across renames is something we have only half\nof the needed support.  We can notice rename points but there is\nno way to tell our usability tools to automatically follow it.\nIOW \"whatchanged r1/hello.c\" will stop at the point the\noriginal project renamed hello.c to r1/hello.c in preparation\nfor the coolest merge and you have to restart whatchanged at\nthat commit with \"whatchanged $that_commit hello.c\" to go\nfurther back; but that is true when no project merge is involved\nso it may be an issue but it is not a new issue.\n\nSo for this case, I think the \"coolest merge\" is the right thing\nto do.\n\n> suggest just something like\n>\n> \tgit clone oldrepo r1\n> \tcd r1\n> \tgit checkout -b join-branch\n> \tcd ..\n> \tgit add r1/.git/refs/heads/join-branch\n> \tgit commit -m \"Join repo 'oldrepo' at 'r1'\"\n>\n> which should actually work except for the fact that we don't like the \n> \".git\" component even as part of a sub-component (ie small hack to git \n> required to make it not ignore that path).\n\nSorry you lost me with this.  The \"join-branch\" is a file with\ncommit object name in it, so is this \"recording the revision of\nthe other project this particular version of the project is\nlinked to\" idea?\n"},{"id":"14830","messageId":"7v7j8xxwym.fsf@assigned-by-dhcp.cox.net","threadId":"3093","inReplyTo":"20060118170536.GS28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-18T17:33:21Z","receivedAt":"2006-01-18T17:33:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis <pasky@suse.cz> writes:\n\n> Well, the recursive merge strategy is at least advertised to be able to\n> merge across renames (as long as they are autodetected). Not that I\n> would have any practical experiences with it and I don't know how smart\n> is it done (if it just runs rename detection between the merge base and\n> current head, yes, that might not give very good results in this case).\n\nIf I understand and remember the code correctly, it is based on\nrename detection between two heads.  When the versions have\ndiverged greatly beyond recognition, obviously this would not\nwork well.\n"},{"id":"14834","messageId":"Pine.LNX.4.64.0601180957460.3240@g5.osdl.org","threadId":"3093","inReplyTo":"20060118170536.GS28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T18:03:22Z","receivedAt":"2006-01-18T18:03:22Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Petr Baudis wrote:\n> \n> Well, the recursive merge strategy is at least advertised to be able to\n> merge across renames (as long as they are autodetected).\n\nYes. It can. However, the git merge approach fundamentally expects that \nthe merged trees _converge_. \n\nSo what happens is that if two trees continue to be developed as separate \ntrees, and the merges never converge (ie they happen just one way, with \nthe other way seldom or never merging back), the merge strategy will get \nincreasingly bad.\n\nThat's pretty unavoidable, btw. I can pretty much guarantee that nobody \nelse will really do it any better, although any particular strategy will \nalways handle one particular case perfectly (*).\n\n\t\t\tLinus\n\n(*) I think merging is like scheduling: you can always find a perfect \nscheduling algorithm _after_the_fact_ when you know the load, but finding \none that works for all loads inevitably means that some cases won't work \nwell. You also find scheduling getting an inordinate amount of time in CS \nliterature, while in real life it's one of the least important problems of \na kernel.\n"},{"id":"14838","messageId":"Pine.LNX.4.64.0601181010410.3240@g5.osdl.org","threadId":"3093","inReplyTo":"7vbqy9xx2r.fsf@assigned-by-dhcp.cox.net","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T18:22:02Z","receivedAt":"2006-01-18T18:22:02Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Junio C Hamano wrote:\n> \n> Tracking history across renames is something we have only half\n> of the needed support.  We can notice rename points but there is\n> no way to tell our usability tools to automatically follow it.\n> IOW \"whatchanged r1/hello.c\" will stop at the point the\n> original project renamed hello.c\n\nNote that \"whatchanged\" really really _really_ must not follow renames.\n\nWhy? The whole _point_ of whatchanged is that it takes a path limiter. A \npath limiter is much more useful (to me) that following files. A path \nlimiter works even when the file doesn't _currently_ exist, and you don't \nknow what happened to it. A path limiter also fundamentally works for \ndirectories and for multiple files.\n\nIf you want \"git annotate\", then do a \"git annotate\". Don't change what \n\"git whatchanged\" does. The \"pathname only\" behaviour is in my opinion THE \nMOST SINGLE BIGGEST FEATURE of git.\n\nI _much_ more often use a directory to \"git-whatchanged\" than I look at an \nindividual file. Maybe other people don't do that, because they don't tend \nto look at the \"big picture\", or because they are stuck in the \"single \nfile only\" mentality, but trust me, especially for maintaining big things, \nthe \"track directories\" is a lot more important than single-file tracking. \n\nA _LOT_ more.\n\nI realize that this is heresy to people who are used to \"annotate\" and \nwant to follow not the path, but the \"conceptual inode\", but the thing is, \npaths really really are a lot more important to a maintainer. Following an \nindividual file is a secondary issue.\n\nNow, the fact that we still don't have an efficient \"git annotate\" is not \nbecause we don't have the capability. It _should_ be perfectly easy to do \n(and efficient) by just parsing the output of\n\n\tgit-rev-list HEAD -- <filename>\n\nand when you see a \"delete\" event, you'd look if it was really a rename.\n\nI think qgit does this, but we don't have the textual version in core git.\n\nSo the reason core git tracks PATHNAMES and not files is:\n\n - it's fundamentally a much more powerful operation\n\n - it's possible (and not that much harder) to track renames efficiently \n   using it anyway, so you can emulate the \"track file\" behaviour using \n   \"track pathname\", but you can NOT emulate it the other way around.\n\nIn other words, tracking pathnames is really _fundamentally_ a more \npowerful operation.\n\nGit does this right. It is - together with multiple branches - the thing \nthat makes git work better than BK did for me.\n\n> > suggest just something like\n> >\n> > \tgit clone oldrepo r1\n> > \tcd r1\n> > \tgit checkout -b join-branch\n> > \tcd ..\n> > \tgit add r1/.git/refs/heads/join-branch\n> > \tgit commit -m \"Join repo 'oldrepo' at 'r1'\"\n> >\n> > which should actually work except for the fact that we don't like the \n> > \".git\" component even as part of a sub-component (ie small hack to git \n> > required to make it not ignore that path).\n> \n> Sorry you lost me with this.  The \"join-branch\" is a file with\n> commit object name in it, so is this \"recording the revision of\n> the other project this particular version of the project is\n> linked to\" idea?\n\nYes. It also means that if you update the other project (commit, pull, \nwhatever), \"git diff\" in the top level will do the right thing: it won't \nbe readable, but you'll always have the SHA1 visible, and you can \"commit\" \nthe fact that you updated the sub-project in the top project. It's \nbasically a poor mans \"gitlink\".\n\n\t\tLinus\n"},{"id":"14843","messageId":"43CE8F36.3090100@michonline.com","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601180813170.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Ryan Anderson","fromEmail":"ryan@michonline.com","sentAt":"2006-01-18T18:55:50Z","receivedAt":"2006-01-18T18:55:50Z","isPatch":false,"sender":{"key":"ryan@michonline.com","avatar":null},"body":"Linus Torvalds wrote:\n> \n> On Wed, 18 Jan 2006, Ryan Anderson wrote:\n> \n>>Assuming both repositories are clean, no extraneous files, and without\n>>testing, of course:\n>>\n>>In r1:\n>>\tmkdir r1\n>>\t# move everything into the subdirectory called r1.\n>>\tgit mv $(ls -a | grep -v -e ^.git$ -e ^r1$) r1/\n>>\tgit commit -a \"Restructure directory\"\n>>\n>>In r2:\n>>\tmkdir r2\n>>\t# move all files into r2/\n>>\tgit mv $(ls -a | grep -v -e ^.git$ -e ^r2$) r2/\n>>\tgit commit -a \"Restructure directory\"\n>>\n>>\tgit fetch ../r1/\n>>\tGIT_INDEX_FILE=.git/tmp-index git-read-tree FETCH_HEAD\n>>\tGIT_INDEX_FILE=.git/tmp-index git-checkout-cache -a -u\n>>\tgit-update-cache --add -- $(GIT_INDEX_FILE=.git/tmp-index git-ls-files)\n>>\tcp .git/FETCH_HEAD .git/MERGE_HEAD\n>>\tgit commit\n>>\n>>No history rewritten,\n> \n> \n> Right.\n> \n> \n>>merging with the old repositories should, at least theoretically, work, \n>>etc.\n> \n> \n> No. This - and the history rewriting - both have a fundamental problem: it \n> becomes totally impossible to merge back any changes of the subprojects, \n> at least automatically. The renaming just ends up making pretty much any \n> merge a manual affair (git can _help_, but it's going to be a matter of \n> luck if it works or not - and it usually won't work very well, because \n> you'll probably end up having files that have the same name in the new and \n> the old repo, and eventually you'll just have tons of confusion).\n\nWhat I meant was that an old repository can continue to be developed in,\nand, in theory, the recursive merge should be able to merge those\nchanges into this new, joined, repository.\n\n\n"},{"id":"14845","messageId":"7vslrluze6.fsf@assigned-by-dhcp.cox.net","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601181010410.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-18T19:09:05Z","receivedAt":"2006-01-18T19:09:05Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds <torvalds@osdl.org> writes:\n\n> On Wed, 18 Jan 2006, Junio C Hamano wrote:\n>> \n>> Tracking history across renames is something we have only half\n>> of the needed support.  We can notice rename points but there is\n>> no way to tell our usability tools to automatically follow it.\n>> IOW \"whatchanged r1/hello.c\" will stop at the point the\n>> original project renamed hello.c\n>\n> Note that \"whatchanged\" really really _really_ must not follow renames.\n\nSorry for having brought up the issue again.  I personally do\nnot disagree with you.  Because I did not want to waste your\ntime rehashing it, I attempted to make it absolutely clear that\nI was talking about an _optional_ way to tell the tool to do so,\nwhen somebody so used to \"annotate\" wants to.  Obviously my\nattempt did not work well X-<.\n\n> I realize that this is heresy to people who are used to \"annotate\" and \n> want to follow not the path, but the \"conceptual inode\", but the thing is, \n> paths really really are a lot more important to a maintainer. Following an \n> individual file is a secondary issue.\n> ...\n> In other words, tracking pathnames is really _fundamentally_ a more \n> powerful operation.\n\nI agree with you including this part. At the same time, I think\nannotate-minded people would find it useful if there were an\noption (or a separate program similar to whatchanged) that\nnotices file or directory renames, and adjusts the initial path\nlimiter, probably by adding not replacing the new \"possibly\nrelevant\" paths it discovered to it.\n"},{"id":"14850","messageId":"Pine.LNX.4.64.0601181128060.3240@g5.osdl.org","threadId":"3093","inReplyTo":"7vslrluze6.fsf@assigned-by-dhcp.cox.net","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T20:09:41Z","receivedAt":"2006-01-18T20:09:41Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Junio C Hamano wrote:\n> \n> Sorry for having brought up the issue again.  I personally do\n> not disagree with you.  Because I did not want to waste your\n> time rehashing it, I attempted to make it absolutely clear that\n> I was talking about an _optional_ way to tell the tool to do so,\n> when somebody so used to \"annotate\" wants to.  Obviously my\n> attempt did not work well X-<.\n\nWell, the main reason I did the \"limit git-rev-list with pathnames\" was \nliterally to try to help the \"annotate one file\".\n\nSo I don't think file following has anything to do with \"git-whatchanged\". \nIt's literally an op we don't have yet - and that we should have. It \nreally should be pretty straighforward to do a \"git annotate\" these days. \nIt will never be as fast as git-whatchanged, but it shouldn't be horribly \ninefficient either.\n\nThe one thing I've considered doing (I really should) is to add a \"stop \nwhen you don't find the file\" option to \"git-rev-list\". This patch does \nsome of the work towards that: it removes the \"parent\" thing when the \nfile disappears, so a \"git annotate\" could do do something like\n\n\tgit-rev-list --remove-empty --parents HEAD -- \"$filename\"\n\nand it would get a good graph that stops when the filename disappears \n(it's not perfect though: it won't remove all the unintersting commits).\n\n\t\tLinus\n\n----\ndiff --git a/rev-list.c b/rev-list.c\nindex d060966..3906674 100644\n--- a/rev-list.c\n+++ b/rev-list.c\n@@ -54,6 +54,7 @@ static int stop_traversal = 0;\n static int topo_order = 0;\n static int no_merges = 0;\n static const char **paths = NULL;\n+static int remove_empty_trees = 0;\n \n static void show_commit(struct commit *commit)\n {\n@@ -424,14 +425,33 @@ static void mark_edges_uninteresting(str\n \t}\n }\n \n-static int is_different = 0;\n+#define TREE_SAME\t0\n+#define TREE_NEW\t1\n+#define TREE_DIFFERENT\t2\n+static int tree_difference = TREE_SAME;\n \n static void file_add_remove(struct diff_options *options,\n \t\t    int addremove, unsigned mode,\n \t\t    const unsigned char *sha1,\n \t\t    const char *base, const char *path)\n {\n-\tis_different = 1;\n+\tint diff = TREE_DIFFERENT;\n+\n+\t/*\n+\t * Is it an add of a new file? It means that\n+\t * the old tree didn't have it at all, so we\n+\t * will turn \"TREE_SAME\" -> \"TREE_NEW\", but\n+\t * leave any \"TREE_DIFFERENT\" alone (and if\n+\t * it already was \"TREE_NEW\", we'll keep it\n+\t * \"TREE_NEW\" of course).\n+\t */\n+\tif (addremove == '+') {\n+\t\tdiff = tree_difference;\n+\t\tif (diff != TREE_SAME)\n+\t\t\treturn;\n+\t\tdiff = TREE_NEW;\n+\t}\n+\ttree_difference = diff;\n }\n \n static void file_change(struct diff_options *options,\n@@ -440,7 +460,7 @@ static void file_change(struct diff_opti\n \t\t const unsigned char *new_sha1,\n \t\t const char *base, const char *path)\n {\n-\tis_different = 1;\n+\ttree_difference = TREE_DIFFERENT;\n }\n \n static struct diff_options diff_opt = {\n@@ -449,12 +469,16 @@ static struct diff_options diff_opt = {\n \t.change = file_change,\n };\n \n-static int same_tree(struct tree *t1, struct tree *t2)\n+static int compare_tree(struct tree *t1, struct tree *t2)\n {\n-\tis_different = 0;\n+\tif (!t1)\n+\t\treturn TREE_NEW;\n+\tif (!t2)\n+\t\treturn TREE_DIFFERENT;\n+\ttree_difference = TREE_SAME;\n \tif (diff_tree_sha1(t1->object.sha1, t2->object.sha1, \"\", &diff_opt) < 0)\n-\t\treturn 0;\n-\treturn !is_different;\n+\t\treturn TREE_DIFFERENT;\n+\treturn tree_difference;\n }\n \n static int same_tree_as_empty(struct tree *t1)\n@@ -474,26 +498,38 @@ static int same_tree_as_empty(struct tre\n \tempty.buf = \"\";\n \tempty.size = 0;\n \n-\tis_different = 0;\n+\ttree_difference = 0;\n \tretval = diff_tree(&empty, &real, \"\", &diff_opt);\n \tfree(tree);\n \n-\treturn retval >= 0 && !is_different;\n+\treturn retval >= 0 && !tree_difference;\n }\n \n-static struct commit *try_to_simplify_merge(struct commit *commit, struct commit_list *parent)\n+static struct commit *try_to_simplify_merge(struct commit *commit)\n {\n+\tstruct commit_list **pp, *parent;\n+\n \tif (!commit->tree)\n \t\treturn NULL;\n \n-\twhile (parent) {\n+\tpp = &commit->parents;\n+\twhile ((parent = *pp) != NULL) {\n \t\tstruct commit *p = parent->item;\n-\t\tparent = parent->next;\n \t\tparse_commit(p);\n-\t\tif (!p->tree)\n-\t\t\tcontinue;\n-\t\tif (same_tree(commit->tree, p->tree))\n+\t\tswitch (compare_tree(p->tree, commit->tree)) {\n+\t\tcase TREE_SAME:\n \t\t\treturn p;\n+\t\tcase TREE_NEW:\n+\t\t\tif (remove_empty_trees) {\n+\t\t\t\t*pp = parent->next;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t/* fallthrough */\n+\t\tcase TREE_DIFFERENT:\n+\t\t\tpp = &parent->next;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tdie(\"bad tree compare for commit %s\", sha1_to_hex(commit->object.sha1));\n \t}\n \treturn NULL;\n }\n@@ -538,7 +574,7 @@ static void add_parents_to_list(struct c\n \tif (paths && parent && parent->next) {\n \t\tstruct commit *preferred;\n \n-\t\tpreferred = try_to_simplify_merge(commit, parent);\n+\t\tpreferred = try_to_simplify_merge(commit);\n \t\tif (preferred) {\n \t\t\tparent->item = preferred;\n \t\t\tparent->next = NULL;\n@@ -576,10 +612,18 @@ static void compress_list(struct commit_\n \t\t * unchanged\n \t\t */\n \t\tif (!parent->next) {\n-\t\t\tstruct tree *t1 = commit->tree;\n-\t\t\tstruct tree *t2 = parent->item->tree;\n-\t\t\tif (!t1 || !t2 || same_tree(t1, t2))\n+\t\t\tstruct tree *t1 = parent->item->tree;\n+\t\t\tstruct tree *t2 = commit->tree;\n+\t\t\tswitch (compare_tree(t1, t2)) {\n+\t\t\tcase TREE_SAME:\n \t\t\t\tcontinue;\n+\t\t\tcase TREE_NEW:\n+\t\t\t\tif (remove_empty_trees)\n+\t\t\t\t\tcommit->parents = NULL;\n+\t\t\t\tbreak;\n+\t\t\tcase TREE_DIFFERENT:\n+\t\t\t\tbreak;\n+\t\t\t}\n \t\t}\n \t\tcommit->object.flags |= TREECHANGE;\n \t}\n@@ -808,6 +852,10 @@ int main(int argc, const char **argv)\n \t\t\tdense = 0;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strcmp(arg, \"--remove-empty\")) {\n+\t\t\tremove_empty_trees = 1;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--\")) {\n \t\t\ti++;\n \t\t\tbreak;\n"},{"id":"14853","messageId":"Pine.LNX.4.64.0601181414530.3240@g5.osdl.org","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601181128060.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T22:47:30Z","receivedAt":"2006-01-18T22:47:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Linus Torvalds wrote:\n> \n> The one thing I've considered doing (I really should) is to add a \"stop \n> when you don't find the file\" option to \"git-rev-list\". This patch does \n> some of the work towards that: it removes the \"parent\" thing when the \n> file disappears, so a \"git annotate\" could do do something like\n> \n> \tgit-rev-list --remove-empty --parents HEAD -- \"$filename\"\n> \n> and it would get a good graph that stops when the filename disappears \n> (it's not perfect though: it won't remove all the unintersting commits).\n\nHere is a somewhat better version of this patch.\n\nIt also simplifies the logic of finding tree differences a bit, at the \ncost of making it a tad less efficient.\n\nThe old logic was two-phase: it would first simplify _only_ merges tree as \nit traversed the tree, and then simplify the linear parts of the remainder \nindependently. That was pretty optimal from an efficiency standpoint \nbecause it avoids doing any comparisons that we can see are unnecessary, \nbut it made it much harder to understand than it really needed to be.\n\nThe new logic is a lot more straightforward, and compares the trees as it \ntraverses the graph (ie everything is a single phase). That makes it much \neasier to stop graph traversal at any point where a file disappears.\n\nAs an example, let's say that you have a git repository that has had a \nfile called \"A\" some time in the past. That file gets renamed to B, and \nthen gets renamed back again to A. The old \"git-rev-list\" would show two \ncommits commits: the commit that renames B to A (because it changes A) \n_and_ as its parent the commit that renames A to B (because it changes A).\n\nWith the new --remove-empty flag, git-rev-list will show just the commit \nthat renames B to A as the \"root\" commit, and stop traversal there \n(because that's what you want for \"annotate\" - you want to stop there, and \nfor every \"root\" commit you then separately see if it really is a new \nfile, or if the paths history disappeared because it was renamed from some \nother file).\n\nWith this patch, you should be able to basically do a \"poor mans 'git \nannotate'\" with a fairly simple loop:\n\n\tpush(\"HEAD\", \"$filename\")\n\twhile (revision,filename = pop()) {\n\t\tfor each i in $(git-rev-list --parents --remove-empty $revision -- \"$filename\")\n\n\t\tpseudo-parents($i) = git-rev-list parents for that line\n\n\t\tif (pseudo-parents($i) is non-empty) {\n\t\t\tshow diff of $i against pseudo-parents\n\t\t\tcontinue\n\t\t}\n\n\t\t/* See if the _real_ parents of $i had a rename */\n\t\tparent($i) = real-parent($i)\n\t\tif (find-rename in $parent($i)->$i)\n\t\t\tpush $parent($i), \"old-name\"\n\t}\n\nwhich should be doable in perl or something (doing stacks in shell is just \ntoo painful to be worth it, so I'm not going to do this).\n\nAnybody want to try?\n\n\t\tLinus\n\n---\ndiff --git a/rev-list.c b/rev-list.c\nindex d060966..912071d 100644\n--- a/rev-list.c\n+++ b/rev-list.c\n@@ -54,6 +54,7 @@ static int stop_traversal = 0;\n static int topo_order = 0;\n static int no_merges = 0;\n static const char **paths = NULL;\n+static int remove_empty_trees = 0;\n \n static void show_commit(struct commit *commit)\n {\n@@ -424,14 +425,33 @@ static void mark_edges_uninteresting(str\n \t}\n }\n \n-static int is_different = 0;\n+#define TREE_SAME\t0\n+#define TREE_NEW\t1\n+#define TREE_DIFFERENT\t2\n+static int tree_difference = TREE_SAME;\n \n static void file_add_remove(struct diff_options *options,\n \t\t    int addremove, unsigned mode,\n \t\t    const unsigned char *sha1,\n \t\t    const char *base, const char *path)\n {\n-\tis_different = 1;\n+\tint diff = TREE_DIFFERENT;\n+\n+\t/*\n+\t * Is it an add of a new file? It means that\n+\t * the old tree didn't have it at all, so we\n+\t * will turn \"TREE_SAME\" -> \"TREE_NEW\", but\n+\t * leave any \"TREE_DIFFERENT\" alone (and if\n+\t * it already was \"TREE_NEW\", we'll keep it\n+\t * \"TREE_NEW\" of course).\n+\t */\n+\tif (addremove == '+') {\n+\t\tdiff = tree_difference;\n+\t\tif (diff != TREE_SAME)\n+\t\t\treturn;\n+\t\tdiff = TREE_NEW;\n+\t}\n+\ttree_difference = diff;\n }\n \n static void file_change(struct diff_options *options,\n@@ -440,7 +460,7 @@ static void file_change(struct diff_opti\n \t\t const unsigned char *new_sha1,\n \t\t const char *base, const char *path)\n {\n-\tis_different = 1;\n+\ttree_difference = TREE_DIFFERENT;\n }\n \n static struct diff_options diff_opt = {\n@@ -449,12 +469,16 @@ static struct diff_options diff_opt = {\n \t.change = file_change,\n };\n \n-static int same_tree(struct tree *t1, struct tree *t2)\n+static int compare_tree(struct tree *t1, struct tree *t2)\n {\n-\tis_different = 0;\n+\tif (!t1)\n+\t\treturn TREE_NEW;\n+\tif (!t2)\n+\t\treturn TREE_DIFFERENT;\n+\ttree_difference = TREE_SAME;\n \tif (diff_tree_sha1(t1->object.sha1, t2->object.sha1, \"\", &diff_opt) < 0)\n-\t\treturn 0;\n-\treturn !is_different;\n+\t\treturn TREE_DIFFERENT;\n+\treturn tree_difference;\n }\n \n static int same_tree_as_empty(struct tree *t1)\n@@ -474,28 +498,55 @@ static int same_tree_as_empty(struct tre\n \tempty.buf = \"\";\n \tempty.size = 0;\n \n-\tis_different = 0;\n+\ttree_difference = 0;\n \tretval = diff_tree(&empty, &real, \"\", &diff_opt);\n \tfree(tree);\n \n-\treturn retval >= 0 && !is_different;\n+\treturn retval >= 0 && !tree_difference;\n }\n \n-static struct commit *try_to_simplify_merge(struct commit *commit, struct commit_list *parent)\n+static void try_to_simplify_commit(struct commit *commit)\n {\n+\tstruct commit_list **pp, *parent;\n+\n \tif (!commit->tree)\n-\t\treturn NULL;\n+\t\treturn;\n \n-\twhile (parent) {\n+\tif (!commit->parents) {\n+\t\tif (!same_tree_as_empty(commit->tree))\n+\t\t\tcommit->object.flags |= TREECHANGE;\n+\t\treturn;\n+\t}\n+\n+\tpp = &commit->parents;\n+\twhile ((parent = *pp) != NULL) {\n \t\tstruct commit *p = parent->item;\n-\t\tparent = parent->next;\n+\n+\t\tif (p->object.flags & UNINTERESTING) {\n+\t\t\tpp = &parent->next;\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tparse_commit(p);\n-\t\tif (!p->tree)\n+\t\tswitch (compare_tree(p->tree, commit->tree)) {\n+\t\tcase TREE_SAME:\n+\t\t\tparent->next = NULL;\n+\t\t\tcommit->parents = parent;\n+\t\t\treturn;\n+\n+\t\tcase TREE_NEW:\n+\t\t\tif (remove_empty_trees && same_tree_as_empty(p->tree)) {\n+\t\t\t\t*pp = parent->next;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t/* fallthrough */\n+\t\tcase TREE_DIFFERENT:\n+\t\t\tpp = &parent->next;\n \t\t\tcontinue;\n-\t\tif (same_tree(commit->tree, p->tree))\n-\t\t\treturn p;\n+\t\t}\n+\t\tdie(\"bad tree compare for commit %s\", sha1_to_hex(commit->object.sha1));\n \t}\n-\treturn NULL;\n+\tcommit->object.flags |= TREECHANGE;\n }\n \n static void add_parents_to_list(struct commit *commit, struct commit_list **list)\n@@ -531,20 +582,14 @@ static void add_parents_to_list(struct c\n \t}\n \n \t/*\n-\t * Ok, the commit wasn't uninteresting. If it\n-\t * is a merge, try to find the parent that has\n-\t * no differences in the path set if one exists.\n+\t * Ok, the commit wasn't uninteresting. Try to\n+\t * simplify the commit history and find the parent\n+\t * that has no differences in the path set if one exists.\n \t */\n-\tif (paths && parent && parent->next) {\n-\t\tstruct commit *preferred;\n-\n-\t\tpreferred = try_to_simplify_merge(commit, parent);\n-\t\tif (preferred) {\n-\t\t\tparent->item = preferred;\n-\t\t\tparent->next = NULL;\n-\t\t}\n-\t}\n+\tif (paths)\n+\t\ttry_to_simplify_commit(commit);\n \n+\tparent = commit->parents;\n \twhile (parent) {\n \t\tstruct commit *p = parent->item;\n \n@@ -558,33 +603,6 @@ static void add_parents_to_list(struct c\n \t}\n }\n \n-static void compress_list(struct commit_list *list)\n-{\n-\twhile (list) {\n-\t\tstruct commit *commit = list->item;\n-\t\tstruct commit_list *parent = commit->parents;\n-\t\tlist = list->next;\n-\n-\t\tif (!parent) {\n-\t\t\tif (!same_tree_as_empty(commit->tree))\n-\t\t\t\tcommit->object.flags |= TREECHANGE;\n-\t\t\tcontinue;\n-\t\t}\n-\n-\t\t/*\n-\t\t * Exactly one parent? Check if it leaves the tree\n-\t\t * unchanged\n-\t\t */\n-\t\tif (!parent->next) {\n-\t\t\tstruct tree *t1 = commit->tree;\n-\t\t\tstruct tree *t2 = parent->item->tree;\n-\t\t\tif (!t1 || !t2 || same_tree(t1, t2))\n-\t\t\t\tcontinue;\n-\t\t}\n-\t\tcommit->object.flags |= TREECHANGE;\n-\t}\n-}\n-\n static struct commit_list *limit_list(struct commit_list *list)\n {\n \tstruct commit_list *newlist = NULL;\n@@ -614,8 +632,6 @@ static struct commit_list *limit_list(st\n \t}\n \tif (tree_objects)\n \t\tmark_edges_uninteresting(newlist);\n-\tif (paths && dense)\n-\t\tcompress_list(newlist);\n \tif (bisect_list)\n \t\tnewlist = find_bisection(newlist);\n \treturn newlist;\n@@ -808,6 +824,10 @@ int main(int argc, const char **argv)\n \t\t\tdense = 0;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strcmp(arg, \"--remove-empty\")) {\n+\t\t\tremove_empty_trees = 1;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--\")) {\n \t\t\ti++;\n \t\t\tbreak;\n"},{"id":"14854","messageId":"Pine.LNX.4.64.0601181513030.3240@g5.osdl.org","threadId":"3093","inReplyTo":"Pine.LNX.4.64.0601181414530.3240@g5.osdl.org","subject":"Re: Joining Repositories","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-01-18T23:15:06Z","receivedAt":"2006-01-18T23:15:06Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 18 Jan 2006, Linus Torvalds wrote:\n> \n> As an example, let's say that you have a git repository that has had a \n> file called \"A\" some time in the past. That file gets renamed to B, and \n> then gets renamed back again to A. The old \"git-rev-list\" would show two \n> commits commits: the commit that renames B to A (because it changes A) \n> _and_ as its parent the commit that renames A to B (because it changes A).\n\nBtw, in case somebody doesn't see what's wrong with that (because those \ntwo commits happen to be the only ones that matter), let's point out that \nthe file may have changed while it was called \"B\". So to do annotation, we \nreally need to _stop_ following the filename, start following the previous\none.\n\nNow, the logic to do that could have been in the \"annotate\" parts, but the \ndecision to stop following the chain actually makes sense as part of \ngit-rev-list, since it does the tree difference tracking anyway.\n\n\t\tLinus\n"},{"id":"14863","messageId":"7v64ohotjj.fsf@assigned-by-dhcp.cox.net","threadId":"3093","inReplyTo":"7vbqy9xx2r.fsf@assigned-by-dhcp.cox.net","subject":"Re: Joining Repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-19T02:12:00Z","receivedAt":"2006-01-19T02:12:00Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <junkio@cox.net> writes:\n\n> Linus Torvalds <torvalds@osdl.org> writes:\n>\n>> Now, let's see what Junio comes up with,...\n>\n> Well, since the reason the original RFI came was...\n> ...\n> So for this case, I think the \"coolest merge\" is the right\n> thing to do.\n\nI said this only because I wanted to recommend a solution\navailable today to the original requester.\n\nIf I were to talk about (one potential) future solution, I would\nsay \"bind\" commit would work naturally for this case, without\ndisrupting the development histories of the projects being\ncombined.  After all, the \"bind\" commit approach is not much\ndifferent from what you did with \"the coolest merge ever\", with\na few twists, such as (1) not overlaying but requiring each\nsubproject to be bound at a non-overlapping subdirectory, and\n(2) letting a subproject keep its own independent history that\ndoes not know about other subprojects even after combination\nhappens (as opposed to the coolest merge, after which there is\nno git history that does not have gitk as its part).\n\nOf course, \"gitlink\" would equally work well and have similar\ncharacteristics.  The implementation details will certainly\ndiffer and the user interaction may somewhat be different as\nwell, but I do not think of any reason either approach would not\nwork well in this situation.\n\nSo would Cogito subprojects support (will it be in the 0.17?),\nbut it might look less powerful when compared with the \"bind\" or\n\"gitlink\", if it comes in the form of the patch Pasky sent to\nthe list, since it does not attempt to tie the versions of two\nprojects together.  Certainly that makes it the easiest to use\nand understand what is going on, though.\n"},{"id":"14879","messageId":"20060119090348.GX28365@pasky.or.cz","threadId":"3093","inReplyTo":"7v64ohotjj.fsf@assigned-by-dhcp.cox.net","subject":"Re: Joining Repositories","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-01-19T09:03:48Z","receivedAt":"2006-01-19T09:03:48Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Thu, Jan 19, 2006 at 03:12:00AM CET, I got a letter\nwhere Junio C Hamano <junkio@cox.net> said that...\n> (2) letting a subproject keep its own independent history that\n> does not know about other subprojects even after combination\n> happens (as opposed to the coolest merge, after which there is\n> no git history that does not have gitk as its part).\n\nWhich can be an advantage or disadvantage depending on the situation,\nbut I suspect that it would be rather a disadvantage here.\n\n> So would Cogito subprojects support (will it be in the 0.17?),\n\nProbably in 0.18. I would have the subprojects support complete when\nit's already in, and it will be some work, while I don't want to\npostpone 0.17 much anymore. But we'll see. ;)\n\n> but it might look less powerful when compared with the \"bind\" or\n> \"gitlink\", if it comes in the form of the patch Pasky sent to\n> the list, since it does not attempt to tie the versions of two\n> projects together.  Certainly that makes it the easiest to use\n> and understand what is going on, though.\n\nBy now (I wanted to note that in the original thread when I get to make\na new patch), I've decided to extend the format to allow extra\ninformation stored for the subprojects:\n\n\tpath\\t...\n\nThis means that I will have to do preprocessing when using this as an\nexclude file, but it will let me do the stuff like read-only\nsubprojects. I could also tie it to a specific version here, but if\ngitlink/binds are after all going to get included in Git, I'd rather use\nthat when it is ready (although the single-git-directory approach does\nnot make sense to me; but I didn't read part of the thread very\ncarefully yet).\n\nI think the approaches might be complementary, Cogito subprojects are\njust designed for different scenarios than gitlinks.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\nOf the 3 great composers Mozart tells us what it's like to be human,\nBeethoven tells us what it's like to be Beethoven and Bach tells us\nwhat it's like to be the universe.  -- Douglas Adams\n"},{"id":"14885","messageId":"200601191236.58725.Mathias.Waack@rantzau.de","threadId":"3093","inReplyTo":"20060118141442.GP28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Mathias Waack","fromEmail":"mathias.waack@rantzau.de","sentAt":"2006-01-19T11:36:58Z","receivedAt":"2006-01-19T11:36:58Z","isPatch":false,"sender":{"key":"mathias.waack@rantzau.de","avatar":null},"body":"Hello Petr, \n\nOn Wednesday 18 January 2006 03:14 pm, Petr Baudis wrote:\n> > ...git-read-tree doesn't know the parameter --prefix (just downloaded and\n> > tried it on 0.99.9i). What version shall I use?\n>\n>   oops, it seems this is only in the latest pu branch of git. If you are\n> not brave enough for that, \n\nI think I am;) \n\n> you will need to use the prefix facility of \n> checkout-index instead, and it'll take much longer:\n>\n> \tgit-read-tree $commit\n> \tgit-checkout-index -a --prefix=r1/\n> \trm .git/index\n> \tcg-add -r r1/\n> \trm -rf r1\n> \t.. then proceed with git-cat-file etc ..\n\nTime doesn't matter (in this case). This solution seems to work. \n\n>   Note that I'm not sure when which feature was introduced. Your best\n> bet is to just use the latest stable GIT/Cogito versions.\n\nThanks for your suggestions, I've got a working solution. Now I'm going to \nspend the remaining days of this week by trying to understand why it works. \n\nMathias\n"},{"id":"14907","messageId":"7vlkxcdqjc.fsf@assigned-by-dhcp.cox.net","threadId":"3093","inReplyTo":"20060118141442.GP28365@pasky.or.cz","subject":"Re: Joining Repositories","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-01-19T18:24:39Z","receivedAt":"2006-01-19T18:24:39Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis <pasky@suse.cz> writes:\n\n>   oops, it seems this is only in the latest pu branch of git. If you are\n> not brave enough for that, you will need to use the prefix facility of\n> checkout-index instead, and it'll take much longer:\n>...\n>   Note that I'm not sure when which feature was introduced. Your best\n> bet is to just use the latest stable GIT/Cogito versions.\n\nThe \"read-tree --prefix\" and friends were only talked about\nduring the \"bind\" commit discussion, and with some experimental\ncode in \"pu\".  Here is my thought on their readiness:\n\n - \"read-tree --prefix=<prefix>/\" may independently be useful\n   even if we choose not to go \"bind\" commit approach.  However\n   what it does when the index already has something at\n   <prefix>/ has room for improvement.  I think it should have\n   an option to do an equivalent of either one-way merge or\n   two-way merge for that part of subtree.  Currently it always\n   rejects if the current index has anything there.\n\n - \"write-tree --prefix=<prefix>/\" is redundant for the purpose\n   of \"bind\" commit, because we would write the whole tree to be\n   recorded in the enclosing project and it is easy to pick the\n   subproject part with ls-tree from such a tree.  So it needs\n   an independent advocate / rationale before graduating from\n   \"pu\".\n\n - \"write-tree --bound=<prefix>/ --bound=<prefix>/ ...\" cannot\n   be emulated by any other way and may be independently useful\n   outside \"bind\" commit context.  Maybe the option should be\n   renamed to --exclude=<prefix>/ or somesuch before going to\n   \"master\".\n\n - \"commit-tree --bind\" should stay in \"pu\" until other pieces\n   to correctly deal with commit objects with \"bind\" lines are\n   ready, including fsck-objects and rev-list.  It _might_ be\n   safer to bump the core.repositoryformatversion automatically\n   once you have such a commit object in your repo.\n"}]}