{"thread":{"id":"18095","subject":"move files between disparate repos and maintain version history","startedAt":"2009-03-02T20:30:58Z","lastAt":"2009-03-03T19:27:43Z","messageCount":7,"participants":["davetron5000","Jeff King","David Copeland","Peter Baumann"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"106723","messageId":"0d8965bb-e2ed-4f49-b323-c110f605ae2c@33g2000yqm.googlegroups.com","threadId":"18095","inReplyTo":null,"subject":"move files between disparate repos and maintain version history","fromName":"davetron5000","fromEmail":"davetron5000@gmail.com","sentAt":"2009-03-02T20:30:58Z","receivedAt":"2009-03-02T20:30:58Z","isPatch":false,"sender":{"key":"davetron5000@gmail.com","avatar":"https://gravatar.com/avatar/9ec09086e36c4e609dfc23a4582d2c7adf7dc2fb1796f74492d52727efa6ba86?d=mp&s=160"},"body":"Using git-svn at work, and had to do some svn moves that I'm wondering\nwould be handled with git?\n\nOur repo is setup like this:\n\n$SVNROOT/main/core/{trunk,tags,branches}\n$SVNROOT/main/app/{trunk,tags,branches}\n\ncore and app are treated as different project roots internally, and my\ngit-svn treats them as such.\n\nWe recently needed to move some files from app to core. I could've\njust moved them in between my git repos, but the version history would\nbe lost, since these are two separate repos. I did the move via svn\nmv, which works simply because our \"separate projects\" are really just\nin the same SVN repo.\n\nSo, is there a way I can move a file between two git repositories,\nmaintaining the change history? I guess it would be something like\n\"apply all patches that this file was involved in, but ONLY apply the\nones affecting this file, to the new repo, then delete from the old\"?\n"},{"id":"106754","messageId":"20090303041300.GA18136@coredump.intra.peff.net","threadId":"18095","inReplyTo":"0d8965bb-e2ed-4f49-b323-c110f605ae2c@33g2000yqm.googlegroups.com","subject":"Re: move files between disparate repos and maintain version history","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-03T04:13:00Z","receivedAt":"2009-03-03T04:13:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Mar 02, 2009 at 12:30:58PM -0800, davetron5000 wrote:\n\n> So, is there a way I can move a file between two git repositories,\n> maintaining the change history? I guess it would be something like\n> \"apply all patches that this file was involved in, but ONLY apply the\n> ones affecting this file, to the new repo, then delete from the old\"?\n\nYes, that's more or less how you would do it. There are actually two\nseparate issues, and each has two possible solutions.\n\nIssue 1 is moving the history to the new repo.\n\nOne solution is, as you guessed, to export as patches and import into\nthe new repo:\n\n cd /path/to/app\n git format-patch --stdout --root -- >~/patches\n cd /path/to/core\n git am ~/patches\n\nwhere can be a file, directory, or a list of either or both.\nThis should work fine if the history of that path is linear, since a\nlist of patches is, by definition, linear. You can see the \"shape\" of\nthe history with:\n\n cd /path/to/app\n gitk -- \n\nThe other solution is to actually rewrite a version of git history that\nsees only those paths, then merge it in. That will retain the actual\nshape of the history. You can do this using \"git filter-branch\":\n\n cd /path/to/app\n # we do our work on a temporary branch\n git branch just-path\n # the actual filter; note you could do more than just grep here\n # if you wanted to munge the pathnames\n git filter-branch --index-filter '\n git ls-files -s | grep path |\n GIT_INDEX_FILE=\"$GIT_INDEX_FILE.new\" git update-index --index-info &&\n mv \"$GIT_INDEX_FILE.new\" \"$GIT_INDEX_FILE\"\n ' --prune-empty just-path\n\n # you can inspect just-path at this point with \"gitk just-path\"\n cd /path/to/core\n # and then pull it into core's history\n git pull /path/to/app just-path\n\nThe second issue is what you want to have happen in the \"app\"\nrepository. Presumably you no longer want the moved files there. You\n_could_ filter-branch to pretend as if they were always in \"core\" and\nnever in \"app\". But that is probably not a good idea because:\n\n 1. You are rewriting history, which will make merging more complex for\n your users.\n\n 2. You may be breaking historical builds which expect the files to be\n there in the past.\n\nSo I think you're probably best to just remove them with a new commit\nand have the duplicated history in both.\n\n-Peff\n\nPS I think you might be able to replace the filter-branch invocation\nwith a fast-export / fast-import pipeline, but I haven't tried.\n"},{"id":"106882","messageId":"f95d47890903030858xb398b5fy1aeabb19166e6077@mail.gmail.com","threadId":"18095","inReplyTo":"20090303041300.GA18136@coredump.intra.peff.net","subject":"Re: move files between disparate repos and maintain version history","fromName":"David Copeland","fromEmail":"davetron5000@gmail.com","sentAt":"2009-03-03T16:58:42Z","receivedAt":"2009-03-03T16:58:42Z","isPatch":false,"sender":{"key":"davetron5000@gmail.com","avatar":"https://gravatar.com/avatar/9ec09086e36c4e609dfc23a4582d2c7adf7dc2fb1796f74492d52727efa6ba86?d=mp&s=160"},"body":"The first option worked, insomuch the history of diffs is preserved,\nbut the dates are all today. The second option was a little over my\nhead; is the idea there that you are setting up a branch that has ONLY\nthe files I care about (with all their history), and then I pull from\nthe other repo as if they are related? That seems like it might\npreserve the dates...\n\nDave\n\nOn Mon, Mar 2, 2009 at 11:13 PM, Jeff King wrote:\n> On Mon, Mar 02, 2009 at 12:30:58PM -0800, davetron5000 wrote:\n>\n>> So, is there a way I can move a file between two git repositories,\n>> maintaining the change history?  I guess it would be something like\n>> \"apply all patches that this file was involved in, but ONLY apply the\n>> ones affecting this file, to the new repo, then delete from the old\"?\n>\n> Yes, that's more or less how you would do it. There are actually two\n> separate issues, and each has two possible solutions.\n>\n> Issue 1 is moving the history to the new repo.\n>\n> One solution is, as you guessed, to export as patches and import into\n> the new repo:\n>\n>  cd /path/to/app\n>  git format-patch --stdout --root -- >~/patches\n>  cd /path/to/core\n>  git am ~/patches\n>\n> where can be a file, directory, or a list of either or both.\n> This should work fine if the history of that path is linear, since a\n> list of patches is, by definition, linear. You can see the \"shape\" of\n> the history with:\n>\n>  cd /path/to/app\n>  gitk -- \n>\n> The other solution is to actually rewrite a version of git history that\n> sees only those paths, then merge it in. That will retain the actual\n> shape of the history. You can do this using \"git filter-branch\":\n>\n>  cd /path/to/app\n>  # we do our work on a temporary branch\n>  git branch just-path\n>  # the actual filter; note you could do more than just grep here\n>  # if you wanted to munge the pathnames\n>  git filter-branch --index-filter '\n>    git ls-files -s | grep path |\n>      GIT_INDEX_FILE=\"$GIT_INDEX_FILE.new\" git update-index --index-info &&\n>    mv \"$GIT_INDEX_FILE.new\" \"$GIT_INDEX_FILE\"\n>  ' --prune-empty just-path\n>\n>  # you can inspect just-path at this point with \"gitk just-path\"\n>  cd /path/to/core\n>  # and then pull it into core's history\n>  git pull /path/to/app just-path\n>\n> The second issue is what you want to have happen in the \"app\"\n> repository. Presumably you no longer want the moved files there. You\n> _could_ filter-branch to pretend as if they were always in \"core\" and\n> never in \"app\". But that is probably not a good idea because:\n>\n>  1. You are rewriting history, which will make merging more complex for\n>     your users.\n>\n>  2. You may be breaking historical builds which expect the files to be\n>     there in the past.\n>\n> So I think you're probably best to just remove them with a new commit\n> and have the duplicated history in both.\n>\n> -Peff\n>\n> PS I think you might be able to replace the filter-branch invocation\n> with a fast-export / fast-import pipeline, but I haven't tried.\n>\n"},{"id":"106889","messageId":"20090303171835.GB454@coredump.intra.peff.net","threadId":"18095","inReplyTo":"f95d47890903030858xb398b5fy1aeabb19166e6077@mail.gmail.com","subject":"Re: move files between disparate repos and maintain version history","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-03T17:18:35Z","receivedAt":"2009-03-03T17:18:35Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 03, 2009 at 11:58:42AM -0500, David Copeland wrote:\n\n> The first option worked, insomuch the history of diffs is preserved,\n> but the dates are all today.\n\nThat's odd. It works fine here. Can you confirm that the correct dates\nin the \"patches\" file (i.e., the output of format-patch)? What are you\nusing to look at the patches? Note that gitk will show you both the\n\"committer\" and the \"author\" fields. The \"author\" field should have the\noriginal author and time of the patch, but the \"committer\" will be you,\ntoday.\n\n> The second option was a little over my head; is the idea there that\n> you are setting up a branch that has ONLY the files I care about (with\n> all their history), and then I pull from the other repo as if they are\n> related? That seems like it might preserve the dates...\n\nYes, that is exactly what is happening in the second example.\n\n-Peff\n"},{"id":"106902","messageId":"f95d47890903031008s36873f6ex94c7096f79cd6de@mail.gmail.com","threadId":"18095","inReplyTo":"20090303171835.GB454@coredump.intra.peff.net","subject":"Re: move files between disparate repos and maintain version history","fromName":"David Copeland","fromEmail":"davetron5000@gmail.com","sentAt":"2009-03-03T18:08:17Z","receivedAt":"2009-03-03T18:08:17Z","isPatch":false,"sender":{"key":"davetron5000@gmail.com","avatar":"https://gravatar.com/avatar/9ec09086e36c4e609dfc23a4582d2c7adf7dc2fb1796f74492d52727efa6ba86?d=mp&s=160"},"body":"The patch file looks correct. I'm wondering if this is a result of\nboth repos being connected to svn?\n\nmy process was:\n\n- format patch\n- go to other repo\n- git svn rebase\n- apply patch\n- git svn dcommit\n\nCould dcommit change the dates since, to svn, they are appear as\ncommits right now?\n\nDave\n\nOn Tue, Mar 3, 2009 at 12:18 PM, Jeff King wrote:\n> On Tue, Mar 03, 2009 at 11:58:42AM -0500, David Copeland wrote:\n>\n>> The first option worked, insomuch the history of diffs is preserved,\n>> but the dates are all today.\n>\n> That's odd. It works fine here. Can you confirm that the correct dates\n> in the \"patches\" file (i.e., the output of format-patch)? What are you\n> using to look at the patches? Note that gitk will show you both the\n> \"committer\" and the \"author\" fields. The \"author\" field should have the\n> original author and time of the patch, but the \"committer\" will be you,\n> today.\n>\n>> The second option was a little over my head; is the idea there that\n>> you are setting up a branch that has ONLY the files I care about (with\n>> all their history), and then I pull from the other repo as if they are\n>> related?  That seems like it might preserve the dates...\n>\n> Yes, that is exactly what is happening in the second example.\n>\n> -Peff\n>\n"},{"id":"106913","messageId":"20090303192224.GB9964@m62s10.vlinux.de","threadId":"18095","inReplyTo":"f95d47890903031008s36873f6ex94c7096f79cd6de@mail.gmail.com","subject":"Re: move files between disparate repos and maintain version history","fromName":"Peter Baumann","fromEmail":"waste.manager@gmx.de","sentAt":"2009-03-03T19:22:24Z","receivedAt":"2009-03-03T19:22:24Z","isPatch":false,"sender":{"key":"waste.manager@gmx.de","avatar":null},"body":"On Tue, Mar 03, 2009 at 01:08:17PM -0500, David Copeland wrote:\n> The patch file looks correct. I'm wondering if this is a result of\n> both repos being connected to svn?\n> \n> my process was:\n> \n> - format patch\n> - go to other repo\n\n> - git svn rebase\n\nThis will change the date of commits already in svn because it uses\ngit rebase (with all its problems, see its manpage)\n\n> - apply patch\n> - git svn dcommit\n> \n> Could dcommit change the dates since, to svn, they are appear as\n> commits right now?\n> \n> Dave\n> \n> On Tue, Mar 3, 2009 at 12:18 PM, Jeff King wrote:\n> > On Tue, Mar 03, 2009 at 11:58:42AM -0500, David Copeland wrote:\n> >\n> >> The first option worked, insomuch the history of diffs is preserved,\n> >> but the dates are all today.\n> >\n> > That's odd. It works fine here. Can you confirm that the correct dates\n> > in the \"patches\" file (i.e., the output of format-patch)? What are you\n> > using to look at the patches? Note that gitk will show you both the\n> > \"committer\" and the \"author\" fields. The \"author\" field should have the\n> > original author and time of the patch, but the \"committer\" will be you,\n> > today.\n> >\n> >> The second option was a little over my head; is the idea there that\n> >> you are setting up a branch that has ONLY the files I care about (with\n> >> all their history), and then I pull from the other repo as if they are\n> >> related?  That seems like it might preserve the dates...\n> >\n> > Yes, that is exactly what is happening in the second example.\n> >\n> > -Peff\n"},{"id":"106916","messageId":"20090303192743.GA3530@coredump.intra.peff.net","threadId":"18095","inReplyTo":"f95d47890903031008s36873f6ex94c7096f79cd6de@mail.gmail.com","subject":"Re: move files between disparate repos and maintain version history","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-03-03T19:27:43Z","receivedAt":"2009-03-03T19:27:43Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 03, 2009 at 01:08:17PM -0500, David Copeland wrote:\n\n> The patch file looks correct. I'm wondering if this is a result of\n> both repos being connected to svn?\n> \n> my process was:\n> \n> - format patch\n> - go to other repo\n> - git svn rebase\n> - apply patch\n> - git svn dcommit\n> \n> Could dcommit change the dates since, to svn, they are appear as\n> commits right now?\n\nI know very little about git-svn. But you could test your theory by\nchecking the dates between the penultimate and ultimate steps.\n\n-Peff\n"}]}