{"thread":{"id":"24121","subject":"Fixing branches from a (really messed up) subversion repository","startedAt":"2010-06-15T16:26:16Z","lastAt":"2010-06-18T21:40:58Z","messageCount":6,"participants":["Joshua Shrader","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"143759","messageId":"AANLkTinmU3tOkv9usjGogHn5cXm07FMde8ryDKuWNKD_@mail.gmail.com","threadId":"24121","inReplyTo":null,"subject":"Fixing branches from a (really messed up) subversion repository","fromName":"Joshua Shrader","fromEmail":"jshrader83@gmail.com","sentAt":"2010-06-15T16:26:16Z","receivedAt":"2010-06-15T16:26:16Z","isPatch":false,"sender":{"key":"jshrader83@gmail.com","avatar":null},"body":"I've finally convinced my project to switch to Git... However, when\nmy project first began, the people responsible for branching/tagging\nwere fairly unfamiliar with standard procedures. As such, the\nbeginnings of our repository look a little something like this:\n\n.\n`----project\n |\n `branch1\n | |\n | `----project\n | `src\n `branch2\n | |\n | `----project\n | `src\n `src\n\nThat is, the branches were essentially inside the trunk. In order to\ncheck out the \"mainline\" of development, you also got all of the\nbranches. After a few branches, they realized that this was a\nproblem, and someone found out about the trunk/branches/tags paradigm.\n Our branches were svn mv'd to separate directories under \"branches\",\nand all was well. Now everything looked like a sane version control\nrepository.\n\nNow, we want to import this into Git. I tried git svn import\n--stdlayout..., but this just stalled. I'm assuming because the first\nversion didn't have a trunk/branches/tags structure. So, I had to\nimport it leaving off the --stdlayout option. Is there any way to\nmanually fix this in Git? After the import is complete, my master\nbranch will have a trunk, branches, and tags directory in it. Can I\ncreate real branches out of the directories under the branches\ndirectory and then remove them from master?\n\nAny help is greatly appreciated. I've been hoping for a while that\nwe'd switch to Git, and I don't want this to hang us up.\n\nThanks,\n\nJosh\n"},{"id":"143760","messageId":"AANLkTink-Rm44xnzBHVo6W_29YCPrjScnolqx_9B8A5G@mail.gmail.com","threadId":"24121","inReplyTo":"AANLkTinmU3tOkv9usjGogHn5cXm07FMde8ryDKuWNKD_@mail.gmail.com","subject":"Re: Fixing branches from a (really messed up) subversion repository","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-15T16:29:47Z","receivedAt":"2010-06-15T16:29:47Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Jun 15, 2010 at 16:26, Joshua Shrader wrote:\n> I've finally convinced my project to switch to Git...  However, when\n> my project first began, the people responsible for branching/tagging\n> were fairly unfamiliar with standard procedures.  As such, the\n> beginnings of our repository look a little something like this:\n>\n> .\n> `----project\n>     |\n>     `branch1\n>     |          |\n>     |          `----project\n>     |                        `src\n>     `branch2\n>     |          |\n>     |          `----project\n>     |                        `src\n>     `src\n>\n> That is, the branches were essentially inside the trunk.  In order to\n> check out the \"mainline\" of development, you also got all of the\n> branches.  After a few branches, they realized that this was a\n> problem, and someone found out about the trunk/branches/tags paradigm.\n>  Our branches were svn mv'd to separate directories under \"branches\",\n> and all was well.  Now everything looked like a sane version control\n> repository.\n>\n> Now, we want to import this into Git.  I tried git svn import\n> --stdlayout..., but this just stalled.  I'm assuming because the first\n> version didn't have a trunk/branches/tags structure.  So, I had to\n> import it leaving off the --stdlayout option.  Is there any way to\n> manually fix this in Git?  After the import is complete, my master\n> branch will have a trunk, branches, and tags directory in it.  Can I\n> create real branches out of the directories under the branches\n> directory and then remove them from master?\n>\n> Any help is greatly appreciated.  I've been hoping for a while that\n> we'd switch to Git, and I don't want this to hang us up.\n\nI have the same problem on another project, and after looking at the\ncomponents involved it seems that the easiest way is to filter the SVN\ndump, then import it into Git.\n\nIt's quite easy to write a filter with SVN::Dump that does this,\ni.e. just do a few string replacements on the paths before a given\nrevision, and drop the revision where you moved things.\n\nA lot easier than convincing git-svn to do what you want, anyway.\n"},{"id":"143770","messageId":"AANLkTim6pBqc1qQhvGrzmf5FDbVkfw1GmNP00BM_99n4@mail.gmail.com","threadId":"24121","inReplyTo":"AANLkTink-Rm44xnzBHVo6W_29YCPrjScnolqx_9B8A5G@mail.gmail.com","subject":"Re: Fixing branches from a (really messed up) subversion repository","fromName":"Joshua Shrader","fromEmail":"jshrader83@gmail.com","sentAt":"2010-06-15T16:40:38Z","receivedAt":"2010-06-15T16:40:38Z","isPatch":false,"sender":{"key":"jshrader83@gmail.com","avatar":null},"body":"Not quite sure if I understand correctly. Can Git import directly\nfrom an SVN dump, or are you suggesting to essentially re-build the\nsvn repository correctly, and then import?\n\nOn Tue, Jun 15, 2010 at 12:29 PM, Ævar Arnfjörð Bjarmason\n wrote:\n> On Tue, Jun 15, 2010 at 16:26, Joshua Shrader wrote:\n>> I've finally convinced my project to switch to Git...  However, when\n>> my project first began, the people responsible for branching/tagging\n>> were fairly unfamiliar with standard procedures.  As such, the\n>> beginnings of our repository look a little something like this:\n>>\n>> .\n>> `----project\n>>     |\n>>     `branch1\n>>     |          |\n>>     |          `----project\n>>     |                        `src\n>>     `branch2\n>>     |          |\n>>     |          `----project\n>>     |                        `src\n>>     `src\n>>\n>> That is, the branches were essentially inside the trunk.  In order to\n>> check out the \"mainline\" of development, you also got all of the\n>> branches.  After a few branches, they realized that this was a\n>> problem, and someone found out about the trunk/branches/tags paradigm.\n>>  Our branches were svn mv'd to separate directories under \"branches\",\n>> and all was well.  Now everything looked like a sane version control\n>> repository.\n>>\n>> Now, we want to import this into Git.  I tried git svn import\n>> --stdlayout..., but this just stalled.  I'm assuming because the first\n>> version didn't have a trunk/branches/tags structure.  So, I had to\n>> import it leaving off the --stdlayout option.  Is there any way to\n>> manually fix this in Git?  After the import is complete, my master\n>> branch will have a trunk, branches, and tags directory in it.  Can I\n>> create real branches out of the directories under the branches\n>> directory and then remove them from master?\n>>\n>> Any help is greatly appreciated.  I've been hoping for a while that\n>> we'd switch to Git, and I don't want this to hang us up.\n>\n> I have the same problem on another project, and after looking at the\n> components involved it seems that the easiest way is to filter the SVN\n> dump, then import it into Git.\n>\n> It's quite easy to write a filter with SVN::Dump that does this,\n> i.e. just do a few string replacements on the paths before a given\n> revision, and drop the revision where you moved things.\n>\n> A lot easier than convincing git-svn to do what you want, anyway.\n>\n"},{"id":"143764","messageId":"AANLkTin268tbt5E2flbzwq8Rik0D21Uf4t2z-5fgb4BZ@mail.gmail.com","threadId":"24121","inReplyTo":"AANLkTim6pBqc1qQhvGrzmf5FDbVkfw1GmNP00BM_99n4@mail.gmail.com","subject":"Re: Fixing branches from a (really messed up) subversion repository","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-15T16:42:31Z","receivedAt":"2010-06-15T16:42:31Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Tue, Jun 15, 2010 at 16:40, Joshua Shrader wrote:\n> Not quite sure if I understand correctly.  Can Git import directly\n> from an SVN dump, or are you suggesting to essentially re-build the\n> svn repository correctly, and then import?\n\nI mean:\n\n 1. svnadmin dump /path/to/repo > svn.dump\n 2. *filter svn.dump*\n 3. svnadmin load ...\n 4. git svn /path/to/loaded/dump\n"},{"id":"143922","messageId":"AANLkTimxivcSShukg0kFPF_7ignIv3d3TUl8KELKUksF@mail.gmail.com","threadId":"24121","inReplyTo":"AANLkTingIuZQ8BB9NW0UpGt7a9uccnBX0_R7YZwJpVsd@mail.gmail.com","subject":"Re: Fixing branches from a (really messed up) subversion repository","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-06-18T21:01:44Z","receivedAt":"2010-06-18T21:01:44Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Fri, Jun 18, 2010 at 20:59, Joshua Shrader wrote:\n> Thanks.  It took a couple days to get everything straightened out, but\n> this seems to have worked very well.\n\nMind sharing the details or code for what you did?\n\nI've only proved to myself that this approach will work, I haven't\nactually done it.\n"},{"id":"143925","messageId":"AANLkTin0zwH8KLr9Cem-piRZ0mn9Q9ZScmFJF2r_DTi4@mail.gmail.com","threadId":"24121","inReplyTo":"AANLkTimxivcSShukg0kFPF_7ignIv3d3TUl8KELKUksF@mail.gmail.com","subject":"Re: Fixing branches from a (really messed up) subversion repository","fromName":"Joshua Shrader","fromEmail":"jshrader83@gmail.com","sentAt":"2010-06-18T21:40:58Z","receivedAt":"2010-06-18T21:40:58Z","isPatch":false,"sender":{"key":"jshrader83@gmail.com","avatar":null},"body":"I started with svnadmin dump /path/to/repo > svn.dump. Then I had to\nmanually edit svn.dump (I'm sure someone smarter then me could roll\nthis into the Perl script I used in the next step) to create a \"trunk\"\nand \"branches\" directory in the first commit. I found the commits\nthat added \"trunk\" and \"branches\" when we re-organized our repo to\nfollow this convention, and deleted them. Then I had to locate where\nall of the branches were created - looked like\n\nNode-path: branchname\nNode-kind: dir\nNode-action: add\nNode-copyfrom-rev: 23\nNode-copyfrom-path:\n\nand change that blank \"Node-copyfrom-path: \" to \"Node-copyfrom-path:\ntrunk\". The blank path is what really caused issues because everytime\na branch was created, it would contain copies of branches which\ncontained copies of branches, etc... So branch2 had a subfolder\nbranch1, and branch3 contained branch2 and branch 1, where branch 2\ncontained branch 1...\n\nThen, I removed all of the contents of the commits (not the commits\nthemselves) where we re-organized the repo. i.e.\n\nNode-path: trunk/src\nNode-action: add\nNode-copyfrom-ref: 23\nNode-copyfrom-path: src\n\nand the ones where we deleted nested branches and moved the individual\nbranches into a \"branches\" directory.\n\nI made these edits and saved them. Then I ran that file through a\nperl filter, tried to svnadmin load newrepo < filteredDumpFile, and if\nI got errors, I iterated - changed the pre-filtered file, blow away\nthe svn repo that gave me an error during load, re-create it, and\nre-load. I iterated this over several days, as I discovered more and\nmore inconsistencies with my edited dump file. I think I have all of\nthe steps above, but I may be missing one or two, and if I am, I\napologize. I'd imagine everyone's situation would be a bit different,\nso adjust accordingly.\n\nNow, I'm quite the Perl hack so I doubt you want to look at the\nscript. It essentially did the following:\n\nRead each line of the dump file.\nIf the line doesn't start with \"Node-path:\" or \"Node-copyfrom-path:\",\nthen just print it un-altered.\nIf it does,\n does what follows the ':' start with \"trunk\", \"branches\", or \"tags\"?\n If so, then just print un-altered.\nOtherwise,\n for each $branchname(@branchnames) if what follows the ':' begins\nwith $branchname, replace it with \"branches/$branchname\", and print.\nIf none of the branch names match,\n prepend \"trunk\" to the beginning of the path, and print.\n\nThis essentially re-writes all of the paths so that everything not in\na branch, or already under \"branches\", \"tags\", or \"trunk\", is put\nunder trunk. It also moves all of the /branchenames to\n/branches/branchnames.\n\nAfter this, we did\nsvnadmin hotcopy repo repo-backup\nrm -rf repo\nsvnadmin create repo\nsvnadmin load repo wrote:\n> On Fri, Jun 18, 2010 at 20:59, Joshua Shrader wrote:\n>> Thanks.  It took a couple days to get everything straightened out, but\n>> this seems to have worked very well.\n>\n> Mind sharing the details or code for what you did?\n>\n> I've only proved to myself that this approach will work, I haven't\n> actually done it.\n>\n"}]}