Fixing branches from a (really messed up) subversion repository

I've finally convinced my project to switch to Git... However, when my project first began, the people responsible for branching/tagging were fairly unfamiliar with standard procedures. As such, the beginnings of our repository look a little something like this:

.
`----project
     |
     `branch1
     |          |
     |          `----project
     |                        `src
     `branch2
     |          |
     |          `----project
     |                        `src
     `src

That is, the branches were essentially inside the trunk.  In order to
check out the "mainline" of development, you also got all of the
branches.  After a few branches, they realized that this was a
problem, and someone found out about the trunk/branches/tags paradigm.
 Our branches were svn mv'd to separate directories under "branches",
and all was well.  Now everything looked like a sane version control
repository.

Now, we want to import this into Git. I tried git svn import --stdlayout..., but this just stalled. I'm assuming because the first version didn't have a trunk/branches/tags structure. So, I had to import it leaving off the --stdlayout option. Is there any way to manually fix this in Git? After the import is complete, my master branch will have a trunk, branches, and tags directory in it. Can I create real branches out of the directories under the branches directory and then remove them from master?

Any help is greatly appreciated. I've been hoping for a while that we'd switch to Git, and I don't want this to hang us up.

Thanks,

Josh

On Tue, Jun 15, 2010 at 16:26, Joshua Shrader <jshrader83@gmail.com> wrote:

I have the same problem on another project, and after looking at the components involved it seems that the easiest way is to filter the SVN dump, then import it into Git.

It's quite easy to write a filter with SVN::Dump that does this, i.e. just do a few string replacements on the paths before a given revision, and drop the revision where you moved things.

A lot easier than convincing git-svn to do what you want, anyway.

Not quite sure if I understand correctly. Can Git import directly from an SVN dump, or are you suggesting to essentially re-build the svn repository correctly, and then import?

On Tue, Jun 15, 2010 at 12:29 PM, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:

On Tue, Jun 15, 2010 at 16:40, Joshua Shrader <jshrader83@gmail.com> wrote:

I mean:

  1. svnadmin dump /path/to/repo > svn.dump
  2. *filter svn.dump*
  3. svnadmin load ...
  4. git svn /path/to/loaded/dump

On Fri, Jun 18, 2010 at 20:59, Joshua Shrader <jshrader83@gmail.com> wrote:

Mind sharing the details or code for what you did?

I've only proved to myself that this approach will work, I haven't actually done it.

I started with svnadmin dump /path/to/repo > svn.dump. Then I had to manually edit svn.dump (I'm sure someone smarter then me could roll this into the Perl script I used in the next step) to create a "trunk" and "branches" directory in the first commit. I found the commits that added "trunk" and "branches" when we re-organized our repo to follow this convention, and deleted them. Then I had to locate where all of the branches were created - looked like

Node-path: branchname
Node-kind: dir
Node-action: add
Node-copyfrom-rev: 23
Node-copyfrom-path:

and change that blank "Node-copyfrom-path: " to "Node-copyfrom-path: trunk". The blank path is what really caused issues because everytime a branch was created, it would contain copies of branches which contained copies of branches, etc... So branch2 had a subfolder branch1, and branch3 contained branch2 and branch 1, where branch 2 contained branch 1...

Then, I removed all of the contents of the commits (not the commits themselves) where we re-organized the repo. i.e.

Node-path: trunk/src
Node-action: add
Node-copyfrom-ref: 23
Node-copyfrom-path: src

and the ones where we deleted nested branches and moved the individual branches into a "branches" directory.

I made these edits and saved them. Then I ran that file through a perl filter, tried to svnadmin load newrepo < filteredDumpFile, and if I got errors, I iterated - changed the pre-filtered file, blow away the svn repo that gave me an error during load, re-create it, and re-load. I iterated this over several days, as I discovered more and more inconsistencies with my edited dump file. I think I have all of the steps above, but I may be missing one or two, and if I am, I apologize. I'd imagine everyone's situation would be a bit different, so adjust accordingly.

Now, I'm quite the Perl hack so I doubt you want to look at the script. It essentially did the following:

Read each line of the dump file.
If the line doesn't start with "Node-path:" or "Node-copyfrom-path:",
then just print it un-altered.
If it does,
  does what follows the ':' start with "trunk", "branches", or "tags"?
 If so, then just print un-altered.
Otherwise,
  for each $branchname(@branchnames) if what follows the ':' begins
with $branchname, replace it with "branches/$branchname", and print.
If none of the branch names match,
  prepend "trunk" to the beginning of the path, and print.

This essentially re-writes all of the paths so that everything not in a branch, or already under "branches", "tags", or "trunk", is put under trunk. It also moves all of the /branchenames to /branches/branchnames.

After this, we did svnadmin hotcopy repo repo-backup rm -rf repo svnadmin create repo svnadmin load repo <filteredDumpFile

Changed the ownership to match the ownership of the original repository, copied over the contents of the repo-backup/hooks directory into the new repo/hooks, and we were all set.

git svn clone --stdlayout... then worked fine. Now I just need to work on a workflow for my co-workers how want to also use git (i.e. create a bare git repo, and keep that in sync with the git-svn repo). Ideally, all of us git users can clone the bare repo and push to it, then every once in a while, I can pull from the bare into the git-svn repo, and do a dcommit, but I've ran into a couple yet-unresolved problems. If you have an references or ideas, I'd appreciate them, but I suppose that's another thread.

On Fri, Jun 18, 2010 at 5:01 PM, Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote: