challenges using fast-import and svn

I have an svn repo containing several small projects that is an odd "shape" (in terms of directories) because of its history; git-svnimport doesn't like the directory structure, and I wasn't able to coax it to work.

I looked around for other options, and discovered fast-import (thanks Shawn!). I decided that the "easiest" approach would be to parse the svn dump file and feed the commits into fast-import.

So I wrote, in Lua, a parser for the (terrible) svn dump file format that feeds commands into fast-import. The parser took a day and a half to write; the fast-import backend took about an hour. ;-)

However, there are issues. I don't currently track branch copies correctly, so branches start out with no history, rather than the with the history of the branch they are copied from; and handling deletes is tricky.

This last thing is my main "question" to the list, although I'm curious if anyone else has played with svn dump files, and whether my approach makes sense.

Here is the problem: if a file or directory is deleted in svn, the dumpfile shows simply this:

Node-path: trunk/project/file-or-directory
Node-action: delete

In the case of a file, I can simply feed a "D" command to fast-import; but if I'm deleting a whole directory, my code knows nothing about what files exist in that directory. Is fast-import smart about this? Will it barf if given a directory argument rather than a file for "D" commands?

I could cache the directory contents in my code, but isn't that partly what fast-import is good for?

Any thoughts are welcome.

Cheers,

- David

David Frech <nimblemachines@gmail.com> wrote:

Heh.  That's about what most folks say.  ;-)
 

Branches are easy to create from the right branch in fast-import, but its hard with the SVN dump file to know where it starts from.

One trick folks have used in the past is to assign a mark in fast-import for each SVN revision. Marks are very cheap and make it easy to reference a commit in a from command when you need to make a new branch. You can just use the SVN revision number you get from the SVN dump file.

I just read the code again. You can delete an entire subdirectory just by sending a D command for that subdirectory, assuming you don't end the name with a '/'. So you should be able to just do:

  D file-or-directory

and whatever file-or-directory is, it goes away. If you were to send a trailing '/':

  D file-or-directory/

its likely bad things will happen because fast-import will try to remove the file or directory named "" (yes, empty string) in the subdirectory called "file-or-directory" but leave the subdirectory.

Another option is you can replace a tree with a file at any point in time, without first deleting it. So you could also just overwrite the entire subdirectory with an empty file via the M command, then delete the file. But that shouldn't be necessary as the D command should already do exactly what you want it to do. Internally the "replace entire directory with single file" is the same implementation as the "delete entire directory" implementation...

So I guess this means a documentation update for the D command would be a good idea?

Yes. fast-import is really quite good at helping you do Git side of the equation. ;-)

On 7/2/07, Shawn O. Pearce <spearce@spearce.org> wrote:

I think I know how to do this. I'm already using marks for each commit.

This is great! I'll update the code and see what happens...

Sounds good to me. Right now it really implies "this only works on files".

Thanks for the snappy reply, and thanks again for writing fast-import! It was a pleasure to use.

- David