{"thread":{"id":"8815","subject":"challenges using fast-import and svn","startedAt":"2007-07-02T19:26:17Z","lastAt":"2007-07-02T23:28:25Z","messageCount":3,"participants":["David Frech","Shawn O. Pearce"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"46266","messageId":"7154c5c60707021226k3a7b82fcl7558cfc66d61dde@mail.gmail.com","threadId":"8815","inReplyTo":null,"subject":"challenges using fast-import and svn","fromName":"David Frech","fromEmail":"nimblemachines@gmail.com","sentAt":"2007-07-02T19:26:17Z","receivedAt":"2007-07-02T19:26:17Z","isPatch":false,"sender":{"key":"nimblemachines@gmail.com","avatar":null},"body":"I have an svn repo containing several small projects that is an odd\n\"shape\" (in terms of directories) because of its history;\ngit-svnimport doesn't like the directory structure, and I wasn't able\nto coax it to work.\n\nI looked around for other options, and discovered fast-import (thanks\nShawn!). I decided that the \"easiest\" approach would be to parse the\nsvn dump file and feed the commits into fast-import.\n\nSo I wrote, in Lua, a parser for the (terrible) svn dump file format\nthat feeds commands into fast-import. The parser took a day and a half\nto write; the fast-import backend took about an hour. ;-)\n\nHowever, there are issues. I don't currently track branch copies\ncorrectly, so branches start out with no history, rather than the with\nthe history of the branch they are copied from; and handling deletes\nis tricky.\n\nThis last thing is my main \"question\" to the list, although I'm\ncurious if anyone else has played with svn dump files, and whether my\napproach makes sense.\n\nHere is the problem: if a file or directory is deleted in svn, the\ndumpfile shows simply this:\n\nNode-path: trunk/project/file-or-directory\nNode-action: delete\n\nIn the case of a file, I can simply feed a \"D\" command to fast-import;\nbut if I'm deleting a whole directory, my code knows nothing about\nwhat files exist in that directory. Is fast-import smart about this?\nWill it barf if given a directory argument rather than a file for \"D\"\ncommands?\n\nI could cache the directory contents in my code, but isn't that partly\nwhat fast-import is good for?\n\nAny thoughts are welcome.\n\nCheers,\n\n- David\n\n-- \nIf I have not seen farther, it is because I have stood in the\nfootsteps of giants.\n"},{"id":"46289","messageId":"20070702222400.GB4495@spearce.org","threadId":"8815","inReplyTo":"7154c5c60707021226k3a7b82fcl7558cfc66d61dde@mail.gmail.com","subject":"Re: challenges using fast-import and svn","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2007-07-02T22:24:00Z","receivedAt":"2007-07-02T22:24:00Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"David Frech wrote:\n> So I wrote, in Lua, a parser for the (terrible) svn dump file format\n> that feeds commands into fast-import. The parser took a day and a half\n> to write; the fast-import backend took about an hour. ;-)\n\nHeh. That's about what most folks say. ;-)\n \n> However, there are issues. I don't currently track branch copies\n> correctly, so branches start out with no history, rather than the with\n> the history of the branch they are copied from; and handling deletes\n> is tricky.\n\nBranches are easy to create from the right branch in fast-import,\nbut its hard with the SVN dump file to know where it starts from.\n\nOne trick folks have used in the past is to assign a mark in\nfast-import for each SVN revision. Marks are very cheap and make\nit easy to reference a commit in a from command when you need to\nmake a new branch. You can just use the SVN revision number you\nget from the SVN dump file.\n\n> Here is the problem: if a file or directory is deleted in svn, the\n> dumpfile shows simply this:\n> \n> Node-path: trunk/project/file-or-directory\n> Node-action: delete\n> \n> In the case of a file, I can simply feed a \"D\" command to fast-import;\n> but if I'm deleting a whole directory, my code knows nothing about\n> what files exist in that directory. Is fast-import smart about this?\n> Will it barf if given a directory argument rather than a file for \"D\"\n> commands?\n\nI just read the code again. You can delete an entire subdirectory\njust by sending a D command for that subdirectory, assuming you\ndon't end the name with a '/'. So you should be able to just do:\n\n D file-or-directory\n\nand whatever file-or-directory is, it goes away. If you were to\nsend a trailing '/':\n\n D file-or-directory/\n\nits likely bad things will happen because fast-import will try to\nremove the file or directory named \"\" (yes, empty string) in the\nsubdirectory called \"file-or-directory\" but leave the subdirectory.\n\n\nAnother option is you can replace a tree with a file at any point in\ntime, without first deleting it. So you could also just overwrite\nthe entire subdirectory with an empty file via the M command, then\ndelete the file. But that shouldn't be necessary as the D command\nshould already do exactly what you want it to do. Internally\nthe \"replace entire directory with single file\" is the same\nimplementation as the \"delete entire directory\" implementation...\n\n\nSo I guess this means a documentation update for the D command\nwould be a good idea?\n\n> I could cache the directory contents in my code, but isn't that partly\n> what fast-import is good for?\n\nYes. fast-import is really quite good at helping you do Git side\nof the equation. ;-)\n\n-- \nShawn.\n"},{"id":"46302","messageId":"7154c5c60707021628t14735e94gc8958670a328be79@mail.gmail.com","threadId":"8815","inReplyTo":"20070702222400.GB4495@spearce.org","subject":"Re: challenges using fast-import and svn","fromName":"David Frech","fromEmail":"david@nimblemachines.com","sentAt":"2007-07-02T23:28:25Z","receivedAt":"2007-07-02T23:28:25Z","isPatch":false,"sender":{"key":"david@nimblemachines.com","avatar":null},"body":"On 7/2/07, Shawn O. Pearce wrote:\n> David Frech wrote:\n> > However, there are issues. I don't currently track branch copies\n> > correctly, so branches start out with no history, rather than the with\n> > the history of the branch they are copied from; and handling deletes\n> > is tricky.\n>\n> Branches are easy to create from the right branch in fast-import,\n> but its hard with the SVN dump file to know where it starts from.\n>\n> One trick folks have used in the past is to assign a mark in\n> fast-import for each SVN revision. Marks are very cheap and make\n> it easy to reference a commit in a from command when you need to\n> make a new branch. You can just use the SVN revision number you\n> get from the SVN dump file.\n\nI think I know how to do this. I'm already using marks for each commit.\n\n\n> > Here is the problem: if a file or directory is deleted in svn, the\n> > dumpfile shows simply this:\n> >\n> > Node-path: trunk/project/file-or-directory\n> > Node-action: delete\n> >\n> > In the case of a file, I can simply feed a \"D\" command to fast-import;\n> > but if I'm deleting a whole directory, my code knows nothing about\n> > what files exist in that directory. Is fast-import smart about this?\n> > Will it barf if given a directory argument rather than a file for \"D\"\n> > commands?\n>\n> I just read the code again. You can delete an entire subdirectory\n> just by sending a D command for that subdirectory, assuming you\n> don't end the name with a '/'. So you should be able to just do:\n>\n> D file-or-directory\n>\n> and whatever file-or-directory is, it goes away. If you were to\n> send a trailing '/':\n>\n> D file-or-directory/\n>\n> its likely bad things will happen because fast-import will try to\n> remove the file or directory named \"\" (yes, empty string) in the\n> subdirectory called \"file-or-directory\" but leave the subdirectory.\n\nThis is great! I'll update the code and see what happens...\n\n\n> So I guess this means a documentation update for the D command\n> would be a good idea?\n\nSounds good to me. Right now it really implies \"this only works on files\".\n\nThanks for the snappy reply, and thanks again for writing fast-import!\nIt was a pleasure to use.\n\n> --\n> Shawn.\n>\n\n- David\n\n-- \nIf I have not seen farther, it is because I have stood in the\nfootsteps of giants.\n"}]}