{"thread":{"id":"30196","subject":"Migrating SVN to Git, and preserve merge information","startedAt":"2012-04-10T15:18:11Z","lastAt":"2012-04-11T17:58:31Z","messageCount":5,"participants":["Nick Douma","Andrew Sayers","Santi Béjar"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"188873","messageId":"4F844F33.5000004@nekoconeko.nl","threadId":"30196","inReplyTo":null,"subject":"Migrating SVN to Git, and preserve merge information","fromName":"Nick Douma","fromEmail":"n.douma@nekoconeko.nl","sentAt":"2012-04-10T15:18:11Z","receivedAt":"2012-04-10T15:18:11Z","isPatch":false,"sender":{"key":"n.douma@nekoconeko.nl","avatar":"https://gravatar.com/avatar/93ce45c81df3adf49c9d5ee27c2e27bcc519ea6b43ad60e31e070119ff2abd92?d=mp&s=160"},"body":"Hi,\n\nI currently have two SVN repositories I'm looking to convert to Git.\nWhile the process itself isn't all that hard (following this guide[1]),\nI'm looking to do a bit more than just convert raw commits.\n\nThe repositories basically have two main branches at a time, for example:\n\n* trunk\n* current version, for example 3.5\n\nOur SVN workflow consisted of working on trunk, and then manually\nmerging single commits from trunk to the other branch. We quite\nconsistently mentioned the merged commits in the SVN commit message, and\nI'm looking to use this information to generate a more accurate tree in\nGit. The commit messages are for example:\n\ntrunk rev 100: Fixed important bug\nbranch rev 101: Merged r100 from trunk\n\nOr more elaborate:\n\nbranch rev 200: Merged r100,r101,r.... from trunk\n\nIn these examples, tools like gitk or git log should show a line from\nrev 100 to rev 101 in the first example, and lines from r100, r101,rn to\nrev 200 in example two.\n\nI have tried to create a custom grafts file to create the parent-child\nrelations above, and basically finds all merge commits and converts them\ninto graft lines like so:\n\n ... \n\nThis all works to a certain extent, but the problem arises when trying\nto view older history in these repositories. If I use the grafts file,\nand do `gitk --all`, gitk freezes. Gitg doesn't show commits before a\ncertain point. tig and `git log --graph` all show a huge amount of\nparentless commits near the bottom. All of this leads me to the\nconclusion that something is wrong with the method of using grafts,\nrather than problems in the individual tools.\n\n\nCan someone find something obvious that I'm missing in the above\napproach? Or alternatively suggest another more appropriate method of\nachieving the same results in Git?\n\nKind regards,\n\nNick Douma\n\n\n\n[1]: http://john.albin.net/git/convert-subversion-to-git\n\n"},{"id":"188915","messageId":"4F84BAE2.5090803@pileofstuff.org","threadId":"30196","inReplyTo":"4F844F33.5000004@nekoconeko.nl","subject":"Re: Migrating SVN to Git, and preserve merge information","fromName":"Andrew Sayers","fromEmail":"andrew-git@pileofstuff.org","sentAt":"2012-04-10T22:57:38Z","receivedAt":"2012-04-10T22:57:38Z","isPatch":false,"sender":{"key":"andrew-git@pileofstuff.org","avatar":null},"body":"Hi Nick,\n\nWould I be right in thinking that a commit like \"Merged r100,r101,r102\nfrom trunk\" will create three grafts? If so, that might be the problem.\n\nGit differentiates between \"merges\" (which include every commit up to\nand including the specified one) and \"cherry-picks\" (which just include\nthe specified commit), whereas SVN calls both of these \"merges\". Grafts\nare a way of creating \"merges\" rather than \"cherry-picks\" (which git\ndoesn't have any metadata for), and it's not at all easy to get \"merge\"\ndata out of SVN in the general case. Having said that, it's often a\ngood enough heuristic to pick the highest revision number mentioned in\nthe commit message and pretend it's a merge.\n\nIncidentally, I'm planning to work on this area of SVN->git conversion\nin the coming months. I don't have anything you could use yet, but I\ndon't suppose the scripts you used are available somewhere? Getting\nrevision information out of log files is particularly tricky, and\neveryone stumbles over a different set of issues. I'd be really\ninterested to pick any nuggets of wisdom out of the approach you took.\n\n\t- Andrew\n"},{"id":"188933","messageId":"4F8531B3.9030501@nekoconeko.nl","threadId":"30196","inReplyTo":"4F84BAE2.5090803@pileofstuff.org","subject":"Re: Migrating SVN to Git, and preserve merge information","fromName":"Nick Douma","fromEmail":"n.douma@nekoconeko.nl","sentAt":"2012-04-11T07:24:35Z","receivedAt":"2012-04-11T07:24:35Z","isPatch":false,"sender":{"key":"n.douma@nekoconeko.nl","avatar":"https://gravatar.com/avatar/93ce45c81df3adf49c9d5ee27c2e27bcc519ea6b43ad60e31e070119ff2abd92?d=mp&s=160"},"body":"Hi Andrew,\n\nOn 11-04-12 00:57, Andrew Sayers wrote:\n> Would I be right in thinking that a commit like \"Merged r100,r101,r102\n> from trunk\" will create three grafts? If so, that might be the problem.\n\nOne SVN 'merge' commit will generate one graft. The graft itself will\ncontain all revisions mentioned in the merge commit, and whatever the\noriginal git parent was before the graft.\n\n> Git differentiates between \"merges\" (which include every commit up to\n> and including the specified one) and \"cherry-picks\" (which just include\n> the specified commit), whereas SVN calls both of these \"merges\". Grafts\n> are a way of creating \"merges\" rather than \"cherry-picks\" (which git\n> doesn't have any metadata for), and it's not at all easy to get \"merge\"\n> data out of SVN in the general case. Having said that, it's often a\n> good enough heuristic to pick the highest revision number mentioned in\n> the commit message and pretend it's a merge.\n\nThe way we 'merged' in SVN was indeed more like cherry-picking, but I'm\nlooking to display this information as a merge in Git. I also would like\nto include all revisions if possible.\n\nThe real problem I seem to be having is not completely understanding how\nGit grafts work, because I think I'm hitting some kind of limitation or\nbug, or just not using it right.\n\n> Incidentally, I'm planning to work on this area of SVN->git conversion\n> in the coming months. I don't have anything you could use yet, but I\n> don't suppose the scripts you used are available somewhere? Getting\n> revision information out of log files is particularly tricky, and\n> everyone stumbles over a different set of issues. I'd be really\n> interested to pick any nuggets of wisdom out of the approach you took.\n\nI don't really have any useful script for you at the moment, but the\nmain approach is this:\n\n* I first tag all SVN Git commits with the original SVN revision, like:\n\"svn/1234\"\n* Then I retrieve all commits with \"merge\" in the message, but not \"unmerge\"\n* Now I filter all revisions from the commit message using a regex or two.\n* Using all relevant revisions, I retrieve the corresponding SHA hashes\nusing the tag names I created in step 1.\n* Finally, I write a graft file in format:\n ... \n\nKind regards,\n\nNick Douma\n\n"},{"id":"188944","messageId":"CA+gHt1DPSkbwyXzt4SHL=1btLJW8mVYrWORU8Wk2riW4v+SCvA@mail.gmail.com","threadId":"30196","inReplyTo":"4F8531B3.9030501@nekoconeko.nl","subject":"Re: Migrating SVN to Git, and preserve merge information","fromName":"Santi Béjar","fromEmail":"santi@agolina.net","sentAt":"2012-04-11T11:13:16Z","receivedAt":"2012-04-11T11:13:16Z","isPatch":false,"sender":{"key":"santi@agolina.net","avatar":null},"body":"On Wed, Apr 11, 2012 at 9:24 AM, Nick Douma wrote:\n> Hi Andrew,\n>\n> On 11-04-12 00:57, Andrew Sayers wrote:\n>> Would I be right in thinking that a commit like \"Merged r100,r101,r102\n>> from trunk\" will create three grafts?  If so, that might be the problem.\n>\n> One SVN 'merge' commit will generate one graft. The graft itself will\n> contain all revisions mentioned in the merge commit, and whatever the\n> original git parent was before the graft.\n\nThis graft does not represent the SVN ´merge´. There are two types of\nSVN ´merges´, and you are describing the cherry-pick one. So you have\nto represent them as such, as cherry-pick. And a cherry-pick in git is\njust a normal commit, it just happens to apply to a different branch\n(commit), it normally has the same commit message (except some people\nprefer to add \"(cherry pick from commit ...)\". And I think this is\njust what you are looking for.\n\nSo, I would recommend you to just rewrite the SVN commit id with git\ncommit sha1. Only in the case where you have a real merge (rev XXX:\nMerge all history up to rXXX-1 from trunk), you use the graft\napproach.\n\n>\n>> Git differentiates between \"merges\" (which include every commit up to\n>> and including the specified one) and \"cherry-picks\" (which just include\n>> the specified commit), whereas SVN calls both of these \"merges\".  Grafts\n>> are a way of creating \"merges\" rather than \"cherry-picks\" (which git\n>> doesn't have any metadata for), and it's not at all easy to get \"merge\"\n>> data out of SVN in the general case.  Having said that, it's often a\n>> good enough heuristic to pick the highest revision number mentioned in\n>> the commit message and pretend it's a merge.\n>\n> The way we 'merged' in SVN was indeed more like cherry-picking, but I'm\n> looking to display this information as a merge in Git. I also would like\n> to include all revisions if possible.\n\nBut git needs a meaningful history!! If you use merges to represent\ncherry-picks it will confuse some of the git tools.\n\nIf the question is to \"display this information\" in a meaningful way\njust use the approach outlined above.\n\n>\n> The real problem I seem to be having is not completely understanding how\n> Git grafts work, because I think I'm hitting some kind of limitation or\n> bug, or just not using it right.\n\nIf gitk --all freeze theres is definitly a bug somewhere. I don´t\nreally know, but maybe the problem is that some parents in your grafts\nare descendent of the others.\n\nAnyway, we need a minimal method to reproduce it.\n\nHTH,\nSanti\n"},{"id":"189005","messageId":"4F85C647.9070302@pileofstuff.org","threadId":"30196","inReplyTo":"4F8531B3.9030501@nekoconeko.nl","subject":"Re: Migrating SVN to Git, and preserve merge information","fromName":"Andrew Sayers","fromEmail":"andrew-git@pileofstuff.org","sentAt":"2012-04-11T17:58:31Z","receivedAt":"2012-04-11T17:58:31Z","isPatch":false,"sender":{"key":"andrew-git@pileofstuff.org","avatar":null},"body":"Thanks Nick for the outline - the main thing I take from this is that\n\"unmerge\" is used in real world revision messages, so it's important for\nme to test that any regexp does the right thing there.\n\nIt sounds like grafts aren't the right solution for your problem. I\ncan't really suggest much without a better understanding of what you're\ntrying to achieve, but you might be interested in `git cherry`, which\ntries to detect cherry-picked commits by comparing the diffs.\n\n\t- Andrew\n"}]}