{"thread":{"id":"23183","subject":"Three issues from a Subversion-to-git migration","startedAt":"2010-03-26T12:09:06Z","lastAt":"2010-03-29T18:01:58Z","messageCount":7,"participants":["Eric Raymond","Thomas Rast","Gabriel Filion"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"137842","messageId":"20100326120906.F03BB20CD21@snark.thyrsus.com","threadId":"23183","inReplyTo":null,"subject":"Three issues from a Subversion-to-git migration","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-26T12:09:06Z","receivedAt":"2010-03-26T12:09:06Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"I joined the git list a few hours ago because of experiences I had\nwhile migrating one of my projects, GPSD, from Subversion to git.\nGPSD haa a logical but unusual use case for distributed version\ncontrol; the best places to test GPS sensors are out-of-doors in cars\nand other places where wire-line Internet is absent and one may well\nbe out of range of a WAP.\n\nIn the course of the migration, I encountered three issues which I\nthink could be addressed with a relatively small amount of work.\n\n1. The git hook scripts for CIA.vc are broken (in a way that is,\nfortunately, easy to fix) and generally seem to be in an unmaintained,\ndusty state.\n\n2. The git-svn migration logic does not handle unmodified SVN tag\ntrees well.\n\n3. I'm prone to typos, so I quickly noticed that the existing\nfacilities for editing comments in commits (before they're pushed)\nare clumsy and dangerous.\n\nI will address these points in detail in separate mails.  Issues 1 and\n3 I can fix with working code.\n-- \n\t\t<a href=\"http://www.catb.org/~esr/\">Eric S. Raymond</a>\n"},{"id":"138062","messageId":"201003291100.13043.trast@student.ethz.ch","threadId":"23183","inReplyTo":"20100326120906.F03BB20CD21@snark.thyrsus.com","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-03-29T09:00:12Z","receivedAt":"2010-03-29T09:00:12Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Eric Raymond wrote:\n> 2. The git-svn migration logic does not handle unmodified SVN tag\n> trees well.\n\nThe problem here is that git-svn is designed to handle incremental\nupdates, where it can't know whether some insane SVN user decides to\nmodify the tag later on.\n\nI've used the following hack to make real tags out of SVN \"tags\":\n\ngit for-each-ref --format=\"%(refname)\" refs/remotes/tags/ |\nwhile read tag; do\n    GIT_COMMITTER_DATE=\"$(git log -1 --pretty=format:\"%ad\" \"$tag\")\" \\\n    GIT_COMMITTER_EMAIL=\"$(git log -1 --pretty=format:\"%ce\" \"$tag\")\" \\\n    GIT_COMMITTER_NAME=\"$(git log -1 --pretty=format:\"%cn\" \"$tag\")\" \\\n    git tag -m \"$(git log -1 --pretty=format:\"%s%n%b\" \"$tag\")\" \\\n    \"${tag#refs/remotes/tags/}\" \"$tag\"\ndone\n\nDisclaimer: it worked last time I used it.  Haven't checked if it got\ndusty since.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"138064","messageId":"20100329091056.GC10538@thyrsus.com","threadId":"23183","inReplyTo":"201003291100.13043.trast@student.ethz.ch","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-29T09:10:56Z","receivedAt":"2010-03-29T09:10:56Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Thomas Rast <trast@student.ethz.ch>:\n> Eric Raymond wrote:\n> > 2. The git-svn migration logic does not handle unmodified SVN tag\n> > trees well.\n> \n> The problem here is that git-svn is designed to handle incremental\n> updates, where it can't know whether some insane SVN user decides to\n> modify the tag later on.\n\nYes.  Ideally, I suppose, git-svn (or whatever replaces it) would have\nbehavior something like this:\n\n1. Turn unmodified tag directories into git tags\n2. Turn odified tags into branches.\n3. Recognize when a formerly unmodified tag has been modified, remove\n   the git tag, and turn it into a branch.\n \n> I've used the following hack to make real tags out of SVN \"tags\":\n> \n> git for-each-ref --format=\"%(refname)\" refs/remotes/tags/ |\n> while read tag; do\n>     GIT_COMMITTER_DATE=\"$(git log -1 --pretty=format:\"%ad\" \"$tag\")\" \\\n>     GIT_COMMITTER_EMAIL=\"$(git log -1 --pretty=format:\"%ce\" \"$tag\")\" \\\n>     GIT_COMMITTER_NAME=\"$(git log -1 --pretty=format:\"%cn\" \"$tag\")\" \\\n>     git tag -m \"$(git log -1 --pretty=format:\"%s%n%b\" \"$tag\")\" \\\n>     \"${tag#refs/remotes/tags/}\" \"$tag\"\n> done\n> \n> Disclaimer: it worked last time I used it.  Haven't checked if it got\n> dusty since.\n\nWow, that's ugly. But it does look like it ought to work.\n-- \n\t\t<a href=\"http://www.catb.org/~esr/\">Eric S. Raymond</a>\n"},{"id":"138065","messageId":"201003291132.53415.trast@student.ethz.ch","threadId":"23183","inReplyTo":"20100329091056.GC10538@thyrsus.com","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2010-03-29T09:32:53Z","receivedAt":"2010-03-29T09:32:53Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Eric Raymond wrote:\n> > I've used the following hack to make real tags out of SVN \"tags\":\n> > \n> > git for-each-ref --format=\"%(refname)\" refs/remotes/tags/ |\n> > while read tag; do\n> >     GIT_COMMITTER_DATE=\"$(git log -1 --pretty=format:\"%ad\" \"$tag\")\" \\\n> >     GIT_COMMITTER_EMAIL=\"$(git log -1 --pretty=format:\"%ce\" \"$tag\")\" \\\n> >     GIT_COMMITTER_NAME=\"$(git log -1 --pretty=format:\"%cn\" \"$tag\")\" \\\n> >     git tag -m \"$(git log -1 --pretty=format:\"%s%n%b\" \"$tag\")\" \\\n> >     \"${tag#refs/remotes/tags/}\" \"$tag\"\n> > done\n> > \n> > Disclaimer: it worked last time I used it.  Haven't checked if it got\n> > dusty since.\n> \n> Wow, that's ugly. But it does look like it ought to work.\n\nBTW, you'll also want to use some treatment that removes the empty\ncommit that is generated from the 'svn copy' SVN commit for tagging.\nOne option is to use 'git filter-branch --prune-empty ...', which will\nalso drop other no-op commits.  If you want to remove only the ones\nthat come from tagging, creative use of git-diff-tree in the above\nloop will work.\n\nI suppose I was never bothered by the lack of automatic tagging\nbecause I rarely found a git-svn import to be immediately fit for\npublishing.  Usually it took some grafting and other filtering to\nbring the history into shape anyway.  Maybe now that the svn:mergeinfo\nsupport obviates the need for grafting, it's worth thinking about the\nrest.\n\n-- \nThomas Rast\ntrast@{inf,student}.ethz.ch\n"},{"id":"138066","messageId":"20100329102656.GA10925@thyrsus.com","threadId":"23183","inReplyTo":"201003291132.53415.trast@student.ethz.ch","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-29T10:26:56Z","receivedAt":"2010-03-29T10:26:56Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Thomas Rast <trast@student.ethz.ch>:\n> I suppose I was never bothered by the lack of automatic tagging\n> because I rarely found a git-svn import to be immediately fit for\n> publishing.  Usually it took some grafting and other filtering to\n> bring the history into shape anyway.  Maybe now that the svn:mergeinfo\n> support obviates the need for grafting, it's worth thinking about the\n> rest.\n\nCan't argue your first point at all, because my only large migration\nso far did in fact need filtering - to fix up artifacts from the Emacs\nVC front end that were fossilized in the Subversion history. And that\nwas all my own fault; I was the original author of VC back in the\nearly 1990s, and should have rewritten it to be changeset-aware years\nsooner than I did. Alas, I was kind of busy being Mr. Famous Geek for\nabout a decade in there, and the VC rewrite was one of several\nprojects that got seriously sidetracked.  I finally got it done in\n2008-2009, and git is one of the backend systems that benefits from\nthat.\n\nStill. Even conceding that point, built-in support to further reduce\nthe amount of hand-work required in SVN conversion would be no bad\nthing.  Tag conversion was unequivocally the biggest pain in the ass\nwhen I migrated GPSD; I'm not claiming that will always be true, but I do\nthink it's the largest pain that could be *reliably mechanized\naway*. That makes it a logical target.\n\nOne of the reasons this is still on my mind after the GPSD migration\nis Battle For Wesnoth <http://www.wesnoth.org/>. I'm one of the senior\ndevs on that project, and it is becoming clear to all that we have\nreached Subversion's limits there.  I'm the project's tools and\ntoolsmithing expert, and I've pretty much got the other devs convinced to\nswitch to a DVCS when we can screw up our courage to move to a forge\nthat supports one. \n\nThis means I'm probably going to be the guy on the spot doing yet\nanother big ugly conversion away from SVN sometime within the next\nyear.  The state of conversion tools at that time might end up\ndetermining whether Wesnoth goes with git or Mercurial.\n-- \n\t\t<a href=\"http://www.catb.org/~esr/\">Eric S. Raymond</a>\n"},{"id":"138100","messageId":"4BB0CDEC.8000708@gmail.com","threadId":"23183","inReplyTo":"20100329091056.GC10538@thyrsus.com","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Gabriel Filion","fromEmail":"lelutin@gmail.com","sentAt":"2010-03-29T15:57:32Z","receivedAt":"2010-03-29T15:57:32Z","isPatch":false,"sender":{"key":"lelutin@gmail.com","avatar":"https://avatars.githubusercontent.com/u/108728?v=4"},"body":"Hello,\n\nOn 2010-03-29 05:10, Eric Raymond wrote:\n> Thomas Rast <trast@student.ethz.ch>:\n>> Eric Raymond wrote:\n>>> 2. The git-svn migration logic does not handle unmodified SVN tag\n>>> trees well.\n>>\n>> The problem here is that git-svn is designed to handle incremental\n>> updates, where it can't know whether some insane SVN user decides to\n>> modify the tag later on.\n> \n> Yes.  Ideally, I suppose, git-svn (or whatever replaces it) would have\n> behavior something like this:\n> \n> 1. Turn unmodified tag directories into git tags\n> 2. Turn odified tags into branches.\n> 3. Recognize when a formerly unmodified tag has been modified, remove\n>    the git tag, and turn it into a branch.\n>  \n\nThe 3rd point seems a bit weird to me.. users don't expect tags to\ndisappear magically. Especially if it's done during a fetch while working.\n\nHere's how I would change the scenario:\n\n1. For each creation of a sub-directory in SVN's tag directory, create a\ngit tag on the revision that was referenced by the directory copy in SVN.\n2. If (and only if) there are later modifications in the tag directory,\ncreate a branch starting from that tag.\n\nThis way, the tag would be there but a branch would hold modifications\nbased on code at this point, if there is any.\n\nThe problem with my scenario, though is that it doesn't take care of tag\ncreation + modification in the same commit (yuukkkk, but it's possible\nthat it exists somewhere). If it could be possible to verify if\nmodifications were made during the tag creation, then we could make the\ncase hit both points.\n\nThe other big \"thing\" is that it expects a certain correct separation\ninto different directories (e.g. trunk/ tags/ branches/ ), which SVN\ndoesn't enforce.\n\n-- \nGabriel Filion\n"},{"id":"138118","messageId":"20100329180158.GB12922@thyrsus.com","threadId":"23183","inReplyTo":"4BB0CDEC.8000708@gmail.com","subject":"Re: Three issues from a Subversion-to-git migration","fromName":"Eric Raymond","fromEmail":"esr@thyrsus.com","sentAt":"2010-03-29T18:01:58Z","receivedAt":"2010-03-29T18:01:58Z","isPatch":false,"sender":{"key":"esr@thyrsus.com","avatar":"https://avatars.githubusercontent.com/u/727961?v=4"},"body":"Gabriel Filion <lelutin@gmail.com>:\n> > 1. Turn unmodified tag directories into git tags\n> > 2. Turn odified tags into branches.\n> > 3. Recognize when a formerly unmodified tag has been modified, remove\n> >    the git tag, and turn it into a branch.\n> \n> The 3rd point seems a bit weird to me.. users don't expect tags to\n> disappear magically. Especially if it's done during a fetch while working.\n\nA reasonable objection.\n \n> Here's how I would change the scenario:\n> \n> 1. For each creation of a sub-directory in SVN's tag directory, create a\n> git tag on the revision that was referenced by the directory copy in SVN.\n> 2. If (and only if) there are later modifications in the tag directory,\n> create a branch starting from that tag.\n> \n> This way, the tag would be there but a branch would hold modifications\n> based on code at this point, if there is any.\n\nThat would work for me.\n \n> The problem with my scenario, though is that it doesn't take care of tag\n> creation + modification in the same commit (yuukkkk, but it's possible\n> that it exists somewhere). If it could be possible to verify if\n> modifications were made during the tag creation, then we could make the\n> case hit both points.\n\nDoesn't seem like that should be difficult.\n\n> The other big \"thing\" is that it expects a certain correct separation\n> into different directories (e.g. trunk/ tags/ branches/ ), which SVN\n> doesn't enforce.\n\nAre you suggesting that branch directory copies should be handled with\nthe same rule? I think I could live with that.\n-- \n\t\t<a href=\"http://www.catb.org/~esr/\">Eric S. Raymond</a>\n"}]}