{"thread":{"id":"25273","subject":"Another way to compare tools: is it possible to transfer full history?","startedAt":"2010-09-28T13:44:53Z","lastAt":"2010-09-29T14:00:56Z","messageCount":11,"participants":["Tuomo","Tomas Carnecky","Michael Haggerty","Jonathan Nieder","Sverre Rabbelier","Andreas Ericsson","Bruce Stephens"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"151912","messageId":"loom.20100928T153519-936@post.gmane.org","threadId":"25273","inReplyTo":null,"subject":"Another way to compare tools: is it possible to transfer full history?","fromName":"Tuomo","fromEmail":"tuo.tie@gmail.com","sentAt":"2010-09-28T13:44:53Z","receivedAt":"2010-09-28T13:44:53Z","isPatch":false,"sender":{"key":"tuo.tie@gmail.com","avatar":null},"body":"I have seen lots of comparisons between source control tools, \nbut have not found a comparison that would explain the fundamental differences \nand similarities in a way that would really let me choose. \nSo I decided to try a new approach: if one tries to transfer the full history \nof an application or a larger product/project, which features can I rely on \nfinding in any decently recent tool?\n\nLet's start from Git vs. Mercurial: is it possible to move the whole history \nof an application (with or without submodules) from Git to Mercurial? \nFrom Mercurial to Git? \nIf it is not always possible, what is the feature that might completely \nprevent the whole attempt? If partial transfer is possible, what information \nmight be missing in the result?\n\nI am not interested in whether it is easy or fast in practice. \nI am only interested in the concepts and finding equivalent classes \nfor the tools or at least their core concepts.\n"},{"id":"151919","messageId":"4CA20169.2040606@dbservice.com","threadId":"25273","inReplyTo":"loom.20100928T153519-936@post.gmane.org","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Tomas Carnecky","fromEmail":"tom@dbservice.com","sentAt":"2010-09-28T14:53:29Z","receivedAt":"2010-09-28T14:53:29Z","isPatch":false,"sender":{"key":"tom@dbservice.com","avatar":"https://gravatar.com/avatar/900a300bdd1a8bbe086008ad78210bbee2ad2803b7d50a5cba04c1e9404bd6d2?d=mp&s=160"},"body":"On 9/28/10 3:44 PM, Tuomo wrote:\n> I have seen lots of comparisons between source control tools, \n> but have not found a comparison that would explain the fundamental differences \n> and similarities in a way that would really let me choose. \n> So I decided to try a new approach: if one tries to transfer the full history \n> of an application or a larger product/project, which features can I rely on \n> finding in any decently recent tool?\n> \n> Let's start from Git vs. Mercurial: is it possible to move the whole history \n> of an application (with or without submodules) from Git to Mercurial? \n> From Mercurial to Git? \n\nfast-import/export (man git-fast-export/import) seems to be the future.\nGit provides excellent support for it and other SCMs are adopting it as\nwell. And then there are custom written conversion tools, just take a\nlook at [1] to see which ones are available.\n\n(shameless plug: just this weekend I started collecting the various fast\nimport/export tools and made a webpage about it:\nhttp://caurea.org/fast-export-import/. It's far from complete though.\nAnd if you know any tools that perform better than those I've listed,\nI'd be glade to update the page).\n\n> If it is not always possible, what is the feature that might completely \n> prevent the whole attempt? If partial transfer is possible, what information \n> might be missing in the result?\n\nNot all SCMs have the same features. Subversion for example doesn't have\nreal branches, tags nor merges (in the same sense that Git does). And\neven in distributed version control systems there can be differences.\nGit history can't be mapped 1:1 to Mercurial (octopus merges come to\nmind). Some of these things can be reasonably emulated, some can't and\nyou loose that information.\n\ntom\n\n[1]\nhttps://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools#Interaction_with_other_Revision_Control_Systems\n"},{"id":"151926","messageId":"4CA20E51.1000503@alum.mit.edu","threadId":"25273","inReplyTo":"4CA20169.2040606@dbservice.com","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2010-09-28T15:48:33Z","receivedAt":"2010-09-28T15:48:33Z","isPatch":false,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"On 09/28/2010 04:53 PM, Tomas Carnecky wrote:\n> (shameless plug: just this weekend I started collecting the various fast\n> import/export tools and made a webpage about it:\n> http://caurea.org/fast-export-import/. It's far from complete though.\n> And if you know any tools that perform better than those I've listed,\n> I'd be glade to update the page).\n\ncvs2svn [1] (in particular the cvs2git [2] and cvs2bzr [3] variants,\nwhich are part of the same project) can convert from CVS to fast-import\nformat. It is mature and actively maintained [4], has a lot of features\n[5], help is available, and hopefully you agree that the documentation\n[6,7] is well-written.\n\nMichael\n\n[1] http://cvs2svn.tigris.org/\n[2] http://cvs2svn.tigris.org/cvs2git.html\n[3] http://cvs2svn.tigris.org/cvs2bzr.html\n[4] https://www.ohloh.net/p/cvs2svn\n[5] http://cvs2svn.tigris.org/features.html\n[6] http://cvs2svn.tigris.org/cvs2svn.html\n[7] http://cvs2svn.tigris.org/faq.html\n"},{"id":"151955","messageId":"AANLkTi=oRv4NnG0yWCpmj+AVXijGU-EK5rAHUZ4dZLQV@mail.gmail.com","threadId":"25273","inReplyTo":"4CA20169.2040606@dbservice.com","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-09-28T20:48:07Z","receivedAt":"2010-09-28T20:48:07Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Tomas Carnecky wrote:\n\n> (shameless plug: just this weekend I started collecting the various fast\n> import/export tools and made a webpage about it:\n> http://caurea.org/fast-export-import/.\n\nThis is awesome --- thank you!\n\nIn development:\n\nsvn-fe (contrib/svn-fe in Git) converts from an svn dump file to a\nfast-import stream. Stale webpage:\nhttp://barrbrain.github.com/svn-dump-fast-export/\n\nremote-hg (see git_remote_helpers/hg/ in\nhttp://github.com/SRabbelier/git) contains a partial hg fast-export\nimplementation in exporter.py. I don't know how it compares to Rocco\nRutte's exporter.\n\nNot sure if these belong on your list yet, but thought you might like\nto know about them. :)\n"},{"id":"151958","messageId":"AANLkTikvA-BkyGxJ14SpLerjsp1JpCDgegdc+9aT929x@mail.gmail.com","threadId":"25273","inReplyTo":"AANLkTi=oRv4NnG0yWCpmj+AVXijGU-EK5rAHUZ4dZLQV@mail.gmail.com","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Sverre Rabbelier","fromEmail":"srabbelier@gmail.com","sentAt":"2010-09-28T20:55:24Z","receivedAt":"2010-09-28T20:55:24Z","isPatch":false,"sender":{"key":"srabbelier@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3098?v=4"},"body":"Heya,\n\nOn Tue, Sep 28, 2010 at 22:48, Jonathan Nieder wrote:\n> remote-hg (see git_remote_helpers/hg/ in\n> http://github.com/SRabbelier/git) contains a partial hg fast-export\n> implementation in exporter.py. I don't know how it compares to Rocco\n> Rutte's exporter.\n\nI wrote the exporter from scratch, I know that it works, but I haven't\ncompared performance against other implementations (such as Rocco\nRutte's)\n\nThe accompanying fast-import is based off the bzr one. I'll use the\nnew python fast-export library that's on pypi in the next iteration\nthough, so only the part that hooks into hg is interesting.\n\n-- \nCheers,\n\nSverre Rabbelier\n"},{"id":"152037","messageId":"loom.20100929T130008-795@post.gmane.org","threadId":"25273","inReplyTo":"4CA20169.2040606@dbservice.com","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Tuomo","fromEmail":"tuo.tie@gmail.com","sentAt":"2010-09-29T11:03:48Z","receivedAt":"2010-09-29T11:03:48Z","isPatch":false,"sender":{"key":"tuo.tie@gmail.com","avatar":null},"body":"Tomas Carnecky dbservice.com> writes:\n\n> fast-import/export (man git-fast-export/import) seems to be the future.\n> Git provides excellent support for it and other SCMs are adopting it as\n> well. And then there are custom written conversion tools, just take a\n> look at [1] to see which ones are available.\n> \n> (shameless plug: just this weekend I started collecting the various fast\n> import/export tools and made a webpage about it:\n> http://caurea.org/fast-export-import/. It's far from complete though.\n> And if you know any tools that perform better than those I've listed,\n> I'd be glade to update the page).\n\nLast time I was looking for conversions between source control tools, I could\nnot find any (but my attempt may have been a bit limited). I am very happy to\nsee that there is now not only a plethora of point-to-point conversion tools,\nbut a common exchange format. It means that the field of source control is\nfinally maturing, finding common concepts that most agree on.\n\nHowever, the scanty documentation for the tools does not answer my question.\nI'll try to formulate it again. You can move history from SCCS to RCS without\nlosing anything, but you cannot move from RCS to SCCS, because SCCS does not\nhandle sub-branches. You can move from RCS to CVS, but since RCS does not record\nevolutionary relationships between tags, the result does not record the history\nin the manner we'd expect in CVS. You can provide additional data to the\nconversion, but that additional data cannot be automatically deduced by any\ngeneral algorithm. You can move stuff from CVS to RCS, but you lose the\nevolution of the whole configuration. So, SCCS, RCS and CVS all belong to a\ndifferent class. Only the latest tools have enough in common that one can find\nenough of information to make a full transfer of history without loss of data\nthat can be deduced by automatic means if a back-conversion is desired.\n\nWhich tools belong to the same class with Git? Strictly speaking, I am talking\nabout conversions that do not require us to inject any additional data,\nconversions that are fully automatic. The page http://wiki.darcs.net/DarcsGit\nmentions that Darcs->Git->Darcs roundtrip loses no information (but also notes\nthat the tool is nowadays broken and needs fixing), but the same is not true for\nthe other direction. That is the kind of information I am looking for.\n\nIs it possible to make a round-trip Mercurial->Git->Mercurial or\nGit->Mercurial->Git without loss of any information? I would expect that\nMercurial->Git->Mercurial might produce some differences if files have been\nrenamed or moved between directories, but other than that?\n\nWhat particularly interests me is how the conversion handles unnamed Mercurial\nbranches? I am asking this because at work, I had to ponder once if it would be\npossible to transfer histories from Synergy (ex Continuus) to some other tool,\nand found it very difficult to imagine how to create named branches from the\nversion DAGs Synergy uses. You can never be sure if a new version is a successor\nof its predecessor on the same branch or the first version on a sub.branch,\nbecause Synergy doesn't treat them any differently. users often try to organize\nthe branches in ways compatible with other tools, but since Synergy has no way\nof enforcing any of these methods, there is no guarantee of consistency. The\nworst-case scenario is that every single version is its own branch. So, I really\nwould like to know how the unnamed branches from Mercurial are transferred to\nnamed branches in Git?\n"},{"id":"152038","messageId":"4CA320C3.6090006@op5.se","threadId":"25273","inReplyTo":"loom.20100929T130008-795@post.gmane.org","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2010-09-29T11:19:31Z","receivedAt":"2010-09-29T11:19:31Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"On 09/29/2010 01:03 PM, Tuomo wrote:\n> Tomas Carnecky dbservice.com> writes:\n> \n> Which tools belong to the same class with Git?\n\nAll repositories from pure distributed version control systems can be\ntransformed into git repositories and git repos can be transformed into\na repository of a pure distributed version control system, but not all\nrepositories can do so without information loss since all dvcs systems\nstore and represent things different and have (slightly) different\ncapabilities.\n\nOctopus merges have been mentioned. I'm sure there are other things,\nsuch as the three different tag-types we have in git, that can't be\nproperly represented by other scm systems.\n\nExactly which those are and how it affects conversions from one\nsystem to another is something you seem to want to learn without\nactually doing the job of finding it out, and I doubt anyone on this\nlist knows every detail you're looking for (although the complete\ninformation you're after might be available in scattered form among\nthe population on this list).\n\n> \n> Is it possible to make a round-trip Mercurial->Git->Mercurial or\n> Git->Mercurial->Git without loss of any information?\n\nThat depends. If the git repository has no octopus merges and no tags\nof a type that can't be represented in mercurial, I believe it should\nbe possible.\n\nTry and find out.\n\n> I would expect that\n> Mercurial->Git->Mercurial might produce some differences if files have been\n> renamed or moved between directories, but other than that?\n> \n\nPossibly. Again though; Try and find out.\n\n> What particularly interests me is how the conversion handles unnamed Mercurial\n> branches?\n\nProbably as detached heads. There's nothing special with having commits with no\nrefs attached to them in git.\n\n> I am asking this because at work, I had to ponder once if it would be\n> possible to transfer histories from Synergy (ex Continuus) to some other tool,\n> and found it very difficult to imagine how to create named branches from the\n> version DAGs Synergy uses. You can never be sure if a new version is a successor\n> of its predecessor on the same branch or the first version on a sub.branch,\n> because Synergy doesn't treat them any differently. users often try to organize\n> the branches in ways compatible with other tools, but since Synergy has no way\n> of enforcing any of these methods, there is no guarantee of consistency. The\n> worst-case scenario is that every single version is its own branch. So, I really\n> would like to know how the unnamed branches from Mercurial are transferred to\n> named branches in Git?\n> \n\nSo now we get to a proper use-case. You want to convert a Synergy repository to\nsomething else, and you've been looking at mercurial and git.\n\nSo let me ask you a question; What have you done so far to find out the answer\nto the questions you're looking for, apart from asking here how a theoretical\nscenario would pan out?\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"152048","messageId":"loom.20100929T145158-59@post.gmane.org","threadId":"25273","inReplyTo":"4CA320C3.6090006@op5.se","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Tuomo","fromEmail":"tuo.tie@gmail.com","sentAt":"2010-09-29T13:03:34Z","receivedAt":"2010-09-29T13:03:34Z","isPatch":false,"sender":{"key":"tuo.tie@gmail.com","avatar":null},"body":"Andreas Ericsson op5.se> writes:\n\n> \n> So now we get to a proper use-case. You want to convert a Synergy repository to\n> something else, and you've been looking at mercurial and git.\n\nNah, I am not working with Synergy. I referred to an earlier encounter with it.\n\nRight now, I am stuck with ClearCase. Most of the projects have been using base \nClearCase, not UCM, and it means that there is no realistic way to transfer the \nhistories anywhere. It can be tried, sure, but who would really want to do it?\nWhat I am looking for is tools that I can recommend to the projects, tools \nthat are available to them, and will not pose a threat in being non-convertible, \ntools that would allow the projects to later make another decision, and not get \nstuck with the tool they choose now.\n\n> So let me ask you a question; What have you done so far to find out the answer\n> to the questions you're looking for, apart from asking here how a theoretical\n> scenario would pan out?\n\nFair enough. I am looking for documentation, and advice on how to find worthy \nsources, before delving into trying out something for no good reason. \nBut, at the same time, I wish to have a look-see on what the general status \nis right now. If there is no summary of it available, then apparently I have to \nmake it myself. But I would hate to find out afterward that someone had in fact \ndone it already. Tomas Carnecky's list is a good start.\n"},{"id":"152052","messageId":"4CA33EE3.3090202@op5.se","threadId":"25273","inReplyTo":"loom.20100929T145158-59@post.gmane.org","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Andreas Ericsson","fromEmail":"ae@op5.se","sentAt":"2010-09-29T13:28:03Z","receivedAt":"2010-09-29T13:28:03Z","isPatch":false,"sender":{"key":"ae@op5.se","avatar":"https://gravatar.com/avatar/426e89595c75a8f5252dd0c989e5fabe5bcac616e68557427ad9aef6b0ca342a?d=mp&s=160"},"body":"On 09/29/2010 03:03 PM, Tuomo wrote:\n> Andreas Ericsson op5.se> writes:\n> \n>>\n>> So now we get to a proper use-case. You want to convert a Synergy repository to\n>> something else, and you've been looking at mercurial and git.\n> \n> Nah, I am not working with Synergy. I referred to an earlier encounter with it.\n> \n> Right now, I am stuck with ClearCase. Most of the projects have been using base\n> ClearCase, not UCM, and it means that there is no realistic way to transfer the\n> histories anywhere. It can be tried, sure, but who would really want to do it?\n> What I am looking for is tools that I can recommend to the projects, tools\n> that are available to them, and will not pose a threat in being non-convertible,\n> tools that would allow the projects to later make another decision, and not get\n> stuck with the tool they choose now.\n> \n\nIn general, it's easier if they decide to change to a distributed version control\nsystem such as git, mercurial or bazaar, since those generally have a much better\ndata model than centralized systems where you can get away with ugly workarounds\nmuch easier.\n\nWorkable conversions are possible between very nearly every scm out there and\nwhatever you want. It's not necessarily possible to convert back in a way that\nmakes it look as if you'd used the original scm all the time, but it's (almost)\nalways possible to create a new repository that is obviously functional and\ncontains all or most of the relevant history. Such conversions generally take\nquite a while for projects with a large history though, so it's not something\nyou'd ponder doing twice a week.\n\nMost larger projects that have switched have done so with the intention of\nusing the switched-to system for at least the foreseeable future and have thus\ndone proper research into what that system has to offer and then made the\ndecision based on featureset, data model and usability of the available\nsystems. If you make the decision based on how easy it is to switch away\nfrom the system you choose, you're bound to end up having to do just that\nsooner or later.\n\nEither way, very nearly every project in the world who in the past 3 years\nhave gone looking for a better version control system has gone with one\nof the two major distributed systems, git and mercurial. Conversion between\nthose two is relatively quick and painless and produces working repositories\nin both directions, even if git -> hg -> git doesn't necessarily produce a\nrepository identical to the one you started out with.\n\n>> So let me ask you a question; What have you done so far to find out the answer\n>> to the questions you're looking for, apart from asking here how a theoretical\n>> scenario would pan out?\n> \n> Fair enough. I am looking for documentation, and advice on how to find worthy\n> sources, before delving into trying out something for no good reason.\n> But, at the same time, I wish to have a look-see on what the general status\n> is right now. If there is no summary of it available, then apparently I have to\n> make it myself. But I would hate to find out afterward that someone had in fact\n> done it already. Tomas Carnecky's list is a good start.\n> \n\nYou're looking at the wrong criteria for switching vcs. If the main goal is to\nbe able to convert the target repository back into the original one, you're up\nfor disappointment in what you'll find. None of the conversion tools destroy\nthe original repository though, so you can experiment with any one you like.\n\nSome general truths that might aid you though:\ngit has the best fast-{import,export} support. Not surprising since the format\nwas engineered by the brilliant minds we're fortunate to have on the git list.\n\nConversion between git and other distributed version control systems produce\nworking repositories in a quick and painfree fashion.\n\nIt is usually impossible to convert from one repository format to the other\nand back again in such a fashion that the resulting repository is identical\nto the one you started with.\n\nIt is almost always possible to create a working repository of any kind from\na repository type that has a fast-export tool.\n\nWriting a fast-import tool is not exactly rocket science, although it could\nget timeconsuming if the target vcs is limited in capabilities and many\nworkarounds are necessary.\n\nMost people don't switch to a vcs with fewer capabilities than the one they're\nalready using, so the previous point is mostly academic.\n\n-- \nAndreas Ericsson andreas.ericsson@op5.se\nOP5 AB www.op5.se\nTel: +46 8-230225 Fax: +46 8-230231\n\nConsidering the successes of the wars on alcohol, poverty, drugs and\nterror, I think we should give some serious thought to declaring war\non peace.\n"},{"id":"152058","messageId":"loom.20100929T155226-300@post.gmane.org","threadId":"25273","inReplyTo":"4CA33EE3.3090202@op5.se","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Tuomo","fromEmail":"tuo.tie@gmail.com","sentAt":"2010-09-29T13:53:57Z","receivedAt":"2010-09-29T13:53:57Z","isPatch":false,"sender":{"key":"tuo.tie@gmail.com","avatar":null},"body":"Andreas Ericsson op5.se> writes:\n\n> You're looking at the wrong criteria for switching vcs.\n\nAll right, since this forum is for those who have to have a very concrete \nproblem at hands, I am clearly in the wrong company.\n\nWhat forum would you suggest for more philosophical/academic ponderings? \nThe old comp.softare.config-mgmt newsgroup is not available to me (can't \nuse newsreader, and it's not on gmane), and it's dead anyway, full of \ncommercial bs instead of discussion. Where has the general discussion \nmoved?\n"},{"id":"152059","messageId":"80ocbgk5t3.fsf@tiny.isode.net","threadId":"25273","inReplyTo":"loom.20100929T155226-300@post.gmane.org","subject":"Re: Another way to compare tools: is it possible to transfer full history?","fromName":"Bruce Stephens","fromEmail":"bruce.stephens@isode.com","sentAt":"2010-09-29T14:00:56Z","receivedAt":"2010-09-29T14:00:56Z","isPatch":false,"sender":{"key":"bruce.stephens@isode.com","avatar":null},"body":"Tuomo writes:\n\n[...]\n\n> Where has the general discussion moved?\n\nDunno. There's a (extremely low-volume) revctrl mailing list with a\nmarginally more active IRC channel. Maybe that's it, or maybe people\nare using blogs?\n"}]}