{"thread":{"id":"12956","subject":"[PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","startedAt":"2008-04-02T01:34:51Z","lastAt":"2008-04-27T08:48:25Z","messageCount":18,"participants":["David Mansfield","Junio C Hamano","Steffen Prohaska","Michael Haggerty","Jean-François Veillette","Ping Yin"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"73525","messageId":"1207100091.10532.64.camel@gandalf.cobite.com","threadId":"12956","inReplyTo":null,"subject":"[PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"David Mansfield","fromEmail":"david@cobite.com","sentAt":"2008-04-02T01:34:51Z","receivedAt":"2008-04-02T01:34:51Z","isPatch":true,"sender":{"key":"david@cobite.com","avatar":null},"body":"Hi Everyone,\n\nThis email addresses a long-standing bug with the cvsimport which is due\nto a bugs in cvsps. The bug is that branches can be forked off too\nlate. \n\nIn case you're wondering, I'm actually the original author of cvsps,\nwhich is behind the scenes for cvsimport. I don't call myself\nmaintainer because I've hardly been that over the last few years.\n\nAnyway, the fix to cvsps is attached (1st 2 patches) as well as the\npatch to git-cvsimport.perl (2nd 2 patches) against the master branch as\nof today's git repo.\n\nThe cvsps patches apply with fuzz against the 2.1 version which is out\nthere.\n\nThe full tarball of the latest cvsps version including this is available\non the website http://www.cobite.com/cvsps as well, the version is\n2.2b1.\n\nI plan to find time in the next week or so to merge all of the\noutstanding patches from Yann Dirson's git repo, publish cvsps via a git\nrepo myself, and fix other bugs as time permits (including adding\nsupport for multiple tags).\n\nI'd mainly like feedback if anyone can test this.\n\nAlso, as I'm actually a newb. to this list, if I'm violating any rules,\nsuch as how to post the patches, let me know.\n\nThanks,\nDavid\n\nP.S Also, as many people may have imported broken branches already, can\nanyone thing of a way to fix the branch, (maybe with git-rebase or\nsomething)? The breakage affects, I believe, files not ever modified on\nthe branch until any given point in time on the branch...\n\n"},{"id":"73555","messageId":"7vprt8838y.fsf@gitster.siamese.dyndns.org","threadId":"12956","inReplyTo":"1207100091.10532.64.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-02T19:29:17Z","receivedAt":"2008-04-02T19:29:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Mansfield writes:\n\n> In case you're wondering, I'm actually the original author of cvsps,\n> which is behind the scenes for cvsimport. I don't call myself\n> maintainer because I've hardly been that over the last few years.\n>\n> Anyway, the fix to cvsps is attached (1st 2 patches) as well as the\n> patch to git-cvsimport.perl (2nd 2 patches) against the master branch as\n> of today's git repo.\n>\n> The cvsps patches apply with fuzz against the 2.1 version which is out\n> there.\n\nWhen output from an unfixed cvsps is fed to the updated cvsimport, does it\ngracefully do the wrong thing (iow, create the same broken history not too\nmuch worse than the original)?\n\n> @@ -826,12 +824,9 @@ while () {\n> \t\t$branch = $_;\n> \t\t$state = 5;\n> \t} elsif ($state == 5 and s/^Ancestor branch:\\s+//) {\n> -\t\ts/\\s+$//;\n> -\t\t$ancestor = $_;\n> -\t\t$ancestor = $opt_o if $ancestor eq \"HEAD\";\n> +\t\t# now ignored. see 'Branches' below\n> \t\t$state = 6;\n> \t} elsif ($state == 5) {\n> -\t\t$ancestor = undef;\n> \t\t$state = 6;\n> \t\tredo;\n> \t} elsif ($state == 6 and s/^Tag:\\s+//) {\n"},{"id":"73570","messageId":"1207187090.17329.15.camel@gandalf.cobite.com","threadId":"12956","inReplyTo":"7vprt8838y.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"David Mansfield","fromEmail":"david@cobite.com","sentAt":"2008-04-03T01:44:50Z","receivedAt":"2008-04-03T01:44:50Z","isPatch":true,"sender":{"key":"david@cobite.com","avatar":null},"body":"\n\nOn Wed, 2008-04-02 at 12:29 -0700, Junio C Hamano wrote:\n> David Mansfield writes:\n> \n> > In case you're wondering, I'm actually the original author of cvsps,\n> > which is behind the scenes for cvsimport. I don't call myself\n> > maintainer because I've hardly been that over the last few years.\n> >\n> > Anyway, the fix to cvsps is attached (1st 2 patches) as well as the\n> > patch to git-cvsimport.perl (2nd 2 patches) against the master branch as\n> > of today's git repo.\n> >\n> > The cvsps patches apply with fuzz against the 2.1 version which is out\n> > there.\n> \n> When output from an unfixed cvsps is fed to the updated cvsimport, does it\n> gracefully do the wrong thing (iow, create the same broken history not too\n> much worse than the original)?\n> \n> > @@ -826,12 +824,9 @@ while () {\n> > \t\t$branch = $_;\n> > \t\t$state = 5;\n> > \t} elsif ($state == 5 and s/^Ancestor branch:\\s+//) {\n> > -\t\ts/\\s+$//;\n> > -\t\t$ancestor = $_;\n> > -\t\t$ancestor = $opt_o if $ancestor eq \"HEAD\";\n> > +\t\t# now ignored. see 'Branches' below\n> > \t\t$state = 6;\n> > \t} elsif ($state == 5) {\n> > -\t\t$ancestor = undef;\n> > \t\t$state = 6;\n> > \t\tredo;\n> > \t} elsif ($state == 6 and s/^Tag:\\s+//) {\n> \n\nNot currently. I'm just searching for failure modes for the feature at\nthe moment (I've already found one myself). \n\nYou're right to point this out though. Maybe someone can help me write\nsome tests for this?\n\nAlso, how does the git packaging (non-rpm version) specify and/or\nguarantee dependencies are at a certain version anyway?\n\nDavid\n"},{"id":"73571","messageId":"7viqyz66b3.fsf@gitster.siamese.dyndns.org","threadId":"12956","inReplyTo":"1207187090.17329.15.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-04-03T02:06:08Z","receivedAt":"2008-04-03T02:06:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Mansfield writes:\n\n> Also, how does the git packaging (non-rpm version) specify and/or\n> guarantee dependencies are at a certain version anyway?\n\nWe cannot really do much with the old cvsimport out in the field, but I\nwas wondering more about automatic detection in new cvsimport.\n\nThe way I read 02-cvsps-implement-branch-point-detection.patch, you have\nthree cases:\n\n - \"Ancestor branch:\" is not followed by \"Branches:\" before \"Log:\"\n (old cvsps);\n\n - \"Ancestor branch:\" is followed by \"Branches:\" before \"Log:\" (new);\n\n - \"Branches:\" without \"Ancestor branch:\" (new);\n\nSo perhaps your 04-cvsimport-redo-branch-creation-process.patch, instead\nof ignoring what \"Ancestor branch:\" said, can remember it has seen what\n\"ancestor\" (which may be a bit off) information it was given, and when you\nsee \"Log:\" (by that time, you either have seen \"Branches:\" from new cvsps,\nor you didn't see it from old cvsps) you can decide which vintage of cvsps\nit is reading from.\n\nOr something like that.\n"},{"id":"73574","messageId":"1207189663.17329.27.camel@gandalf.cobite.com","threadId":"12956","inReplyTo":"7viqyz66b3.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"David Mansfield","fromEmail":"david@cobite.com","sentAt":"2008-04-03T02:27:43Z","receivedAt":"2008-04-03T02:27:43Z","isPatch":true,"sender":{"key":"david@cobite.com","avatar":null},"body":"\nOn Wed, 2008-04-02 at 19:06 -0700, Junio C Hamano wrote:\n> David Mansfield writes:\n> \n> > Also, how does the git packaging (non-rpm version) specify and/or\n> > guarantee dependencies are at a certain version anyway?\n> \n> We cannot really do much with the old cvsimport out in the field, but I\n> was wondering more about automatic detection in new cvsimport.\n> \n> The way I read 02-cvsps-implement-branch-point-detection.patch, you have\n> three cases:\n> \n> - \"Ancestor branch:\" is not followed by \"Branches:\" before \"Log:\"\n> (old cvsps);\n> \n> - \"Ancestor branch:\" is followed by \"Branches:\" before \"Log:\" (new);\n> \n> - \"Branches:\" without \"Ancestor branch:\" (new);\n> \n> So perhaps your 04-cvsimport-redo-branch-creation-process.patch, instead\n> of ignoring what \"Ancestor branch:\" said, can remember it has seen what\n> \"ancestor\" (which may be a bit off) information it was given, and when you\n> see \"Log:\" (by that time, you either have seen \"Branches:\" from new cvsps,\n> or you didn't see it from old cvsps) you can decide which vintage of cvsps\n> it is reading from.\n> \n> Or something like that.\n\nQuite right. And also, one of the failure modes I've found is based on\nreal abuse of cvs, and the result is that cvsps shows the branch as\noccurring AFTER the first commit on that branch. Anyway, it's all\nnonsense and hand waiving after all. cvsps just creates an illusion\nanyway.\n\nBut to fix it, I'll need something similar to the code that I removed\nanyway, so that will definitely be in the mix when all is said and done.\n\nDavid\n"},{"id":"73576","messageId":"0C7AA499-56AD-4D20-AED0-9E7DDD0C77DF@zib.de","threadId":"12956","inReplyTo":"1207100091.10532.64.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-04-03T05:47:54Z","receivedAt":"2008-04-03T05:47:54Z","isPatch":true,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"\nOn Apr 2, 2008, at 3:34 AM, David Mansfield wrote:\n\n> P.S Also, as many people may have imported broken branches already, \n> can\n> anyone thing of a way to fix the branch, (maybe with git-rebase or\n> something)? The breakage affects, I believe, files not ever \n> modified on\n> the branch until any given point in time on the branch...\n>\n\nThe breakage you describe might be the same breakage that I recognized\nin June 2007:\n\n http://article.gmane.org/gmane.comp.version-control.git/50736\n\nAt that time, I wrote a script (git-transplant) that fixed a broken\nimport from CVS for me:\n\n http://article.gmane.org/gmane.comp.version-control.git/50746\n\nThe discussion in\n\n http://article.gmane.org/gmane.comp.version-control.git/50789\n\nexplains the reason for the script a bit more detailed.\n\nBut note that I never finished git-transplant and I also failed to\nconvince anyone that the idea behind the script is of any general value.\nInstead, I decided that git-cvsimport is not the right tools for me; and\nsince then I use parsecvs to convert my repositories.\n\n Steffen\n"},{"id":"73602","messageId":"1207230582.17329.39.camel@gandalf.cobite.com","threadId":"12956","inReplyTo":"0C7AA499-56AD-4D20-AED0-9E7DDD0C77DF@zib.de","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"David Mansfield","fromEmail":"david@cobite.com","sentAt":"2008-04-03T13:49:42Z","receivedAt":"2008-04-03T13:49:42Z","isPatch":true,"sender":{"key":"david@cobite.com","avatar":null},"body":"\nOn Thu, 2008-04-03 at 07:47 +0200, Steffen Prohaska wrote:\n> On Apr 2, 2008, at 3:34 AM, David Mansfield wrote:\n> \n> > P.S Also, as many people may have imported broken branches already, \n> > can\n> > anyone thing of a way to fix the branch, (maybe with git-rebase or\n> > something)? The breakage affects, I believe, files not ever \n> > modified on\n> > the branch until any given point in time on the branch...\n> >\n> \n> The breakage you describe might be the same breakage that I recognized\n> in June 2007:\n> \n> http://article.gmane.org/gmane.comp.version-control.git/50736\n> \n> At that time, I wrote a script (git-transplant) that fixed a broken\n> import from CVS for me:\n> \n> http://article.gmane.org/gmane.comp.version-control.git/50746\n> \n> The discussion in\n> \n> http://article.gmane.org/gmane.comp.version-control.git/50789\n> \n> explains the reason for the script a bit more detailed.\n> \n> But note that I never finished git-transplant and I also failed to\n> convince anyone that the idea behind the script is of any general value.\n> Instead, I decided that git-cvsimport is not the right tools for me; and\n> since then I use parsecvs to convert my repositories.\n> \n\n\nYes. It's the same problem. It will be fixed with the above patches\nonce they stabilize. I'll look at the transplant thing too. It looks\nlike a good idea.\n\nThe main issue with git-cvsimport stems from an unfixable problem.\ncvsps's design goal is to show commits in chronological order. Based\nsolely on this data, it's impossible to always reconstruct a branch\npoint (or a tag) since a person could have committed files after someone\nelse's commit, but not done an update then tagged. \n\nSo some files are from before the 'other' user's commit, and some files\nafter. What can you do? \n\nIt's not per se a flaw in cvsps, it always wanted to show commits in\nchronological order, but it is a severe limitation in using cvsps to\ngenerate changesets for git.\n\nBy engineering a direct tool (such as parsecvs, I presume) these\nobstacles can be overcome by constructing some commits that were never\nmade by the actual users of the cvs repo in order to get it right.\n\nI'm not sure exactly how this is done, because I've never looked at\nparsecvs.\n\nDavid\n"},{"id":"73647","messageId":"47F5FA79.8010604@alum.mit.edu","threadId":"12956","inReplyTo":"1207230582.17329.39.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2008-04-04T09:52:57Z","receivedAt":"2008-04-04T09:52:57Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"David Mansfield wrote:\n> The main issue with git-cvsimport stems from an unfixable problem.\n> cvsps's design goal is to show commits in chronological order. Based\n> solely on this data, it's impossible to always reconstruct a branch\n> point (or a tag) since a person could have committed files after someone\n> else's commit, but not done an update then tagged. \n\nJust to be more explicit, I think you are talking about a situation like\nthis:\n\n1. Add file1:1.1 and file2:1.1 to repository.\n2. User1 modifies file1 and commits file1:1.2.\n ...some non-negligible amount of time passes...\n3. User2 modifies file2 and commits file2:1.2.\n4. User2, without updating file1 to revision 1.2, adds a tag.\n\nThis results in a tag that refers to file1:1.1 and file2:1.2, even\nthough those two revisions never appeared in the repository at the same\ntime.\n\n> So some files are from before the 'other' user's commit, and some files\n> after. What can you do? \n\nYou can do the only thing that is consistent with the CVS\nhistory--create the tag not from a single source revision but from\nmultiple revisions. Unfortunately, git cannot handle this directly, but\nthere is a workaround using a \"fixup branch\" [1].\n\ncvs2svn/cvs2git [2] creates a \"fixup branch\", copies file1:1.1 and\nfile2:1.2 onto that branch, then creates the tag from the fixup branch.\n This ensures that checking the tag out of git gives the same file\ncontents as checking the tag out of CVS. I think that git-cvsimport\ngets this wrong (!?!)\n\nIt is your framing of the problem that is leading to the impossibility.\n CVS's design does *not* require that a tag or branch is created in a\nsingle commit, nor that it is created from a single source revision.\nTrying to impose these artificial constraints means that the resulting\ngit repository is inconsistent with the CVS repository in quite common\ncircumstances.\n\n> It's not per se a flaw in cvsps, it always wanted to show commits in\n> chronological order, but it is a severe limitation in using cvsps to\n> generate changesets for git.\n\ncvs2git always creates commits in chronological order too, but its\noutput is by design always consistent with the CVS record.\n\n> By engineering a direct tool (such as parsecvs, I presume) these\n> obstacles can be overcome by constructing some commits that were never\n> made by the actual users of the cvs repo in order to get it right.\n> \n> I'm not sure exactly how this is done, because I've never looked at\n> parsecvs.\n\ncvs2git's design is documented quite extensively, if you are interested\n[3]. Parsecvs, AFAIK, uses a similar approach.\n\nMichael\n\n[1] http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html\n[2] http://cvs2svn.tigris.org/cvs2git.html\n[3] http://cvs2svn.tigris.org/svn/cvs2svn/trunk/doc/design-notes.txt\n"},{"id":"73831","messageId":"1207590845.17329.98.camel@gandalf.cobite.com","threadId":"12956","inReplyTo":"47F5FA79.8010604@alum.mit.edu","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"David Mansfield","fromEmail":"david@cobite.com","sentAt":"2008-04-07T17:54:05Z","receivedAt":"2008-04-07T17:54:05Z","isPatch":true,"sender":{"key":"david@cobite.com","avatar":null},"body":"\nOn Fri, 2008-04-04 at 11:52 +0200, Michael Haggerty wrote:\n> David Mansfield wrote:\n> > The main issue with git-cvsimport stems from an unfixable problem.\n> > cvsps's design goal is to show commits in chronological order. Based\n> > solely on this data, it's impossible to always reconstruct a branch\n> > point (or a tag) since a person could have committed files after someone\n> > else's commit, but not done an update then tagged. \n> \n> Just to be more explicit, I think you are talking about a situation like\n> this:\n> \n> 1. Add file1:1.1 and file2:1.1 to repository.\n> 2. User1 modifies file1 and commits file1:1.2.\n> ...some non-negligible amount of time passes...\n> 3. User2 modifies file2 and commits file2:1.2.\n> 4. User2, without updating file1 to revision 1.2, adds a tag.\n> \n> This results in a tag that refers to file1:1.1 and file2:1.2, even\n> though those two revisions never appeared in the repository at the same\n> time.\n> \n\nMore or less, yes. It gets worse if a user does 'cvs update' in a\ndirectory, or on an individual file.\n\n\n> > So some files are from before the 'other' user's commit, and some files\n> > after. What can you do? \n> \n> You can do the only thing that is consistent with the CVS\n> history--create the tag not from a single source revision but from\n> multiple revisions. Unfortunately, git cannot handle this directly, but\n> there is a workaround using a \"fixup branch\" [1].\n> \n> cvs2svn/cvs2git [2] creates a \"fixup branch\", copies file1:1.1 and\n> file2:1.2 onto that branch, then creates the tag from the fixup branch.\n> This ensures that checking the tag out of git gives the same file\n> contents as checking the tag out of CVS. I think that git-cvsimport\n> gets this wrong (!?!)\n> \n> It is your framing of the problem that is leading to the impossibility.\n> CVS's design does *not* require that a tag or branch is created in a\n> single commit, nor that it is created from a single source revision.\n> Trying to impose these artificial constraints means that the resulting\n> git repository is inconsistent with the CVS repository in quite common\n> circumstances.\n> \n\nIt's not 'my framing of the problem.' It's 'the design goal of cvsps is\nnot compatible with the desire to use the output of cvsps to create a\ngit repository.' See the difference?\n\n> > It's not per se a flaw in cvsps, it always wanted to show commits in\n> > chronological order, but it is a severe limitation in using cvsps to\n> > generate changesets for git.\n> \n> cvs2git always creates commits in chronological order too, but its\n> output is by design always consistent with the CVS record.\n> \n\nYes. That's what cvs2git was designed for. Look at the name. In order\nto create the 'fixup' branch, you have to make some out of operations,\nwhich is fine if that's what your design goal is.\n\nThe design goal of cvsps was always simply to show who did what and in\nwhat chronological order. However, just with that, it's impossible to\nuse for the purpose it is currently being used for.\n\nThe 'fixup branch' sounds like a really great idea and an elegant\nsolution.\n\n> > By engineering a direct tool (such as parsecvs, I presume) these\n> > obstacles can be overcome by constructing some commits that were never\n> > made by the actual users of the cvs repo in order to get it right.\n> > \n> > I'm not sure exactly how this is done, because I've never looked at\n> > parsecvs.\n> \n> cvs2git's design is documented quite extensively, if you are interested\n> [3]. Parsecvs, AFAIK, uses a similar approach.\n> \n\nI'm quite happy that there are other tools, and even more happy if they\nalready fix every bug that git-cvsimport has.\n\nI was simply addressing the bug from the standpoint of: this issue can\nbe fixed without compromising what cvsps wants to be as a tool.\n\nThe place where the fixup branch logic needs to be is in git-cvsimport,\nnot in cvsps. Better yet, get rid of git-cvsimport and replace it with\ncvs2git if it works better. \n\nHowever, if possible, I'd like to fix problems with the\ncvsps/git-cvsimport if possible, unless someone can tell me for sure\nthat it's obsolete and noone uses it.\n\nThanks,\nDavid\n"},{"id":"73832","messageId":"72D530CB-3228-42BC-A4F7-253E37F6EAAB@yahoo.ca","threadId":"12956","inReplyTo":"1207590845.17329.98.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Jean-François Veillette","fromEmail":"jean_francois_veillette@yahoo.ca","sentAt":"2008-04-07T18:07:37Z","receivedAt":"2008-04-07T18:07:37Z","isPatch":true,"sender":{"key":"jean_francois_veillette@yahoo.ca","avatar":null},"body":"\n> However, if possible, I'd like to fix problems with the cvsps/git- \n> cvsimport if possible, unless someone can tell me for sure that \n> it's obsolete and noone uses it.\n\nI do use it, please fix the duo cvsps/git-cvsimport (if possible).\nThe fact that it's integrated with git make it very useful and handy.\nIt is working well for almost all of my cvs repo that I track with git.\nIf it would work for all of them it would be wonderful !\n\n- jfv\n"},{"id":"73914","messageId":"47FC2190.3070303@alum.mit.edu","threadId":"12956","inReplyTo":"1207590845.17329.98.camel@gandalf.cobite.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2008-04-09T01:53:20Z","receivedAt":"2008-04-09T01:53:20Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"David Mansfield wrote:\n> The design goal of cvsps was always simply to show who did what and in\n> what chronological order. However, just with that, it's impossible to\n> use for the purpose it is currently being used for.\n\nGood point.\n\nI re-read the cvsps manpage and found the information about \"FUNKY\" and\n\"INVALID\" tags. I'd forgotten that cvsps does the right thing in some\ncases by warning the user about tags that are beyond its abilities to\ndescribe. (But there are other problems that cvsps doesn't warn about;\nsee below.)\n\nThen I looked in the git-cvsimport code to see how it deals with FUNKY\nand INVALID tags. It does the *wrong* thing by explicitly ignoring\nthese warnings (!). IMHO git-cvsimport should notice the **FUNKY** and\n**INVALID** annotations and at least output a warning to the user that\nthe associated tags may not have been converted correctly.\n\nBut cvsps makes some other symbol-related mistakes, presumably in the\nname of simplification. These problems make it impossible for\ngit-cvsimport to generate accurate branches and tags, even if it were to\nuse fixup branches internally. Moreover, many of these are silent\nfailures; there is no way that git-cvsimport could even determine that\nthe cvsps output is inadequate. For example, if I understand correctly:\n\n- cvsps pretends that a tag or branch is applied to a single snapshot of\nthe repository on a single branch, even though in reality:\n\n - some files might have been left out of the tag/branch (cvsps doesn't\ngive any indication if this was the case). If this tag/branch is\nchecked out, the files that were not tagged are erroneously included.\n\n - the revisions not being tagged might not have all existed\ncontemporaneously (cvsps indicates these cases by marking the tags\n**FUNKY** or **INVALID**).\n\n - a tag can be applied to different files on different branches; e.g.,\na tag can contain file1:1.3 (from trunk) and file2:1.2.2.1 (from some\nother branch). cvsps seems to pick one branch as source without\nindicating a problem. The inevitable result in cvsps is that the tag\nincludes the wrong contents for some files with no way to detect the error.\n\n- If there is no commit on a branch, cvsps ignores the branch entirely.\n (Maybe this is fixed by your recent patch?)\n\n- If there are multiple tags applied to the same set of file revisions\n(for example, a daily tag and a release tag), cvsps silently ignores all\nbut one of them. This causes unavoidable data loss in git-cvsimport.\n\nThere are lots of more complicated scenarios that I haven't tested\nagainst cvsps...\n\nGranted, cvsps was not written to be usable for converters. But\nregardless of whether the output is being read by a human or by another\nprogram, its output can be wrong, and there is often no way to tell from\nthe output that it had a problem. Maybe cvsps could emit warning\nannotations in more of the situations that it punts on, and\ngit-cvsimport could pass these warnings along to the end user?\nOtherwise people will believe that git-cvsimport is converting their\nrepository accurately when in fact it often silently produces incorrect\noutput.\n\n> The place where the fixup branch logic needs to be is in git-cvsimport,\n> not in cvsps. Better yet, get rid of git-cvsimport and replace it with\n> cvs2git if it works better. \n\ncvs2git hopefully gives a more accurate conversion of a CVS repository\n-- it handles all of the cases described above, plus many more [1] --\nbut it is much slower and can't work incrementally. So there is\ndefinitely still demand for something like git-cvsimport.\n\nMichael\n\n[1] http://cvs2svn.tigris.org/features.html\n"},{"id":"75253","messageId":"46dff0320804262206p76941ee8la6a784ed0b6f8294@mail.gmail.com","threadId":"12956","inReplyTo":"47FC2190.3070303@alum.mit.edu","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T05:06:53Z","receivedAt":"2008-04-27T05:06:53Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Wed, Apr 9, 2008 at 9:53 AM, Michael Haggerty wrote:\n> > The place where the fixup branch logic needs to be is in git-cvsimport,\n> > not in cvsps. Better yet, get rid of git-cvsimport and replace it with\n> > cvs2git if it works better.\n>\n> cvs2git hopefully gives a more accurate conversion of a CVS repository\n> -- it handles all of the cases described above, plus many more [1] --\n> but it is much slower and can't work incrementally. So there is\n> definitely still demand for something like git-cvsimport.\n>\n\nThese days i tried to convert the cvs repository into git. I really\nwant the conversion to be as accurate as possible. However, the cvs\nrepository has been tagged in a very bad style which makes\ngit-cvsimport or cvsps not work well.\n\ncvs2git sounds to be the right tool i should try. Unfortualely, i\ncan't touch the cvs repository directly. So is it possible to use\ncvs2git in the remote host instead of the host of the cvs repository\njust as git-cvsimport does? Yes, i know it can't now. I just wonder\nwhether it is possible to implement.\n\nI choose to reply to this thread instead of opening a new one because\ni think this reply of Michael has told much shortcommings of\ngit-cvsimport or cvsps but had got no replies yet.\n\n-- \nPing Yin\n"},{"id":"75256","messageId":"48141379.9030306@alum.mit.edu","threadId":"12956","inReplyTo":"46dff0320804262206p76941ee8la6a784ed0b6f8294@mail.gmail.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Michael Haggerty","fromEmail":"mhagger@alum.mit.edu","sentAt":"2008-04-27T05:47:37Z","receivedAt":"2008-04-27T05:47:37Z","isPatch":true,"sender":{"key":"mhagger@alum.mit.edu","avatar":"https://avatars.githubusercontent.com/u/119718?v=4"},"body":"Ping Yin wrote:\n> These days i tried to convert the cvs repository into git. I really\n> want the conversion to be as accurate as possible. However, the cvs\n> repository has been tagged in a very bad style which makes\n> git-cvsimport or cvsps not work well.\n> \n> cvs2git sounds to be the right tool i should try. Unfortualely, i\n> can't touch the cvs repository directly. So is it possible to use\n> cvs2git in the remote host instead of the host of the cvs repository\n> just as git-cvsimport does? Yes, i know it can't now. I just wonder\n> whether it is possible to implement.\n\ncvs2svn/cvs2git itself can't work with remote repositories. It would be\nenough if you could just get a copy of the repository; obviously you\ndon't need to use the original.\n\nIf you can't get a copy of the CVS repository directly, you might be\nable to recreate it indirectly via information read over the CVS\nprotocol using a tool like CVSsuck [1,2]. I have no experience with\nCVSsuck, so if you try it out, please let us know whether you were\nsuccessful.\n\nPresumably some CVSsuck-like functionality could be built directly into\ncvs2git, but given that this request hasn't come up very often and that\nthe two tools can presumably be used in concert, it doesn't seem worth\nthe effort.\n\nMichael\n\n[1] http://cvs.m17n.org/~akr/cvssuck/\n[2] http://cvs2svn.tigris.org/faq.html#repoaccess\n"},{"id":"75257","messageId":"46dff0320804262251g3a0f12b2rb9197fc07ea57c51@mail.gmail.com","threadId":"12956","inReplyTo":"48141379.9030306@alum.mit.edu","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T05:51:07Z","receivedAt":"2008-04-27T05:51:07Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sun, Apr 27, 2008 at 1:47 PM, Michael Haggerty wrote:\n> Ping Yin wrote:\n> > These days i tried to convert the cvs repository into git. I really\n> > want the conversion to be as accurate as possible. However, the cvs\n> > repository has been tagged in a very bad style which makes\n> > git-cvsimport or cvsps not work well.\n> >\n> > cvs2git sounds to be the right tool i should try. Unfortualely, i\n> > can't touch the cvs repository directly. So is it possible to use\n> > cvs2git in the remote host instead of the host of the cvs repository\n> > just as git-cvsimport does? Yes, i know it can't now. I just wonder\n> > whether it is possible to implement.\n>\n> cvs2svn/cvs2git itself can't work with remote repositories. It would be\n> enough if you could just get a copy of the repository; obviously you\n> don't need to use the original.\n>\n> If you can't get a copy of the CVS repository directly, you might be\n> able to recreate it indirectly via information read over the CVS\n> protocol using a tool like CVSsuck [1,2]. I have no experience with\n> CVSsuck, so if you try it out, please let us know whether you were\n> successful.\n>\n\nTHX. If i try out cvssuck, i will let you know.\n\n\n-- \nPing Yin\n"},{"id":"75275","messageId":"46dff0320804270038t651d94d4t3c63fe0cb5240e0d@mail.gmail.com","threadId":"12956","inReplyTo":"46dff0320804262251g3a0f12b2rb9197fc07ea57c51@mail.gmail.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T07:38:39Z","receivedAt":"2008-04-27T07:38:39Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sun, Apr 27, 2008 at 1:51 PM, Ping Yin wrote:\n> On Sun, Apr 27, 2008 at 1:47 PM, Michael Haggerty wrote:\n> > Ping Yin wrote:\n> > > These days i tried to convert the cvs repository into git. I really\n> > > want the conversion to be as accurate as possible. However, the cvs\n> > > repository has been tagged in a very bad style which makes\n> > > git-cvsimport or cvsps not work well.\n> > >\n> > > cvs2git sounds to be the right tool i should try. Unfortualely, i\n> > > can't touch the cvs repository directly. So is it possible to use\n> > > cvs2git in the remote host instead of the host of the cvs repository\n> > > just as git-cvsimport does? Yes, i know it can't now. I just wonder\n> > > whether it is possible to implement.\n> >\n> > cvs2svn/cvs2git itself can't work with remote repositories. It would be\n> > enough if you could just get a copy of the repository; obviously you\n> > don't need to use the original.\n> >\n> > If you can't get a copy of the CVS repository directly, you might be\n> > able to recreate it indirectly via information read over the CVS\n> > protocol using a tool like CVSsuck [1,2]. I have no experience with\n> > CVSsuck, so if you try it out, please let us know whether you were\n> > successful.\n> >\n>\n> THX. If i try out cvssuck, i will let you know.\n>\n\nGreat, i succeed. And the result is exactly what i want!\n\nHowever, it is so so slow.\n\nHere is a example to convert a module util from cvs to git\n--------------------------------------------------------------------------------------------\n$ cvssuck $CVSROOT util <1>\n$ mkdir util/CVSROOT <2>\n$ edit cvs2svn-git.options and cvs2svn-example.options\n ( change run_options.add_project and ctx.cvs_log_decorder)\n$ cvs2svn --options=cvs2svn-git.options\n$ mkdir util.git && cd util.git && git init\n$ cat ../cvs2svn-tmp/git-{blob,dump}.dat | git-fast-import\n-------------------------------------------------------------------------------------------\n<1> very slow, about 30 minutes for a very small module.\n Other steps are fast enough.\n<2> I have to create a dir util/CVSROOT to avoid the error\n\"util is not a CVS repository, nor a path within a CVS repository. A\nCVS repository contains a CVSROOT directory within its root\ndirectory.\"\n\n\n-- \nPing Yin\n"},{"id":"75276","messageId":"46dff0320804270043k7e978abcoe44f192cd49e5df4@mail.gmail.com","threadId":"12956","inReplyTo":"46dff0320804270038t651d94d4t3c63fe0cb5240e0d@mail.gmail.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T07:43:16Z","receivedAt":"2008-04-27T07:43:16Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sun, Apr 27, 2008 at 3:38 PM, Ping Yin wrote:\n\n> <1> very slow, about 30 minutes for a very small module.\n\nMore accurate, about 500 commits and 300 files\n\n\n-- \nPing Yin\n"},{"id":"75277","messageId":"46dff0320804270048h6938b8c9scdc1f3e922e41231@mail.gmail.com","threadId":"12956","inReplyTo":"46dff0320804270038t651d94d4t3c63fe0cb5240e0d@mail.gmail.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T07:48:06Z","receivedAt":"2008-04-27T07:48:06Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sun, Apr 27, 2008 at 3:38 PM, Ping Yin wrote:\n>\n> On Sun, Apr 27, 2008 at 1:51 PM, Ping Yin wrote:\n> > On Sun, Apr 27, 2008 at 1:47 PM, Michael Haggerty wrote:\n> > > Ping Yin wrote:\n> > > > These days i tried to convert the cvs repository into git. I really\n> > > > want the conversion to be as accurate as possible. However, the cvs\n> > > > repository has been tagged in a very bad style which makes\n> > > > git-cvsimport or cvsps not work well.\n> > > >\n> > > > cvs2git sounds to be the right tool i should try. Unfortualely, i\n> > > > can't touch the cvs repository directly. So is it possible to use\n> > > > cvs2git in the remote host instead of the host of the cvs repository\n> > > > just as git-cvsimport does? Yes, i know it can't now. I just wonder\n> > > > whether it is possible to implement.\n> > >\n> > > cvs2svn/cvs2git itself can't work with remote repositories. It would be\n> > > enough if you could just get a copy of the repository; obviously you\n> > > don't need to use the original.\n> > >\n> > > If you can't get a copy of the CVS repository directly, you might be\n> > > able to recreate it indirectly via information read over the CVS\n> > > protocol using a tool like CVSsuck [1,2]. I have no experience with\n> > > CVSsuck, so if you try it out, please let us know whether you were\n> > > successful.\n> > >\n> >\n> > THX. If i try out cvssuck, i will let you know.\n> >\n>\n> Great, i succeed. And the result is exactly what i want!\n>\n\nNot exactly, for my another conversion.\n\n$ git log --pretty=online --name-status x64_UI_071204\n724eb47 \\\nThis commit was manufactured by cvs2svn to create tag 'x64_UI_071204'.\n9362987 add support of writing cookies of 'fromid';\nM logqueue.c\nM ui.c\nM ui.h\n\nShould we avoid recording the commit 724eb47 since it is the same with\ncommit 9362987 (no content change)？\n\n-- \nPing Yin\n"},{"id":"75278","messageId":"46dff0320804270148y52a593fbt57b8a8fc3184267d@mail.gmail.com","threadId":"12956","inReplyTo":"46dff0320804270048h6938b8c9scdc1f3e922e41231@mail.gmail.com","subject":"Re: [PATCH] cvsps/cvsimport: fix branch point calculation and broken branch imports","fromName":"Ping Yin","fromEmail":"pkufranky@gmail.com","sentAt":"2008-04-27T08:48:25Z","receivedAt":"2008-04-27T08:48:25Z","isPatch":true,"sender":{"key":"pkufranky@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5346?v=4"},"body":"On Sun, Apr 27, 2008 at 3:48 PM, Ping Yin wrote:\n> > >\n> > > THX. If i try out cvssuck, i will let you know.\n> > >\n> >\n> > Great, i succeed. And the result is exactly what i want!\n> >\n>\n> Not exactly, for my another conversion.\n>\n> $ git log --pretty=online --name-status x64_UI_071204\n> 724eb47 \\\n> This commit was manufactured by cvs2svn to create tag 'x64_UI_071204'.\n> 9362987 add support of writing cookies of 'fromid';\n> M logqueue.c\n> M ui.c\n> M ui.h\n>\n> Should we avoid recording the commit 724eb47 since it is the same with\n> commit 9362987 (no content change)？\n>\n\nI have found the related issue in\nhttp://cvs2svn.tigris.org/issues/show_bug.cgi?id=117 and the solution\ncontrib/git-move-tags.pl (in cvs2svn trunk, not released yet). Sorry\nfor the noise.\n\n\n-- \nPing Yin\n"}]}