{"thread":{"id":"11390","subject":"cvsimport: trying to convert freebsd cvs to git","startedAt":"2007-12-22T17:18:01Z","lastAt":"2007-12-24T12:43:18Z","messageCount":9,"participants":["Miklos Vajna","Anand Kumria","David Soria Parra","Jeff King","Martin Langhoff"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"64005","messageId":"20071222171801.GE15286@genesis.frugalware.org","threadId":"11390","inReplyTo":null,"subject":"cvsimport: trying to convert freebsd cvs to git","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2007-12-22T17:18:01Z","receivedAt":"2007-12-22T17:18:01Z","isPatch":false,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"hi,\n\nrecently Stefan reported that he tried to convert the freebsd cvs' src\nmodule to git and he failed. i tried to help him, but i failed, too.\nhere are my efforts:\n\nhe made the cvs available on a (relatively) fast rsync mirror at:\n\nrsync://ftp.spline.de/FreeBSD-CVS\n\n(so you don't have to use cvsup if it's a problem for you)\n\nso after mirroring it, i had:\n\n$ ls cvs\nCVSROOT/ CVSROOT-ports/ CVSROOT-src/ ports/ src/\n\nthen i tried:\n\n$ time git cvsimport -d `pwd`/cvs -C src.git src\nInitialized empty Git repository in /home/vmiklos/git/freebsd/src.git/.git/\nmalformed revision\nfatal: refs/heads/origin: not a valid SHA1\nfatal: master: not a valid SHA1\nwarning: You appear to be on a branch yet to be born.\nwarning: Forcing checkout of HEAD.\nfatal: just how do you expect me to merge 0 trees?\ncheckout failed: 256\n\nreal 15m11.529s\nuser 0m46.212s\nsys 0m6.680s\n\nmy questions:\n\n1) does cvsimport supports the case when the source if on the local\nfilesystem, and not in not on a cvs server?\n\nfirst i wanted to make sure that cvsimport itself works properly here:\n\n$ git cvsimport -d :pserver:anonymous@tcpflow.cvs.sourceforge.net:/cvsroot/tcpflow -C tcpflow tcpflow\n\nand it converted this small repo fine\n\n2) if it supports, then i think the real error message is 'malformed\nrevision'. what is the proper way to see where is that revision?\n\nof course if cvsimport is not the right tool to incrementally convert\nsuch a big repo, then i would be interested in other advices, too.\n\nthanks,\n- VMiklos\n"},{"id":"64020","messageId":"pan.2007.12.23.01.47.29@progsoc.org","threadId":"11390","inReplyTo":"20071222171801.GE15286@genesis.frugalware.org","subject":"Re: cvsimport: trying to convert freebsd cvs to git","fromName":"Anand Kumria","fromEmail":"wildfire@progsoc.org","sentAt":"2007-12-23T01:47:35Z","receivedAt":"2007-12-23T01:47:35Z","isPatch":false,"sender":{"key":"wildfire@progsoc.org","avatar":null},"body":"On Sat, 22 Dec 2007 18:18:01 +0100, Miklos Vajna wrote:\n\n> hi,\n> \n> recently Stefan reported that he tried to convert the freebsd cvs' src\n> module to git and he failed. i tried to help him, but i failed, too.\n> here are my efforts:\n> \n> he made the cvs available on a (relatively) fast rsync mirror at:\n> \n> rsync://ftp.spline.de/FreeBSD-CVS\n> \n> (so you don't have to use cvsup if it's a problem for you)\n> \n> so after mirroring it, i had:\n> \n> $ ls cvs\n> CVSROOT/ CVSROOT-ports/ CVSROOT-src/ ports/ src/\n> \n\nA few days prior to your email, someone else asked a similar question:\n\nSome tools you can use:\n\ngit-cvsimport (via cvsps)\nparsecvs\nfromcvs\ncvs2svn (development branch)\n\nThere is a more detailed listing on the wiki at\n\n\nIf you have access to the raw \",v\" files as you do, I would suggest you \ntry out parsecvs. It will not do incremental imports but will you a good \nidea of how you can expect the converted repository to behave / act.\n\nAnand\n"},{"id":"64035","messageId":"fklivc$cio$1@ger.gmane.org","threadId":"11390","inReplyTo":"pan.2007.12.23.01.47.29@progsoc.org","subject":"Re: cvsimport: trying to convert freebsd cvs to git","fromName":"David Soria Parra","fromEmail":"sn_@gmx.net","sentAt":"2007-12-23T12:06:05Z","receivedAt":"2007-12-23T12:06:05Z","isPatch":false,"sender":{"key":"sn_@gmx.net","avatar":"https://gravatar.com/avatar/b1075ecdd33ea094cbc23798fe8b95c73ec1ccf7bb213ac8260c719e2dd97b55?d=mp&s=160"},"body":"\n> If you have access to the raw \",v\" files as you do, I would suggest you \n> try out parsecvs. It will not do incremental imports but will you a good \n> idea of how you can expect the converted repository to behave / act.\n\n\nParseCVS works out fine with the PHP repository for us while cvsimport\nhas some problems with branches. For me it looks like doing parsecvs on\nan existing repository tries to check all ,v files but in fact producing\na more or less incremental update.\n\nIf you try to use parsecvs make sure that your branchname doesnot\ncontain / or ~ as this will cause an error on parsecvs. If you need it,\nwe have a simple patch for that with a small workaround on that.\n"},{"id":"64049","messageId":"20071223172913.GA9297@sigill.intra.peff.net","threadId":"11390","inReplyTo":"20071222171801.GE15286@genesis.frugalware.org","subject":"Re: cvsimport: trying to convert freebsd cvs to git","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-12-23T17:29:13Z","receivedAt":"2007-12-23T17:29:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 22, 2007 at 06:18:01PM +0100, Miklos Vajna wrote:\n\n> then i tried:\n> \n> $ time git cvsimport -d `pwd`/cvs -C src.git src\n> Initialized empty Git repository in /home/vmiklos/git/freebsd/src.git/.git/\n> malformed revision\n\nThis is the actual error; everything else is git-cvsimport failing to\nnotice the problem and continuing anyway.\n\nI will take a look and see if it is simple to stop the process here and\nproduce a better error message.\n\n> 1) does cvsimport supports the case when the source if on the local\n> filesystem, and not in not on a cvs server?\n\nYes, it should work fine (it will still call cvsps and CVS, so it should\nlook the same to git-cvsimport).\n\n> 2) if it supports, then i think the real error message is 'malformed\n> revision'. what is the proper way to see where is that revision?\n\nTry git-cvsimport -v.\n\n-Peff\n"},{"id":"64058","messageId":"46a038f90712231313k3d17a30bwa910dd9679851c92@mail.gmail.com","threadId":"11390","inReplyTo":"20071223172913.GA9297@sigill.intra.peff.net","subject":"Re: cvsimport: trying to convert freebsd cvs to git","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2007-12-23T21:13:34Z","receivedAt":"2007-12-23T21:13:34Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Dec 24, 2007 6:29 AM, Jeff King wrote:\n> On Sat, Dec 22, 2007 at 06:18:01PM +0100, Miklos Vajna wrote:\n>\n> > then i tried:\n> >\n> > $ time git cvsimport -d `pwd`/cvs -C src.git src\n> > Initialized empty Git repository in /home/vmiklos/git/freebsd/src.git/.git/\n> > malformed revision\n>\n> This is the actual error; everything else is git-cvsimport failing to\n> notice the problem and continuing anyway.\n>\n> I will take a look and see if it is simple to stop the process here and\n> produce a better error message.\n\nIndeed. Running verbose will name the branch name that triggers that\nerror. One thing I find useful is to capture the commandline passed to\ncvsps (using ps), stopping cvsimport+cvsps, and running cvsps by hand\nwith the same commandline (remember to set TZ=UTC in the\nenvironment!), saving STDOUT to a file.\n\nOnce you have the file, you can feed it to cvsimport with the -P\nparameter (IIRC). Usually cvsimport errors are all about wonky cvsps\noutput.\n\nFor the record, I normally do my initial imports with Keith's\nparsecvs, and then run incrementals using git-cvsimport. These\nincrementals aren't 100% perfect, so I routinely check for 'drift'.\n\ncheers,\n\n\nm\n"},{"id":"64061","messageId":"20071224030551.GA12495@sigill.intra.peff.net","threadId":"11390","inReplyTo":"20071222171801.GE15286@genesis.frugalware.org","subject":"Re: cvsimport: trying to convert freebsd cvs to git","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-12-24T03:05:51Z","receivedAt":"2007-12-24T03:05:51Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Dec 22, 2007 at 06:18:01PM +0100, Miklos Vajna wrote:\n\n> $ time git cvsimport -d `pwd`/cvs -C src.git src\n> Initialized empty Git repository in /home/vmiklos/git/freebsd/src.git/.git/\n> malformed revision\n> fatal: refs/heads/origin: not a valid SHA1\n> fatal: master: not a valid SHA1\n> warning: You appear to be on a branch yet to be born.\n> warning: Forcing checkout of HEAD.\n> fatal: just how do you expect me to merge 0 trees?\n> checkout failed: 256\n\nI looked into this a bit more. The error is definitely coming from\ncvsps. I can't even get a reasonable cvsps output for this repo\n(presumably because of behind-the-scenes hackery on the RCS files).\nE.g., try making a module with _just_\n\"src/release/Attic/MIRROR.SITES,v\"; I can't even get a reasonable cvsps\noutput on that.\n\nYou might have better results with one of the other importers that have\nbeen suggested, most of which are more robust in the face of CVS\noddities.\n\nThat being said, it's awful that git-cvsimport doesn't stop when cvsps\nexits with an error, producing all of those meaningless errors. Patch\nwill follow.\n\n-Peff\n"},{"id":"64062","messageId":"20071224030819.GA15485@sigill.intra.peff.net","threadId":"11390","inReplyTo":"20071224030551.GA12495@sigill.intra.peff.net","subject":"[PATCH] cvsimport: die on cvsps errors","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2007-12-24T03:08:19Z","receivedAt":"2007-12-24T03:08:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"We were not previously checking the exit status of cvsps at\nall. If it exited before producing any useful output, we\nended up with an empty import, which caused a spew of\nconfusing error messages from other parts of git:\n\n$ git-cvsimport foo\nInitialized empty Git repository in ...\nsome error from cvsps\nfatal: refs/heads/origin: not a valid SHA1\nfatal: master: not a valid SHA1\nwarning: You appear to be on a branch yet to be born.\nwarning: Forcing checkout of HEAD.\nfatal: just how do you expect me to merge 0 trees?\ncheckout failed: 256\n\nNow we get:\n\n$ git-cvsimport foo\nInitialized empty Git repository in ...\nsome error from cvsps\ngit-cvsimport: fatal: cvsps reported error\n\nSigned-off-by: Jeff King \n---\nOn Sun, Dec 23, 2007 at 10:05:51PM -0500, Jeff King wrote:\n\n> That being said, it's awful that git-cvsimport doesn't stop when cvsps\n> exits with an error, producing all of those meaningless errors. Patch\n> will follow.\n\nAnd here it is.\n\n git-cvsimport.perl | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/git-cvsimport.perl b/git-cvsimport.perl\nindex 1fa9a22..6d8ff93 100755\n--- a/git-cvsimport.perl\n+++ b/git-cvsimport.perl\n@@ -632,6 +632,7 @@ unless ($opt_P) {\n \t print $cvspsfh $_;\n \t}\n \tclose CVSPS;\n+\t$? == 0 or die \"git-cvsimport: fatal: cvsps reported error\\n\";\n \tclose $cvspsfh;\n } else {\n \t$cvspsfile = $opt_P;\n-- \n1.5.4.rc1.1119.g1e6bc-dirty\n"},{"id":"64063","messageId":"46a038f90712231921m396479c8k9ecb4ed4c01659f7@mail.gmail.com","threadId":"11390","inReplyTo":"20071224030819.GA15485@sigill.intra.peff.net","subject":"Re: [PATCH] cvsimport: die on cvsps errors","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2007-12-24T03:21:14Z","receivedAt":"2007-12-24T03:21:14Z","isPatch":true,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"> We were not previously checking the exit status of cvsps\n> at all.\n\nAcked. We should have done this ages ago. Thanks!\n\nm\n"},{"id":"64073","messageId":"20071224124318.GB23659@genesis.frugalware.org","threadId":"11390","inReplyTo":"20071224030819.GA15485@sigill.intra.peff.net","subject":"Re: [PATCH] cvsimport: die on cvsps errors","fromName":"Miklos Vajna","fromEmail":"vmiklos@frugalware.org","sentAt":"2007-12-24T12:43:18Z","receivedAt":"2007-12-24T12:43:18Z","isPatch":true,"sender":{"key":"vmiklos@frugalware.org","avatar":"https://gravatar.com/avatar/401c1cbbb3a5d13e650c691a2c71d6fd0b80df1a01bc74d9f1972675dd58f2bd?d=mp&s=160"},"body":"On Sun, Dec 23, 2007 at 10:08:19PM -0500, Jeff King wrote:\n> > That being said, it's awful that git-cvsimport doesn't stop when cvsps\n> > exits with an error, producing all of those meaningless errors. Patch\n> > will follow.\n> \n> And here it is.\n\ngreat, thanks :)\n\n- VMiklos\n"}]}