cvsimport: trying to convert freebsd cvs to git

hi,

recently Stefan reported that he tried to convert the freebsd cvs' src module to git and he failed. i tried to help him, but i failed, too. here are my efforts:

he made the cvs available on a (relatively) fast rsync mirror at:

rsync://ftp.spline.de/FreeBSD-CVS

(so you don't have to use cvsup if it's a problem for you)

so after mirroring it, i had:

$ ls cvs CVSROOT/ CVSROOT-ports/ CVSROOT-src/ ports/ src/

then i tried:

$ time git cvsimport -d `pwd`/cvs -C src.git src Initialized empty Git repository in /home/vmiklos/git/freebsd/src.git/.git/ malformed revision fatal: refs/heads/origin: not a valid SHA1 fatal: master: not a valid SHA1 warning: You appear to be on a branch yet to be born. warning: Forcing checkout of HEAD. fatal: just how do you expect me to merge 0 trees? checkout failed: 256

real 15m11.529s user 0m46.212s sys 0m6.680s

my questions:

1) does cvsimport supports the case when the source if on the local
filesystem, and not in not on a cvs server?

first i wanted to make sure that cvsimport itself works properly here:

$ git cvsimport -d :pserver:anonymous@tcpflow.cvs.sourceforge.net:/cvsroot/tcpflow -C tcpflow tcpflow

and it converted this small repo fine

2) if it supports, then i think the real error message is 'malformed
revision'. what is the proper way to see where is that revision?

of course if cvsimport is not the right tool to incrementally convert such a big repo, then i would be interested in other advices, too.

thanks,
- VMiklos

On Sat, 22 Dec 2007 18:18:01 +0100, Miklos Vajna wrote:

A few days prior to your email, someone else asked a similar question:

Some tools you can use:

git-cvsimport (via cvsps) parsecvs fromcvs cvs2svn (development branch)

There is a more detailed listing on the wiki at <http://git.or.cz/gitwiki/ InterfacesFrontendsAndTools#head-8870e1c81cc93f9a7a7acb5e969924ee60182d6b>

If you have access to the raw ",v" files as you do, I would suggest you try out parsecvs. It will not do incremental imports but will you a good idea of how you can expect the converted repository to behave / act.

Anand

ParseCVS works out fine with the PHP repository for us while cvsimport has some problems with branches. For me it looks like doing parsecvs on an existing repository tries to check all ,v files but in fact producing a more or less incremental update.

If you try to use parsecvs make sure that your branchname doesnot contain / or ~ as this will cause an error on parsecvs. If you need it, we have a simple patch for that with a small workaround on that.

On Sat, Dec 22, 2007 at 06:18:01PM +0100, Miklos Vajna wrote:

This is the actual error; everything else is git-cvsimport failing to notice the problem and continuing anyway.

I will take a look and see if it is simple to stop the process here and produce a better error message.

Yes, it should work fine (it will still call cvsps and CVS, so it should look the same to git-cvsimport).

Try git-cvsimport -v.

-Peff

On Dec 24, 2007 6:29 AM, Jeff King <peff@peff.net> wrote:

Indeed. Running verbose will name the branch name that triggers that error. One thing I find useful is to capture the commandline passed to cvsps (using ps), stopping cvsimport+cvsps, and running cvsps by hand with the same commandline (remember to set TZ=UTC in the environment!), saving STDOUT to a file.

Once you have the file, you can feed it to cvsimport with the -P parameter (IIRC). Usually cvsimport errors are all about wonky cvsps output.

For the record, I normally do my initial imports with Keith's parsecvs, and then run incrementals using git-cvsimport. These incrementals aren't 100% perfect, so I routinely check for 'drift'.

cheers,

m

On Sat, Dec 22, 2007 at 06:18:01PM +0100, Miklos Vajna wrote:

I looked into this a bit more. The error is definitely coming from cvsps. I can't even get a reasonable cvsps output for this repo (presumably because of behind-the-scenes hackery on the RCS files). E.g., try making a module with _just_ "src/release/Attic/MIRROR.SITES,v"; I can't even get a reasonable cvsps output on that.

You might have better results with one of the other importers that have been suggested, most of which are more robust in the face of CVS oddities.

That being said, it's awful that git-cvsimport doesn't stop when cvsps exits with an error, producing all of those meaningless errors. Patch will follow.

-Peff

We were not previously checking the exit status of cvsps at all. If it exited before producing any useful output, we ended up with an empty import, which caused a spew of confusing error messages from other parts of git:

$ git-cvsimport foo Initialized empty Git repository in ... some error from cvsps fatal: refs/heads/origin: not a valid SHA1 fatal: master: not a valid SHA1 warning: You appear to be on a branch yet to be born. warning: Forcing checkout of HEAD. fatal: just how do you expect me to merge 0 trees? checkout failed: 256

Now we get:

$ git-cvsimport foo Initialized empty Git repository in ... some error from cvsps git-cvsimport: fatal: cvsps reported error

Signed-off-by: Jeff King <peff@peff.net>
---
On Sun, Dec 23, 2007 at 10:05:51PM -0500, Jeff King wrote:

And here it is.

 git-cvsimport.perl |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

Acked. We should have done this ages ago. Thanks!

m

On Sun, Dec 23, 2007 at 10:08:19PM -0500, Jeff King <peff@peff.net> wrote:

great, thanks :)

- VMiklos