threads / discuss / 8922

git cvsimport branches not consistent with CVS branches

Subject: git cvsimport branches not consistent with CVS branches

## tl;dr

8 messages between Jul 8, 2007 and Jul 9, 2007.

replies: 7people: 6as markdown or json

Gordon Heydon· Jul 8, 2007, 00:45 UTC · lore
Hi,

After some investigation I found that git cvsimport was not importing branches 100% correctly with CVS.

Git and CVS do branching very differently in that CVS this is done at the file level (like everything else) and git does it repository wide.

So if I have a CVS repository with the files a, b and c and I branch b with a `cvs tab -b BRANCH test` on the branch test I will just have the file b.

If I do a git cvsimport on the test branch there will actually be the files a, b and c

What I think needs to happen is that when git cvsimport created the branch in the git repository it needs to delete all files from the branch that were not branched.

Thanks in advance. Gordon.

Brian Downing· Jul 8, 2007, 05:45 UTC · re: Gordon Heydon · lore

Re: git cvsimport branches not consistent with CVS branches

On Sun, Jul 08, 2007 at 10:45:10AM +1000, Gordon Heydon wrote:
Show 16 quoted lines
> After some investigation I found that git cvsimport was not importing 
> branches 100% correctly with CVS.
> 
> Git and CVS do branching very differently in that CVS this is done at 
> the file level (like everything else) and git does it repository wide.
> 
> So if I have a CVS repository with the files a, b and c and I branch b 
> with a `cvs tab -b BRANCH test` on the branch test I will just have the 
> file b.
> 
> If I do a git cvsimport on the test branch there will actually be the 
> files a, b and c
> 
> What I think needs to happen is that when git cvsimport created the 
> branch in the git repository it needs to delete all files from the 
> branch that were not branched.

I've been vaguely working on Yet Another CVS Importer (an incremental one; both git-cvsimport (thanks to cvsps) and tailor take about ten minutes and a gigabyte of RAM to figure out that nothing has to happen with my repository. I think I can do better than that).

In thinking about this case, I think I've decided that you want an option on what to do here. For some repositories you're not going to care about having extra files with the tag, and would greatly prefer that to having to create a branch for each and every tag (assuming you can arrange to have the correct files present otherwise; this isn't always possible.)

For other cases, you really only want to get the subset of the files that are tagged. For this, I think the best arrangement would be to make your branch, then make a commit that only deletes the files that are not present in the CVS branch, as you said. Then immediately make an empty commit to the mainline (wherever you branched from) merging from the deletion commit. Then proceed to commit normally to the branch. This way, if a user chooses to merge from the branch in Git, it won't try to delete a bunch of files in the target branch. (I'm having a hard time coming up with a situation where you'd want that behavior assuming the missing files were never tagged in CVS. A sane usage pattern would be to tag the whole repository then "cvs rm" the files you don't want otherwise...)

A fun third case is when a file in CVS doesn't have a branch tag, but then you go and "cvs add" a file with the same name in a branch. What CVS does then is to place the branch tag off the current HEAD. I /think/ you can detect this situation by noticing that the date of the HEAD revision and the date of the "file was created on branch ..." revision are exactly the same. You really need to detect this case, because when this happens there is no real parent relationship between the branch and the branch point. Since CVS has that "wait for a second to tick" delay at the end this might even be safe. Sigh.

Don't get me started about the abomination that is vendor branches.
I /hate/ CVS.
-bcd
Robin Rosenberg· Jul 8, 2007, 10:53 UTC · re: Brian Downing · lore

Re: git cvsimport branches not consistent with CVS branches

söndag 08 juli 2007 skrev Brian Downing:
Show 22 quoted lines
> On Sun, Jul 08, 2007 at 10:45:10AM +1000, Gordon Heydon wrote:
> > After some investigation I found that git cvsimport was not importing 
> > branches 100% correctly with CVS.
> > 
> > Git and CVS do branching very differently in that CVS this is done at 
> > the file level (like everything else) and git does it repository wide.
> > 
> > So if I have a CVS repository with the files a, b and c and I branch b 
> > with a `cvs tab -b BRANCH test` on the branch test I will just have the 
> > file b.
> > 
> > If I do a git cvsimport on the test branch there will actually be the 
> > files a, b and c
> > 
> > What I think needs to happen is that when git cvsimport created the 
> > branch in the git repository it needs to delete all files from the 
> > branch that were not branched.
> 
> I've been vaguely working on Yet Another CVS Importer (an incremental
> one; both git-cvsimport (thanks to cvsps) and tailor take about ten
> minutes and a gigabyte of RAM to figure out that nothing has to happen
> with my repository.  I think I can do better than that).

Corecode's fromcvs is pretty fast and incremental and AFAIK accurate. I had plenty problems with cvsimport, but fromcvs keeps in sync with the CVS repo. Get it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .

It does not convert regular tags, only branches, however so there is something to do for those that want a complete cvs import.

Show 11 quoted lines
> In thinking about this case, I think I've decided that you want an
> option on what to do here.  For some repositories you're not going to
> care about having extra files with the tag, and would greatly prefer
> that to having to create a branch for each and every tag (assuming you
> can arrange to have the correct files present otherwise; this isn't
> always possible.)
> 
> For other cases, you really only want to get the subset of the files
> that are tagged.  For this, I think the best arrangement would be to
> make your branch, then make a commit that only deletes the files that
> are not present in the CVS branch, as you said.  Then immediately make

fromcvs drops the files that do not have the branch tag. It is pretty simple to change the behaviour ( http://rosenberg.homelinux.net/gitweb/gitweb.cgi?p=FROMCVS.git;a=commitdiff;h=eb1c159bcc0d79eab182da3d7040ac62b52fd297 ) although this should be a switch and not hard coded.

-- robin
Johannes Schindelin· Jul 8, 2007, 11:47 UTC · re: Robin Rosenberg · lore

Re: git cvsimport branches not consistent with CVS branches

Hi,
On Sun, 8 Jul 2007, Robin Rosenberg wrote:
> Corecode's fromcvs is pretty fast and incremental and AFAIK accurate.

The only problem is that it is a misnomer: it is not fromcvs, but fromrcs, since you have to have access at the _files_. This is not always possible.

Ciao, Dscho

Robin Rosenberg· Jul 8, 2007, 12:22 UTC · re: Johannes Schindelin · lore

Re: git cvsimport branches not consistent with CVS branches

söndag 08 juli 2007 skrev Johannes Schindelin:
Show 9 quoted lines
> Hi,
> 
> On Sun, 8 Jul 2007, Robin Rosenberg wrote:
> 
> > Corecode's fromcvs is pretty fast and incremental and AFAIK accurate.
> 
> The only problem is that it is a misnomer: it is not fromcvs, but fromrcs, 
> since you have to have access at the _files_. This is not always 
> possible.

A small problem considering that fromcvs works and the other don't. My guess is that fromcvs kan be "fixed" to do remote access much more easily than cvsimport can be "fixed" to import an archive properly.

-- robin
Brian Downing· Jul 8, 2007, 21:02 UTC · re: Johannes Schindelin · lore

Re: git cvsimport branches not consistent with CVS branches

On Sun, Jul 08, 2007 at 12:47:16PM +0100, Johannes Schindelin wrote:
> The only problem is that it is a misnomer: it is not fromcvs, but
> fromrcs, since you have to have access at the _files_. This is not
> always possible.

For the problem space I'm interested in (a /fast/ incremental CVS importer) I think access to the RCS files is a must. You can use inode signatures to know which files you have to look at at all since the last run, and you can abort your parse at the end of the delta section if no new revisions appeared (rather than continuing to parse the deltatext section, the largest part of the file).

For the repository I'm interested in (work), running a "real" cvs update takes two or three minutes on a LAN. Running a full rlog takes about 10 or so. With the tricks above, I can import a single file change in seconds instead. (That includes the time to rsync the CVS repository to my local mirror.)

Granted, though, not everyone can get access to the RCS files.

(The name's not really a misnomer though. There's tons of CVS-specific crap you have to deal with in the RCS file to successfully import it -- magic branch revision symbols, horrible vendor branch behavior, etc.)

I will try fromcvs; I hadn't looked at it, since from the documentation it looked like it wasn't incremental.

-bcd
Steffen Prohaska· Jul 8, 2007, 14:38 UTC · re: Robin Rosenberg · lore

Re: git cvsimport branches not consistent with CVS branches

On Jul 8, 2007, at 12:53 PM, Robin Rosenberg wrote:
Show 9 quoted lines
> Corecode's fromcvs is pretty fast and incremental and AFAIK  
> accurate. I had
> plenty problems with cvsimport, but fromcvs keeps in sync with the  
> CVS repo.
> Get  it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .
>
> It does not convert regular tags, only branches, however so there  
> is something to
> do for those that want a complete cvs import.
Did anyone compare
    * git-cvsimport with cvsps patches from [1]
    * parsecvs [2]
    * fromcvs
and can give a recommendation?

My experience with plain git-cvsimport (without cvsps patches from [1]) is that it has a lot of problems. I'd recommend not to use it for incremental import and be very suspicious about the git repository created by git-cvsimport. You need to carefully validate the repository. It's likely that you need to fix imported branches. The trunk seems to be ok.

	Steffen

[1] http://ydirson.free.fr/en/software/scm/cvsps.html [2] http://anongit.freedesktop.org/git/users/keithp/repos/parsecvs.git/

Josh Triplett· Jul 9, 2007, 04:31 UTC · re: Steffen Prohaska · lore

Re: git cvsimport branches not consistent with CVS branches

Steffen Prohaska wrote:
Show 23 quoted lines
> On Jul 8, 2007, at 12:53 PM, Robin Rosenberg wrote:
>> Corecode's fromcvs is pretty fast and incremental and AFAIK  
>> accurate. I had
>> plenty problems with cvsimport, but fromcvs keeps in sync with the  
>> CVS repo.
>> Get  it at http://ww2.fs.ei.tum.de/~corecode/hg/fromcvs/ .
>>
>> It does not convert regular tags, only branches, however so there  
>> is something to
>> do for those that want a complete cvs import.
> 
> Did anyone compare
>     * git-cvsimport with cvsps patches from [1]
>     * parsecvs [2]
>     * fromcvs
> and can give a recommendation?
> 
> My experience with plain git-cvsimport (without cvsps patches from [1])
> is that it has a lot of problems. I'd recommend not to use it for
> incremental import and be very suspicious about the git repository
> created by git-cvsimport. You need to carefully validate the repository.
> It's likely that you need to fix imported branches. The trunk seems to
> be ok.

I've used both parsecvs and git-cvsimport, and I can definitely say from experience that cvsimport badly munges history, and parsecvs seems to handle it correctly. parsecvs does have the limitations that it requires the ,v files and doesn't work incrementally, making it great for project repository conversions and painful for just wanting to use git on a CVS-(mis)managed project.

I've never used fromcvs.

One idea to mitigate one of the limitations: how about modifying the CVS server to add an extension that supports downloading full ,v files? CVS has had server extensions in the past, such as for ls/rls. Once support for this extension makes its way out to most CVS servers, this problem goes away.

- Josh Triplett

← back to recent threads