Re: Importing Mozilla CVS into git

From: Jon Smirl <jonsmirl@gmail.com>
Date: Jun 7, 2006, 00:40 UTC
Message-ID: <9e4733910606061740v797886baif6b1edd969dfab2a@mail.gmail.com>
In-Reply-To: <9e4733910606060813r41037467u74235f7a9386c1e0@mail.gmail.com>

On 6/6/06, Jon Smirl <jonsmirl@gmail.com> wrote:

Show 14 quoted lines

> On 6/6/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:
> > On 6/3/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > On 6/1/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> > > > With the attached patch you can parse the entire Mozilla tree. The
> > > > tree has over 100,000 files in it and about 300 branches.
> > >
> > > I was a little low with these counts, more like 110,000 files and some
> > > parts of the tree have 1,000 branches. Total tree size is 3GB.
> >
> > I don't think it really has that many branches. If I am to believe
> > cvsps (which took 3GB to walk the history), it has some branches with
> > recursive loops in their ancestry (MANG_MATH_BRANCH and
> > SpiderMonkey140_BRANCH have eachother as ancestors!?), 197969 commits
> > and 796 branches.

My full import to svn just finished after a day and a half. Here are the stats:

cvs2svn Statistics: ------------------ Total CVS Files: 99851 Total CVS Revisions: 948580 Total Unique Tags: 1505 Total Unique Branches: 1577 CVS Repos Size in KB: 2725843 Total SVN Commits: 205787 First Revision Date: Fri Mar 27 21:13:08 1998 Last Revision Date: Tue May 30 19:28:10 2006 ------------------ Timings: ------------------ pass 1: 3602 seconds pass 2: 227 seconds pass 3: 66 seconds pass 4: 1070 seconds pass 8:124650 seconds total: 124650 seconds [jonsmirl@jonsmirl ~]$

[jonsmirl@jonsmirl svn]$ du -h 4.0K ./svntest/dav 12K ./svntest/locks 40K ./svntest/hooks 16K ./svntest/conf 7.4G ./svntest/db/revs 808M ./svntest/db/revprops 4.0K ./svntest/db/transactions 8.2G ./svntest/db 8.2G ./svntest 8.2G .

[jonsmirl@jonsmirl svn]$ find | wc
 411607  411607 10891057

There are two directories that each contain about 205k files. 205K files in a single directory is causing svn problems on Ext3.

Bottom line, cvs2svn import tool works quite well. Highest memory consumption I saw was 100MB and it used 6GB of extra disk while running plus space need by svn.

I don't know quite enough about git yet to replace the svn commands it uses with git equivalents but if that were done I think most of the cvs import problems would be solved. Obviously the svn team has put a great deal of work into this program.

I don't think replacing the svn commands is very hard, I just haven't figured out the right way to build branches with low-level git yet and I don't know Python. I'll bet someone already familiar with git and cvs import could convert it in a couple of hours.

-- 
Jon Smirl
jonsmirl@gmail.com