Re: Importing Mozilla CVS into git

From: Martin Langhoff <martin.langhoff@gmail.com>
Date: Jun 6, 2006, 19:57 UTC
Message-ID: <46a038f90606061257v569aefackc4920a20f2970b0f@mail.gmail.com>
In-Reply-To: <9e4733910606060813r41037467u74235f7a9386c1e0@mail.gmail.com>

On 6/7/06, Jon Smirl <jonsmirl@gmail.com> wrote:

> Have you looked at the SVN CVS import tool? It imported Mozilla on the
> first try. If you download the source they have built about 40 test
> repositories with various errors. Those would make a good test suite
> for cvsps.  http://cvs2svn.tigris.org

Haven't yet, but I'll do. I'm currently quite busy at work, but I'm running these imports (Moz, Gentoo) and trying to address issues arising. Will look into SVN's tool it when I have a bit more time.

What I'll probably do is steal those test cases! ;-)

> I have been working on converting the svn tool to do git commands but
> my git knowledge is limited so it has been slow going. The last stage,
> pass 8, is very similar to what the git tools do. The svn commands
> just need to be swapped for git ones.

That would be interesting. And yet, I would have to evaluate how to transition gateways running git-cvsimport incrementally to a different importer.

Does it do incremental imports?

> If you get git-cvsimport working I'll use it instead. Will the cvsps
> process stay small enough to run on a 32b machine? The svn tools are

Currently not, but I do hope that the moz team has access to at least one machine with more than 32MB ;-)

With the current code, you will want a 3GB machine to run the git-cvsimport

git-cvsimport has a memory leak that I've been chasing for a while and I'll eventually fix, so it should fit in 32MB comfortably. cvsps is memory bound, and will probably take quite a bit of work to fix that. However, I suspect we can make it a lot more efficient.

> very RAM efficient since they use an external db. Can cvsps read from
> a local copy of the repository without using a CVS server?

> We are going to have to develop some kind of incremental mechanism for
> updating the new git tree. It can take up to two days to convert the
> repository, Mozilla development can't be shut down that long for a

You don't have to. Run an initial import, and then freeze development and run an incremental -- which will take an hour at the most. And then your mozilla.git repo is ready and up to date.

> transition. Git will also need to mirror the CVS repository (check-in
> still going to CVS) for a long time while we convince everyone on the
> merits of switching.

That's easy -- run git-cvsimport on a cronjob.

Show 5 quoted lines

> My imported svn version of Mozilla has a lot of performance problems.
> One of the directories has over 200,000 files in it slowing downing
> the filesystem. The repository went from 3GB CVS to 8GB svn, probably
> due to svn using 1000s of tiny files. I'll look around and see if svn
> has a pack feature like git.

At least they got a good importer ;-)

cheers,

martin