Re: working with a large repository and git svn

From: Wesley J. Landaker <wjl@icecavern.net>
Date: Jan 12, 2011, 15:30 UTC
Message-ID: <201101120830.47016.wjl@icecavern.net>
In-Reply-To: <AANLkTimKbS3ECzOaGtNgvx7DThJGH_DkPmg4ehKXGtwc@mail.gmail.com>

On Tuesday, January 11, 2011 18:27:10 Joe Corneli wrote:

Show 15 quoted lines

> I am experiencing trouble with git svn, trying to import a
> large repository (7.9 gigs, ~54000 commits) from Git into
> SVN.
> 
> This has failed in a couple of different ways, depending
> on the operating environment.  With Git version 1.7.3.5
> running on Ubuntu 9.10, in the final step
> 
>   git svn dcommit --no-rebase
> 
> of the formula described below, I get:
> 
>  failing with "Can't fork at /usr/share/perl/5.10.0/Git.pm line 1261."
> 
> after committing just over 2000 revisions.

I haven't tried importing 8 GB from Git to Subversion, but I have used Git against existing huge Subversion repositories that are >= 10 GB with little trouble, other than that it takes forever because Subversion is slow.

Here are some thoughts on how I'd approach what you are doing. Realize that no matter what, it's still probably going to take "forever" (e.g. run it over the weekend).

  1) Sounds like git-svn is running out of resources on your machine -- 
that's probably a bug, but work around it: Don't dcommit all 20000 revisions 
at once. Maybe write a shell script that goes through and dcommits a 100 
commits at a time.

  2) Do you need the full history to be in SVN? Can you rebase/squash large 
parts together and thus need to commit less revisions in the first place?

  3) I love git-svn for working with Subversion repositories, but you could 
consider a different tool, like tailor, if you can't make git-svn do what 
you want. I have also heard talk (but I don't know the state of things) of 
people working on a fast-import tool for SVN, so you could git-fast-export 
and svn-fast-import in a big batch.

  4) Does 8 GB of data really belong in the same repository? Maybe it should 
really be split up and used with git submodules or SVN externals? That may 
make things easier to work with in the long term.

  5) Do you really want to be going from Git, to Subversion? That seems like 
a big step backwards. =)

In any case, good luck!