Re: Excruciatingly slow git-svn imports
- From
- Geert Bosch <bosch@adacore.com>
- Date
- May 5, 2008, 04:29 UTC
- Message-ID
- <2B9E6C04-69F1-42BD-AE60-AFCE401E093E@adacore.com>
- In-Reply-To
- <20080429071122.GG24171@muzzle>
On Apr 29, 2008, at 03:11, Eric Wong wrote:
Show 16 quoted lines
>> I've found that git-svn gets slower as it runs. Try interrupting the >> clone and running "git svn fetch" -- it should pick up where it left >> off and will be MUCH faster if my experience is any indication. >> When I >> clone the big svn repository at work I usually restart it every 1000 >> revisions or so and it finishes in a fraction of the time it takes if >> I let it do everything in a single run. > > That's really strange to hear... The git-svn process itself does not > store much state other than the current revision and the log > information > for the next 100 or so revisions it needs to import. > > Are you packing the repository? Which SVN protocol are you using? > Does > memory usage of git-svn stay stable throughout the run?
I found the same. After about 5 days (with maybe 10 break/restarts), I had a converted repository with all 135K commits and a total size of just under 1 GB. The last 100K commits took (much?) less than a day, almost all the time was spend in the earlier ones. These commits seemed all to have thousands of files, even though most were probably the same. I'm sure this repositor, which covers 15 years of development of a multi-million line project, has a lot of tags and it seemed that it just had to chew through many copies of the complete set of files to find out that they're all the same.
It's great git-svn can be restarted so well and doesn't get confused by uncleanly terminated runs. My final repository is fast and small. I'm still struggling with how to properly synchronize branches, but that probably is mostly a matter of user education.
Thanks all for these great tools.
-Geert