threads / discuss / 22473

Speed of conversion from Subversion repository to Git

Subject: Speed of conversion from Subversion repository to Git

## tl;dr

4 messages between Feb 1, 2010 and Feb 11, 2010.

replies: 3people: 2as markdown or json

Markus Elfring· Feb 1, 2010, 18:05 UTC · lore
Hello,

I am trying to convert the YaST Subversion repository into a Git variant on my system. Unfortunately, a command like "svn2git http://svn.opensuse.org/svn/yast/" does not finish this attempt after 1,5 days (> 715:13.24 CPU time) here. I would appreciate suggestions for improvements of the execution speed. (Renicing of the process "git-svn" does not help to increase its processor utilisation in noticeable ways.)

Regards, Markus

Jacob Helwig· Feb 1, 2010, 18:17 UTC · re: Markus Elfring · lore

Re: Speed of conversion from Subversion repository to Git

On Mon, Feb 1, 2010 at 10:05, Markus Elfring <Markus.Elfring@web.de> wrote:
Show 13 quoted lines
> Hello,
>
> I am trying to convert the YaST Subversion repository into a Git variant on my
> system. Unfortunately, a command like "svn2git
> http://svn.opensuse.org/svn/yast/" does not finish this attempt after 1,5 days
> (> 715:13.24 CPU time) here. I would appreciate suggestions for improvements of
> the execution speed.
> (Renicing of the process "git-svn" does not help to increase its processor
> utilisation in noticeable ways.)
>
> Regards,
> Markus
>

We're in the process of converting to Git at work, and I've found that svn-all-fast-export was the best option for us. git-svn's behavior of a branch's entire history every time it found a new branch (even though it has already seen 3/4 of the revisions involved) caused it to be far too slow for our needs. Admittedly, this was ~1 year ago that I tried git-svn to do a test conversion, and things might have improved since then.

svn-all-fast-export does require access to the raw repository, instead of just (http|svn|svn+ssh):// access, but it does allow you to split out one SVN "repository" into multiple Git repositories in one go.

-Jacob
Markus Elfring· Feb 2, 2010, 11:20 UTC · re: Markus Elfring · lore

Re: Speed of conversion from Subversion repository to Git

Hello,
The execution of the command "svn2git http://svn.opensuse.org/svn/yast/" was finished today somewhere during the night. I used the version 1.3.1 together with Git 1.6.6.1 on my openSUSE 11.2 system. Now I hope that my conversion try was really successful.
elfring@Sonne:~/Projekte/YaST/lokal> du -hs .git . && git branch
518M    .git
653M    .
  Moblin
  REFACTORING-10_3
  SuSE-Code-11-Branch
  SuSE-Code-11-SP1-Branch
  SuSE-Linux-10_0-Branch
  SuSE-Linux-10_1-Branch
  SuSE-Linux-10_1-Branch@30668
  SuSE-Linux-10_2-Branch
  SuSE-Linux-10_3-Branch
  SuSE-Linux-11_0-Branch
  SuSE-Linux-11_2-Branch
  SuSE-Linux-9_3-Branch
  SuSE-SLE-10-Branch
  SuSE-SLE-10-SP1-Branch
  SuSE-SLE-10-SP1-Features-Branch
  SuSE-SLE-10-SP2-Branch
  SuSE-SLE-10-SP3-Branch
  YaST2-X11-2-13-SuSE-10-1
  YaST2-X11-2-13-SuSE-10-1@29086
  YaST2-X11-2-14-SuSE-10-2
  YaST2-X11-2-14-SuSE-10-2@34577
* master
  moblin
  research
  selection_widgets_cleanup-core
  selection_widgets_cleanup-core@22138
  selection_widgets_cleanup-qt
  selection_widgets_cleanup-qt@22138
  tmp
  unsupported

The network activity was mostly low during this lengthy process. Are more software developers interested to speed this procedure up? Do know ways to improve the processor utilisation so that such a data import will be completed faster?

Regards, Markus

Markus Elfring· Feb 11, 2010, 17:10 UTC · re: Markus Elfring · lore

Re: Speed of conversion from Subversion repository to Git

Hello,

Are there any chances to increase the processor utilisation in noticeable ways by multi-threaded programming? http://git.kernel.org/?p=git/git.git;a=blob;f=git-svn.perl;h=265852f4596bfe5aeca12be06f78631320b8ebb4;hb=HEAD

Do any conversion tasks exist which would benefit from parallel processing?

Regards, Markus

← back to recent threads