threads / patch / 22674

patchgit svn slowness on win32

Subject: [patch] git svn slowness on win32

## tl;dr

3 messages between Feb 16, 2010 and Feb 17, 2010. Diffs are folded; open one to read it.

replies: 2people: 3as markdown or json

josh robb· Feb 16, 2010, 10:04 UTC · lore

Hi,

I'm using git svn (1.6.4.msysgit.0) within a WindowsXP VM running under VMWare fusion on OSX 10.5. For me (and at least one other person I know) git svn has been unbearably slow.

My profiling shows that almost all of the git-svn runtime is being spent in DynaLoader (via SVN::Base::import) which I guess is slow in a virtualized windows environment. For example:

$ time perl /libexec/git-core/git-svn rebase Current branch master is up to date.

real 2m56.750s user 0m3.129s sys 2m39.232s

I've been able to make significant (an order of magnitude) improvements to it's performance by delaying SVN::Base::import from running until it's actually needed. After making this change:

$ time perl /libexec/git-core/git-svn rebase Current branch master is up to date.

real 0m33.407s user 0m1.409s sys 0m23.054s

git svn rebase -n goes from 3m7.046s to 0m10.312s.

Would love to get some feedback/thoughts etc...

j.

--- git-svn.orig Wed Jul 29 22:55:26 2009 +++ git-svn Fri Dec 4 13:38:52 2009

Show changes to diff +9 −5

@@ -27,11 +27,13 @@
 $| = 1; # unbuffer STDOUT
 
 sub fatal (@) { print STDERR "@_\n"; exit 1 }
-require SVN::Core; # use()-ing this causes segfaults for me... *shrug*
-require SVN::Ra;
-require SVN::Delta;
-if ($SVN::Core::VERSION lt '1.1.0') {
-	fatal "Need SVN::Core 1.1.0 or better (got $SVN::Core::VERSION)";
+sub _req_svn {
+	require SVN::Core; # use()-ing this causes segfaults for me... *shrug*
+	require SVN::Ra;
+	require SVN::Delta;
+	if ($SVN::Core::VERSION lt '1.1.0') {
+		fatal "Need SVN::Core 1.1.0 or better (got $SVN::Core::VERSION)";
+	}
 }
 my $can_compress = eval { require Compress::Zlib; 1};
 push @Git::SVN::Ra::ISA, 'SVN::Ra';
@@ -4379,6 +4381,8 @@
 	my ($class, $url) = @_;
 	$url =~ s!/+$!!;
 	return $RA if ($RA && $RA->{url} eq $url);
+
+	::_req_svn();
 
 	SVN::_Core::svn_config_ensure($config_dir, undef);
 	my ($baton, $callbacks) = SVN::Core::auth_open_helper(_auth_providers);

Johannes Schindelin· Feb 16, 2010, 10:34 UTC · re: josh robb · lore

Re: [patch] git svn slowness on win32

Hi,

On Tue, 16 Feb 2010, josh robb wrote:

Show 29 quoted lines

> I'm using git svn (1.6.4.msysgit.0) within a WindowsXP VM running
> under VMWare fusion on OSX 10.5. For me (and at least one other person
> I know) git svn has been unbearably slow.
> 
> My profiling shows that almost all of the git-svn runtime is being
> spent in DynaLoader (via SVN::Base::import) which I guess is slow in a
> virtualized windows environment. For example:
> 
> $ time perl /libexec/git-core/git-svn rebase
> Current branch master is up to date.
> 
> real 2m56.750s
> user 0m3.129s
> sys 2m39.232s
> 
> I've been able to make significant (an order of magnitude)
> improvements to it's performance by delaying SVN::Base::import from
> running until it's actually needed. After making this change:
> 
> $ time perl /libexec/git-core/git-svn rebase
> Current branch master is up to date.
> 
> real 0m33.407s
> user 0m1.409s
> sys 0m23.054s
> 
> git svn rebase -n goes from 3m7.046s to 0m10.312s.
> 
> Would love to get some feedback/thoughts etc...

How about the following commit message (trying to follow http://repo.or.cz/w/git.git?a=blob;f=Documentation/SubmittingPatches;hb=HEAD):

-- snip -- git svn: delay importing SVN::Base until it is needed

Importing functions from a .dll into Git for Windows' perl is pretty slow, so let's avoid importing if it is not necessary.

[... timing statistics here...]

Signed-off-by: Josh Robb <josh_robb@fastmail.fm>
-- snap --

Hmm?

Ciao, Dscho

Eric Wong· Feb 17, 2010, 23:15 UTC · re: Johannes Schindelin · lore

Re: [patch] git svn slowness on win32

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

Show 47 quoted lines

> Hi,
> 
> On Tue, 16 Feb 2010, josh robb wrote:
> 
> > I'm using git svn (1.6.4.msysgit.0) within a WindowsXP VM running
> > under VMWare fusion on OSX 10.5. For me (and at least one other person
> > I know) git svn has been unbearably slow.
> > 
> > My profiling shows that almost all of the git-svn runtime is being
> > spent in DynaLoader (via SVN::Base::import) which I guess is slow in a
> > virtualized windows environment. For example:
> > 
> > $ time perl /libexec/git-core/git-svn rebase
> > Current branch master is up to date.
> > 
> > real 2m56.750s
> > user 0m3.129s
> > sys 2m39.232s
> > 
> > I've been able to make significant (an order of magnitude)
> > improvements to it's performance by delaying SVN::Base::import from
> > running until it's actually needed. After making this change:
> > 
> > $ time perl /libexec/git-core/git-svn rebase
> > Current branch master is up to date.
> > 
> > real 0m33.407s
> > user 0m1.409s
> > sys 0m23.054s
> > 
> > git svn rebase -n goes from 3m7.046s to 0m10.312s.
> > 
> > Would love to get some feedback/thoughts etc...
> 
> How about the following commit message (trying to follow 
> http://repo.or.cz/w/git.git?a=blob;f=Documentation/SubmittingPatches;hb=HEAD):
> 
> -- snip --
> git svn: delay importing SVN::Base until it is needed
> 
> Importing functions from a .dll into Git for Windows' perl is pretty slow,
> so let's avoid importing if it is not necessary.
> 
> [... timing statistics here...]
> 
> Signed-off-by: Josh Robb <josh_robb@fastmail.fm>
> -- snap --

Hi Josh,

Your patch was missing a call to ::req_svn() in cmd_branch(), but otherwise it appears fine and passes all tests.

Can you write a proper commit message + patch so we can apply it cleanly as Johannes suggested? Thanks!

(full diff below)

Show changes to git-svn.perl +10 −5

diff --git a/git-svn.perl b/git-svn.perl
index 473a0b9..224c29f 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -36,11 +36,13 @@ $ENV{TZ} = 'UTC';
 $| = 1; # unbuffer STDOUT
 
 sub fatal (@) { print STDERR "@_\n"; exit 1 }
-require SVN::Core; # use()-ing this causes segfaults for me... *shrug*
-require SVN::Ra;
-require SVN::Delta;
-if ($SVN::Core::VERSION lt '1.1.0') {
-	fatal "Need SVN::Core 1.1.0 or better (got $SVN::Core::VERSION)";
+sub _req_svn {
+	require SVN::Core; # use()-ing this causes segfaults for me... *shrug*
+	require SVN::Ra;
+	require SVN::Delta;
+	if ($SVN::Core::VERSION lt '1.1.0') {
+		fatal "Need SVN::Core 1.1.0 or better (got $SVN::Core::VERSION)";
+	}
 }
 my $can_compress = eval { require Compress::Zlib; 1};
 push @Git::SVN::Ra::ISA, 'SVN::Ra';
@@ -730,6 +732,7 @@ sub cmd_branch {
 		$src=~s/^http:/https:/;
 	}
 
+	::_req_svn();
 	my $ctx = SVN::Client->new(
 		auth    => Git::SVN::Ra::_auth_providers(),
 		log_msg => sub {
@@ -4859,6 +4862,8 @@ sub new {
 	$url =~ s!/+$!!;
 	return $RA if ($RA && $RA->{url} eq $url);
 
+	::_req_svn();
+
 	SVN::_Core::svn_config_ensure($config_dir, undef);
 	my ($baton, $callbacks) = SVN::Core::auth_open_helper(_auth_providers);
 	my $config = SVN::Core::config_get_config($config_dir);

-- 
Eric Wong