threads / discuss / 22601

git-svn taking a long time

Subject: git-svn taking a long time

## tl;dr

5 messages between Feb 10, 2010 and Feb 10, 2010.

replies: 4people: 2as markdown or json

David Kågedal· Feb 10, 2010, 12:41 UTC · lore

I compiled the perl bindings from subversion 1.6.x, and git git version 1.7.0.rc0.52.g64ba.

I have imported a largish svn repo using "git svn fetch", which takes quite a long time (at least a day).

But my problems is that if I rerun "git svn fetch" again right after fetching, it will chew for hours before figuring out there are just a few new revisions to import. And the same if I try again.

I have another import from the same repo (on another machine, using slightly different versions of stuff) that doesn't have this problem.

So, does anyone have any suggestions for what could be done about this?

This is what the entry i .git/config looks like (url edited to protect the innocent)

[svn-remote "svn"]
        url = svn://svn.server
        branches = foo/branches/*:refs/remotes/svn/*
        tags = foo/tags/*:refs/remotes/svn/tags/*

-- 
David Kågedal

Andrew Myrick· Feb 10, 2010, 16:59 UTC · re: David Kågedal · lore

Re: git-svn taking a long time

Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.

Note 1: git did not support subversion merge tickets before git-1.6.6, so there would be no slowdown if you use prior versions.

Note 2: you shouldn't need to clone your subversion repository from scratch.  git will create the caches the first time you fetch after upgrading (so the first fetch will be slow), and it will continue to use and update the caches thereafter.

-Andrew

On Feb 10, 2010, at 4:41 AM, David Kågedal wrote:

Show 29 quoted lines

> I compiled the perl bindings from subversion 1.6.x, and git git version
> 1.7.0.rc0.52.g64ba.
> 
> I have imported a largish svn repo using "git svn fetch", which takes
> quite a long time (at least a day).
> 
> But my problems is that if I rerun "git svn fetch" again right after
> fetching, it will chew for hours before figuring out there are just a
> few new revisions to import. And the same if I try again.
> 
> I have another import from the same repo (on another machine, using
> slightly different versions of stuff) that doesn't have this problem.
> 
> So, does anyone have any suggestions for what could be done about this?
> 
> This is what the entry i .git/config looks like (url edited to protect
> the innocent)
> 
> [svn-remote "svn"]
>        url = svn://svn.server
>        branches = foo/branches/*:refs/remotes/svn/*
>        tags = foo/tags/*:refs/remotes/svn/tags/*
> 
> -- 
> David Kågedal
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

David Kågedal· Feb 10, 2010, 17:29 UTC · re: Andrew Myrick · lore

Re: git-svn taking a long time

Andrew Myrick <amyrick@apple.com> writes:

> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.

By "merge tickets", are you talking about the merge functionality that appeared in subversion 1.5? We don't use that.

But I had another idea. I pecularity of our subversion repo is that we no longer use the foo/trunk branch, but only foo/branches/*. But we did once upon a time have a foo/trunk. And since I didn't include a "fetch = foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch that information every time. For instance, the first revision is on trunk.

I'm rerunning the fetch now with the trunk added, so see if it helps.

And another note is that "git svn fetch --parent" was always quick.

-- 
David Kågedal

Andrew Myrick· Feb 10, 2010, 17:39 UTC · re: David Kågedal · lore

Re: git-svn taking a long time

On Feb 10, 2010, at 9:29 AM, David Kågedal wrote:

Show 6 quoted lines

> Andrew Myrick <amyrick@apple.com> writes:
> 
>> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  
> 
> By "merge tickets", are you talking about the merge functionality that
> appeared in subversion 1.5? We don't use that.

I do mean that.  If you don't use them, then I'm stumped.

Show 10 quoted lines

> But I had another idea. I pecularity of our subversion repo is that we
> no longer use the foo/trunk branch, but only foo/branches/*. But we did
> once upon a time have a foo/trunk. And since I didn't include a "fetch =
> foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch
> that information every time. For instance, the first revision is on
> trunk.
> 
> I'm rerunning the fetch now with the trunk added, so see if it helps.
> 
> And another note is that "git svn fetch --parent" was always quick.

Sounds reasonable.  Good luck!

-Andrew

David Kågedal· Feb 10, 2010, 22:25 UTC · re: David Kågedal · lore

Re: git-svn taking a long time

David Kågedal <davidk@lysator.liu.se> writes:

Show 15 quoted lines

> Andrew Myrick <amyrick@apple.com> writes:
>
>> Give 1.7.0-rc2 a try.  It includes commit 8bff7c5383ed833bd1df9c8d85c00a27af3e5b02, which attempts to persistently cache a lot of the processing that git-svn has to do on subversion's merge tickets, which has improved my fetch times significantly.  
>
> By "merge tickets", are you talking about the merge functionality that
> appeared in subversion 1.5? We don't use that.
>
> But I had another idea. I pecularity of our subversion repo is that we
> no longer use the foo/trunk branch, but only foo/branches/*. But we did
> once upon a time have a foo/trunk. And since I didn't include a "fetch =
> foo/trunk:refs/remotes/svn/trunk" in my config, it might need to refetch
> that information every time. For instance, the first revision is on
> trunk.
>
> I'm rerunning the fetch now with the trunk added, so see if it helps.

That seems to have done the trick. Now it's quick when there is nothing to do.

Perhaps there is something in the git-svn caching that could still be improved?

-- 
David Kågedal