threads / discuss / 7796

speeding up git-svn when directories are copied?

Subject: speeding up git-svn when directories are copied?

## tl;dr

4 messages between Apr 23, 2007 and Apr 26, 2007.

replies: 3people: 2as markdown or json

Karl Hasselström· Apr 23, 2007, 14:16 UTC · lore

When importing a whole repository, git-svn currently takes a _lot_ of time. Almost all of it seems to be spent getting the full text of each and every file when a tag or new branch is created.

If I remember correctly (but it was quite a while since I looked at it), git-svnimport didn't ask the Subversion server for all the files; it just noticed when the server said that a new directory was a copy of an old directory, which makes them just as fast as any other commit that doesn't change a gazillion files. (I've had an import running for a few hours now, and have checked on it about a dozen times; every single time, it was importing a new tag or branch. This suggests that making these go as fast as regular commits would speed up the import by more than a factor 10.)

Have you looked at doing this with git-svn? If it's not on your to-do list, I might take a stab at it at some point. But I only speak the copy-n-paste dialect of perl, so I'll need to find a largish block of uninterrupted time first. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Eric Wong· Apr 24, 2007, 01:54 UTC · re: Karl Hasselström · lore

Re: speeding up git-svn when directories are copied?

Karl Hasselström <kha@treskal.com> wrote:

> When importing a whole repository, git-svn currently takes a _lot_ of
> time. Almost all of it seems to be spent getting the full text of each
> and every file when a tag or new branch is created.

Try SVN 1.4.3 with my patch[1] to get do_switch() working. trunk should work without the patch.

[1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml

do_switch() is broken otherwise, and do_update() is extremely inefficient.

-- 
Eric Wong

Karl Hasselström· Apr 24, 2007, 10:01 UTC · re: Eric Wong · lore

Re: speeding up git-svn when directories are copied?

On 2007-04-23 18:54:05 -0700, Eric Wong wrote:

Show 13 quoted lines

> Karl Hasselström <kha@treskal.com> wrote:
>
> > When importing a whole repository, git-svn currently takes a _lot_
> > of time. Almost all of it seems to be spent getting the full text
> > of each and every file when a tag or new branch is created.
>
> Try SVN 1.4.3 with my patch[1] to get do_switch() working. trunk
> should work without the patch.
>
> [1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml
>
> do_switch() is broken otherwise, and do_update() is extremely
> inefficient.

Thanks for the help. I applied your patch to a clean 1.4.3 tree, and built subversion itself plus the perl bindings. But git-svn is still slow, and the testcase you gave in that thread still fails.

(I'm pretty sure that I'm actually using the bindings I've built; I install them to a nonstandard location, so if I don't set PERL5LIB they aren't found at all, and perl complains. When I do set it to my patched bindings, perl accepts it but your testcase fails.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Eric Wong· Apr 26, 2007, 15:45 UTC · re: Karl Hasselström · lore

Re: speeding up git-svn when directories are copied?

Karl Hasselström <kha@treskal.com> wrote:

Show 24 quoted lines

> On 2007-04-23 18:54:05 -0700, Eric Wong wrote:
> 
> > Karl Hasselström <kha@treskal.com> wrote:
> >
> > > When importing a whole repository, git-svn currently takes a _lot_
> > > of time. Almost all of it seems to be spent getting the full text
> > > of each and every file when a tag or new branch is created.
> >
> > Try SVN 1.4.3 with my patch[1] to get do_switch() working. trunk
> > should work without the patch.
> >
> > [1] - http://svn.haxx.se/dev/archive-2007-01/0936.shtml
> >
> > do_switch() is broken otherwise, and do_update() is extremely
> > inefficient.
> 
> Thanks for the help. I applied your patch to a clean 1.4.3 tree, and
> built subversion itself plus the perl bindings. But git-svn is still
> slow, and the testcase you gave in that thread still fails.
> 
> (I'm pretty sure that I'm actually using the bindings I've built; I
> install them to a nonstandard location, so if I don't set PERL5LIB
> they aren't found at all, and perl complains. When I do set it to my
> patched bindings, perl accepts it but your testcase fails.)

Hmm... I've been using 1.4.3 + patch since January on at least one of my machines and do_switch() has worked correctly every time.

git-svn will display a message telling you that it's using do_switch() when following a branch. Otherwise, it'll say it's using do_update().

-- 
Eric Wong