threads / discuss / 24273

[GSoC update extra!] git-remote-svn: Week 9

Subject: [GSoC update extra!] git-remote-svn: Week 9

## tl;dr

3 messages between Jul 2, 2010 and Jul 3, 2010.

replies: 2people: 2as markdown or json

Ramkumar Ramachandra· Jul 2, 2010, 21:57 UTC · lore

Hi,

I figured that I haven't posted a recent update about the RA svnclient, so here it is. The good news is that using a combination of trial-and-error and twisted logic, I seem to have to fix the memory leaks and pool lifetime issues. I also managed to ouput the text-content-length header by writing the dump to a temporary file and stat'ing it. Unfortunately, the headers are still malformed, and this requires some more work to get right. The client runs successfully and manages to dump 5000 revisions of the ASF repository:

11.34s user 3.50s system 21% cpu 1:07.69 total

That's about 75 revisions per second, which isn't too bad I hope. This is over the network, and my internet connection isn't all that fast.

The bad news is that the client seems to signal SIGABRT when it finishes its work for some strange reason. Fortunately, I suspect that this is a trivial issue where I'm leaking some error or forgetting to clean up something. Hopefully, I'll be able to resolve this and fix the dump headers within the next few days. After that, I'll scrub the history clean and roll a series.

-- Ram

Tay Ray Chuan· Jul 3, 2010, 08:36 UTC · re: Ramkumar Ramachandra · lore

Re: [GSoC update extra!] git-remote-svn: Week 9

Hi,

On Sat, Jul 3, 2010 at 5:57 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:

Show 8 quoted lines

> [snip]
> The client runs successfully and
> manages to dump 5000 revisions of the ASF repository:
>
> 11.34s user 3.50s system 21% cpu 1:07.69 total
>
> That's about 75 revisions per second, which isn't too bad I hope. This
> is over the network, and my internet connection isn't all that fast.

I wonder if you could mirror the test SVN repository locally with svnsync so that we can a) spare their bandwidth b) minimize the effect of network bottlenecks on stats - I'm sure you could get more impressive stats that way.

Even better, having the SVN repo and the "exported" git repo on separate hard disks to minimize I/O as a bottleneck.

(Great work, by the way.)

-- 
Cheers,
Ray Chuan

Ramkumar Ramachandra· Jul 3, 2010, 19:47 UTC · re: Tay Ray Chuan · lore

Re: [GSoC update extra!] git-remote-svn: Week 9

Hi Tay,

Tay Ray Chuan writes:

Show 16 quoted lines

> Hi,
> 
> On Sat, Jul 3, 2010 at 5:57 AM, Ramkumar Ramachandra <artagnon@gmail.com> wrote:
> > [snip]
> > The client runs successfully and
> > manages to dump 5000 revisions of the ASF repository:
> >
> > 11.34s user 3.50s system 21% cpu 1:07.69 total
> >
> > That's about 75 revisions per second, which isn't too bad I hope. This
> > is over the network, and my internet connection isn't all that fast.
> 
> I wonder if you could mirror the test SVN repository locally with
> svnsync so that we can a) spare their bandwidth b) minimize the effect
> of network bottlenecks on stats - I'm sure you could get more
> impressive stats that way.

Excellent idea. Unfortunately, I don't have enough disk space to mirror the whole ASF repository on my laptop- I'll try it when I get back to my desktop (in 15~20 days).

In my prelimiary benchmark, I also found that I beat `svnsync` over the same network connection :)

> Even better, having the SVN repo and the "exported" git repo on
> separate hard disks to minimize I/O as a bottleneck.

Unfortunately, a lot of middleware components are missing and we cannot export it to a Git repository just yet- see my previous email about this [1]. Great idea though- will try it in future.

> (Great work, by the way.)

Thanks, and thanks for the feedback! :)

[1]: http://article.gmane.org/gmane.comp.version-control.git/150035

-- Ram