threads / discuss / 19765

git-svn and lots of ssh connections

Subject: git-svn and *lots* of ssh connections

## tl;dr

3 messages between Jun 10, 2009 and Jun 25, 2009.

replies: 2people: 3as markdown or json

Jon Nelson· Jun 10, 2009, 14:09 UTC · lore

The company I work for (but do not speak for) uses svn (via ssh). I tried to use it; I really did. But I missed git and found svn quite honestly painful to use so I went to git-svn, and other than some caveats I'm pretty happy. However, one of the issues I've had with git-svn is fairly troublesome.

Basically, during the fetch stage (or clone) git-svn uses a *ton* of ssh connections. Many dozens and in even with smaller projects well over a hundred. By "small" I mean a .git of less than 6MB and less than 60 files in the checkout. I've got 4MBit bandwidth available but frequently see only a single digit fraction of that going to the fetch/clone process. Is there anything that can be done to reduce the number of ssh connections involved? Why can't a single connection simple be re-used? I can't use "ssh connection sharing" for a variety of reasons that aren't relevant here.

I'm using 1.6.0.2.

-- 
Jon

Pascal Obry· Jun 11, 2009, 07:52 UTC · re: Jon Nelson · lore

Re: git-svn and *lots* of ssh connections

Jon Nelson a écrit :

Show 9 quoted lines

> Basically, during the fetch stage (or clone) git-svn uses a *ton* of
> ssh connections. Many dozens and in even with smaller projects well
> over a hundred. By "small" I mean a .git of less than 6MB and less
> than 60 files in the checkout. I've got 4MBit bandwidth available but
> frequently see only a single digit fraction of that going to the
> fetch/clone process. Is there anything that can be done to reduce the
> number of ssh connections involved? Why can't a single connection
> simple be re-used? I can't use "ssh connection sharing" for a variety
> of reasons that aren't relevant here.

I don't know if git-svn can be improved but I would go with a reusable ssh connection (this is not working with Cygwin though).

See for example (but lot of web site talk about this): http://www.cyberciti.biz/faq/linux-unix-reuse-openssh-connection/

Pascal.

-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

Eric Wong· Jun 25, 2009, 09:49 UTC · re: Jon Nelson · lore

Re: git-svn and *lots* of ssh connections

Jon Nelson <jnelson@jamponi.net> wrote:

Show 17 quoted lines

> The company I work for (but do not speak for) uses svn (via ssh). I
> tried to use it; I really did. But I missed git and found svn quite
> honestly painful to use so I went to git-svn, and other than some
> caveats I'm pretty happy.  However, one of the issues I've had with
> git-svn is fairly troublesome.
> 
> Basically, during the fetch stage (or clone) git-svn uses a *ton* of
> ssh connections. Many dozens and in even with smaller projects well
> over a hundred. By "small" I mean a .git of less than 6MB and less
> than 60 files in the checkout. I've got 4MBit bandwidth available but
> frequently see only a single digit fraction of that going to the
> fetch/clone process. Is there anything that can be done to reduce the
> number of ssh connections involved? Why can't a single connection
> simple be re-used? I can't use "ssh connection sharing" for a variety
> of reasons that aren't relevant here.
> 
> I'm using 1.6.0.2.

Hi Jon,

The actual connection setup/teardown is handled by the SVN Perl bindings which gives git-svn little control over handling them. We actually try to force a zero reference count to close a connection to an open SVN repository.

Which OS/Perl version/SVN bindings version are you using?

Could the ssh processes just be zombies that didn't get reaped?

Does this happen when you clone a single directory with --no-follow-parent? (no --stdlayout/-s option or --branches/--tags switches, either). Can you test against an http/https/plain-svn server and see if it happens there?

I could be talked into putting in a (nasty!) hack to start zapping integer file descriptors when switching paths/repos and hope SVN closes those; but that may not work...

-- 
Eric Wong