{"thread":{"id":"19765","subject":"git-svn and *lots* of ssh connections","startedAt":"2009-06-10T14:09:39Z","lastAt":"2009-06-25T09:49:20Z","messageCount":3,"participants":["Jon Nelson","Pascal Obry","Eric Wong"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"116010","messageId":"cccedfc60906100709r18364bc2h82d8e1a7ee0b8fd1@mail.gmail.com","threadId":"19765","inReplyTo":null,"subject":"git-svn and *lots* of ssh connections","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2009-06-10T14:09:39Z","receivedAt":"2009-06-10T14:09:39Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"The company I work for (but do not speak for) uses svn (via ssh). I\ntried to use it; I really did. But I missed git and found svn quite\nhonestly painful to use so I went to git-svn, and other than some\ncaveats I'm pretty happy. However, one of the issues I've had with\ngit-svn is fairly troublesome.\n\nBasically, during the fetch stage (or clone) git-svn uses a *ton* of\nssh connections. Many dozens and in even with smaller projects well\nover a hundred. By \"small\" I mean a .git of less than 6MB and less\nthan 60 files in the checkout. I've got 4MBit bandwidth available but\nfrequently see only a single digit fraction of that going to the\nfetch/clone process. Is there anything that can be done to reduce the\nnumber of ssh connections involved? Why can't a single connection\nsimple be re-used? I can't use \"ssh connection sharing\" for a variety\nof reasons that aren't relevant here.\n\nI'm using 1.6.0.2.\n\n-- \nJon\n"},{"id":"116061","messageId":"4A30B7A2.709@obry.net","threadId":"19765","inReplyTo":"cccedfc60906100709r18364bc2h82d8e1a7ee0b8fd1@mail.gmail.com","subject":"Re: git-svn and *lots* of ssh connections","fromName":"Pascal Obry","fromEmail":"pascal@obry.net","sentAt":"2009-06-11T07:52:02Z","receivedAt":"2009-06-11T07:52:02Z","isPatch":false,"sender":{"key":"pascal@obry.net","avatar":"https://avatars.githubusercontent.com/u/467069?v=4"},"body":"Jon Nelson a écrit :\n> Basically, during the fetch stage (or clone) git-svn uses a *ton* of\n> ssh connections. Many dozens and in even with smaller projects well\n> over a hundred. By \"small\" I mean a .git of less than 6MB and less\n> than 60 files in the checkout. I've got 4MBit bandwidth available but\n> frequently see only a single digit fraction of that going to the\n> fetch/clone process. Is there anything that can be done to reduce the\n> number of ssh connections involved? Why can't a single connection\n> simple be re-used? I can't use \"ssh connection sharing\" for a variety\n> of reasons that aren't relevant here.\n\nI don't know if git-svn can be improved but I would go with a reusable\nssh connection (this is not working with Cygwin though).\n\nSee for example (but lot of web site talk about this):\nhttp://www.cyberciti.biz/faq/linux-unix-reuse-openssh-connection/\n\nPascal.\n\n-- \n\n--|------------------------------------------------------\n--| Pascal Obry Team-Ada Member\n--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE\n--|------------------------------------------------------\n--| http://www.obry.net - http://v2p.fr.eu.org\n--| \"The best way to travel is by means of imagination\"\n--|\n--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B\n"},{"id":"116913","messageId":"20090625094919.GC2901@dcvr.yhbt.net","threadId":"19765","inReplyTo":"cccedfc60906100709r18364bc2h82d8e1a7ee0b8fd1@mail.gmail.com","subject":"Re: git-svn and *lots* of ssh connections","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2009-06-25T09:49:20Z","receivedAt":"2009-06-25T09:49:20Z","isPatch":false,"sender":{"key":"e@80x24.org","avatar":null},"body":"Jon Nelson wrote:\n> The company I work for (but do not speak for) uses svn (via ssh). I\n> tried to use it; I really did. But I missed git and found svn quite\n> honestly painful to use so I went to git-svn, and other than some\n> caveats I'm pretty happy. However, one of the issues I've had with\n> git-svn is fairly troublesome.\n> \n> Basically, during the fetch stage (or clone) git-svn uses a *ton* of\n> ssh connections. Many dozens and in even with smaller projects well\n> over a hundred. By \"small\" I mean a .git of less than 6MB and less\n> than 60 files in the checkout. I've got 4MBit bandwidth available but\n> frequently see only a single digit fraction of that going to the\n> fetch/clone process. Is there anything that can be done to reduce the\n> number of ssh connections involved? Why can't a single connection\n> simple be re-used? I can't use \"ssh connection sharing\" for a variety\n> of reasons that aren't relevant here.\n> \n> I'm using 1.6.0.2.\n\nHi Jon,\n\nThe actual connection setup/teardown is handled by the SVN Perl bindings\nwhich gives git-svn little control over handling them. We actually try\nto force a zero reference count to close a connection to an open SVN\nrepository.\n\nWhich OS/Perl version/SVN bindings version are you using?\n\nCould the ssh processes just be zombies that didn't get reaped?\n\nDoes this happen when you clone a single directory with\n--no-follow-parent? (no --stdlayout/-s option or --branches/--tags\nswitches, either). Can you test against an http/https/plain-svn\nserver and see if it happens there?\n\nI could be talked into putting in a (nasty!) hack to start zapping\ninteger file descriptors when switching paths/repos and hope SVN closes\nthose; but that may not work...\n\n-- \nEric Wong\n"}]}