{"thread":{"id":"3935","subject":"PATCH RESEND: git-svnimport memory leak plug","startedAt":"2006-04-21T02:12:12Z","lastAt":"2006-04-21T02:12:12Z","messageCount":1,"participants":["Auke Kok"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"18962","messageId":"19fdff02f87c86575bfb6a4016aea8a4@localhost","threadId":"3935","inReplyTo":null,"subject":"PATCH RESEND: git-svnimport memory leak plug","fromName":"Auke Kok","fromEmail":"sofar@foo-projects.org","sentAt":"2006-04-21T02:12:12Z","receivedAt":"2006-04-21T02:12:12Z","isPatch":false,"sender":{"key":"sofar@foo-projects.org","avatar":null},"body":"\nI was unable with todays git-svnimport to convert a modest 140mb svn repository to git using svn-git-import. the process would bomb out after 2000 revisions and I have 20000. It consumed over 1.5gb vm space on my 1gb machine, and died with a 'cannot fork error'. this also killed my project server today after I discivered it the wrong way.\n\nThe original patch send earlier by Santi Bejar fixes the problem. I include the patch again here so it can be merged. my mailer probably will nuke it so here's the link to the archive post:\n\nhttp://marc.theaimsgroup.com/?l=git&m=114345884526971&w=2\n\n\nPlease include this patch.\n\nCheers,\n\nAuke\n\n\n\nOn 3/24/06, Santi Béjar <sbejar@gmail.com> wrote:\n> Jan-Benedict Glaw <jbglaw@lug-owl.de> writes:\n>\n> > On Wed, 2006-03-22 14:33:37 +0100, Jan-Benedict Glaw <jbglaw@lug-owl.de> wrote:\n> >\n> > Since it seems nobody looked at the GCC import run (which means to use\n> > the svnimport), I ran it again, under strace control:\n> >\n> >> GCC\n> >> ~~~\n> >> $ /home/jbglaw/bin/git svnimport -C gcc -v svn://gcc.gnu.org/svn/gcc\n> >\n> >> Committed change 3936:/ 1993-03-31 05:44:03)\n> >> Commit ID ceff85145f8671fb2a9d826a761cedc2a507bd1e\n> >> Writing to refs/heads/origin\n> >> DONE: 3936 origin ceff85145f8671fb2a9d826a761cedc2a507bd1e\n> >> ... 3937 trunk/gcc/final.c ...\n> >> Can't fork at /home/jbglaw/bin/git-svnimport line 379.\n> >\n>\n> I have the same (?) problem with one of my svn repository. It worked\n> before (I've redone the import with the -r flag), so I bisected it.\n> The problematic commit seems to be:\n>\n> diff-tree 4802426... (from 525c0d7...)\n> Author: Karl  Hasselström <kha@treskal.com>\n> Date:   Sun Feb 26 06:11:27 2006 +0100\n>\n>     svnimport: Convert executable flag\n>\n>     Convert the svn:executable property to file mode 755 when converting\n>     an SVN repository to GIT.\n>\n>     Signed-off-by: Karl Hasselström <kha@treskal.com>\n>     Signed-off-by: Junio C Hamano <junkio@cox.net>\n>\n> :100755 100755 ee2940f... 6603b96... M  git-svnimport.perl\n>\n> I think it has a memory leak, it used up to 140m of memory.\n>\n> $ git reset --hard 4802426^\n> $ time ../git-svnimport.perl file:///path/\n> Use of uninitialized value in string eq at ../git-svnimport.perl line 463.\n> Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466.\n> real    0m55.801s\n> user    0m30.578s\n> sys     0m23.084s\n>\n> $ git reset --hard 4802426\n> $ time ../git-svnimport.perl file:///path/\n> Use of uninitialized value in string eq at ../git-svnimport.perl line 463.\n> Use of uninitialized value in substitution (s///) at ../git-svnimport.perl line 466.\n> Can't fork at /home/santi/usr/src/scm/git/git-svnimport.perl line 331.\n> real    6m2.163s\n> user    0m20.332s\n> sys     0m50.180s\n>\n> and it didn't finished. Hope it helps.\n\nAnd this patch fixes my problems.\n\n---\n\nIntroduced in 4802426.\n\nSigned-off-by: Santi Béjar <sbejar@gmail.com>\n---\n git-svnimport.perl |    4 +++-\n 1 file changed, 3 insertions(+), 1 deletion(-)\n\ndiff --git a/git-svnimport.perl b/git-svnimport.perl\nindex 639aa41..f2cf062 100755\n--- a/git-svnimport.perl\n+++ b/git-svnimport.perl\n@@ -135,8 +135,10 @@\n\n        print \"... $rev $path ...\\n\" if $opt_v;\n        my (undef, $properties);\n+       my $pool = SVN::Pool->new();\n        eval { (undef, $properties)\n-                  = $self->{'svn'}->get_file($path,$rev,$fh); };\n+                  = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };\n+       $pool->clear;\n        if($@) {\n                return undef if $@ =~ /Attempted to get checksum/;\n                die $@;\n"}]}