{"thread":{"id":"14204","subject":"git-svn messed up import, badly","startedAt":"2008-06-28T19:48:08Z","lastAt":"2008-06-29T11:54:17Z","messageCount":11,"participants":["Björn Steinbrink","Avery Pennarun","Mikael Magnusson","Eric Wong","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"81577","messageId":"20080628194808.GA29908@atjola.homenet","threadId":"14204","inReplyTo":null,"subject":"git-svn messed up import, badly","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-06-28T19:48:08Z","receivedAt":"2008-06-28T19:48:08Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"Hi,\n\nSamuel reported on #git that a git-svn import was failing for him.  I\ntried to reproduce that, and while it failed on a different svn revision\nfor me, it still failed pretty badly.\n\nThe repo is at:\n\tsvn://scm.gforge.inria.fr/svn/coq/\n\nI cloned with:\n\tgit svn clone -s svn://scm.gforge.inria.fr/svn/coq/\n\nAnd with my current git-svn import, I'm getting this error message:\n\nChecksum mismatch: trunk/.depend 16e748c219f9f95bf3d05c6b2af5444290bc8471\nexpected: 05fb5edb8c8057be006c7e913ae0c764\n     got: 763b9a426c5bd61e0a85252459d37cfa\n\nI got that when it bailed out from \"git svn clone\" and I get it on each\n\"git svn fetch\" run, trying to resume the import.\n\nLooking at the history of trunk/.depend and comparing the diffs that git\ngives me to those that \"svn diff\" gives me, I noticed that the diffs for\nchanges to .depend introduced in svn revision 2314 differ heavily\nbetween svn and my git-svn import.\n\nTarball of the repo and the two diffs is at:\nhttp://people.linux-vserver.org/~doener/broken-git-svn-import.tbz\n[4.8MB]\n\n$ git --version\ngit version 1.5.6.1.78.gde8d9\n\ngit svn --version\ngit-svn version 1.5.6.1.78.gde8d9 (svn 1.4.6)\n\nAvery and Adam on Cc:, because Avery reported a somewhat similar issue\nthat he bisected down to the git-svn speed-up patch by Adam.\n\nUnfortunately, I currently don't have enough time on my hands to perform\na bisection.\n\nIf you need anything else, let me know.\n\nThanks,\nBjörn\n"},{"id":"81581","messageId":"1214686673-28099-1-git-send-email-apenwarr@gmail.com","threadId":"14204","inReplyTo":"20080628194808.GA29908@atjola.homenet","subject":"[PATCH] Revert \"git-svn: Speed up fetch\"","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-06-28T20:57:53Z","receivedAt":"2008-06-28T20:57:53Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"This reverts commit ffe256f9bac8a40ff751a9341a5869d98f72c285, because it\nwas causing errors of the form:\n\nChecksum mismatch: trunk/.depend 16e748c219f9f95bf3d05c6b2af5444290bc8471\nexpected: 05fb5edb8c8057be006c7e913ae0c764\n     got: 763b9a426c5bd61e0a85252459d37cfa\n     \nNote that the exact failing file and checksum seems to vary if you clear\nthe repository and try again.\n\nConflicts:\n\n\tgit-svn.perl\n---\nObviously it would be better to actually fix the bug here than the revert\nthe patch (because the patch really *does* make fetch go a lot faster), but\nI don't know where to begin, and it's a pain to debug because of the\nvariability.\n\n git-svn.perl |   42 ++++++++++++++++++++++--------------------\n 1 files changed, 22 insertions(+), 20 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 4c9c59b..a02bcf4 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -4,7 +4,7 @@\n use warnings;\n use strict;\n use vars qw/\t$AUTHOR $VERSION\n-\t\t$sha1 $sha1_short $_revision $_repository\n+\t\t$sha1 $sha1_short $_revision\n \t\t$_q $_authors %users/;\n $AUTHOR = 'Eric Wong <normalperson@yhbt.net>';\n $VERSION = '@@GIT_VERSION@@';\n@@ -223,7 +223,6 @@ unless ($cmd && $cmd =~ /(?:clone|init|multi-init)$/) {\n \t\t}\n \t\t$ENV{GIT_DIR} = $git_dir;\n \t}\n-\t$_repository = Git->repository(Repository => $ENV{GIT_DIR});\n }\n \n my %opts = %{$cmd{$cmd}->[2]} if (defined $cmd);\n@@ -305,7 +304,6 @@ sub do_git_init_db {\n \t\t\t}\n \t\t}\n \t\tcommand_noisy(@init_db);\n-\t\t$_repository = Git->repository(Repository => \".git\");\n \t}\n \tmy $set;\n \tmy $pfx = \"svn-remote.$Git::SVN::default_repo_id\";\n@@ -322,7 +320,6 @@ sub init_subdir {\n \tmkpath([$repo_path]) unless -d $repo_path;\n \tchdir $repo_path or die \"Couldn't chdir to $repo_path: $!\\n\";\n \t$ENV{GIT_DIR} = '.git';\n-\t$_repository = Git->repository(Repository => $ENV{GIT_DIR});\n }\n \n sub cmd_clone {\n@@ -3040,7 +3037,6 @@ use vars qw/@ISA/;\n use strict;\n use warnings;\n use Carp qw/croak/;\n-use File::Temp qw/tempfile/;\n use IO::File qw//;\n \n # file baton members: path, mode_a, mode_b, pool, fh, blob, base\n@@ -3196,9 +3192,14 @@ sub apply_textdelta {\n \tmy $base = IO::File->new_tmpfile;\n \t$base->autoflush(1);\n \tif ($fb->{blob}) {\n-\t\tprint $base 'link ' if ($fb->{mode_a} == 120000);\n-\t\tmy $size = $::_repository->cat_blob($fb->{blob}, $base);\n-\t\tdie \"Failed to read object $fb->{blob}\" if ($size < 0);\n+\t\tdefined (my $pid = fork) or croak $!;\n+\t\tif (!$pid) {\n+\t\t\topen STDOUT, '>&', $base or croak $!;\n+\t\t\tprint STDOUT 'link ' if ($fb->{mode_a} == 120000);\n+\t\t\texec qw/git-cat-file blob/, $fb->{blob} or croak $!;\n+\t\t}\n+\t\twaitpid $pid, 0;\n+\t\tcroak $? if $?;\n \n \t\tif (defined $exp) {\n \t\t\tseek $base, 0, 0 or croak $!;\n@@ -3239,18 +3240,14 @@ sub close_file {\n \t\t\t\tsysseek($fh, 0, 0) or croak $!;\n \t\t\t}\n \t\t}\n-\n-\t\tmy ($tmp_fh, $tmp_filename) = File::Temp::tempfile(UNLINK => 1);\n-\t\tmy $result;\n-\t\twhile ($result = sysread($fh, my $string, 1024)) {\n-\t\t\tsyswrite($tmp_fh, $string, $result);\n+\t\tdefined(my $pid = open my $out,'-|') or die \"Can't fork: $!\\n\";\n+\t\tif (!$pid) {\n+\t\t\topen STDIN, '<&', $fh or croak $!;\n+\t\t\texec qw/git-hash-object -w --stdin/ or croak $!;\n \t\t}\n-\t\tdefined $result or croak $!;\n-\t\tclose $tmp_fh or croak $!;\n-\n+\t\tchomp($hash = do { local $/; <$out> });\n+\t\tclose $out or croak $!;\n \t\tclose $fh or croak $!;\n-\n-\t\t$hash = $::_repository->hash_and_insert_object($tmp_filename);\n \t\t$hash =~ /^[a-f\\d]{40}$/ or die \"not a sha1: $hash\\n\";\n \t\tclose $fb->{base} or croak $!;\n \t} else {\n@@ -3576,8 +3573,13 @@ sub chg_file {\n \t} elsif ($m->{mode_a} =~ /^120/ && $m->{mode_b} !~ /^120/) {\n \t\t$self->change_file_prop($fbat,'svn:special',undef);\n \t}\n-\tmy $size = $::_repository->cat_blob($m->{sha1_b}, $fh);\n-\tcroak \"Failed to read object $m->{sha1_b}\" if ($size < 0);\n+\tdefined(my $pid = fork) or croak $!;\n+\tif (!$pid) {\n+\t\topen STDOUT, '>&', $fh or croak $!;\n+\t\texec qw/git-cat-file blob/, $m->{sha1_b} or croak $!;\n+\t}\n+\twaitpid $pid, 0;\n+\tcroak $? if $?;\n \t$fh->flush == 0 or croak $!;\n \tseek $fh, 0, 0 or croak $!;\n \n-- \n1.5.4.3\n"},{"id":"81586","messageId":"1214696036-8294-1-git-send-email-apenwarr@gmail.com","threadId":"14204","inReplyTo":"1214686673-28099-1-git-send-email-apenwarr@gmail.com","subject":"[PATCH v2] git-svn: avoid filling up the disk with temp files.","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-06-28T23:33:56Z","receivedAt":"2008-06-28T23:33:56Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"Commit ffe256f9bac8a40ff751a9341a5869d98f72c285 (\"git-svn: Speed up fetch\")\nintroduced changes that create a temporary file for each object fetched by\nsvn.  These files should be deleted automatically, but perl apparently\ndoesn't do this until the process exits (or perhaps when its garbage\ncollector runs).\n\nThis means that on a large fetch, especially with lots of branches, we\nsometimes fill up /tmp completely, which prevents the next temp file from\nbeing written completely.  This is aggravated by the fact that a new temp\nfile is created for each updated file, even if that update produces a file\nidentical to one already in git.  Thus, it can happen even if there's lots\nof disk space to store the finished repository.\n\nWe weren't adequately checking for write errors, so this would result in an\ninvalid file getting committed, which caused git-svn to fail later with an\ninvalid checksum.\n\nThis patch adds a check to syswrite() so similar problems don't lead to\ncorruption in the future.  It also unlink()'s each temp file explicitly\nwhen we're done with it, so the disk doesn't need to fill up.\n\nSigned-off-by: Avery Pennarun <apenwarr@gmail.com>\n---\n\nPlease use this in favour of the \"Revert \"git-svn: Speed up fetch\" I sent\nearlier.  I ended up having a surprise inspiration that led to a real fix :)\n\n git-svn.perl |    5 ++++-\n 1 files changed, 4 insertions(+), 1 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 263d66c..0011387 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -3243,7 +3243,9 @@ sub close_file {\n \t\tmy ($tmp_fh, $tmp_filename) = File::Temp::tempfile(UNLINK => 1);\n \t\tmy $result;\n \t\twhile ($result = sysread($fh, my $string, 1024)) {\n-\t\t\tsyswrite($tmp_fh, $string, $result);\n+\t\t\tmy $wrote = syswrite($tmp_fh, $string, $result);\n+\t\t\tdefined($wrote) && $wrote == $result\n+\t\t\t\tor croak(\"write $tmp_filename: $!\\n\");\n \t\t}\n \t\tdefined $result or croak $!;\n \t\tclose $tmp_fh or croak $!;\n@@ -3251,6 +3253,7 @@ sub close_file {\n \t\tclose $fh or croak $!;\n \n \t\t$hash = $::_repository->hash_and_insert_object($tmp_filename);\n+\t\tunlink($tmp_filename);\n \t\t$hash =~ /^[a-f\\d]{40}$/ or die \"not a sha1: $hash\\n\";\n \t\tclose $fb->{base} or croak $!;\n \t} else {\n-- \n1.5.4.3\n"},{"id":"81587","messageId":"237967ef0806281651p3abf6ae4j9169cdc5bbf8f85d@mail.gmail.com","threadId":"14204","inReplyTo":"1214686673-28099-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH] Revert \"git-svn: Speed up fetch\"","fromName":"Mikael Magnusson","fromEmail":"mikachu@gmail.com","sentAt":"2008-06-28T23:51:09Z","receivedAt":"2008-06-28T23:51:09Z","isPatch":true,"sender":{"key":"mikachu@gmail.com","avatar":null},"body":"2008/6/28 Avery Pennarun <apenwarr@gmail.com>:\n> This reverts commit ffe256f9bac8a40ff751a9341a5869d98f72c285, because it\n> was causing errors of the form:\n>\n> Checksum mismatch: trunk/.depend 16e748c219f9f95bf3d05c6b2af5444290bc8471\n> expected: 05fb5edb8c8057be006c7e913ae0c764\n>     got: 763b9a426c5bd61e0a85252459d37cfa\n>\n> Note that the exact failing file and checksum seems to vary if you clear\n> the repository and try again.\n>\n> Conflicts:\n>\n>        git-svn.perl\n\nWasn't this the problem that was fixed by\nd683a0e00cd4734b4fab704baef1ee76205722be[1]?\n\n-- \nMikael Magnusson\n\n[1]\n\ncommit d683a0e00cd4734b4fab704baef1ee76205722be\nAuthor: Junio C Hamano <gitster@pobox.com>\nDate:   Tue May 27 23:33:22 2008 -0700\n\n    Git::cat_blob: allow using an empty blob to fix git-svn breakage\n\n    Recent \"git-svn optimization\" series introduced Git::cat_blob() subroutine\n    whose interface was broken in that it returned the size of the blob but\n    signalled an error by returning 0.  You can never use an empty blob with\n    such an interface.\n\n    This fixes the interface to return a negative value to signal an error.\n\n    Reported by Björn Steinbrink.\n\n    Signed-off-by: Junio C Hamano <gitster@pobox.com>\n"},{"id":"81588","messageId":"32541b130806281658v512c1afco9de32428db6cbae7@mail.gmail.com","threadId":"14204","inReplyTo":"237967ef0806281651p3abf6ae4j9169cdc5bbf8f85d@mail.gmail.com","subject":"Re: [PATCH] Revert \"git-svn: Speed up fetch\"","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-06-28T23:58:45Z","receivedAt":"2008-06-28T23:58:45Z","isPatch":true,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 6/28/08, Mikael Magnusson <mikachu@gmail.com> wrote:\n> Wasn't this the problem that was fixed by\n>  d683a0e00cd4734b4fab704baef1ee76205722be?\n>\n>  commit d683a0e00cd4734b4fab704baef1ee76205722be\n>  Author: Junio C Hamano <gitster@pobox.com>\n>  Date:   Tue May 27 23:33:22 2008 -0700\n>\n>     Git::cat_blob: allow using an empty blob to fix git-svn breakage\n> [...]\n\nNo, I have that patch and the problem still occurred.\n\nAvery\n"},{"id":"81590","messageId":"20080629005858.GA2036@atjola.homenet","threadId":"14204","inReplyTo":"1214696036-8294-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH v2] git-svn: avoid filling up the disk with temp files.","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-06-29T00:58:58Z","receivedAt":"2008-06-29T00:58:58Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.06.28 19:33:56 -0400, Avery Pennarun wrote:\n> Commit ffe256f9bac8a40ff751a9341a5869d98f72c285 (\"git-svn: Speed up fetch\")\n> introduced changes that create a temporary file for each object fetched by\n> svn.  These files should be deleted automatically, but perl apparently\n> doesn't do this until the process exits (or perhaps when its garbage\n> collector runs).\n> \n> This means that on a large fetch, especially with lots of branches, we\n> sometimes fill up /tmp completely, which prevents the next temp file from\n> being written completely.  This is aggravated by the fact that a new temp\n> file is created for each updated file, even if that update produces a file\n> identical to one already in git.  Thus, it can happen even if there's lots\n> of disk space to store the finished repository.\n> \n> We weren't adequately checking for write errors, so this would result in an\n> invalid file getting committed, which caused git-svn to fail later with an\n> invalid checksum.\n> \n> This patch adds a check to syswrite() so similar problems don't lead to\n> corruption in the future.  It also unlink()'s each temp file explicitly\n> when we're done with it, so the disk doesn't need to fill up.\n\nOh sweet! That also means that I don't have to fear about the repos I\nalready track with git-svn being corrupted (at least I hope so). :-)\n\nAnd yeah, looking at /tmp, there are still 58K temp files belonging to\nthe failed import. Temporarily mounting a 8k tmpfs there triggers the\nbug immediately, while with this patch I get a error message.\n\nI'm running another import of the coq repo that failed earlier, just to\nmake sure, but I also just noticed that cat-file --batch is leaking\nmemory. For example this one shoots up to about 700M RSS usage with\ngit.git:\n\ngit rev-list --objects origin/master | \\\n\tsed 's/ .*//' | \\\n\tgit cat-file --batch > /dev/null\n\nI'll follow-up with a patch that at least fixes the worst part of that,\ngetting the RSS usage for the above test down to about 40M.\n\nThanks!\nBjörn\n"},{"id":"81591","messageId":"20080629012125.GA1722@atjola.homenet","threadId":"14204","inReplyTo":"20080629005858.GA2036@atjola.homenet","subject":"[PATCH] git cat-file: Fix memory leak in batch mode","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-06-29T01:21:25Z","receivedAt":"2008-06-29T01:21:25Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"When run in batch mode, git cat-file never frees the memory for the blob\ncontents it is printing. This quickly adds up and causes git-svn to be\nhardly usable for imports of large svn repos, because it uses cat-file in\nbatch mode and cat-file's memory usage easily reaches several hundred MB\nwithout any good reason.\n\nSigned-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>\n---\n builtin-cat-file.c |    1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-cat-file.c b/builtin-cat-file.c\nindex bd343ef..f966dcb 100644\n--- a/builtin-cat-file.c\n+++ b/builtin-cat-file.c\n@@ -183,6 +183,7 @@ static int batch_one_object(const char *obj_name, int print_contents)\n \t\tfflush(stdout);\n \t}\n \n+\tfree(contents);\n \treturn 0;\n }\n \n-- \n1.5.6.1.94.gd3899.dirty\n"},{"id":"81592","messageId":"20080629022400.GA12112@atjola.homenet","threadId":"14204","inReplyTo":"20080629005858.GA2036@atjola.homenet","subject":"Re: [PATCH v2] git-svn: avoid filling up the disk with temp files.","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-06-29T02:24:00Z","receivedAt":"2008-06-29T02:24:00Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.06.29 02:58:58 +0200, Björn Steinbrink wrote:\n> On 2008.06.28 19:33:56 -0400, Avery Pennarun wrote:\n> > Commit ffe256f9bac8a40ff751a9341a5869d98f72c285 (\"git-svn: Speed up fetch\")\n> > introduced changes that create a temporary file for each object fetched by\n> > svn.  These files should be deleted automatically, but perl apparently\n> > doesn't do this until the process exits (or perhaps when its garbage\n> > collector runs).\n> > \n> > This means that on a large fetch, especially with lots of branches, we\n> > sometimes fill up /tmp completely, which prevents the next temp file from\n> > being written completely.  This is aggravated by the fact that a new temp\n> > file is created for each updated file, even if that update produces a file\n> > identical to one already in git.  Thus, it can happen even if there's lots\n> > of disk space to store the finished repository.\n> > \n> > We weren't adequately checking for write errors, so this would result in an\n> > invalid file getting committed, which caused git-svn to fail later with an\n> > invalid checksum.\n> > \n> > This patch adds a check to syswrite() so similar problems don't lead to\n> > corruption in the future.  It also unlink()'s each temp file explicitly\n> > when we're done with it, so the disk doesn't need to fill up.\n> \n> I'm running another import of the coq repo that failed earlier, just to\n> make sure, ...\n\nImport completed successfully, so:\n\nTested-by: Björn Steinbrink <B.Steinbrink@gmx.de>\n\nThanks,\nBjörn\n"},{"id":"81593","messageId":"20080629023804.GA6768@untitled","threadId":"14204","inReplyTo":"1214696036-8294-1-git-send-email-apenwarr@gmail.com","subject":"Re: [PATCH v2] git-svn: avoid filling up the disk with temp files.","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2008-06-29T02:38:37Z","receivedAt":"2008-06-29T02:38:37Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Avery Pennarun <apenwarr@gmail.com> wrote:\n> Commit ffe256f9bac8a40ff751a9341a5869d98f72c285 (\"git-svn: Speed up fetch\")\n> introduced changes that create a temporary file for each object fetched by\n> svn.  These files should be deleted automatically, but perl apparently\n> doesn't do this until the process exits (or perhaps when its garbage\n> collector runs).\n> \n> This means that on a large fetch, especially with lots of branches, we\n> sometimes fill up /tmp completely, which prevents the next temp file from\n> being written completely.  This is aggravated by the fact that a new temp\n> file is created for each updated file, even if that update produces a file\n> identical to one already in git.  Thus, it can happen even if there's lots\n> of disk space to store the finished repository.\n> \n> We weren't adequately checking for write errors, so this would result in an\n> invalid file getting committed, which caused git-svn to fail later with an\n> invalid checksum.\n> \n> This patch adds a check to syswrite() so similar problems don't lead to\n> corruption in the future.  It also unlink()'s each temp file explicitly\n> when we're done with it, so the disk doesn't need to fill up.\n> \n> Signed-off-by: Avery Pennarun <apenwarr@gmail.com>\n> ---\n \n> Please use this in favour of the \"Revert \"git-svn: Speed up fetch\" I sent\n> earlier.  I ended up having a surprise inspiration that led to a real fix :)\n\nOuch, I didn't noticed these unchecked syscalls :x\n\nVery graciously\nAcked-by: Eric Wong <normalperson@yhbt.net>\n\nApologies to all users who were bitten by this bug.\n\n>  git-svn.perl |    5 ++++-\n>  1 files changed, 4 insertions(+), 1 deletions(-)\n> \n> diff --git a/git-svn.perl b/git-svn.perl\n> index 263d66c..0011387 100755\n> --- a/git-svn.perl\n> +++ b/git-svn.perl\n> @@ -3243,7 +3243,9 @@ sub close_file {\n>  \t\tmy ($tmp_fh, $tmp_filename) = File::Temp::tempfile(UNLINK => 1);\n>  \t\tmy $result;\n>  \t\twhile ($result = sysread($fh, my $string, 1024)) {\n> -\t\t\tsyswrite($tmp_fh, $string, $result);\n> +\t\t\tmy $wrote = syswrite($tmp_fh, $string, $result);\n> +\t\t\tdefined($wrote) && $wrote == $result\n> +\t\t\t\tor croak(\"write $tmp_filename: $!\\n\");\n>  \t\t}\n>  \t\tdefined $result or croak $!;\n>  \t\tclose $tmp_fh or croak $!;\n> @@ -3251,6 +3253,7 @@ sub close_file {\n>  \t\tclose $fh or croak $!;\n>  \n>  \t\t$hash = $::_repository->hash_and_insert_object($tmp_filename);\n> +\t\tunlink($tmp_filename);\n>  \t\t$hash =~ /^[a-f\\d]{40}$/ or die \"not a sha1: $hash\\n\";\n>  \t\tclose $fb->{base} or croak $!;\n>  \t} else {\n> -- \n> 1.5.4.3\n"},{"id":"81595","messageId":"7v1w2gkj3l.fsf@gitster.siamese.dyndns.org","threadId":"14204","inReplyTo":"20080629012125.GA1722@atjola.homenet","subject":"Re: [PATCH] git cat-file: Fix memory leak in batch mode","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-06-29T03:36:46Z","receivedAt":"2008-06-29T03:36:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Björn Steinbrink <B.Steinbrink@gmx.de> writes:\n\n> When run in batch mode, git cat-file never frees the memory for the blob\n> contents it is printing. This quickly adds up and causes git-svn to be\n> hardly usable for imports of large svn repos, because it uses cat-file in\n> batch mode and cat-file's memory usage easily reaches several hundred MB\n> without any good reason.\n>\n> Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>\n> ---\n>  builtin-cat-file.c |    1 +\n>  1 files changed, 1 insertions(+), 0 deletions(-)\n>\n> diff --git a/builtin-cat-file.c b/builtin-cat-file.c\n> index bd343ef..f966dcb 100644\n> --- a/builtin-cat-file.c\n> +++ b/builtin-cat-file.c\n> @@ -183,6 +183,7 @@ static int batch_one_object(const char *obj_name, int print_contents)\n>  \t\tfflush(stdout);\n>  \t}\n>  \n> +\tfree(contents);\n>  \treturn 0;\n>  }\n\nThanks, except that it should go inside the \"if (print_contents == BATCH)\"\nblock to avoid freeing an uninitialized pointer.\n"},{"id":"81633","messageId":"20080629115417.GA14806@atjola.homenet","threadId":"14204","inReplyTo":"7v1w2gkj3l.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] git cat-file: Fix memory leak in batch mode","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2008-06-29T11:54:17Z","receivedAt":"2008-06-29T11:54:17Z","isPatch":true,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2008.06.28 20:36:46 -0700, Junio C Hamano wrote:\n> Björn Steinbrink <B.Steinbrink@gmx.de> writes:\n> \n> > When run in batch mode, git cat-file never frees the memory for the blob\n> > contents it is printing. This quickly adds up and causes git-svn to be\n> > hardly usable for imports of large svn repos, because it uses cat-file in\n> > batch mode and cat-file's memory usage easily reaches several hundred MB\n> > without any good reason.\n> >\n> > Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>\n> > ---\n> >  builtin-cat-file.c |    1 +\n> >  1 files changed, 1 insertions(+), 0 deletions(-)\n> >\n> > diff --git a/builtin-cat-file.c b/builtin-cat-file.c\n> > index bd343ef..f966dcb 100644\n> > --- a/builtin-cat-file.c\n> > +++ b/builtin-cat-file.c\n> > @@ -183,6 +183,7 @@ static int batch_one_object(const char *obj_name, int print_contents)\n> >  \t\tfflush(stdout);\n> >  \t}\n> >  \n> > +\tfree(contents);\n> >  \treturn 0;\n> >  }\n> \n> Thanks, except that it should go inside the \"if (print_contents == BATCH)\"\n> block to avoid freeing an uninitialized pointer.\n\nAh crap, I even wondered about the kill-a-warning initialization of\n\"contents\", but my brain was already asleep.\n\nThanks,\nBjörn\n"}]}