git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] cvsimport: introduce _fetchfile() method and used a 1M buffer to read()

From
MLMartin Langhoff <martin@catalyst.net.nz>
Date
May 23, 2006, 08:08 UTC
Message-ID
<11483717381774-git-send-email-martin@catalyst.net.nz>

File retrieval from the socket is now moved to _fetchfile() and we now cap reads at 1MB. This should limit the memory growth of the cvsimport process.

Signed-off-by: Martin Langhoff <martin@catalyst.net.nz>
---
 git-cvsimport.perl |   36 +++++++++++++++++++-----------------
 1 files changed, 19 insertions(+), 17 deletions(-)
c06eea55b2b061abe6c09fa0737d6bb87afa9d39
diff --git a/git-cvsimport.perl b/git-cvsimport.perl
index 5b68671..ace7087 100755
--- a/git-cvsimport.perl
+++ b/git-cvsimport.perl
@@ -315,15 +315,7 @@ sub _line {
 			chomp $cnt;
 			die "Duh: Filesize $cnt" if $cnt !~ /^\d+$/;
 			$line="";
-			$res=0;
-			while($cnt) {
-				my $buf;
-				my $num = $self->{'socketi'}->read($buf,$cnt);
-				die "Server: Filesize $cnt: $num: $!\n" if not defined $num or $num<=0;
-				print $fh $buf;
-				$res += $num;
-				$cnt -= $num;
-			}
+			$res = $self->_fetchfile($fh, $cnt);
 		} elsif($line =~ s/^ //) {
 			print $fh $line;
 			$res += length($line);
@@ -335,14 +327,7 @@ sub _line {
 			chomp $cnt;
 			die "Duh: Mbinary $cnt" if $cnt !~ /^\d+$/ or $cnt<1;
 			$line="";
-			while($cnt) {
-				my $buf;
-				my $num = $self->{'socketi'}->read($buf,$cnt);
-				die "S: Mbinary $cnt: $num: $!\n" if not defined $num or $num<=0;
-				print $fh $buf;
-				$res += $num;
-				$cnt -= $num;
-			}
+			$res += $self->_fetchfile($fh, $cnt);
 		} else {
 			chomp $line;
 			if($line eq "ok") {
@@ -384,6 +369,23 @@ sub file {
 
 	return ($name, $res);
 }
+sub _fetchfile {
+	my ($self, $fh, $cnt) = @_;
+	my $res;
+	my $bufsize = 1024 * 1024;
+	while($cnt) {
+	    if ($bufsize > $cnt) {
+		$bufsize = $cnt;
+	    }
+	    my $buf;      
+	    my $num = $self->{'socketi'}->read($buf,$bufsize);
+	    die "Server: Filesize $cnt: $num: $!\n" if not defined $num or $num<=0;
+	    print $fh $buf;
+	    $res += $num;
+	    $cnt -= $num;
+	}
+	return $res;
+}
 
 
 package main;
-- 
1.3.2.g82000
Message 1 of 1 in “cvsimport: introduce _fetchfile() method and used a 1M buffer to read()”
  1. cvsimport: introduce _fetchfile() method and used a 1M buffer to read()Martin Langhoff, May 23, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.