Re: git-cvsexportcommit fails for huge commits

From: Jeff King <peff@peff.net>
Date: Dec 12, 2007, 08:31 UTC
Message-ID: <20071212083154.GB7676@coredump.intra.peff.net>
In-Reply-To: <20071211200418.GA13815@mkl-desktop>

On Tue, Dec 11, 2007 at 09:04:18PM +0100, Markus Klinik wrote:

Show 11 quoted lines

> git-cvsexportcommit fails for huge commits, that is commits with lots
> of files.  To be exact, the problem arises if the generated 'cvs
> status' command exceeds the maximum length for commands.
> 
> Here is the error:
> 
> Can't exec "cvs": Argument list too long at
> /home/mkl/bin/git-cvsexportcommit line 334.  Argument list too long 0
> at /home/mkl/bin/git-cvsexportcommit line 334.  cvs status [snip:
> long, long list of files]  1792 at /home/mkl/bin/git-cvsexportcommit
> line 332.

Yuck. Unfortunately, CVS doesn't offer a more scalable interface, so we are stuck splitting the arguments across multiple invocations.

However, I think this should work. The output of "cvs status foo bar" is the same as "cvs status foo; cvs status bar". We will make our commits in two CVS invocations, but since CVS isn't atomic _anyway_, we shouldn't mind losing the atomicity.

Does the patch below clear up your problem?

---

diff --git a/git-cvsexportcommit.perl b/git-cvsexportcommit.perl
index 92e4162..20e432b 100755
--- a/git-cvsexportcommit.perl
+++ b/git-cvsexportcommit.perl
@@ -195,11 +195,11 @@ foreach my $f (@files) {
 my %cvsstat;
 if (@canstatusfiles) {
     if ($opt_u) {
-      my @updated = safe_pipe_capture(@cvs, 'update', @canstatusfiles);
+      my @updated = xargs_safe_pipe_capture([@cvs, 'update'], @canstatusfiles);
       print @updated;
     }
     my @cvsoutput;
-    @cvsoutput= safe_pipe_capture(@cvs, 'status', @canstatusfiles);
+    @cvsoutput = xargs_safe_pipe_capture([@cvs, 'status'], @canstatusfiles);
     my $matchcount = 0;
     foreach my $l (@cvsoutput) {
         chomp $l;
@@ -295,7 +295,7 @@ if ($dirtypatch) {
 
 if ($opt_c) {
     print "Autocommit\n  $cmd\n";
-    print safe_pipe_capture(@cvs, 'commit', '-F', '.msg', @files);
+    print xargs_safe_pipe_capture([@cvs, 'commit', '-F', '.msg'], @files);
     if ($?) {
 	die "Exiting: The commit did not succeed";
     }
@@ -335,6 +335,22 @@ sub safe_pipe_capture {
     return wantarray ? @output : join('',@output);
 }
 
+sub xargs_safe_pipe_capture {
+	my $MAX_ARG_LENGTH = 1024;
+	my $cmd = shift;
+	my @output;
+	while(@_) {
+		my @args;
+		my $length = 0;
+		while(@_ && $length < $MAX_ARG_LENGTH) {
+			push @args, shift;
+			$length += length($args[$#args]);
+		}
+		push @output, safe_pipe_capture(@$cmd, @args);
+	}
+	return @output;
+}
+
 sub safe_pipe_capture_blob {
     my $output;
     if (my $pid = open my $child, '-|') {