threads / patch / 3211

patchgit-svnimport.perl: fix for 'arg list too long...'

Subject: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

## tl;dr

6 messages between Feb 1, 2006 and Feb 2, 2006. Diffs are folded; open one to read it.

replies: 5people: 3as markdown or json

Sasha Khapyorsky· Feb 1, 2006, 15:53 UTC · lore
Hello,
This fixes 'arg list too long' problem when passed to git-ls-files.
Sasha.
This fixes 'arg list too long..' problem with git-ls-files.

Note that second arg list separation loop (with 'git-update-index') is needed since git-ls-files arguments can be directories.

Signed-off-by: Sasha Khapyorsky <sashak@voltaire.com>
Show changes to git-svnimport.perl +16 −9
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 6e3a44a..b6799d8 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -546,23 +546,30 @@ sub commit {
 			}
 		}
 
-		if(@old) {
-			open my $F, "-|", "git-ls-files", "-z", @old or die $!;
-			@old = ();
+		while(@old) {
+			my @o1;
+			if(@old > 55) {
+				@o1 = splice(@old,0,50);
+			} else {
+				@o1 = @old;
+				@old = ();
+			}
+			open my $F, "-|", "git-ls-files", "-z", @o1 or die $!;
+			@o1 = ();
 			local $/ = "\0";
 			while(<$F>) {
 				chomp;
-				push(@old,$_);
+				push(@o1,$_);
 			}
 			close($F);
 
-			while(@old) {
+			while(@o1) {
 				my @o2;
-				if(@old > 55) {
-					@o2 = splice(@old,0,50);
+				if(@o1 > 55) {
+					@o2 = splice(@o1,0,50);
 				} else {
-					@o2 = @old;
-					@old = ();
+					@o2 = @o1;
+					@o1 = ();
 				}
 				system("git-update-index","--force-remove","--",@o2);
 				die "Cannot remove files: $?\n" if $?;
Junio C Hamano· Feb 1, 2006, 20:27 UTC · re: Sasha Khapyorsky · lore

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

Sasha Khapyorsky <sashak@voltaire.com> writes:
> This fixes 'arg list too long..' problem with git-ls-files.

I wonder if feeding the list into a pipe to 'update-index --stdin' would be an option.

BTW, please do not do this:
        Mail-Followup-To: Junio C Hamano <junkio@cox.net>,
                Matthias Urlichs <smurf@smurf.noris.de>, git@vger.kernel.org
It is simply rude.
 * I and/or Smurf may be uninterested in hearing the whole
   discussion thread started by you.  We might well be, but that
   is not something you shoudl decide upfront in your message.
 * When a recipient of your message tries to "[R]eply" to *you*,
   that Mail-Followup-To: would cause the MUA to address that
   message to me and/or smurf by default, not to you.  I had to
   edit the resulting To: header by hand to address this
   response to you.  Why force extra work on others?

I know why you did it --- you subscribe to git list and otherwise you would get two copies.

That is _not_ a good excuse. That is how mailing lists work; either you filter the duplicates on your end, or you learn to live with it. Please don't force extra work on others.

I think the only header mucking of this kind that is acceptable is to remove yourself from CC: line when the mailing list you subscribe to is already on the CC: line.

Sasha Khapyorsky· Feb 1, 2006, 21:54 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

On 12:27 Wed 01 Feb     , Junio C Hamano wrote:
Show 6 quoted lines
> Sasha Khapyorsky <sashak@voltaire.com> writes:
> 
> > This fixes 'arg list too long..' problem with git-ls-files.
> 
> I wonder if feeding the list into a pipe to 'update-index --stdin' 
> would be an option.

This can be option for git-update-index, but my problem was with git-ls-files, git-update-index was ok (arg list processing were splitted already).

Show 7 quoted lines
> 
> BTW, please do not do this:
> 
>         Mail-Followup-To: Junio C Hamano <junkio@cox.net>,
>                 Matthias Urlichs <smurf@smurf.noris.de>, git@vger.kernel.org
> 
> It is simply rude.
OK, will not. Thanks for pointing.
Sasha.
Junio C Hamano· Feb 1, 2006, 22:50 UTC · re: Sasha Khapyorsky · lore

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

Sasha Khapyorsky <sashak@voltaire.com> writes:
Show 11 quoted lines
> On 12:27 Wed 01 Feb     , Junio C Hamano wrote:
>> Sasha Khapyorsky <sashak@voltaire.com> writes:
>> 
>> > This fixes 'arg list too long..' problem with git-ls-files.
>> 
>> I wonder if feeding the list into a pipe to 'update-index --stdin' 
>> would be an option.
>
> This can be option for git-update-index, but my problem was with
> git-ls-files, git-update-index was ok (arg list processing were splitted
> already).

I was commenting on the "while (@o1)" loop that splices at 50 when the list has more than 55 items to feed update-index. You could accumulate output from multiple invocations of ls-files and feed everything to a single "update-index --stdin" after the "while (@old)" loop is done. Batch of 50 vs a single batch may not matter though [*1*].

But you are right; that git-ls-files can get too many arguments unless you split like that.

A casual skimming over the rest of the code tells me that this fixes the last instance of such a command invocation with too many arguments. Good catch.

Smurf, I do not have problems with Sasha's patch. Are you OK if I apply it?

-- >8 --

*1* I do not think this makes much of a difference but here is what I mean.

Show changes to git-svnimport.perl +9 −15
diff --git a/git-svnimport.perl b/git-svnimport.perl
index b6799d8..d76a595 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -546,7 +546,8 @@ sub commit {
 			}
 		}
 
-		while(@old) {
+		my @u = ();
+		while (@old) {
 			my @o1;
 			if(@old > 55) {
 				@o1 = splice(@old,0,50);
@@ -555,26 +556,19 @@ sub commit {
 				@old = ();
 			}
 			open my $F, "-|", "git-ls-files", "-z", @o1 or die $!;
-			@o1 = ();
 			local $/ = "\0";
 			while(<$F>) {
 				chomp;
-				push(@o1,$_);
+				push(@u,$_);
 			}
 			close($F);
-
-			while(@o1) {
-				my @o2;
-				if(@o1 > 55) {
-					@o2 = splice(@o1,0,50);
-				} else {
-					@o2 = @o1;
-					@o1 = ();
-				}
-				system("git-update-index","--force-remove","--",@o2);
-				die "Cannot remove files: $?\n" if $?;
-			}
 		}
+		open my $F, "|-",
+			qw(git-update-index --force-remove -z --stdin)
+				or die $!;
+		print $F "$_\0" for @u;
+		close $F or die $!;
+
 		while(@new) {
 			my @n2;
 			if(@new > 12) {
Matthias Urlichs· Feb 1, 2006, 23:50 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

Hi,
Junio C Hamano:
> Smurf, I do not have problems with Sasha's patch.  Are you OK
> if I apply it?
> 
Go ahead.
-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Circumstances can force a generalized incompetent to become competent, at
least in a specialized field.
					-- Frank R. Freemon
Sasha Khapyorsky· Feb 2, 2006, 14:26 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-svnimport.perl: fix for 'arg list too long...'

On 14:50 Wed 01 Feb     , Junio C Hamano wrote:
Show 5 quoted lines
> 
> *1* I do not think this makes much of a difference but here is what
> I mean.
> 
> diff --git a/git-svnimport.perl b/git-svnimport.perl
Tested. Works good. I think your version is cleaner. Cosmetic comment btw:
Show 5 quoted lines
> +		open my $F, "|-",
> +			qw(git-update-index --force-remove -z --stdin)
> +				or die $!;
> +		print $F "$_\0" for @u;
> +		close $F or die $!;

It is not critical, but this block may be under if(@u) {..} to prevent "empty" git-update-index invocations.

Sasha.

← back to recent threads