Re: [PATCH] git-mv is not able to handle big directories

From: Junio C Hamano <junkio@cox.net>
Date: Nov 23, 2005, 06:14 UTC
Message-ID: <7voe4b7uw7.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <200511231141.57683.lan@ac-sw.com>

Alexander Litvinov <lan@ac-sw.com> writes:

Show 10 quoted lines

> When moving directory with large number of files git-mv says:
>> git-mv jsp* .
> Can't exec "git-update-index": Argument list too long at /usr/local/bin/git-mv 
> line 193.
> git-update-index failed to add new names with code -1
>
> This patch fixes this by building list of files with limited len (currently 
> 5000) and executing git-update-index few times until all files will be 
> processed. I don't know how to determinate limit of command line but 5000 
> seems safe enougth to me.

Two comments.

(1) the argument limit is enforced by the operating system in
    bytes (including environment size unfortunately) so we might
    want to count bytes not number of paths.  I heard GNU xargs
    uses 131072 as the default limit.

(2) I wonder if we can detect this particular failure case and
    then fall back on splitting the arguments dynamically, maybe
    something like this:

	sub xargs_system {
        	my ($cmd, @args) = @_;
                my $rc = system(@$cmd, @args);
                if ($rc == 'argument list too long error') {
                	my (@args0) = splice(@args, 0, @args/2);
			$rc = xargs_system($cmd, @args0);
                        return $c if ($rc);
                        return xargs_system($cmd, @args);
		}
		return $rc;
	}

    and:

	$rc = xargs_system([qw(git-update-index --)], @changedfiles);
	$rc = xargs_system([qw(git-update-index --add --)], @addedfiles);
	...