Re: [PATCH-v2] Allow git-filter-branch to process large repositories with lots of branches.

From: Stefano Lattarini <stefano.lattarini@gmail.com>
Date: Sep 7, 2013, 23:06 UTC
Message-ID: <522BB170.8040507@gmail.com>
In-Reply-To: <CE50E29B.191B8%lee.carver@corp.service-now.com>

On 07/09/13 22:03, Lee Carver wrote:

Show 44 quoted lines

> As noted in several forums, a recommended way to move trees between
> repositories
> is to use git-filter-branch to revise the history for a single tree:
>
> http://gbayer.com/development/moving-files-from-one-git-repository-to-anoth
> er-preserving-history/
> http://stackoverflow.com/questions/1365541/how-to-move-files-from-one-git-r
> epo-to-another-not-a-clone-preserving-history
>
> However, this can lead to argument list too long errors when the original
> repository has many retained branches (>6k)
>
> /usr/local/git/libexec/git-core/git-filter-branch: line 270:
> /usr/local/git/libexec/git-core/git: Argument list too long
> Could not get the commits
>
> Piping the saved output from git rev-parse into git rev-list avoids this
> problem, since the rev-parse output is not processed as a command line
> argument.
> ---
>   git-filter-branch.sh | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index ac2a005..60d239b 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -255,7 +255,7 @@ else
>   	remap_to_ancestor=t
>   fi
>
> -rev_args=$(git rev-parse --revs-only "$@")
> +git rev-parse --revs-only "$@" > ../parse
>
>   case "$filter_subdir" in
>   "")
> @@ -267,8 +267,9 @@ case "$filter_subdir" in
>   	;;
>   esac
>
> +cat ../parse | \
>   git rev-list --reverse --topo-order --default HEAD \
> -	--parents --simplify-merges $rev_args "$@" > ../revs ||
> +	--parents --simplify-merges --stdin "$@" > ../revs ||

 >
Useless use of cat IMO.  I'd suggest using a redirection instead:

   git rev-list --reverse --topo-order --default HEAD \
-	--parents --simplify-merges $rev_args "$@" > ../revs ||
+	--parents --simplify-merges --stdin "$@" > ../revs < ../parse ||

>   	die "Could not get the commits"
>   commits=$(wc -l <../revs | tr -d " ")
>
>

Regards,
   Stefano