Re: [PATCH v2] hash binary sha1 into patch id

From: Clemens Buchacher <drizzd@aon.at>
Date: Sep 10, 2010, 08:16 UTC
Message-ID: <20100910081626.GA30507@localhost>
In-Reply-To: <1284095850.1728.5.camel@n900.home.ru>

On Fri, Sep 10, 2010 at 09:17:30AM +0400, Marat Radchenko wrote:

Show 8 quoted lines

> 
> However, I can't say I'm completely satisfied with git rebase
> speed (I'd like it to be < 1 min). So, are there known
> performance hotspots in 'git rev-list --cherry-pick'/'git
> format-patch --ignore-if-in-upstream' (these are two flags have
> biggest impact on git rebase [-i] speed) or I should run some
> kind of profiles in order to determine what else could be
> improved?

git format-patch is still slow if you have large text files (>10000 lines). This patch only helps for binary files or files marked as binary in .gitattributes. There is also a patch

 http://mid.gmane.org/645d8a9bf671937c1a6962b49cf1c512e0af0d82.1279008702.git.git@drmicha.warpmail.net

to turn off patch ID computation entirely.

Other than that, I am not aware of performance issues with rebase that would cause it to take more than a minute. Possibly you can use 'perf' to profile your use case. I used it to test this patch in the script below (pass -p to enable). But it's only testing format-patch right now, so you would have to modify it to profile rebase as a whole.

Clemens ---

#!/bin/bash

set -e

NLINES=50000
NRAND=32768
# probability of change in percent
CHG_PROB=30
perf=
while test $# -gt 0
do
	case $1 in
	-p)
		perf=t
		;;
	-n)
		shift
		NLINES=$1
		;;
	*)
		break
		;;
	esac
	shift
done

if test $# -gt 0
then
	export PATH="$1:$PATH"
	shift
fi

if test $# -gt 0
then
	echo "too many arguments" >&2
	exit 1
fi

dir=$(mktemp -d)

scramble()
{
	while read x
	do
		if test $RANDOM -lt $((($CHG_PROB * $NRAND)/100))
		then
			echo $RANDOM
		else
			echo "$x"
		fi
	done < "$1" > "$1.new"
	mv -f "$1.new" "$1"
}

run()
{
	echo \$ "$@"
	if test -n "$perf"
	then
		perf record -g -f "$@" >/dev/null
		perf report -g
	else
		time "$@" >/dev/null
	fi
}

cd "$dir" git init -q

for i in $(seq $NLINES)
do
	echo $i
done > file
git add file
echo "file binary" >.gitattributes
git add .gitattributes
git commit -q -m initial
git branch other

scramble file git add file git commit -q -m 'change big file'

git checkout -q other : >newfile git add newfile git commit -q -m 'add small file'

gfp="git format-patch --stdout"

run $gfp master run $gfp --ignore-if-in-upstream master

git cherry-pick master >/dev/null 2>&1

git checkout -q master scramble file git add file git commit -q -m 'change big file again'

git checkout -q other^{}
run git rebase master
if test -n "$(git rev-list master...HEAD~)"
then
	echo "patch not identified" >&2
	exit 1
fi

git checkout -q -b squashed master
git reset -q --soft HEAD~2
git commit -q -m squashed
git checkout -q other^{}
if git rebase squashed >/dev/null
then
	echo "patch dropped" >&2
	exit 1
fi

cd - >/dev/null rm -rf "$dir"