Why is git-clone --reference so slow?

David Woodhouse· Feb 5, 2007, 11:46 UTC · lore

My DSL line sucks; I know this. But why is git-clone so bad at using it?

pmac /pmac/git $ git-clone --reference /pmac/git/linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git
remote: Generating pack...
remote: Done counting 14428 objects.
remote: Deltifying 14428 objects.
remote:  100% (14428/14428) done
Indexing 14428 objects.
  25% (3662/14428) done

hera /home/dwmw2/x $ git-clone -n --bare --reference /pub/scm/linux/kernel/git/torvalds/linux-2.6.git /pub/scm/linux/kernel/git/geoff/ps3-linux.git 
remote: Generating pack...
remote: Done counting 14490 objects.
remote: Deltifying 14490 objects.
remote:  100% (14490/14490) done
Indexing 14490 objects.
remote: Total 14490, written 14490 (delta 579), reused 5598 (delta 367)
 100% (14490/14490) done
Resolving 579 deltas.
 100% (579/579) done

-- 
dwmw2

Johannes Schindelin· Feb 5, 2007, 15:31 UTC · re: David Woodhouse · lore

Re: Why is git-clone --reference so slow?

Hi,

On Mon, 5 Feb 2007, David Woodhouse wrote:

> My DSL line sucks; I know this. But why is git-clone so bad at using it?
> 
> This is slow and seems to be downloading a lot of stuff it doesn't
> need...
> 
> pmac /pmac/git $ git-clone --reference /pmac/git/linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/geoff/ps3-linux.git

Is this a Mac? Is it Mac OS X? Which git version do you use?

David Woodhouse· Feb 5, 2007, 15:35 UTC · re: Johannes Schindelin · lore

Re: Why is git-clone --reference so slow?

On Mon, 2007-02-05 at 16:31 +0100, Johannes Schindelin wrote:

> Is this a Mac? Is it Mac OS X? Which git version do you use?
> 
> We recently had a case like this, and it was fixed: apparently, mmap()
> on Mac OS X is less than fast... 

-- 
dwmw2

Junio C Hamano· Feb 7, 2007, 10:03 UTC · re: David Woodhouse · lore

Re: Why is git-clone --reference so slow?

David Woodhouse <dwmw2@infradead.org> writes:

> My DSL line sucks; I know this. But why is git-clone so bad at using it?

I did the same and it initially got 45MB pack (14.5k objects).

diff --git a/git-clone.sh b/git-clone.sh
index 1710996..1bd54de 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -178,46 +178,32 @@ esac && export GIT_DIR && git-init ${template+"$template"} || usage
 
 if test -n "$reference"
 then
+	ref_git=
 	if test -d "$reference"
 	then
 		if test -d "$reference/.git/objects"
 		then
-			reference="$reference/.git"
+			ref_git="$reference/.git"
+		elif test -d "$reference/objects"
+		then
+			ref_git="$reference"
 		fi
-		reference=$(cd "$reference" && pwd)
-		echo "$reference/objects" >"$GIT_DIR/objects/info/alternates"
-		(cd "$reference" && tar cf - refs) |
-		(cd "$GIT_DIR/refs" &&
-		 mkdir reference-tmp &&
-		 cd reference-tmp &&
-		 tar xf - &&
-		 find refs ! -type d -print |
-		 while read ref
-		 do
-			if test -h "$ref"
-			then
-				# Old-style symbolic link ref.  Not likely
-				# to appear under refs/ but we might as well
-				# deal with them.
-				:
-			elif test -f "$ref"
-			then
-				point=$(cat "$ref") &&
-					case "$point" in
-					'ref: '*) ;;
-					*) continue ;;
-					esac
-			fi
-			# The above makes true ref to 'continue' and
-			# we will come here when we are looking at
-			# symbolic link ref or a textual symref (or
-			# garbage, like fifo).
-			# The true ref pointed at by it is enough to
-			# ensure that we do not fetch objects reachable
-			# from it.
-			rm -f "$ref"
-		 done
-		)
+	fi
+	if test -n "$ref_git"
+	then
+		ref_git=$(cd "$ref_git" && pwd)
+		echo "$ref_git/objects" >"$GIT_DIR/objects/info/alternates"
+		(
+			GIT_DIR="$ref_git" git for-each-ref \
+				--format='%(objectname) %(*objectname)'
+		) |
+		while read a b
+		do
+			test -z "$a" ||
+			git update-ref "refs/reference-tmp/$a" "$a"
+			test -z "$b" ||
+			git update-ref "refs/reference-tmp/$b" "$b"
+		done
 	else
 		die "reference repository '$reference' is not a local directory."
 	fi