Re: Slow fetches of tags

From: Linus Torvalds <torvalds@osdl.org>
Date: May 24, 2006, 17:21 UTC
Message-ID: <Pine.LNX.4.64.0605240947580.5623@g5.osdl.org>
In-Reply-To: <Pine.LNX.4.64.0605240931480.5623@g5.osdl.org>

On Wed, 24 May 2006, Linus Torvalds wrote:

Show 7 quoted lines

> 
> Can you add a printout to show what the "taglist" is for you in 
> git-fetch.sh (just before the thing that does that
> 
> 	fetch_main "$taglist"
> 
> thing?). It _should_ have pruned out all the tags you already have.

Actually, looking at that tag-fetching logic, we already know that we have the objects that the tags point to (because those are the only kinds that we should auto-follow). I wonder if the slowness is because of all the have/want commit following, which walks the whole tree to say "I have this", when in this case we really should directly say "I have these" for the objects that the tags point to.

So the problem may be that we basically send a totally unnecessary list of all the objects we have, when the other end really only cares about the fact that we have the objects that the tags point to. Which we know we do, but we didn't say so, because "git-fetch" didn't really mark them that way.

And instead of sending the commits that we know we have, and that we know are the interesting ones and that will cut off the tag-object-walk, we start from all the local tips, and use the regular "parse commits in date order" thing and send "have" lines for everything we see that isn't common. Walking a lot of unnecessary crud.

Junio? Any ideas? I didn't want to do that tag-auto-following, and while I admit it's damn convenient, it's really quite broken, methinks.

I almost suspect that we need to have a syntax where-by the local fetch-list ends up doing

	"$tagname:$tagname:$sha1wehave"

as the argument to fetch-pack, and then fetch-pack would be modified to send those "$sha1wehave" objects early as "have" objects. Ie start from something like

	diff --git a/git-fetch.sh b/git-fetch.sh
	index 280f62e..dce3812 100755
	--- a/git-fetch.sh
	+++ b/git-fetch.sh
	@@ -400,7 +400,7 @@ case "$no_tags$tags" in
	 			}
	 			git-cat-file -t "$sha1" >/dev/null 2>&1 || continue
	 			echo >&2 "Auto-following $name"
	-			echo ".${name}:${name}"
	+			echo ".${name}:${name}:${sha1}"
	 		done)
	 	esac
	 	case "$taglist" in

and then pass the info all the way up (the above patch will obviously result in a totally broken script, everything downstream from that point would have to be taught about the "already have this" part too).

Ralf, which repo is this, so that others (me, if I get the time and energy, Junio or some other hapless sucker^W^Whero if I'm lucky) can try things out?

		Linus