Re: Slow fetches of tags

From: Junio C Hamano <junkio@cox.net>
Date: May 25, 2006, 01:32 UTC
Message-ID: <7vd5e23n5a.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <Pine.LNX.4.64.0605241641250.5623@g5.osdl.org>

Linus Torvalds <torvalds@osdl.org> writes:

Show 12 quoted lines

> On Wed, 24 May 2006, Linus Torvalds wrote:
>> 
>> IOW, I think there's something more fundamentally wrong with the tag 
>> following. We _should_ have figured out much more quickly that we have it 
>> all.
>
> Actually, maybe the problem is that Ralf's tree has two roots, because of 
> the old CVS history. It might be following the other root down for the 
> "have" part, since that one doesn't exist at all in the target and the 
> other side will never acknowledge any of it. 
>
> I'll play with it.

I think I know what is going on. You are exactly right -- the two-root ness is what is causing this.

We used to stop sending "have" immediately after we get an ACK. This was troublesome for trees with many long branches, so we introduced multi_ack protocol extension to let the server side (upload-pack) say "Ok, enough on this branch -- I know this object so do not tell me any more about objects reachable from it, but do tell me about other development tracks if you have one". If you run "fetch-pack -v" after priming a repository with Ralf's tree and Chris's tree, you will see many "have" with occasional "got ack 2 [0-9a-f]{40}". The latter is upload-pack acking this way.

This was done to prevent already-known-to-be-common objects filling up the list of known common commits on the server side. The remaining slots can be used to discover common commits on other branches, so that we can minimize the transfer. It was an important optimization when dealing with sets of branches that are long.

This unfortunately breaks down quite badly in this case, since the remaining "branch" it keeps following is the other history Chris's tree has never heard of down to its root in vain.

It might be worth changing fetch-pack to note that it has sent many "have"s after it got an "continue" ACK, and give up early, say using a heuristic between the age of the commit that did got an ACK and the one we are about to send out as a "have".