Re: [BUG?] fetch into shallow sends a large number of objects

From: Duy Nguyen <pclouds@gmail.com>
Date: Mar 10, 2016, 12:20 UTC
Message-ID: <20160310122020.GA24007@lanh>
In-Reply-To: <20160308132524.GA22866@sigill.intra.peff.net>

On Tue, Mar 08, 2016 at 08:25:24AM -0500, Jeff King wrote:

Show 23 quoted lines

> I think this patch does roughly the right thing:
> 
> diff --git a/upload-pack.c b/upload-pack.c
> index 4859535..da76f70 100644
> --- a/upload-pack.c
> +++ b/upload-pack.c
> @@ -833,12 +833,41 @@ static void receive_needs(void)
>  		deepen_by_rev_list(av.argc, av.argv, &shallows);
>  		argv_array_clear(&av);
>  	}
> -	else
> -		if (shallows.nr > 0) {
> -			int i;
> -			for (i = 0; i < shallows.nr; i++)
> -				register_shallow(shallows.objects[i].item->oid.hash);
> +	else if (shallows.nr > 0) {
> +		struct rev_info revs;
> +		struct argv_array av = ARGV_ARRAY_INIT;
> +		struct commit *c;
> +		int i;
> +
> +		argv_array_push(&av, "rev-list");
> +		argv_array_push(&av, "--boundary");

Nice. I didn't know about --boundary. But will it work correctly in this case?

       --- B ---- C ---- F
          /      /
     --- D ---- E ---- G

C and D will be current shallow cut points. People "want" F and G. "rev-list --boundary F G ^C ^D" would mark E as boundary/shallow too, correct? If so the history from G will be one depth short on a normal fetch.

Show 21 quoted lines

> +		for (i = 0; i < want_obj.nr; i++) {
> +			struct object *o = want_obj.objects[i].item;
> +			argv_array_push(&av, oid_to_hex(&o->oid));
>  		}
> +		for (i = 0; i < shallows.nr; i++) {
> +			struct object *o = shallows.objects[i].item;
> +			argv_array_pushf(&av, "^%s", oid_to_hex(&o->oid));
> +		}
> +
> +		init_revisions(&revs, NULL);
> +		setup_revisions(av.argc, av.argv, &revs, NULL);
> +		if (prepare_revision_walk(&revs))
> +			die("revision walk setup failed");
> +
> +		while ((c = get_revision(&revs))) {
> +			if (!(c->object.flags & BOUNDARY))
> +				continue;
> +			register_shallow(c->object.oid.hash);
> +			packet_write(1, "shallow %s",
> +				     oid_to_hex(&c->object.oid));
> +		}

Show 5 quoted lines

>  ...
> _But_, the client is not prepared to handle this. We send "shallow"
> lines that it is not expecting, since it did not ask for any depth. So I
> think this logic would have to kick in only when the client tells us to
> do so.

Urgh.. not good. Perhaps a new extension to let the server know the client can handle spontaneous "deepen" commands and only activate new mode when the extension is present?

Show 5 quoted lines

> So what next? I think there's some protocol work here, and I think the
> overall design of that needs to be considered alongside the other
> "deepen" options your topic in pu adds (and of which I'm largely
> ignorant). Does this sufficiently interest you to pick up and roll into
> your other shallow work?

I can pick it up if you are busy with other stuff. But I'm also having a couple other topics at the moment, so it may not progress very fast. -- Duy