Re: [PATCH] [RFC] list-objects-filter: introduce new filter sparse:buffer=<spec>

From: Jeff King <peff@peff.net>
Date: Aug 14, 2022, 06:54 UTC
Message-ID: <YvicFCS45cPp9khe@coredump.intra.peff.net>
In-Reply-To: <CAOLTT8QfmVXHDVfWZf0U73PXOQ--ZUmm3N=a60fv_Wo_5UnqmA@mail.gmail.com>

On Fri, Aug 12, 2022 at 11:49:06PM +0800, ZheNing Hu wrote:

Show 12 quoted lines

> > ...I agree with this. It is nice to put more power in the hands of the
> > clients, but we have to balance that with other issues like server
> > resource use. The approach so far has been to implement the simplest and
> > most efficient operations at the client-server level, and then have the
> > client build local features on top of that. So in this case, probably
> > requesting that _no_ trees are sent in the initial clone, and then
> > faulting them in as the client explores the tree using its own local
> > sparse definition. And I think that mostly works now.
> 
> Agree. But we have to fetch these blobs one by one after partial clone,
> why not reduce some extra network overhead If we can get those blobs
> that are *most* needed in the first partial clone, right?

Ideally we wouldn't be getting them one-by-one, but rather batching them. I'm not sure of the current state of the code. But we should at least be able to batch by tree depth.

You're right that we'll never be able to be as efficient as a server to whom we can say "I care about these paths", though.

-Peff