git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [Design RFC] Being more defensive about fetching commits in partial clone

From
Jeff King <peff@peff.net>
Date
Nov 29, 2022, 01:31 UTC
Message-ID
<Y4Vg2FkoX/Z8G2Cd@coredump.intra.peff.net>
In-Reply-To
<20221128185320.2735382-1-jonathantanmy@google.com>
On Mon, Nov 28, 2022 at 10:53:19AM -0800, Jonathan Tan wrote:
Show 28 quoted lines
> > In general, I think partial clones tend to know which filters were used
> > to create them, because we save that filter in the config. Would it be
> > reasonable before lazy-fetching to say "I am looking for an object of
> > type X; would my configured filters have skipped such an object?".
> > 
> > Then not only would you get the behavior you want for commits (which are
> > never skipped), but a blob:none clone would not try to lazy-fetch trees
> > (but a tree:depth one would lazy-fetch both trees and blobs).
> > 
> > The gotcha I'd worry about is that the config doesn't necessarily match
> > how the repository was originally created. There is nothing right now
> > saying you cannot partial-clone with one filter, then change the config
> > going forward.
> 
> Thanks for weighing in. In the general case, we indeed do know what kind of
> object we're fetching, so it does make sense to generalize my idea to trees and
> blobs as well. On the other hand, though, besides the issue that the user may
> subsequently change the config, the benefits of distinguishing between a blob
> and tree are not that great, I think - we would fail fast when, say, a tree
> is missing due to object store corruption when trying to check out something
> in a blob:none clone, but the same object store corruption probably would
> mean that commits are missing too, so even if we just did this for commits, we
> would already fail equally as fast. Because of this (and the user being able
> to change the config), it makes sense to me to handle only the commit case, at
> least for now.
> 
> Having said that, this is not forwards incompatible with restricting lazy fetch
> for trees and blobs, so if we see fit later to do that, we can still do it.

Yeah, I think your reasoning is sound. Certainly it seems reasonable to start with commit objects, since we know they can never be filtered, and see how that fares in practice. And then think about handling other object types later (or never; this is really just a "we might catch corruption sooner" nice-to-have, and not a correctness problem in normal use).

-Peff
Previous: Jonathan Tan
Message 4 of 4 in “[Design RFC] Being more defensive about fetching commits in partial clone”
  1. Jonathan TanNov 24, 2022
  2. Jeff KingNov 24, 2022
  3. Jonathan TanNov 28, 2022
  4. Jeff KingNov 29, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.