git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Partial clone design (with connectivity check for locally-created objects)

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 5, 2017, 00:21 UTC
Message-ID
<20170804172137.42f27653@twelve2.svl.corp.google.com>
In-Reply-To
<xmqqtw1nrlpf.fsf@gitster.mtv.corp.google.com>

On Fri, 04 Aug 2017 15:51:08 -0700 Junio C Hamano <gitster@pobox.com> wrote:

Show 26 quoted lines
> Jonathan Tan <jonathantanmy@google.com> writes:
> 
> > "Imported" objects must be in a packfile that has a "<pack name>.remote"
> > file with arbitrary text (similar to the ".keep" file). They come from
> > clones, fetches, and the object loader (see below).
> > ...
> > A "homegrown" object is valid if each object it references:
> >  1. is a "homegrown" object,
> >  2. is an "imported" object, or
> >  3. is referenced by an "imported" object.
> 
> Overall it captures what was discussed, and I think it is a good
> start.
> 
> I doubt you want to treat all fetches/clones the same way as the
> "lazy object" loading, though.  You may be critically rely on the
> corporate central server that will give the objects it "promised"
> when you cloned from it lazily (i.e. it may have given you a commit,
> but not its parents or objects contained in its tree--you still know
> that the parents and the tree and its contents will later be
> available and rely on that fact).  You trust that and build on top,
> so the packfile you obtained when you cloned from such a server
> should count as "imported".  But if you exchanged wip changes with
> your colleages by fetching or pushing peer-to-peer, without the
> corporate central server knowing, you would want to treat objects in
> packs (or loose objects) you obtained that way as "not imported".

That's true. I discussed this with a teammate and we might need to make extensions.lazyObject be the name of the "corporate central server" remote instead, and have a "loader" setting within that remote, so that we can distinguish that objects from this server are "imported" but objects from other servers are not.

The connectivity check shouldn't be slow in this case because fetches are usually onto tips that we have (so we don't hit case 3).

Show 12 quoted lines
> Also I think "imported" vs "homegrown" may be a bit misnomer; the
> idea to split objects into two camps sounds like a good idea, and
> "imported" probably is an OK name to use for the category that is a
> group of objects to which you know/trust are backed by your lazy
> loader.  But the other one does not have to be "home"-grown.
> 
> Well, the names are not that important, but I think the line between
> the two classes should not be "everything that came from clone and
> fetch is imported", which is a more important point I am trying to
> make.
> 
> Thanks.

Maybe "imported" vs "non-imported" would be better. I agree that the objects in the non-"imported" group could still be obtained from elsewhere.

Thanks for your comments.
Previous: Junio C HamanoNext: Ben Peart
Message 3 of 18 in “Partial clone design (with connectivity check for locally-created objects)”
  1. Jonathan TanAug 4, 2017
  2. Junio C HamanoAug 4, 2017
  3. Jonathan TanAug 5, 2017
  4. Ben PeartAug 7, 2017
  5. Jonathan NiederAug 7, 2017
  6. Ben PeartAug 8, 2017
  7. Junio C HamanoAug 7, 2017
  8. Ben PeartAug 8, 2017
  9. Jonathan NiederAug 8, 2017
  10. Jonathan TanAug 7, 2017
  11. Updated "imported object" designJonathan Tan, Aug 16, 2017
  12. Junio C HamanoAug 16, 2017
  13. Jonathan TanAug 16, 2017
  14. Ben PeartAug 17, 2017
  15. Jonathan TanAug 17, 2017
  16. Ben PeartAug 18, 2017
  17. Jonathan TanAug 18, 2017
  18. Ben PeartAug 17, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.