git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 3/4] fsck: support referenced lazy objects

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 29, 2017, 16:04 UTC
Message-ID
<xmqq379fkz4x.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<1d841f0e4188f0ae9fa3c238c58d3e1b5fb232e8.1501111615.git.jonathantanmy@google.com>
Jonathan Tan <jonathantanmy@google.com> writes:
> Teach fsck to not treat missing objects indirectly pointed to by refs as
> an error when extensions.lazyobject is set.

I forgot to mention a potential flaw in this approach in my previous message.

If you are a pure sightseer, then this is perfectly fine. The object store in your local Git client working in that mode is purely a cache, lazily populated while browsing the object store backed by the source of what lazy-object "hook" talks with. As long as that cache does not give us a corrupt object, we are OK, because missing objects do not matter.

But once you start using the repository as more than a sightseer, you will have objects that only exist in your local "cache" and are not yet in that backing store behind the lazy-object "hook". You need to notice it when any of them goes corrupt or missing, or your next "git push" to send them over to a remote location will fail by definition because you are the only one with these objects.

If we had the "promise" thing, then we could say that it is OK if traversal terminated at a "promised but not fetched yet" boundary, but we cannot afford the "promise", and more importantly, I do not think "promise" has to be the only approach to ensure that the objects that exist only in the local repository are all connected.

For example, if we know that the remote 'origin' is the actual backing store lazy-object "hook" talks with, a validation rule to ensure that we haven't lost any local commit is to ensure that a traversal from our local branch tips down to remote-tracking branches taken from 'origin' must not hit _any_ missing commit.

That covers only the commit objects. I do not know offhand if we can and how we extend this concept to protect the tags, trees and blobs we have locally generated and haven't pushed out, but you and Ben hopefully can come up with ways to cover them.

Previous: Jonathan TanNext: Jonathan Tan
Message 15 of 40 in “Some patches for fsck for missing objects”
  1. 0/4 Some patches for fsck for missing objectsJonathan Tan, Jul 26, 2017
  2. 1/4 environment, fsck: introduce lazyobject extensionJonathan Tan, Jul 26, 2017
  3. Junio C HamanoJul 27, 2017
  4. Ben PeartJul 28, 2017
  5. Jonathan TanJul 28, 2017
  6. Junio C HamanoJul 29, 2017
  7. 2/4 fsck: support refs pointing to lazy objectsJonathan Tan, Jul 26, 2017
  8. Junio C HamanoJul 27, 2017
  9. Jonathan TanJul 27, 2017
  10. Ben PeartJul 28, 2017
  11. tests: ensure fsck fails on corrupt packfilesJonathan Tan, Jul 28, 2017
  12. 3/4 fsck: support referenced lazy objectsJonathan Tan, Jul 26, 2017
  13. Junio C HamanoJul 27, 2017
  14. Jonathan TanJul 27, 2017
  15. Junio C HamanoJul 29, 2017
  16. 4/4 fsck: support lazy objects as CLI argumentJonathan Tan, Jul 26, 2017
  17. brian m. carlsonJul 26, 2017
  18. Stefan BellerJul 27, 2017
  19. Jonathan TanJul 27, 2017
  20. Ben PeartJul 28, 2017
  21. 0/5 Fsck for lazy objects, and (now) actual invocation of loaderJonathan Tan, Jul 31, 2017
  22. 1/5 environment, fsck: introduce lazyobject extensionJonathan Tan, Jul 31, 2017
  23. 2/5 fsck: support refs pointing to lazy objectsJonathan Tan, Jul 31, 2017
  24. 4/5 fsck: support lazy objects as CLI argumentJonathan Tan, Jul 31, 2017
  25. 3/5 fsck: support referenced lazy objectsJonathan Tan, Jul 31, 2017
  26. 5/5 sha1_file: support loading lazy objectsJonathan Tan, Jul 31, 2017
  27. Junio C HamanoJul 31, 2017
  28. Ben PeartAug 8, 2017
  29. Junio C HamanoJul 31, 2017
  30. Jonathan TanJul 31, 2017
  31. Junio C HamanoAug 1, 2017
  32. Jonathan NiederAug 1, 2017
  33. Junio C HamanoAug 1, 2017
  34. Jonathan TanAug 2, 2017
  35. Junio C HamanoAug 2, 2017
  36. Jonathan NiederAug 2, 2017
  37. Junio C HamanoAug 2, 2017
  38. Jonathan NiederAug 2, 2017
  39. Jonathan TanAug 3, 2017
  40. Ben PeartAug 8, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.