git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 06/10] rev-list: add --allow-partial option to relax connectivity checks

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Mar 8, 2017, 20:10 UTC
Message-ID
<2cc503b4-f6cf-336a-bc3f-f44b1d187eaf@jeffhostetler.com>
In-Reply-To
<xmqqd1dr38f0.fsf@gitster.mtv.corp.google.com>
On 3/8/2017 1:55 PM, Junio C Hamano wrote:
Show 12 quoted lines
> Jeff Hostetler <jeffhost@microsoft.com> writes:
>
>> From: Jeff Hostetler <git@jeffhostetler.com>
>>
>> Teach rev-list to optionally not complain when there are missing
>> blobs.  This is for use following a partial clone or fetch when
>> the server omitted certain blobs.
>
> This makes it impossible to tell from objects missing by design
> (because we did an --partial-by-size clone earlier, expecting we can
> later fetch from elsewhere when necessary) and objects inaccessible
> by accident (because you have a repository corruption), no?

Right. It will effectively neuter several commands like index-pack, gc, and fsck WRT missing blobs.

Show 12 quoted lines
> Even though I do very much like the basic "high level" premise to
> omit often useless large blobs that are buried deep in the history
> we would not necessarily need from the initial cloning and
> subsequent fetches, I find it somewhat disturbing that the code
> "Assume"s that any missing blob is due to an previous partial clone.
> Adding this option smells like telling the users that they are not
> supposed to run "git fsck" because a partially cloned repository is
> inherently a corrupt repository.
>
> Can't we do a bit better?  If we want to make the world safer again,
> what additional complexity is required to allow us to tell the
> "missing by design" and "corrupt repository" apart?

I'm open to suggestions here. It would be nice to extend the fetch-pack/upload-pack protocol to return a list of the SHAa (and maybe the sizes) of the omitted blobs, so that a partial clone or fetch would still be able to be integrity checked.

Jeff
Previous: Junio C HamanoNext: Jeff King
Message 20 of 31 in “RFC Partial Clone and Fetch”
  1. 00/10 RFC Partial Clone and FetchJeff Hostetler, Mar 8, 2017
  2. 02/10 pack-objects: add --partial-by-size=n --partial-specialJeff Hostetler, Mar 8, 2017
  3. Junio C HamanoMar 8, 2017
  4. Jeff HostetlerMar 8, 2017
  5. Jeff KingMar 9, 2017
  6. Brandon WilliamsMar 10, 2017
  7. Jeff KingMar 10, 2017
  8. Junio C HamanoMar 10, 2017
  9. Jeff KingMar 10, 2017
  10. Jeff KingMar 9, 2017
  11. Jeff HostetlerMar 9, 2017
  12. 07/10 index-pack: add --allow-partial option to relax blob existence checksJeff Hostetler, Mar 8, 2017
  13. 05/10 fetch-pack: add partial-by-size and partial-specialJeff Hostetler, Mar 8, 2017
  14. 04/10 upload-pack: add partial (sparse) fetchJeff Hostetler, Mar 8, 2017
  15. Jeff KingMar 9, 2017
  16. Jeff HostetlerMar 9, 2017
  17. Jeff KingMar 9, 2017
  18. 06/10 rev-list: add --allow-partial option to relax connectivity checksJeff Hostetler, Mar 8, 2017
  19. Junio C HamanoMar 8, 2017
  20. Jeff HostetlerMar 8, 2017
  21. Jeff KingMar 9, 2017
  22. Jeff HostetlerMar 9, 2017
  23. 03/10 pack-objects: test for --partial-by-size --partial-specialJeff Hostetler, Mar 8, 2017
  24. Jeff KingMar 9, 2017
  25. Johannes SixtMar 9, 2017
  26. 08/10 fetch: add partial-by-size and partial-special argumentsJeff Hostetler, Mar 8, 2017
  27. 01/10 pack-objects: eat CR in addition to LF after fgets.Jeff Hostetler, Mar 8, 2017
  28. Jeff KingMar 9, 2017
  29. Jeff HostetlerMar 9, 2017
  30. 09/10 clone: add partial-by-size and partial-special argumentsJeff Hostetler, Mar 8, 2017
  31. 10/10 ls-partial: created command to list missing blobsJeff Hostetler, Mar 8, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.