git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches)

From
JHJeff Hostetler <git@jeffhostetler.com>
Date
Oct 4, 2017, 13:30 UTC
Message-ID
<6ee48dca-5483-9c7f-4c10-f9ca6fba7239@jeffhostetler.com>
In-Reply-To
<CAGf8dgLu-TeK8KbHv-U+18O+L2TxKcGv5vFFHy38J6a_YXRfew@mail.gmail.com>
On 10/3/2017 7:42 PM, Jonathan Tan wrote:
Show 39 quoted lines
> On Tue, Oct 3, 2017 at 7:39 AM, Jeff Hostetler <git@jeffhostetler.com> wrote:
>>
>> As I see it there are the following major parts to partial clone:
>> 1. How to let git-clone (and later git-fetch) specify the desired
>>     subset of objects that it wants?  (A ref-relative request.)
>> 2. How to let the server and git-pack-objects build that incomplete
>>     packfile?
>> 3. How to remember in the local config that a partial clone (or
>>     fetch) was used and that missing object should be expected?
>> 4. How to dynamically fetch individual missing objects individually?
>>      (Not a ref-relative request.)
>> 5. How to augment the local ODB with partial clone information and
>>     let git-fsck (and friends) perform limited consistency checking?
>> 6. Methods to bulk fetching missing objects (whether in a pre-verb
>>     hook or in unpack-tree)
>> 7. Miscellaneous issues (e.g. fixing places that accidentally cause
>>     a missing object to be fetched that don't really need it).
> 
> Thanks for the enumeration.
> 
>> As was suggested above, I think we should merge our efforts:
>> using my filtering for 1 and 2 and Jonathan's code for 3, 4, and 5.
>> I would need to eliminate the "relax" options in favor of his
>> is_promised() functionality for index-pack and similar.  And omit
>> his blob-max-bytes changes from pack-objects, the protocol and
>> related commands.
>>
>> That should be a good first step.
> 
> This sounds good to me. Jeff Hostetler's filtering (all blobs, blobs
> by size, blobs by sparse checkout specification) is more comprehensive
> than mine, so removing blob-max-bytes from my code is not a problem.
> 
>> We both have thoughts on bulk fetching (mine in pre-verb hooks and
>> his in unpack-tree).  We don't need this immediately, but can wait
>> until the above is working to revisit.
> 
> Agreed.
> 
Thanks.

I'll make a first pass at merging our efforts then and post something shortly.

Jeff
Previous: Jonathan Tan
Message 28 of 28 in “Partial clone (from clone to lazy fetch in 18 patches)”
  1. 00/18 Partial clone (from clone to lazy fetch in 18 patches)Jonathan Tan, Sep 29, 2017
  2. 01/18 fsck: introduce partialclone extensionJonathan Tan, Sep 29, 2017
  3. 04/18 fsck: support promisor objects as CLI argumentJonathan Tan, Sep 29, 2017
  4. 06/18 introduce fetch-object: fetch one promisor objectJonathan Tan, Sep 29, 2017
  5. 07/18 sha1_file: support lazily fetching missing objectsJonathan Tan, Sep 29, 2017
  6. Christian CouderOct 12, 2017
  7. Christian CouderOct 12, 2017
  8. 08/18 rev-list: support termination at promisor objectsJonathan Tan, Sep 29, 2017
  9. 02/18 fsck: support refs pointing to promisor objectsJonathan Tan, Sep 29, 2017
  10. 13/18 fetch: refactor calculation of remote listJonathan Tan, Sep 29, 2017
  11. 17/18 unpack-trees: batch fetching of missing blobsJonathan Tan, Sep 29, 2017
  12. 15/18 clone: support excluding large blobsJonathan Tan, Sep 29, 2017
  13. 14/18 fetch: support excluding large blobsJonathan Tan, Sep 29, 2017
  14. 16/18 clone: configure blobmaxbytes in created reposJonathan Tan, Sep 29, 2017
  15. 18/18 fetch-pack: restore save_commit_buffer after useJonathan Tan, Sep 29, 2017
  16. 03/18 fsck: support referenced promisor objectsJonathan Tan, Sep 29, 2017
  17. 12/18 fetch-pack: support excluding large blobsJonathan Tan, Sep 29, 2017
  18. 05/18 index-pack: refactor writing of .keep filesJonathan Tan, Sep 29, 2017
  19. 09/18 gc: do not repack promisor packfilesJonathan Tan, Sep 29, 2017
  20. 10/18 pack-objects: rename want_.* to ignore_.*Jonathan Tan, Sep 29, 2017
  21. 11/18 pack-objects: support --blob-max-bytesJonathan Tan, Sep 29, 2017
  22. Johannes SchindelinSep 29, 2017
  23. Junio C HamanoOct 2, 2017
  24. Christian CouderOct 3, 2017
  25. Junio C HamanoOct 3, 2017
  26. Jeff HostetlerOct 3, 2017
  27. Jonathan TanOct 3, 2017
  28. Jeff HostetlerOct 4, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.