git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 19, 2017, 05:51 UTC
Message-ID
<xmqq8thbqlqf.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<20170915134343.3814dc38@twelve2.svl.corp.google.com>
Jonathan Tan <jonathantanmy@google.com> writes:
Show 48 quoted lines
> For those interested in partial clones and/or missing objects in repos,
> I've updated my original partialclone patches to not require an explicit
> list of promises. Fetch/clone still only permits exclusion of blobs, but
> the infrastructure is there for a local repo to support missing trees
> and commits as well.
> ...
> Demo
> ====
>
> Obtain a repository.
>
>     $ make prefix=$HOME/local install
>     $ cd $HOME/tmp
>     $ git clone https://github.com/git/git
>
> Make it advertise the new feature and allow requests for arbitrary blobs.
>
>     $ git -C git config uploadpack.advertiseblobmaxbytes 1
>     $ git -C git config uploadpack.allowanysha1inwant 1
>
> Perform the partial clone and check that it is indeed smaller. Specify
> "file://" in order to test the partial clone mechanism. (If not, Git will
> perform a local clone, which unselectively copies every object.)
>
>     $ git clone --blob-max-bytes=100000 "file://$(pwd)/git" git2
>     $ git clone "file://$(pwd)/git" git3
>     $ du -sh git2 git3
>     116M	git2
>     129M	git3
>
> Observe that the new repo is automatically configured to fetch missing objects
> from the original repo. Subsequent fetches will also be partial.
>
>     $ cat git2/.git/config
>     [core]
>     	repositoryformatversion = 1
>     	filemode = true
>     	bare = false
>     	logallrefupdates = true
>     [remote "origin"]
>     	url = [snip]
>     	fetch = +refs/heads/*:refs/remotes/origin/*
>     	blobmaxbytes = 100000
>     [extensions]
>     	partialclone = origin
>     [branch "master"]
>     	remote = origin
>     	merge = refs/heads/master

The above sequence of events make quite a lot of sense. And the following description of how it is designed (snipped) is clear enough (at least to me) to allow me to say that I quite like it.

Previous: Jonathan TanNext: Jeff Hostetler
Message 2 of 21 in “RFC: Design and code of partial clones (now, missing commits and trees OK)”
  1. Jonathan TanSep 15, 2017
  2. Junio C HamanoSep 19, 2017
  3. Jeff HostetlerSep 21, 2017
  4. Jonathan TanSep 21, 2017
  5. Jeff HostetlerSep 22, 2017
  6. Jonathan TanSep 22, 2017
  7. Michael HaggertySep 26, 2017
  8. Jonathan TanSep 29, 2017
  9. Jeff HostetlerSep 21, 2017
  10. Jonathan TanSep 21, 2017
  11. Jeff HostetlerSep 22, 2017
  12. Jonathan TanSep 22, 2017
  13. Jeff HostetlerSep 26, 2017
  14. Jeff HostetlerSep 21, 2017
  15. Jonathan TanSep 21, 2017
  16. Jeff HostetlerSep 22, 2017
  17. Jonathan TanSep 22, 2017
  18. Jeff HostetlerSep 26, 2017
  19. Jonathan TanSep 26, 2017
  20. Jonathan TanSep 29, 2017
  21. Junio C HamanoSep 29, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.