Re: RFC: Design and code of partial clones (now, missing commits and trees OK)

From: Jonathan Tan <jonathantanmy@google.com>
Date: Sep 29, 2017, 00:53 UTC
Message-ID: <20170928175352.9a490564d47172568ea2f416@google.com>
In-Reply-To: <20170915134343.3814dc38@twelve2.svl.corp.google.com>

On Fri, 15 Sep 2017 13:43:43 -0700 Jonathan Tan <jonathantanmy@google.com> wrote:

Show 9 quoted lines

> For those interested in partial clones and/or missing objects in repos,
> I've updated my original partialclone patches to not require an explicit
> list of promises. Fetch/clone still only permits exclusion of blobs, but
> the infrastructure is there for a local repo to support missing trees
> and commits as well.
> 
> They can be found here:
> 
> https://github.com/jonathantanmy/git/tree/partialclone2

I've pushed a new version:

https://github.com/jonathantanmy/git/tree/partialclone3

Besides some small changes as requested by comments on the GitHub
repository, I've also updated the code to do the following:
 - clarified terminology - in particular, I've tried to avoid
   "promised", only using "promisor object" to denote objects that the
   local repo knows that the promisor remote has, whether the local repo
   has it or not
 - restored bulk checkout functionality (so now you can clone with
   --blob-max-bytes=0)
 - a fix to fetch-pack to restore a global flag after it uses it, so
   commands like "git log -S" still work (but to test this, I used
   --blob-max-bytes=200000 with the Git repository, because batch
   fetching is not implemented for commands like these)

In its current form, the code is already useful for situations like:
 - a large repository with many blobs in which the client only needs to
   checkout, at most, and does not need to search through history
   locally, and
 - a repository with a few large blobs, where the client still can
   search through history as long as the client is online