Re: [RFC PATCH 0/3] Partial clone: promised blobs (formerly "missing blobs")

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Jul 17, 2017, 18:03 UTC
Message-ID: <20170717180322.GM93855@aiede.mtv.corp.google.com>
In-Reply-To: <C299C45128634A21AF9D65E1B2B52C5B@PhilipOakley>

Hi Philip,

Philip Oakley wrote:

> From: "Jonathan Tan" <jonathantanmy@google.com>

>> These patches are part of a set of patches implementing partial clone,
>> as you can see here:
>>
>> https://github.com/jonathantanmy/git/tree/partialclone

[...]

Show 13 quoted lines

> If I understand correctly, this method doesn't give any direct user
> visibility of missing blobs in the file system. Is that correct?
>
> I was hoping that eventually the various 'on demand' approaches
> would still allow users to continue to work as they go off-line such
> that they can see directly (in the FS) where the missing blobs (and
> trees) are located, so that they can continue to commit new work on
> existing files.
>
> I had felt that some sort of 'gitlink' should be present (huma
> readable) as a place holder for the missing blob/tree. e.g.
> 'gitblob: 1234abcd' (showing the missing oid, jsut like sub-modules
> can do - it's no different really.

That's a reasonable thing to want, but it's a little different from the use cases that partial clone work so far has aimed to support. They are:

 A. Avoiding downloading all blobs (and likely trees as well) that are
    not needed in the current operation (e.g. checkout).  This blends
    well with the sparse checkout feature, which allows the current
    checkout to be fairly small in a large repository.

    GVFS uses a trick that makes it a little easier to widen a sparse
    checkout upon access of a directory.  But the same building blocks
    should work fine with a sparse checkout that has been set up
    explicitly.

 B. Avoiding downloading large blobs, except for those needed in the
    current operation (e.g. checkout).

    When not using sparse checkout, the main benefit out of the box is
    avoiding downloading *historical versions* of large blobs.

It sounds like you are looking for a sort of placeholder outside the sparse checkout area. In a way, that's orthogonal to these patches: even if you have all relevant blobs, you may want to avoid inflating them to check them out and reading them to compare to the index (i.e. the usual benefits of sparse checkout). In a sparse checkout, you still might like to be able to get a listing of files outside the sparse area (which you can get with "git ls-tree") and you may even want to be able to get such a listing with plain "ls" (as with your proposal).

Thanks and hope that helps, Jonathan