Re: [PATCH v2 4/4] bundle v3: the beginning

From: Duy Nguyen <pclouds@gmail.com>
Date: Jun 1, 2016, 13:37 UTC
Message-ID: <CACsJy8AxPKuDUZzWdaWg8tfcWBhqJBADc=ia-Y8cM+zyJ1NFTQ@mail.gmail.com>
In-Reply-To: <CAP8UFD3jPQFk2deSk5JXC3PTz5yWcvXJ4=Qjam5Qw6P9SrLzFQ@mail.gmail.com>

On Tue, May 31, 2016 at 8:18 PM, Christian Couder <christian.couder@gmail.com> wrote:

Show 39 quoted lines

>>> I wonder if this mechanism could also be used or extended to clone and
>>> fetch an alternate object database.
>>>
>>> In [1], [2] and [3], and this was also discussed during the
>>> Contributor Summit last month, Peff says that he started working on
>>> alternate object database support a long time ago, and that the hard
>>> part is a protocol extension to tell remotes that you can access some
>>> objects in a different way.
>>>
>>> If a Git client would download a "$name.bndl" v3 bundle file that
>>> would have a "data: $URL/alt-odb-$name.odb" extended header, the Git
>>> client would just need to download "$URL/alt-odb-$name.odb" and use
>>> the alternate object database support on this file.
>>
>> What does this file contain exactly? A list of SHA-1 that can be
>> retrieved from this remote/alternate odb?
>
> It would depend on the external odb. Git could support different
> external odb that have different trade-offs.
>
>> I wonder if we could just
>> git-replace for this marking. The replaced content could contain the
>> uri pointing to the alt odb.
>
> Yeah, interesting!
> That's indeed another possibility that might not need the transfer of
> any external odb.
>
> But in this case it might be cleaner to just have a separate ref hierarchy like:
>
> refs/external-odbs/my-ext-odb/<sha1>
>
> instead of using the replace one.
>
> Or maybe:
>
> refs/replace/external-odbs/my-ext-odb/<sha1>
>
> if we really want to use the replace hierarchy.

Yep. replace hierarchy crossed my mind. But then I thought about performance degradation when there are more than one pack (we have to search through them all for every SHA-1) and discarded it because we would need to do the same linear search here. I guess we will most likely have one or two name spaces so it probably won't matter.

Show 5 quoted lines

>> We could optionally contact alt odb to
>> retrieve real content, or just show the replaced/fake data when alt
>> odb is out of reach.
>
> Yeah, I wonder if that really needs the replace mechanism.

Replace mechanism provides good hook point. But it really depends how invasive this remote odb is. If a fake content is enough to avoid breakages up high, git-replace is enough. If you really need to pass remote odb info up so higher levels can do something more fancy, then it's insufficient.

Show 9 quoted lines

> By the way this makes me wonder if we could implement resumable clone
> using some kind of replace ref.
>
> The client while cloning nearly as usual would download one or more
> special replace refs that would points to objects with links to
> download bundles using standard protocols.
> Just after the clone, the client would read these objects and download
> the bundles from these objects.
> And then it would clone from these bundles.

I thought we have settled on resumable clone, just waiting for an implementation :) Doing it your way, you would need to download these special objects too (in a pack?) and come back download some more bundles. It would be more efficient to show the bundle uri early and go download the bundle on the side while you go on to get the addition/smaller pack that contains the rest.

-- 
Duy