Re: [PATCH 1/6] docs: document bundle URI standard

From: Junio C Hamano <gitster@pobox.com>
Date: Jun 8, 2022, 21:01 UTC
Message-ID: <xmqq5ylarhsg.fsf@gitster.g>
In-Reply-To: <48e722dc-f860-f7a6-36d0-b0106087aef4@github.com>

Derrick Stolee <derrickstolee@github.com> writes:

Show 9 quoted lines

>> Is it an error if bundle.<id>.list and the contents disagree?
>
> I think this can be flexible. It is not difficult to treat the .list
> value as advisory, so we can ignore disagreements. If we start making
> decisions that hinge on the value of .list, then we can start treating
> it as an error.
>
> Or: maybe the .list value is so "advisory" that it is useless, and I
> should just drop it from the schema.

I was wondering about the same thing ;-)

Show 25 quoted lines

>>> +Related Work: GVFS Cache Servers
>>> +--------------------------------
>>> ...
>>> +During a `git fetch`, a hook requests the prefetch endpoint using the
>>> +most-recent timestamp from a previously-downloaded prefetch packfile.
>>> +Only the list of packfiles with later timestamps are downloaded.
>> 
>> That sounds quite straight-forward.  Do you envision that their
>> incremental snapshot packfile chains can somehow be shared with the
>> bundle URI implementations?  Doesn't it make it more cumbersome that
>> this proposal uses the bundles as the encapsulation format, rather
>> than packfiles?  As you are sending extra pieces of information on
>> top of the payload in the form of table-of-contents already, I
>> wonder if bundle.<id>.uri should point at a bare packfile (instead
>> of a bundle), while multi-valued bundle.<id>.prerequisite give the
>> prerequisite objects?  The machinery that is already generating the
>> prefetch packfiles already know which packfile has what
>> prerequisites in it, so it rather looks simpler if the solution did
>> not involve bundles.
>
> The prefetch packfiles could be replaced with bundle URIs, if desired.
> ...
> So in this world, the bundle URIs could be used as a replacement for
> downloading these prefetch packfiles (bundles with filter=blob:none)
> but the bundled refs become useless to the client.

That's all understandable, but what I was alluding to was to go in the other direction. Since "bundle URI" thing is new, while the GVFS Cache Servers already use these prefetch packfiles, it could be beneficial if the new thing can be done without bundle files and instead with packfiles. You are already generating these snapshot packfiles for GVFS Cache Servers. So if we can reuse them to also serve "git clone" and "git fetch" clients, we can do so without doubling the disk footprint.

Even if you scrapped the "bundle URI" and rebuilt it as the "packfile URI" mechanism, the only change you need is to make positive and negative refs, which were available in bundle files but not stored in packfiles, available as a part of the metadata for each packfile, no? You'd be keeping track of associated metadata (like the .timestamp and .requires fields) in addition to what is in the bundle anyway, so...