Re: [PATCH v3 2/2] bundle-uri: add example bundle organization

From: Derrick Stolee <derrickstolee@github.com>
Date: Aug 5, 2022, 18:29 UTC
Message-ID: <0d31cca1-ee4c-e33f-fdff-80a85bce7981@github.com>
In-Reply-To: <220804.86y1w3sozy.gmgdl@evledraar.gmail.com>

On 8/4/2022 4:29 PM, Ævar Arnfjörð Bjarmason wrote:

> 
> On Thu, Aug 04 2022, Derrick Stolee wrote:
> 
>> On 8/4/2022 12:09 PM, Matthew John Cheetham wrote:

Show 26 quoted lines

>>> Just to confirm, in this example the origin server advertises a single
>>> URL (over v2 protocol) that points to this example "list of lists"?
>>
>> No, here the origin server provides the list of lists using the 'bundle-uri'
>> protocol v2 command. Using the config file format was an unfortunate choice
>> on my part because that actually uses "key=value" lines.
>>
>> This could be more clear by using that format:
>>
>>   bundle.version=1
>>   bundle.mode=any
>>   bundle.eastus.uri=https://eastus.example.com/<domain>/<org>/<repo>
>>   bundle.europe.uri=https://europe.example.com/<domain>/<org>/<repo>
>>   bundle.apac.uri=https://apac.example.com/<domain>/<org>/<repo>
> 
> [I've tried to stay away from the bundle-uri topic for a while, to give
> others some space to comment]
> 
> On it generally: Your CL goes into some of the saga of it, but briefly
> the design I put forward initially assumed that these sort of things
> would be offloaded to other protocols.
> 
> So, just to take an example of a prominent URL from your "From"
> address. AFAICT there isn't a eastus.api.github.com, or
> europe.api.github.com, instead it just uses DNS load-balancing for
> api.github.com.

...

>  We've had some back & fourths on that before. You clearly think this
> sort of thing is needed in (some version of) a bundle-uri. I don't
> really see why. This sort of load spreading by different DNS naming
> hasn't been common in serious production use for a decade or two.

The difference in our proposals is that yours _requires_ using DNS load-balancing while mine does not require it.

The places where I have deviated from your design are almost entirely because your design forces certain things on the side of the bundle provider that I found unacceptable requirements.

Show 15 quoted lines

> But let's leave that aside, and other things I think we've had diverging
> ideas about before (e.g. your spec's explicit cache management, which I
> imagined offloading to standard HTTP features).
> 
> I do think that:
> 
> 1) This proposed version would be much stronger if it generally tried to
>    justify the features it's putting forward. E.g. just in this case
>    (but it applies more generally) it seems to be taken as a given that
>    {eastus,europe,apac}.<domain> etc. is the natural way to do that sort
>    of load-balancing.
> 
>    But the spec doesn't really go into it. Why would someone use that
>    instead of setting up GeoDNS (or similar), why does it need to be in
>    git's protocol, and not in DNS?

Again, just because you _could_ do something using another technology doesn't mean that we should require all users to do that to solve their problems.

Everything is about trade-offs, and one of the most important things that your proposal lacked was the ability to provide a bundle list that updates independently of the origin Git server. That requires that the Git server can advertise a location that contains a bundle list. Once we have a way to parse a bundle list outside of the Git protocol and a way to advertise bundle lists in the Git protocol, it is incredibly easy to provide this ability to advertise multiple geographies in a list of URIs. Here, the complexity of providing the flexibility is small compared to the flexibility available to users.

Show 6 quoted lines

> 2) I'd really like it clarified in the doc whether it considers itself a
>    "living document" amenable to change, or a "spec" that we have to
>    stick to.
> 
>    I'd like it to be the former, and I think it should be prominently
>    noted there (e.g. that it's "EXPERIMENTAL" or whatever).

The living document portion is because the code is not implemented at the same time as this design document is merged. Having actual code solidifies the spec a lot more than a design document does on its own. Having the code implemented provides a way to test things, but also a released version of the spec becomes something that others depend on.

I'd be fine starting off with "EXPERIMENTAL" just like the sparse- checkout builtin has, but the longer the code is out in the wild, the less that EXPERIMENTAL tag offers us to shift things.

Thanks, -Stolee