git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 13/13] bundle-uri docs: add design notes

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Aug 24, 2021, 21:48 UTC
Message-ID
<YSVpLjAhhS8v2dR7@camp.crustytoothpaste.net>
In-Reply-To
<RFC-patch-13.13-1e657ed27a-20210805T150534Z-avarab@gmail.com>
On 2021-08-05 at 15:07:29, Ævar Arnfjörð Bjarmason wrote:
Show 58 quoted lines
> Add a design doc for the bundle-uri protocol extension to go along
> with the packfile-uri extension added in cd8402e0fd8 (Documentation:
> add Packfile URIs design doc, 2020-06-10).
> 
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  Documentation/technical/bundle-uri.txt  | 119 ++++++++++++++++++++++++
>  Documentation/technical/protocol-v2.txt |   5 +
>  2 files changed, 124 insertions(+)
>  create mode 100644 Documentation/technical/bundle-uri.txt
> 
> diff --git a/Documentation/technical/bundle-uri.txt b/Documentation/technical/bundle-uri.txt
> new file mode 100644
> index 0000000000..5ae9a15eaf
> --- /dev/null
> +++ b/Documentation/technical/bundle-uri.txt
> @@ -0,0 +1,119 @@
> +Bundle URI Design Notes
> +=======================
> +
> +Protocol
> +--------
> +
> +See `bundle-uri` in the link:protocol-v2.html[protocol-v2]
> +documentation for a discussion of the bundle-uri command, and the
> +expectations of clients and servers.
> +
> +This document is a a more general discussion of how the `bundle-uri`
> +command fits in with the rest of the git ecosystem, its design goals
> +and non-goals, comparison to alternatives etc.
> +
> +Comparison with Packfile URIs
> +-----------------------------
> +
> +There is a similar "Packfile URIs" facility, see the
> +link:packfile-uri.html[packfile-uri] documentation for details.
> +
> +The Packfile URIs facility requires a much closer cooperation between
> +CDN and server than the bundle URI facility.
> +
> +I.e. the server MUST know what objects exist in the packfile URI it's
> +pointing to, as well as its pack checksum. Failure to do so will not
> +only result in a client error (the packfile hash won't match), but
> +even if it got past that would likely result in a corrupt repository
> +with tips pointing to unreachable objects.
> +
> +By comparison the bundle URIs are meant to be a "dumb" solution
> +friendly to e.g. having a weekly cronjob take a snapshot of a git
> +repository, that snapshot being uploaded to a network of FTP mirrors
> +(which may be inconsistent or out of date).
> +
> +The server does not need to know what state the side-channel download
> +is at, because the client will first validate it, and then optionally
> +negotiate with the server using what it discovers there.
> +
> +Using the local `transfer.injectBundleURI` configuration variable (see
> +linkgit:git-config[1]) the `bundle-uri` mechanism doesn't even need
> +the server to support it.

One thing I'm not seeing with this doc that I brought up during the packfile URI discussion is that HTTPS is broken for a decent number of Git users, and for them SSH is the only viable option. This is true for users of certain antivirus programs on Windows, as well as people who have certain corporate proxies in their workplace. For those people, as soon as the server offers a bundle URI, their connection will stop working.

I know that you're probably thinking, "Gee, how often does that happen?" but judging by the number of people on StackOverflow, this is actually very common. The antivirus programs that break Git are actually not uncommon and they are widely deployed on corporate machines, plus the fact that lots of companies sell TLS intercepting proxies, which are almost always broken in this way. Many of these users don't even know what's going on, so they simply lack the knowledge to take any action or ask their network administrator for a fix. For them, HTTPS just doesn't work with Git, while it does for a web browser.

So we will probably want to make this behavior opt-in with a config option for SSH, or just not available for SSH at all, so that we don't magically break users on upgrade who are relying on the SSH protocol not using HTTPS under the hood[0], especially the users who won't even know what's wrong.

[0] I can't tell you how many times users have complained about the Git LFS SSH protocol also using HTTPS implicitly.

-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 17 of 26 in “Add bundle-uri: resumably clones, static "dumb" CDN etc.”
  1. 00/13 Add bundle-uri: resumably clones, static "dumb" CDN etc.Ævar Arnfjörð Bjarmason, Aug 5, 2021
  2. 01/13 serve: add command to advertise bundle URIsÆvar Arnfjörð Bjarmason, Aug 5, 2021
  3. Derrick StoleeAug 10, 2021
  4. Ævar Arnfjörð BjarmasonAug 23, 2021
  5. 02/13 bundle-uri client: add "bundle-uri" parsing + testsÆvar Arnfjörð Bjarmason, Aug 5, 2021
  6. 04/13 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason, Aug 5, 2021
  7. 03/13 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason, Aug 5, 2021
  8. 05/13 bundle-uri client: add "git ls-remote-bundle-uri"Ævar Arnfjörð Bjarmason, Aug 5, 2021
  9. 07/13 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason, Aug 5, 2021
  10. 09/13 fetch-pack: add a deref_without_lazy_fetch_extended()Ævar Arnfjörð Bjarmason, Aug 5, 2021
  11. 06/13 bundle-uri client: add transfer.injectBundleURI supportÆvar Arnfjörð Bjarmason, Aug 5, 2021
  12. 08/13 bundle.h: make "fd" version of read_bundle_header() publicÆvar Arnfjörð Bjarmason, Aug 5, 2021
  13. 10/13 fetch-pack: move --keep=* option filling to a functionÆvar Arnfjörð Bjarmason, Aug 5, 2021
  14. 11/13 index-pack: add --progress-title optionÆvar Arnfjörð Bjarmason, Aug 5, 2021
  15. 12/13 bundle-uri client: support for bundle-uri with "clone"Ævar Arnfjörð Bjarmason, Aug 5, 2021
  16. 13/13 bundle-uri docs: add design notesÆvar Arnfjörð Bjarmason, Aug 5, 2021
  17. brian m. carlsonAug 24, 2021
  18. Ævar Arnfjörð BjarmasonAug 24, 2021
  19. Jonathan NiederAug 6, 2021
  20. Ævar Arnfjörð BjarmasonAug 6, 2021
  21. Jonathan NiederAug 6, 2021
  22. Ævar Arnfjörð BjarmasonAug 7, 2021
  23. Derrick StoleeAug 10, 2021
  24. Ævar Arnfjörð BjarmasonAug 23, 2021
  25. Derrick StoleeAug 24, 2021
  26. Ævar Arnfjörð BjarmasonAug 24, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.