git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 00/25] [RFC] Bundle URIs

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Mar 4, 2022, 14:49 UTC
Message-ID
<220304.86a6e5g44z.gmgdl@evledraar.gmail.com>
In-Reply-To
<a6981d6e-16b0-b0e1-a94d-a87ec20871bd@github.com>
On Fri, Mar 04 2022, Derrick Stolee wrote:
Show 54 quoted lines
> On 2/24/2022 9:11 AM, Derrick Stolee wrote:
>> On 2/23/2022 5:17 PM, Ævar Arnfjörð Bjarmason wrote:
>>>
>>> On Wed, Feb 23 2022, Derrick Stolee via GitGitGadget wrote:
>
>>>> There have been several suggestions to improve Git clone speeds and
>>>> reliability by supplementing the Git protocol with static content. The
>>>> Packfile URI [0] feature lets the Git response include URIs that point to
>>>> packfiles that the client must download to complete the request.
>>>>
>>>> Last year, Ævar suggested using bundles instead of packfiles [1] [2]. This
>>>> design has the same benefits to the packfile URI feature because it offloads
>>>> most object downloads to static content fetches. The main advantage over
>>>> packfile URIs is that the remote Git server does not need to know what is in
>>>> those bundles. The Git client tells the server what it downloaded during the
>>>> fetch negotiation afterwards. This includes any chance that the client did
>>>> not have access to those bundles or otherwise failed to access them. I
>>>> agreed that this was a much more desirable way to serve static content, but
>>>> had concerns about the flexibility of that design [3]. I have not heard more
>>>> on the topic since October, so I started investigating this idea myself in
>>>> December, resulting in this RFC.
>>>
>>> This timing is both quite fortunate & unfortunate for me, since I'd been
>>> blocked / waiting on various things until very recently to submit a
>>> non-RFC re-roll of (a larger version of) that series you mentioned from
>>> October.
>>>
>>> I guess the good news is that we'll have at least one guaranteed very
>>> interested reviewer for each other's patches, and that the design that
>>> makes it into git.git in the end will definitely be well hashed out :)
>>>
>>> I won't be able to review this in any detail right at this hour, but
>>> will be doing so. I'd also like to submit what I've got in some form
>>> soon for hashing the two out.
>>>
>>> It will be some 50+ patches on the ML in total though related to this
>>> topic, so I think the two of us coming up with some way to manage all of
>>> that for both ourselves & others would be nice. Perhaps we could also
>>> have an off-list (video) chat in real time to clarify/discuss various
>>> thing related to this.
>> 
>> I look forward to seeing your full implementation. There are many things
>> about your RFC that left me confused and not fully understanding your
>> vision.
>
> I am genuinely curious to see your full implementation of bundle URIs.
> I've been having trouble joining the Git IRC chats, but I saw from the
> logs that you are working on getting patches together.
>
> Do you have an expected timeline on that progress?
>
> I would like to move forward in getting bundle URIs submitted as a full
> feature, but it is important to see your intended design so we can take
> the best parts of both to create a version that satisfies us both.

Hi. Very sorry about the late reply. I relly meant to have something read to send at the end of this week, but it's been a bit of a whirlwhind of random things coming up & distracting me too much. Sorry.

I'm also totally on board with that goal, are you OK with waiting until the end of next week at the latest?

Also, as noted in the upthread <220224.86czjdb22l.gmgdl@evledraar.gmail.com> it might be useful to chat in a more voice/video medium in parallel (maybe mid-next-week) about the high-level ideas & to get a feel for our goals, conflicts etc. Doing that over very long E-Mail exchanges (and the fault of "long" there is mostly on my side:) can be a bit harder...

Previous: Derrick StoleeNext: Derrick Stolee
Message 38 of 46 in “[RFC] Bundle URIs”
  1. 00/25 [RFC] Bundle URIsDerrick Stolee via GitGitGadget, Feb 23, 2022
  2. 02/25 bundle: alphabetize subcommands betterDerrick Stolee via GitGitGadget, Feb 23, 2022
  3. Ævar Arnfjörð BjarmasonMar 11, 2022
  4. 05/25 remote: allow relative_url() to return an absolute urlDerrick Stolee via GitGitGadget, Feb 23, 2022
  5. 03/25 dir: extract starts_with_dot[_dot]_slash()Derrick Stolee via GitGitGadget, Feb 23, 2022
  6. 04/25 remote: move relative_url()Derrick Stolee via GitGitGadget, Feb 23, 2022
  7. 01/25 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Feb 23, 2022
  8. Elijah NewrenMar 2, 2022
  9. Derrick StoleeMar 2, 2022
  10. Elijah NewrenMar 3, 2022
  11. Derrick StoleeMar 3, 2022
  12. 06/25 http: make http_get_file() externalDerrick Stolee via GitGitGadget, Feb 23, 2022
  13. 07/25 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  14. 08/25 bundle: implement 'fetch' command for direct bundlesDerrick Stolee via GitGitGadget, Feb 23, 2022
  15. 09/25 bundle: parse table of contents during 'fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  16. 11/25 bundle: allow relative URLs in table of contentsDerrick Stolee via GitGitGadget, Feb 23, 2022
  17. Ævar Arnfjörð BjarmasonMar 11, 2022
  18. 10/25 bundle: add --filter option to 'fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  19. Ævar Arnfjörð BjarmasonMar 11, 2022
  20. 12/25 bundle: make it easy to call 'git bundle fetch'Derrick Stolee via GitGitGadget, Feb 23, 2022
  21. 13/25 clone: add --bundle-uri optionDerrick Stolee via GitGitGadget, Feb 23, 2022
  22. 14/25 clone: --bundle-uri cannot be combined with --depthDerrick Stolee via GitGitGadget, Feb 23, 2022
  23. 15/25 config: add git_config_get_timestamp()Derrick Stolee via GitGitGadget, Feb 23, 2022
  24. Ævar Arnfjörð BjarmasonMar 11, 2022
  25. 18/25 connect.c: refactor sending of agent & object-formatÆvar Arnfjörð Bjarmason via GitGitGadget, Feb 23, 2022
  26. 17/25 fetch: fetch bundles before fetching original dataDerrick Stolee via GitGitGadget, Feb 23, 2022
  27. 16/25 bundle: only fetch bundles if timestamp is newDerrick Stolee via GitGitGadget, Feb 23, 2022
  28. 19/25 protocol-caps: implement cap_features()Derrick Stolee via GitGitGadget, Feb 23, 2022
  29. 21/25 serve: advertise 'features' when config existsDerrick Stolee via GitGitGadget, Feb 23, 2022
  30. 20/25 serve: understand but do not advertise 'features' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  31. 22/25 connect: implement get_recommended_features()Derrick Stolee via GitGitGadget, Feb 23, 2022
  32. 23/25 transport: add connections for 'features' capabilityDerrick Stolee via GitGitGadget, Feb 23, 2022
  33. 24/25 clone: use server-recommended bundle URIDerrick Stolee via GitGitGadget, Feb 23, 2022
  34. 25/25 t5601: basic bundle URI testDerrick Stolee via GitGitGadget, Feb 23, 2022
  35. Ævar Arnfjörð BjarmasonFeb 23, 2022
  36. Derrick StoleeFeb 24, 2022
  37. Derrick StoleeMar 4, 2022
  38. Ævar Arnfjörð BjarmasonMar 4, 2022
  39. Derrick StoleeMar 4, 2022
  40. Derrick StoleeMar 8, 2022
  41. Johannes SchindelinMar 10, 2022
  42. Derrick StoleeApr 7, 2022
  43. Ævar Arnfjörð BjarmasonApr 8, 2022
  44. Derrick StoleeApr 8, 2022
  45. Junio C HamanoApr 8, 2022
  46. Teng LongMar 8, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.