git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/6] docs: document bundle URI standard

From
Derrick Stolee <derrickstolee@github.com>
Date
Jun 9, 2022, 19:39 UTC
Message-ID
<1ad2435b-f6a5-dc17-1ee1-ffa8c127f04a@github.com>
In-Reply-To
<xmqqk09plnze.fsf@gitster.g>
On 6/9/2022 1:56 PM, Junio C Hamano wrote:
Show 23 quoted lines
> Derrick Stolee <derrickstolee@github.com> writes:
> 
>> Ævar's observation that bundles also add ref tips to the packfile is
>> the key to breaking down this concern: these ref tips give us a way
>> to negotiate the difference between what the client already has
>> (including the bundles downloaded from a bundle provider) and what it
>> wants from the origin Git server. This all happens without any change
>> necessary to the origin Git server.
>>
>> And thus, this bundle URI design came about. It takes all of the best
>> things about the GVFS Cache Server but then layers refs on top of the
>> time-based prefetch packfiles so a normal Git client can do that
>> "catch-up fetch" afterwards.
> 
> Yup.  My observation was that (1) you would need ref tips in some
> way, (2) you are conveying not just "here are the set of bundle
> files", but "this bundle file has these associated attributes" (like
> .timestamp, and .uri to fetch it from), in the table-of-contents the
> clients are expected to obtain anyway, hence (3) you could, but you
> do not need to, use bundle as a way to convey "packfile contents
> plus refs" to the clients (iow, instead you can use packfile and
> then report these refs information in the table-of-contents as more
> "associated attributes" to the items listed in the table-of-contents).
...
Show 18 quoted lines
> Starting from "bundle URI standard" document at the beginning of the
> thread, if we replace all the mentions of "bundle file" with
> "packfile" in it, and then add .positiveRefs and .negativeRefs to
> each "packfile" (renamed from "bundle file") as additional
> "packfile.<id>.*" (renamed from "bundle.<id>.*") attributes, without
> changing anything else, the result would be feature equivalent to
> the original "bundle URI standard", I would think, but without
> having to wrap a packfile in a bundle file?
> 
>> I hope I am going in the right direction here, but I likely
>> misunderstood some of your proposed alternatives.
> 
> I wasn't seriously "proposing" an alternative.  It was just that it
> looked wasteful to go to a separate format (i.e. bundle) when packfiles
> should suffice, as you would be adding extra information that is not
> in bundles via the table-of-contents anyway, and what is given by a
> bundle that is missing in a packfile is only the refs information,
> which should be trivial to add to the table-of-contents.
 
Ok, I've trimmed your latest reply to focus on the main point:
"Why bundles?"

You are right that we could use a table of contents to list the metadata that we need (that is currently stored in the bundle header) except for one case: the single bundle. If the provider wants to skip the table of contents/bundle list and only provide one file that bootstraps clones, then we need something more than just a packfile.

This could be remedied by _requiring_ the table of contents with the ref list, but it does lead to separation of the packfile from the important ref information.

Further, the provider might want to cover a large number of refs, not just the default ref. That would increase the size of the table of contents more than necessary.

With these things in mind, I do still think bundles are a good way to store and share this data.

Thanks, -Stolee

Previous: Ævar Arnfjörð BjarmasonNext: Junio C Hamano
Message 19 of 64 in “bundle URIs: design doc and initial git fetch --bundle-uri implementation”
  1. 0/6 bundle URIs: design doc and initial git fetch --bundle-uri implementationDerrick Stolee via GitGitGadget, Jun 6, 2022
  2. 6/6 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, Jun 6, 2022
  3. 4/6 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, Jun 6, 2022
  4. 5/6 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, Jun 6, 2022
  5. 3/6 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, Jun 6, 2022
  6. 1/6 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Jun 6, 2022
  7. Junio C HamanoJun 6, 2022
  8. Derrick StoleeJun 8, 2022
  9. Junio C HamanoJun 8, 2022
  10. Junio C HamanoJun 8, 2022
  11. Junio C HamanoJun 8, 2022
  12. Derrick StoleeJun 8, 2022
  13. Junio C HamanoJun 7, 2022
  14. Derrick StoleeJun 8, 2022
  15. Junio C HamanoJun 8, 2022
  16. Derrick StoleeJun 9, 2022
  17. Junio C HamanoJun 9, 2022
  18. Ævar Arnfjörð BjarmasonJun 9, 2022
  19. Derrick StoleeJun 9, 2022
  20. Junio C HamanoJun 9, 2022
  21. Derrick StoleeJun 21, 2022
  22. Junio C HamanoJun 21, 2022
  23. Derrick StoleeJun 21, 2022
  24. Junio C HamanoJun 21, 2022
  25. 2/6 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Jun 6, 2022
  26. Junio C HamanoJul 21, 2022
  27. 0/6 bundle URIs: design doc and initial git fetch --bundle-uri implementationDerrick Stolee via GitGitGadget, Jun 29, 2022
  28. 2/6 remote-curl: add 'get' capabilityDerrick Stolee via GitGitGadget, Jun 29, 2022
  29. Josh SteadmonJul 21, 2022
  30. 3/6 bundle-uri: create basic file-copy logicDerrick Stolee via GitGitGadget, Jun 29, 2022
  31. Josh SteadmonJul 21, 2022
  32. Derrick StoleeJul 22, 2022
  33. 1/6 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Jun 29, 2022
  34. SZEDER GáborJul 18, 2022
  35. Matthew John CheethamJul 21, 2022
  36. Derrick StoleeJul 22, 2022
  37. Derrick StoleeJul 22, 2022
  38. Josh SteadmonJul 21, 2022
  39. Derrick StoleeJul 22, 2022
  40. Derrick StoleeJul 22, 2022
  41. 4/6 fetch: add --bundle-uri optionDerrick Stolee via GitGitGadget, Jun 29, 2022
  42. 5/6 bundle-uri: add support for http(s):// and file://Derrick Stolee via GitGitGadget, Jun 29, 2022
  43. 6/6 fetch: add 'refs/bundle/' to log.excludeDecorationDerrick Stolee via GitGitGadget, Jun 29, 2022
  44. Josh SteadmonJul 21, 2022
  45. Derrick StoleeJul 22, 2022
  46. Josh SteadmonJul 21, 2022
  47. Junio C HamanoJul 21, 2022
  48. 0/2 bundle URIs: design doc and initial git fetch --bundle-uri implementationDerrick Stolee via GitGitGadget, Jul 25, 2022
  49. 2/2 bundle-uri: add example bundle organizationDerrick Stolee via GitGitGadget, Jul 25, 2022
  50. Matthew John CheethamAug 4, 2022
  51. Derrick StoleeAug 4, 2022
  52. Ævar Arnfjörð BjarmasonAug 4, 2022
  53. Derrick StoleeAug 5, 2022
  54. 1/2 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Jul 25, 2022
  55. 1/2 docs: document bundle URI standardtenglong.tl, Jul 28, 2022
  56. Derrick StoleeAug 1, 2022
  57. Josh SteadmonJul 25, 2022
  58. 0/2 bundle URIs: design docDerrick Stolee via GitGitGadget, Aug 9, 2022
  59. 1/2 docs: document bundle URI standardDerrick Stolee via GitGitGadget, Aug 9, 2022
  60. Philip OakleyOct 4, 2022
  61. 2/2 bundle-uri: add example bundle organizationDerrick Stolee via GitGitGadget, Aug 9, 2022
  62. Phillip WoodAug 9, 2022
  63. Derrick StoleeAug 9, 2022
  64. Phillip WoodAug 11, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.