git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/9] protocol v2: add server-side "bundle-uri" skeleton

From
Derrick Stolee <derrickstolee@github.com>
Date
Nov 16, 2022, 14:08 UTC
Message-ID
<ca11478b-7b44-3018-04d8-0b84c4f43b56@github.com>
In-Reply-To
<508ba524-09d2-81f1-c5f8-815ab766791a@github.com>
On 11/10/22 8:59 PM, Victoria Dye wrote:
Show 16 quoted lines
> Ævar Arnfjörð Bjarmason via GitGitGadget wrote:
>> +
>> +PROTOCOL for bundle-uri
>> +^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +A `bundle-uri` request takes no arguments, and as noted above does not
>> +currently advertise a capability value. Both may be added in the
>> +future.
>> +
>> +When the client issues a `command=bundle-uri` the response is a list of
> 
> nit: comma after `command=bundle-uri`
> 
> I misread this a couple times dropping the "the", so it read like the
> `command=bundle-uri` was the *response*, not the request. I think the comma
> would help avoid that?
I think this should be
  When the client issues a `command=bundle-uri` request, the response is...
Show 8 quoted lines
>> +key-value pairs provided as packet lines with value `<key>=<value>`. The
>> +meaning of these key-value pairs are provided by the config keys in the
>> +`bundle.*` namespace (see linkgit:git-config[1]).
> 
> What does this ("the meaning of these key-value pairs are provided by the
> config keys...") mean? Are the response keys in the `bundle.*` namespace? Or
> do the client-side `bundle.*` keys provide some kind of translation of what
> the keys mean?

I can elaborate more here, but the intention is that the protocol defines only how these key-value pairs are delivered, and how the client assigns meaning to those values and acts upon them is defined elsewhere.

Show 6 quoted lines
>> +
>> +Clients are still expected to fully parse the line according to the
>> +above format, lines that do not conform to the format SHOULD be
>> +discarded. The user MAY be warned in such a case.
> 
> Why "still" - is there some reason they *wouldn't* parse the response line? 
"still" is not needed here.
Show 5 quoted lines
> Is "the above format" referring to `<key>=<value>` in general, or restricted
> to/guaranteed that the `<key>`'s defined by the `bundle.*` namespace? I'm
> guessing "still expected to fully parse" == "MUST parse" (using
> MUST/SHOULD/MAY nomeclature), it would help to call that out explicitly to
> be consistent with the rest of this doc.
Using MUST simplifies things a lot.
Show 14 quoted lines
>> +
>> +bundle-uri CLIENT AND SERVER EXPECTATIONS
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +URI CONTENTS::
>> +The advertised URIs MUST be in one of two possible formats.
>> ++
>> +The first possible format is a bundle file that `git bundle verify`
> 
> I don't think "format" is the right word to describe this (I'd reserve
> "format" for the literal format of the URI string). Maybe something like
> this?
> 
> 	The advertised URIs MUST contain one of two types of content.
How about...
"The content at the advertised URIs MUST be one of two types" ?
Show 17 quoted lines
> 	The advertised URI may contain a bundle file that `git bundle
> 	verify` would accept...
> 
> 	...
> 	
> 	Alternatively, the advertised URI may provide a plaintext file...
> 
>> +would accept. I.e. they MUST contain one or more reference tips for
>> +use by the client, MUST indicate prerequisites (in any) with standard
>> +"-" prefixes, and MUST indicate their "object-format", if
>> +applicable. Create "*.bundle" files with `git bundle create`.
> 
> The last sentence doesn't add anything that you don't know from the `git
> bundle verify` note in the first doesn't already tell you, and feels like a
> bit of a non-sequitur as a result. Although, it tangentially raises a
> question: do bundle files *have to* have the '.bundle' suffix to pass `git
> bundle verify`? If not, are they expected to when coming from these URIs?
 
The files do not need that extension. This sentence can be removed.
Show 6 quoted lines
>> +WHEN ADVERTISED BUNDLE(S) REQUIRE NO FURTHER NEGOTIATION::
>> +If after issuing `bundle-uri` and `ls-refs`, and getting the header(s)
>> +of the bundle(s) the client finds that the ref tips it wants can be
>> +retrieved entirety from advertised bundle(s), it MAY disconnect. The
> 
> s/entirety/entirely
Thanks.
> And to clarify, by "it MAY disconnect", you mean it may disconnect from the
> main repository server? Or the bundle server? 

The main repository server, since the bundle server is not speaking the Git protocol, but there is definitely room to clarify here.

Show 11 quoted lines
>> +bundle-uri PROTOCOL FEATURES
>> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> +
>> +As noted above the `<key>=<value>` definitions are documented by the
>> +`bundle.*` config namespace.
> 
> Same comment as earlier - this is a confusing way to phrase this. If you
> mean "the keys are part of the `bundle.*` namespace documented in
> linkgit:git-config[1]", I think you can just say that directly. If not, it
> would help to clarify the relationship between the `bundle.*` namespace and
> these keys.
Will do.
Show 12 quoted lines
> Overall, this document is quite thorough, especially with respect to edge
> cases/error handling. I found it a bit confusing at times (at least
> partially due to my unfamiliarity with protocol v2), including some
> potentially ambiguous phrasing or scenarios (especially those in the
> disconnection & error recovery sections) that are difficult to clearly
> describe in generic terms.
> 
> I think some sections (especially "PROTOCOL for bundle-uri" and "bundle-uri
> CLIENT AND SERVER EXPECTATIONS") would benefit from examples of what "good"
> and "bad" request/response values & behaviors look like; they would help
> illustrate some of those more complex situations. The rest of the patch (the
> implementation & tests) looked good to me. 

This is an interesting idea, although the examples of "good" and "bad" are probably best left as the test cases. Looking through the rest of this document, this section is already much more verbose than the others, so I hesitate to add these examples at this point. Perhaps there is room to improve the whole document with such examples as a follow-up.

Thanks, -Stolee

Previous: Victoria DyeNext: Ævar Arnfjörð Bjarmason via GitGitGadget
Message 5 of 87 in “Bundle URIs IV: advertise over protocol v2”
  1. 0/9 Bundle URIs IV: advertise over protocol v2Derrick Stolee via GitGitGadget, Nov 1, 2022
  2. 1/9 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 1, 2022
  3. SZEDER GáborNov 8, 2022
  4. Victoria DyeNov 11, 2022
  5. Derrick StoleeNov 16, 2022
  6. 2/9 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 1, 2022
  7. 3/9 bundle-uri client: add helper for testing serverÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 1, 2022
  8. 4/9 bundle-uri: serve bundle.* keys from configDerrick Stolee via GitGitGadget, Nov 1, 2022
  9. 5/9 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 1, 2022
  10. 6/9 strbuf: reintroduce strbuf_parent_directory()Derrick Stolee via GitGitGadget, Nov 1, 2022
  11. Phillip WoodNov 3, 2022
  12. Ævar Arnfjörð BjarmasonNov 3, 2022
  13. 8/9 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, Nov 1, 2022
  14. 7/9 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, Nov 1, 2022
  15. 9/9 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, Nov 1, 2022
  16. 0/9 Bundle URIs IV: advertise over protocol v2Derrick Stolee via GitGitGadget, Nov 16, 2022
  17. 1/9 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 16, 2022
  18. 3/9 bundle-uri client: add helper for testing serverÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 16, 2022
  19. Victoria DyeNov 29, 2022
  20. Derrick StoleeDec 2, 2022
  21. 2/9 bundle-uri client: add minimal NOOP clientÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 16, 2022
  22. Victoria DyeNov 29, 2022
  23. Derrick StoleeDec 2, 2022
  24. 6/9 strbuf: introduce strbuf_strip_file_from_path()Derrick Stolee via GitGitGadget, Nov 16, 2022
  25. Victoria DyeNov 29, 2022
  26. Derrick StoleeDec 2, 2022
  27. Ævar Arnfjörð BjarmasonDec 2, 2022
  28. Derrick StoleeDec 5, 2022
  29. 5/9 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, Nov 16, 2022
  30. Victoria DyeNov 29, 2022
  31. Derrick StoleeDec 2, 2022
  32. 8/9 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, Nov 16, 2022
  33. Victoria DyeNov 29, 2022
  34. 7/9 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, Nov 16, 2022
  35. Victoria DyeNov 29, 2022
  36. Derrick StoleeDec 2, 2022
  37. 4/9 bundle-uri: serve bundle.* keys from configDerrick Stolee via GitGitGadget, Nov 16, 2022
  38. Victoria DyeNov 29, 2022
  39. 9/9 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, Nov 16, 2022
  40. Victoria DyeNov 29, 2022
  41. Derrick StoleeDec 2, 2022
  42. 00/11 Bundle URIs IV: advertise over protocol v2Derrick Stolee via GitGitGadget, Dec 5, 2022
  43. 10/11 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, Dec 5, 2022
  44. Jeff KingDec 7, 2022
  45. Derrick StoleeDec 7, 2022
  46. Derrick StoleeDec 7, 2022
  47. Jeff KingDec 8, 2022
  48. Jeff KingDec 8, 2022
  49. Derrick StoleeDec 8, 2022
  50. 09/11 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, Dec 5, 2022
  51. Victoria DyeDec 5, 2022
  52. Derrick StoleeDec 7, 2022
  53. 01/11 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 5, 2022
  54. Victoria DyeDec 5, 2022
  55. 07/11 bundle-uri: serve bundle.* keys from configDerrick Stolee via GitGitGadget, Dec 5, 2022
  56. 04/11 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 5, 2022
  57. Victoria DyeDec 5, 2022
  58. Derrick StoleeDec 7, 2022
  59. 02/11 t: create test harness for 'bundle-uri' commandÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 5, 2022
  60. 11/11 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, Dec 5, 2022
  61. 06/11 bundle-uri client: add helper for testing serverÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 5, 2022
  62. Victoria DyeDec 5, 2022
  63. 03/11 clone: request the 'bundle-uri' command when availableÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 5, 2022
  64. 08/11 strbuf: introduce strbuf_strip_file_from_path()Derrick Stolee via GitGitGadget, Dec 5, 2022
  65. Ævar Arnfjörð BjarmasonDec 6, 2022
  66. Ævar Arnfjörð BjarmasonDec 6, 2022
  67. Derrick StoleeDec 7, 2022
  68. Ævar Arnfjörð BjarmasonDec 8, 2022
  69. 05/11 transport: rename got_remote_headsDerrick Stolee via GitGitGadget, Dec 5, 2022
  70. Victoria DyeDec 5, 2022
  71. 00/11 Bundle URIs IV: advertise over protocol v2Derrick Stolee via GitGitGadget, Dec 22, 2022
  72. 01/11 protocol v2: add server-side "bundle-uri" skeletonÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 22, 2022
  73. 02/11 t: create test harness for 'bundle-uri' commandÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 22, 2022
  74. 03/11 clone: request the 'bundle-uri' command when availableÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 22, 2022
  75. 05/11 transport: rename got_remote_headsDerrick Stolee via GitGitGadget, Dec 22, 2022
  76. 04/11 bundle-uri client: add boolean transfer.bundleURI settingÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 22, 2022
  77. 07/11 bundle-uri: serve bundle.* keys from configDerrick Stolee via GitGitGadget, Dec 22, 2022
  78. 06/11 bundle-uri client: add helper for testing serverÆvar Arnfjörð Bjarmason via GitGitGadget, Dec 22, 2022
  79. Jeff KingDec 30, 2022
  80. Derrick StoleeJan 5, 2023
  81. test-bundle-uri: drop unused variablesJeff King, Jan 6, 2023
  82. Derrick StoleeJan 6, 2023
  83. 08/11 strbuf: introduce strbuf_strip_file_from_path()Derrick Stolee via GitGitGadget, Dec 22, 2022
  84. 09/11 bundle-uri: allow relative URLs in bundle listsDerrick Stolee via GitGitGadget, Dec 22, 2022
  85. 10/11 bundle-uri: download bundles from an advertised listDerrick Stolee via GitGitGadget, Dec 22, 2022
  86. 11/11 clone: unbundle the advertised bundlesDerrick Stolee via GitGitGadget, Dec 22, 2022
  87. Junio C HamanoDec 25, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.