git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/3] protocol v2 and hidden refs

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Dec 15, 2018, 19:53 UTC
Message-ID
<87d0q21s8w.fsf@evledraar.gmail.com>
In-Reply-To
<20181214083507.GB11777@sigill.intra.peff.net>
On Fri, Dec 14 2018, Jeff King wrote:
Show 70 quoted lines
> On Thu, Dec 13, 2018 at 11:53:05AM -0800, Jonathan Tan wrote:
>
>> >     I don't know if there's a good solution. I tried running the whole
>> >     test suite with v2 as the default. It does find this bug, but it has
>> >     a bunch of other problems (notably fetch-pack won't run as v2, but
>> >     some other tests I think also depend on v0's reachability rules,
>> >     which v2 is documented not to enforce).
>>
>> I think Aevar's patches (sent after you wrote this) is a good start, and
>> I have started looking at it too.
>
> Yeah, I'm excited to see it working with fetch-pack, as the current
> behavior is to complain if you've tried to enable v2 config:
>
>   $ git config protocol.version 2
>   $ git fetch-pack git://github.com/git/git
>   fatal: support for protocol v2 not implemented yet
>
> I haven't actually run into it in the real world, but somebody might if
> they have scripted around fetch-pack and are experimenting with v2. A
> much friendlier behavior would be falling back to v1, but actually
> supporting v2 is better still. :)
>
>> >   - The "serve" command is funky, because it has no concept of whether
>> >     the "ls-refs" is for fetching or pushing. Is git-serve even a thing
>> >     that we want to support going forward?  I know part of the original
>> >     v2 conception was that one would be able to just connect to
>> >     "git-serve" and do a number of operations. But in practice the v2
>> >     probing requires saying "I'd like to git-upload-pack, and v2 if you
>> >     please". So no client ever calls git-serve.
>> >
>> >     Is this something we plan to eventually move to? Or can it be
>> >     considered a funny vestige of the development? In the latter case, I
>> >     think we should consider removing it.
>>
>> Personally, I lean towards removing it, but there are arguments on both
>> sides. In particular, removing "serve" means that both developers and
>> users of Git need not be concerned with a 3rd endpoint, but preserving
>> "serve" (and planning to migrate away from "upload-pack" and
>> "receive-pack") means that we will only have one endpoint, eliminating
>> confusion about which endpoint to use when making certain requests (when
>> we add requests other than "fetch" and "push").
>
> Yeah, at first glance I like the simplicity of a unified model. But the
> separate fetch/push endpoints have been useful in the past. Separate
> uploadpack/receive.hiderefs that I dealt with here are one form. Another
> is that many people do HTTP access control using the endpoints. For
> example, if I have a repo which is public-read and private-write, the
> config we recommend in git-http-backend(1) is to lock down the
> receive-pack access using webserver config.
>
> If all the webserver sees is "somebody wants to connect to git-serve",
> it doesn't know if it should be challenging them for authentication or
> not. It would have to start peeking into the git-serve conversation to
> see what the client actually wants to do. That's _possible_ to do, but
> it gets pretty awkward with existing webserver tools (whereas matching
> the URI endpoint is pretty easy).
>
> Ditto for locked down ssh sessions like git-shell (or custom solutions
> like gitolite). Right now we can "git-upload-pack is OK on this repo,
> git-receive-pack is not". But blindly running "git serve" would be
> dangerous. In this case I think we have a few more options, because the
> user has always already authenticated. So we can just tell "git serve"
> via the environment whether the user is authorized for push. It's harder
> with HTTP because most setups avoid even challenging for auth unless
> it's necessary.
>
> So I'm a bit worried that the unified endpoint model is going to be a
> dead end, at which point carrying around git-serve just makes things
> more complicated.

This is from wetware memory of something discussed at a previous Git Merge, so I may have (inadvertently) made it up, but wasn't part of the idea of "git-serve" to have an extensible next-gen protocol where we could add new actions & verbs unrelated to sending or receiving packs?

E.g. client<->server optimistic cooperation like offloading a long-running "git-grep", "git log -G" etc. to a more powerful workhorse server, which would use "git-serve" as a routing layer.

Of course that's not in itself an argument for having a general "serve" command, actually the opposite for the reasons you mention with locking down things. E.g. maybe I want to support server-side git-grep on my server, but not git-log, and if it's one command it becomes a hassle to do that via SSH config or HTTPD config for the reasons you mention.

The upside would be that once a host understands "git serve" I'm more likely to be able to get past whatever middle layer there is between my client and the "git" binary on the other side. E.g. if I have a newly compiled "git" client/server binary, but something like GitLab's "gitaly" sitting between the two of us.

Previous: Jeff KingNext: Jeff King
Message 71 of 73 in “protocol v2 and hidden refs”
  1. 0/3 protocol v2 and hidden refsJeff King, Dec 11, 2018
  2. 1/3 serve: pass "config context" through to individual commandsJeff King, Dec 11, 2018
  3. Junio C HamanoDec 14, 2018
  4. Jeff KingDec 14, 2018
  5. Junio C HamanoDec 15, 2018
  6. Jeff KingDec 16, 2018
  7. Junio C HamanoDec 16, 2018
  8. Jeff KingDec 18, 2018
  9. Jonathan NiederDec 14, 2018
  10. Jeff KingDec 14, 2018
  11. Jonathan NiederDec 14, 2018
  12. Jeff KingDec 14, 2018
  13. 2/3 parse_hide_refs_config: handle NULL sectionJeff King, Dec 11, 2018
  14. Junio C HamanoDec 14, 2018
  15. 3/3 upload-pack: support hidden refs with protocol v2Jeff King, Dec 11, 2018
  16. Ævar Arnfjörð BjarmasonDec 11, 2018
  17. Jeff KingDec 11, 2018
  18. 0/3 Add a GIT_TEST_PROTOCOL_VERSION=X test modeÆvar Arnfjörð Bjarmason, Dec 11, 2018
  19. Ævar Arnfjörð BjarmasonDec 11, 2018
  20. 1/3 tests: add a special setup where for protocol.versionÆvar Arnfjörð Bjarmason, Dec 11, 2018
  21. 0/3 Some fixes and improvementsJonathan Tan, Dec 12, 2018
  22. 1/3 squash this into your patchJonathan Tan, Dec 12, 2018
  23. 3/3 also squash this into your patchJonathan Tan, Dec 12, 2018
  24. 2/3 builtin/fetch-pack: support protocol version 2Jonathan Tan, Dec 12, 2018
  25. Junio C HamanoDec 13, 2018
  26. 0/8 protocol v2 fixesÆvar Arnfjörð Bjarmason, Dec 13, 2018
  27. 0/4 protocol v2 fixesÆvar Arnfjörð Bjarmason, Dec 17, 2018
  28. Jeff KingDec 18, 2018
  29. 1/4 serve: pass "config context" through to individual commandsÆvar Arnfjörð Bjarmason, Dec 17, 2018
  30. 3/4 upload-pack: support hidden refs with protocol v2Ævar Arnfjörð Bjarmason, Dec 17, 2018
  31. 2/4 parse_hide_refs_config: handle NULL sectionÆvar Arnfjörð Bjarmason, Dec 17, 2018
  32. 4/4 fetch-pack: support protocol version 2Ævar Arnfjörð Bjarmason, Dec 17, 2018
  33. Junio C HamanoJan 8, 2019
  34. Jonathan TanJan 8, 2019
  35. Jeff KingJan 8, 2019
  36. 1/8 serve: pass "config context" through to individual commandsÆvar Arnfjörð Bjarmason, Dec 13, 2018
  37. 2/8 parse_hide_refs_config: handle NULL sectionÆvar Arnfjörð Bjarmason, Dec 13, 2018
  38. 3/8 upload-pack: support hidden refs with protocol v2Ævar Arnfjörð Bjarmason, Dec 13, 2018
  39. 4/8 tests: add a check for unportable env --unsetÆvar Arnfjörð Bjarmason, Dec 13, 2018
  40. 5/8 tests: add a special setup where for protocol.versionÆvar Arnfjörð Bjarmason, Dec 13, 2018
  41. Jonathan TanDec 13, 2018
  42. 6/8 tests: mark & fix tests broken under GIT_TEST_PROTOCOL_VERSION=1Ævar Arnfjörð Bjarmason, Dec 13, 2018
  43. 7/8 builtin/fetch-pack: support protocol version 2Ævar Arnfjörð Bjarmason, Dec 13, 2018
  44. Jeff KingDec 14, 2018
  45. 8/8 tests: mark tests broken under GIT_TEST_PROTOCOL_VERSION=2Ævar Arnfjörð Bjarmason, Dec 13, 2018
  46. Ævar Arnfjörð BjarmasonDec 13, 2018
  47. Junio C HamanoDec 14, 2018
  48. Jeff KingDec 14, 2018
  49. Ævar Arnfjörð BjarmasonDec 14, 2018
  50. Ævar Arnfjörð BjarmasonDec 14, 2018
  51. Jeff KingDec 17, 2018
  52. Jeff KingDec 17, 2018
  53. upload-pack: turn on uploadpack.allowAnySHA1InWant=trueÆvar Arnfjörð Bjarmason, Dec 17, 2018
  54. David TurnerDec 17, 2018
  55. Ævar Arnfjörð BjarmasonDec 17, 2018
  56. David TurnerDec 17, 2018
  57. Jonathan NiederDec 17, 2018
  58. Ævar Arnfjörð BjarmasonDec 17, 2018
  59. Jonathan NiederDec 18, 2018
  60. Ævar Arnfjörð BjarmasonDec 18, 2018
  61. Jeff KingDec 18, 2018
  62. Jeff KingDec 18, 2018
  63. Ævar Arnfjörð BjarmasonDec 18, 2018
  64. Junio C HamanoDec 26, 2018
  65. Ævar Arnfjörð BjarmasonDec 27, 2018
  66. Jonathan NiederDec 27, 2018
  67. 2/3 tests: mark tests broken under GIT_TEST_PROTOCOL_VERSION=1Ævar Arnfjörð Bjarmason, Dec 11, 2018
  68. 3/3 tests: mark tests broken under GIT_TEST_PROTOCOL_VERSION=2Ævar Arnfjörð Bjarmason, Dec 11, 2018
  69. Jonathan TanDec 13, 2018
  70. Jeff KingDec 14, 2018
  71. Ævar Arnfjörð BjarmasonDec 15, 2018
  72. Jeff KingDec 16, 2018
  73. Ævar Arnfjörð BjarmasonDec 16, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.