git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH 0/3] protocol v2

From
Junio C Hamano <gitster@pobox.com>
Date
Mar 1, 2015, 23:06 UTC
Message-ID
<xmqqwq30fhbg.fsf@gitster.dls.corp.google.com>
In-Reply-To
<CACsJy8DKbfU7TBHhT5_qpL0QM3zbxkaF+B4x3hQDpomQ_9OSEg@mail.gmail.com>
Duy Nguyen <pclouds@gmail.com> writes:
Show 12 quoted lines
> On Sun, Mar 1, 2015 at 3:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>>>  - Because the protocol exchange starts by the server side
>>>    advertising all its refs, even when the fetcher is interested in
>>>    a single ref, the initial overhead is nontrivial, especially when
>>>    you are doing a small incremental update.  The worst case is an
>>>    auto-builder that polls every five minutes, even when there is no
>>>    new commits to be fetched [*3*].
>
> Maybe you can elaborate about how to handle states X, Y... in your
> footnote 3. I just don't see how it's actually implemented.  Or is
> it optional feature that will be provided (via hooks, maybe) by
> admin?

These footnotes are not the important part (I wanted us to agree on the problem, and the ideas outlined with the footnotes are only an example that illustrates how a potential solution and the problem to be solved are described in relation to each other), but I'll give a shot anyway ;-)

I am actually torn on how the names X, Y, etc. should be defined.

One side of me wants to leave its computation entirely up to the server side. The client says "Last time I talked with you asking for refs/heads/* and successfully updated from you, you told me to call that state X" without knowing how X is computed, and then the server will update you and then tell you your state is now Y. That way, large hosting sites and server implementations can choose to implement it any way they like.

On the other hand, we could rigidly define it, perhaps like this:
 - Imagine that you saved the output from ls-remote that is run
   against that server, limited to the refs hierarchy you are
   requesting, the last time you talked with it.
 - Concatenate the above to the list of patterns the client used to
   ask the refs.  This step is optional.
 - E.g. if you are asking it for "refs/heads/*", then we are talking
   something like this (illustrated with optional pattern in front):
        refs/heads/*
        8004647...      refs/heads/maint
        7f4ba4b...      refs/heads/master
 - Run SHA-1 hash over that.  And that is the state name.
I.e. if you as a client are doing
    [remote "origin"]
        fetch = refs/heads/*:refs/remotes/origin/*

and if the only time your refs/remotes/origin/* hierarchy changes is when you fetch from there (which should be the norm), you can look into remote.origin.fetch refspec (to learn that "refs/heads*" is what you are asking) and your refs/remotes/origin/* refs (and reverse the mapping you make when you fetch to make them talk about refs/heads/* hierarchy on the server side), you can compute it locally.

The latter will have one benefit over "opaque thing the client does not know how to compute". Because I want us avoid sending unchanged refs over connection, but I do want to see the protocol has some validation mechanism built in, even if we go the latter "client can compute what the state name ought to be" route, I want the servrer to tell the client what to call that state. That way, the client side can tell when it goes out of sync for any reason and attempt to recover.

> Do we need to worry about load balancers?

Unless you are allowing multiple backend servers to serve a same repository behind a set of load balancers in an inconsistent way (e.g. you push to one while I push to two and you fetch from one and you temporarily see my push but then my push will be rejected as conflicting and you fetch from one and now you see your push), I do not think there is anything you need to worry about them more than what you should be worrying about already.

There would be a point where all backend servers would agree "This is the set of values of these refs" at some point (e.g. a majority of surviving servers vote to decide, laggers that later join the party will update to the concensus value before serving the end-user traffic), and they would not be showing half-updated values that haven't ratified by other servers to end users (otherwise they may end up showing reversion).

Show 12 quoted lines
>>    - Is band #2 meant for human consumption, or do we expect the
>>      other end to interpret and act on it?  If the former, would it
>>      make sense to send locale information from the client side and
>>      ask the server side to produce its output with _("message")?
>
> No producing _(...) is a bad idea. First the client has to verify
> placeholders and stuff, we can't just feed data from server straight
> to printf(). Producing _() could complicate server code a lot. And I
> don't like the idea of using client .po files to translate server
> strings. There could be custom strings added by admin, which are not
> available in client .po. String translation should happen at server
> side.

What I meant to say was (1) the client says "I want the human readable message in Vietnamese" and (2) the server uses .po on _("message") in its code and send the result to sideband #2. There is no parsing, interpolation, or anything of that sort necessary on the server end.

Because potential problem you mention looks to me about a different design where the server talks "message" and client side applies _($msg) to it, I suspect that you misread what I meant to say.

Previous: David LangNext: David Lang
Message 75 of 80 in “protocol v2”
  1. 0/3 protocol v2Stefan Beller, Feb 24, 2015
  2. 1/3 Document protocol capabilities extensionStefan Beller, Feb 24, 2015
  3. 2/3 receive-pack: add advertisement of different protocol optionsStefan Beller, Feb 24, 2015
  4. 3/3 receive-pack: enable protocol v2Stefan Beller, Feb 24, 2015
  5. Duy NguyenFeb 24, 2015
  6. Stefan BellerFeb 24, 2015
  7. Junio C HamanoFeb 24, 2015
  8. Stefan BellerFeb 24, 2015
  9. Duy NguyenFeb 25, 2015
  10. Junio C HamanoFeb 25, 2015
  11. Stefan BellerFeb 26, 2015
  12. Duy NguyenFeb 26, 2015
  13. Stefan BellerFeb 26, 2015
  14. Junio C HamanoFeb 26, 2015
  15. Stefan BellerFeb 27, 2015
  16. Stefan BellerFeb 27, 2015
  17. Junio C HamanoFeb 27, 2015
  18. Stefan BellerFeb 27, 2015
  19. Junio C HamanoFeb 28, 2015
  20. Stefan BellerFeb 28, 2015
  21. 0/5 protocol v2 for upload-packStefan Beller, Feb 28, 2015
  22. 1/5 upload-pack: only accept capabilities on the first "want" lineStefan Beller, Feb 28, 2015
  23. 2/5 upload-pack: support out of band client capability requestsStefan Beller, Feb 28, 2015
  24. Kyle J. McKayFeb 28, 2015
  25. Duy NguyenFeb 28, 2015
  26. Kyle J. McKayFeb 28, 2015
  27. Duy NguyenMar 1, 2015
  28. Duy NguyenFeb 28, 2015
  29. 3/5 connect.c: connect to a remote service with some flagsStefan Beller, Feb 28, 2015
  30. Torsten BögershausenFeb 28, 2015
  31. Junio C HamanoMar 1, 2015
  32. 4/5 daemon.c: accept extra service argumentsStefan Beller, Feb 28, 2015
  33. Junio C HamanoMar 1, 2015
  34. 5/5 WIP/Document the http protocol changeStefan Beller, Feb 28, 2015
  35. Duy NguyenFeb 28, 2015
  36. Johannes SixtMar 1, 2015
  37. Junio C HamanoMar 2, 2015
  38. Junio C HamanoMar 2, 2015
  39. Duy NguyenMar 2, 2015
  40. Duy NguyenMar 2, 2015
  41. Duy NguyenMar 3, 2015
  42. Junio C HamanoMar 3, 2015
  43. Junio C HamanoMar 3, 2015
  44. Duy NguyenMar 4, 2015
  45. Shawn PearceMar 4, 2015
  46. Duy NguyenMar 4, 2015
  47. Shawn PearceMar 4, 2015
  48. Stefan BellerMar 5, 2015
  49. Stefan BellerMar 5, 2015
  50. Stefan BellerMar 24, 2015
  51. Junio C HamanoMar 24, 2015
  52. protocol upload-pack-v2Stefan Beller, Mar 6, 2015
  53. Stefan BellerMar 6, 2015
  54. Duy NguyenMar 6, 2015
  55. Duy NguyenMar 7, 2015
  56. Junio C HamanoMar 7, 2015
  57. Stefan BellerMar 7, 2015
  58. Duy NguyenMar 7, 2015
  59. Junio C HamanoMar 8, 2015
  60. Junio C HamanoMar 31, 2015
  61. Duy NguyenApr 2, 2015
  62. Junio C HamanoApr 2, 2015
  63. Martin FickApr 2, 2015
  64. Junio C HamanoApr 2, 2015
  65. Stefan BellerApr 2, 2015
  66. Stefan BellerApr 2, 2015
  67. Duy NguyenMar 10, 2015
  68. Kyle J. McKayMar 10, 2015
  69. Duy NguyenFeb 28, 2015
  70. Junio C HamanoFeb 28, 2015
  71. Junio C HamanoMar 1, 2015
  72. Duy NguyenMar 1, 2015
  73. Stefan BellerMar 1, 2015
  74. David LangMar 2, 2015
  75. Junio C HamanoMar 1, 2015
  76. David LangMar 2, 2015
  77. Junio C HamanoMar 2, 2015
  78. Philip OakleyMar 1, 2015
  79. Duy NguyenMar 2, 2015
  80. Stefan BellerFeb 27, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.