git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC/PATCH v4 27/49] Documentation: add read-object-protocol.txt

From
Christian Couder <christian.couder@gmail.com>
Date
Jun 20, 2017, 07:55 UTC
Message-ID
<20170620075523.26961-28-chriscool@tuxfamily.org>
In-Reply-To
<20170620075523.26961-1-chriscool@tuxfamily.org>
From: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 Documentation/technical/read-object-protocol.txt | 102 +++++++++++++++++++++++
 1 file changed, 102 insertions(+)
 create mode 100644 Documentation/technical/read-object-protocol.txt
diff --git a/Documentation/technical/read-object-protocol.txt b/Documentation/technical/read-object-protocol.txt
new file mode 100644
index 0000000000..a893b46e7c
--- /dev/null
+++ b/Documentation/technical/read-object-protocol.txt
@@ -0,0 +1,102 @@
+Read Object Process
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The read-object process enables Git to read all missing blobs with a
+single process invocation for the entire life of a single Git command.
+This is achieved by using a packet format (pkt-line, see technical/
+protocol-common.txt) based protocol over standard input and standard
+output as follows. All packets, except for the "*CONTENT" packets and
+the "0000" flush packet, are considered text and therefore are
+terminated by a LF.
+
+Git starts the process when it encounters the first missing object that
+needs to be retrieved. After the process is started, Git sends a welcome
+message ("git-read-object-client"), a list of supported protocol version
+numbers, and a flush packet. Git expects to read a welcome response
+message ("git-read-object-server"), exactly one protocol version number
+from the previously sent list, and a flush packet. All further
+communication will be based on the selected version.
+
+The remaining protocol description below documents "version=1". Please
+note that "version=42" in the example below does not exist and is only
+there to illustrate how the protocol would look with more than one
+version.
+
+After the version negotiation Git sends a list of all capabilities that
+it supports and a flush packet. Git expects to read a list of desired
+capabilities, which must be a subset of the supported capabilities list,
+and a flush packet as response:
+------------------------
+packet: git> git-read-object-client
+packet: git> version=1
+packet: git> version=42
+packet: git> 0000
+packet: git< git-read-object-server
+packet: git< version=1
+packet: git< 0000
+packet: git> capability=get
+packet: git> capability=have
+packet: git> capability=put
+packet: git> capability=not-yet-invented
+packet: git> 0000
+packet: git< capability=get
+packet: git< 0000
+------------------------
+The only supported capability in version 1 is "get".
+
+Afterwards Git sends a list of "key=value" pairs terminated with a flush
+packet. The list will contain at least the command (based on the
+supported capabilities) and the sha1 of the object to retrieve. Please
+note, that the process must not send any response before it received the
+final flush packet.
+
+When the process receives the "get" command, it should make the requested
+object available in the git object store and then return success. Git will
+then check the object store again and this time find it and proceed.
+------------------------
+packet: git> command=get
+packet: git> sha1=0a214a649e1b3d5011e14a3dc227753f2bd2be05
+packet: git> 0000
+------------------------
+
+The process is expected to respond with a list of "key=value" pairs
+terminated with a flush packet. If the process does not experience
+problems then the list must contain a "success" status.
+------------------------
+packet: git< status=success
+packet: git< 0000
+------------------------
+
+In case the process cannot or does not want to process the content, it
+is expected to respond with an "error" status.
+------------------------
+packet: git< status=error
+packet: git< 0000
+------------------------
+
+In case the process cannot or does not want to process the content as
+well as any future content for the lifetime of the Git process, then it
+is expected to respond with an "abort" status at any point in the
+protocol.
+------------------------
+packet: git< status=abort
+packet: git< 0000
+------------------------
+
+Git neither stops nor restarts the process in case the "error"/"abort"
+status is set.
+
+If the process dies during the communication or does not adhere to the
+protocol then Git will stop the process and restart it with the next
+object that needs to be processed.
+
+After the read-object process has processed an object it is expected to
+wait for the next "key=value" list containing a command. Git will close
+the command pipe on exit. The process is expected to detect EOF and exit
+gracefully on its own. Git will wait until the process has stopped.
+
+A long running read-object process demo implementation can be found in
+`contrib/long-running-read-object/example.pl` located in the Git core
+repository. If you develop your own long running process then the
+`GIT_TRACE_PACKET` environment variables can be very helpful for
+debugging (see linkgit:git[1]).
-- 
2.13.1.565.gbfcd7a9048
Previous: Christian CouderNext: Christian Couder
Message 15 of 64 in “Add initial experimental external ODB support”
  1. 00/49 Add initial experimental external ODB supportChristian Couder, Jun 20, 2017
  2. 01/49 builtin/clone: get rid of 'value' strbufChristian Couder, Jun 20, 2017
  3. 03/49 t0021/rot13-filter: improve 'if .. elsif .. else' styleChristian Couder, Jun 20, 2017
  4. 02/49 t0021/rot13-filter: refactor packet reading functionsChristian Couder, Jun 20, 2017
  5. 06/49 Git/Packet.pm: improve error messageChristian Couder, Jun 20, 2017
  6. 05/49 t0021/rot13-filter: use Git/Packet.pmChristian Couder, Jun 20, 2017
  7. 13/49 external-odb: accept only blobs for nowChristian Couder, Jun 20, 2017
  8. 15/49 Add GIT_NO_EXTERNAL_ODB env variableChristian Couder, Jun 20, 2017
  9. 12/49 external odb: add write supportChristian Couder, Jun 20, 2017
  10. 16/49 Add t0410 to test external ODB transferChristian Couder, Jun 20, 2017
  11. 18/49 lib-httpd: add upload.shChristian Couder, Jun 20, 2017
  12. 14/49 t0400: add test for external odb write supportChristian Couder, Jun 20, 2017
  13. 19/49 lib-httpd: add list.shChristian Couder, Jun 20, 2017
  14. 26/49 odb-helper: add script_modeChristian Couder, Jun 20, 2017
  15. 27/49 Documentation: add read-object-protocol.txtChristian Couder, Jun 20, 2017
  16. 32/49 t04*: add 'get_cap' support to helpersChristian Couder, Jun 20, 2017
  17. 30/49 odb-helper: add read_object_process()Christian Couder, Jun 20, 2017
  18. Ben PeartJul 10, 2017
  19. 34/49 odb-helper: fix odb_helper_fetch_object() for read_objectChristian Couder, Jun 20, 2017
  20. 33/49 odb-helper: call odb_helper_lookup() with 'have' capabilityChristian Couder, Jun 20, 2017
  21. 38/49 Add t0470 to test passing plain objectsChristian Couder, Jun 20, 2017
  22. 37/49 odb-helper: add read_packetized_plain_object_to_fd()Christian Couder, Jun 20, 2017
  23. 40/49 Add t0480 to test "have" capability and plain objectsChristian Couder, Jun 20, 2017
  24. 31/49 external-odb: add external_odb_get_capabilities()Christian Couder, Jun 20, 2017
  25. 29/49 Add t0410 to test read object mechanismChristian Couder, Jun 20, 2017
  26. 45/49 clone: add initial param to write_remote_refs()Christian Couder, Jun 20, 2017
  27. 46/49 clone: add --initial-refspec optionChristian Couder, Jun 20, 2017
  28. 48/49 Add test for 'clone --initial-refspec'Christian Couder, Jun 20, 2017
  29. 43/49 odb-helper: advertise 'put' capabilityChristian Couder, Jun 20, 2017
  30. 41/49 external-odb: add external_odb_do_fetch_object()Christian Couder, Jun 20, 2017
  31. 47/49 clone: disable external odb before initial cloneChristian Couder, Jun 20, 2017
  32. 42/49 odb-helper: advertise 'have' capabilityChristian Couder, Jun 20, 2017
  33. 49/49 t: add t0430 to test cloning using bundlesChristian Couder, Jun 20, 2017
  34. 39/49 odb-helper: add write_object_process()Christian Couder, Jun 20, 2017
  35. 44/49 odb-helper: add have_object_process()Christian Couder, Jun 20, 2017
  36. 35/49 Add t0460 to test passing git objectsChristian Couder, Jun 20, 2017
  37. 36/49 odb-helper: add read_packetized_git_object_to_fd()Christian Couder, Jun 20, 2017
  38. 23/49 t0420: add test with HTTP external odbChristian Couder, Jun 20, 2017
  39. 24/49 odb-helper: start fault in implementationChristian Couder, Jun 20, 2017
  40. 28/49 contrib: add long-running-read-object/example.plChristian Couder, Jun 20, 2017
  41. 25/49 external-odb: add external_odb_fault_in_object()Christian Couder, Jun 20, 2017
  42. 22/49 pack-objects: don't pack objects in external odbsChristian Couder, Jun 20, 2017
  43. 20/49 lib-httpd: add apache-e-odb.confChristian Couder, Jun 20, 2017
  44. 09/49 Add initial external odb supportChristian Couder, Jun 20, 2017
  45. Ben PeartJun 23, 2017
  46. 17/49 lib-httpd: pass config file to start_httpd()Christian Couder, Jun 20, 2017
  47. 21/49 odb-helper: add 'store_plain_objects' to 'struct odb_helper'Christian Couder, Jun 20, 2017
  48. 07/49 Git/Packet.pm: add packet_initialize()Christian Couder, Jun 20, 2017
  49. Ben PeartJun 23, 2017
  50. 10/49 external odb foreachChristian Couder, Jun 20, 2017
  51. 08/49 Git/Packet: add capability functionsChristian Couder, Jun 20, 2017
  52. 11/49 t0400: add 'put' command to odb-helper scriptChristian Couder, Jun 20, 2017
  53. 04/49 Add Git/Packet.pm from parts of t0021/rot13-filter.plChristian Couder, Jun 20, 2017
  54. Christian CouderJun 20, 2017
  55. Ben PeartJun 23, 2017
  56. Christian CouderJul 1, 2017
  57. Christian CouderJul 1, 2017
  58. Junio C HamanoJul 1, 2017
  59. Christian CouderJul 2, 2017
  60. Junio C HamanoJul 3, 2017
  61. Ben PeartJul 6, 2017
  62. Christian CouderSep 15, 2017
  63. Jonathan TanJul 12, 2017
  64. Christian CouderSep 15, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.