git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH v4 00/49] Add initial experimental external ODB support

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 3, 2017, 16:56 UTC
Message-ID
<xmqqlgo5mpc1.fsf@gitster.mtv.corp.google.com>
In-Reply-To
<CAP8UFD3YPXRDc8ThSuTWjKDQ2i+zqNrhBi6JfEuTPXnNA6LUiw@mail.gmail.com>
Christian Couder <christian.couder@gmail.com> writes:
Show 19 quoted lines
> On Sat, Jul 1, 2017 at 10:33 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> Christian Couder <christian.couder@gmail.com> writes:
>>
>>>> I think it would be good to ensure the
>>>> interface is robust and performant enough to actually replace the current
>>>> object store interface (even if we don't actually do that just yet).
>>>
>>> I agree that it should be robust and performant, but I don't think it
>>> needs to be as performant in all cases as the current object store
>>> right now.
>>
>> That sounds like starting from a defeatest position.  Is there a
>> reason why you think using an external interface could never perform
>> well enough to be usable in everyday work?
>
> Perhaps in the future we will be able to make it as performant as, or
> perhaps even more performant, than the current object store, but in
> the current implementation the following issues mean that it will be
> less performant

That might be an answer to a different question; I was hoping to hear that it should be performant enough for everyday work, but never thought it would perform as well as local disk.

I haven't used network filesystem quite a while, but a repository on NFS may still usable, and we know our own access pattern bettern than NFS which cannot anticipate what paths the next operations by its client happen, so it is not inconceivable that a well designed external object database interface would let us outperform "repo on NFS" scenario.

Previous: Christian CouderNext: Ben Peart
Message 60 of 64 in “Add initial experimental external ODB support”
  1. 00/49 Add initial experimental external ODB supportChristian Couder, Jun 20, 2017
  2. 01/49 builtin/clone: get rid of 'value' strbufChristian Couder, Jun 20, 2017
  3. 03/49 t0021/rot13-filter: improve 'if .. elsif .. else' styleChristian Couder, Jun 20, 2017
  4. 02/49 t0021/rot13-filter: refactor packet reading functionsChristian Couder, Jun 20, 2017
  5. 06/49 Git/Packet.pm: improve error messageChristian Couder, Jun 20, 2017
  6. 05/49 t0021/rot13-filter: use Git/Packet.pmChristian Couder, Jun 20, 2017
  7. 13/49 external-odb: accept only blobs for nowChristian Couder, Jun 20, 2017
  8. 15/49 Add GIT_NO_EXTERNAL_ODB env variableChristian Couder, Jun 20, 2017
  9. 12/49 external odb: add write supportChristian Couder, Jun 20, 2017
  10. 16/49 Add t0410 to test external ODB transferChristian Couder, Jun 20, 2017
  11. 18/49 lib-httpd: add upload.shChristian Couder, Jun 20, 2017
  12. 14/49 t0400: add test for external odb write supportChristian Couder, Jun 20, 2017
  13. 19/49 lib-httpd: add list.shChristian Couder, Jun 20, 2017
  14. 26/49 odb-helper: add script_modeChristian Couder, Jun 20, 2017
  15. 27/49 Documentation: add read-object-protocol.txtChristian Couder, Jun 20, 2017
  16. 32/49 t04*: add 'get_cap' support to helpersChristian Couder, Jun 20, 2017
  17. 30/49 odb-helper: add read_object_process()Christian Couder, Jun 20, 2017
  18. Ben PeartJul 10, 2017
  19. 34/49 odb-helper: fix odb_helper_fetch_object() for read_objectChristian Couder, Jun 20, 2017
  20. 33/49 odb-helper: call odb_helper_lookup() with 'have' capabilityChristian Couder, Jun 20, 2017
  21. 38/49 Add t0470 to test passing plain objectsChristian Couder, Jun 20, 2017
  22. 37/49 odb-helper: add read_packetized_plain_object_to_fd()Christian Couder, Jun 20, 2017
  23. 40/49 Add t0480 to test "have" capability and plain objectsChristian Couder, Jun 20, 2017
  24. 31/49 external-odb: add external_odb_get_capabilities()Christian Couder, Jun 20, 2017
  25. 29/49 Add t0410 to test read object mechanismChristian Couder, Jun 20, 2017
  26. 45/49 clone: add initial param to write_remote_refs()Christian Couder, Jun 20, 2017
  27. 46/49 clone: add --initial-refspec optionChristian Couder, Jun 20, 2017
  28. 48/49 Add test for 'clone --initial-refspec'Christian Couder, Jun 20, 2017
  29. 43/49 odb-helper: advertise 'put' capabilityChristian Couder, Jun 20, 2017
  30. 41/49 external-odb: add external_odb_do_fetch_object()Christian Couder, Jun 20, 2017
  31. 47/49 clone: disable external odb before initial cloneChristian Couder, Jun 20, 2017
  32. 42/49 odb-helper: advertise 'have' capabilityChristian Couder, Jun 20, 2017
  33. 49/49 t: add t0430 to test cloning using bundlesChristian Couder, Jun 20, 2017
  34. 39/49 odb-helper: add write_object_process()Christian Couder, Jun 20, 2017
  35. 44/49 odb-helper: add have_object_process()Christian Couder, Jun 20, 2017
  36. 35/49 Add t0460 to test passing git objectsChristian Couder, Jun 20, 2017
  37. 36/49 odb-helper: add read_packetized_git_object_to_fd()Christian Couder, Jun 20, 2017
  38. 23/49 t0420: add test with HTTP external odbChristian Couder, Jun 20, 2017
  39. 24/49 odb-helper: start fault in implementationChristian Couder, Jun 20, 2017
  40. 28/49 contrib: add long-running-read-object/example.plChristian Couder, Jun 20, 2017
  41. 25/49 external-odb: add external_odb_fault_in_object()Christian Couder, Jun 20, 2017
  42. 22/49 pack-objects: don't pack objects in external odbsChristian Couder, Jun 20, 2017
  43. 20/49 lib-httpd: add apache-e-odb.confChristian Couder, Jun 20, 2017
  44. 09/49 Add initial external odb supportChristian Couder, Jun 20, 2017
  45. Ben PeartJun 23, 2017
  46. 17/49 lib-httpd: pass config file to start_httpd()Christian Couder, Jun 20, 2017
  47. 21/49 odb-helper: add 'store_plain_objects' to 'struct odb_helper'Christian Couder, Jun 20, 2017
  48. 07/49 Git/Packet.pm: add packet_initialize()Christian Couder, Jun 20, 2017
  49. Ben PeartJun 23, 2017
  50. 10/49 external odb foreachChristian Couder, Jun 20, 2017
  51. 08/49 Git/Packet: add capability functionsChristian Couder, Jun 20, 2017
  52. 11/49 t0400: add 'put' command to odb-helper scriptChristian Couder, Jun 20, 2017
  53. 04/49 Add Git/Packet.pm from parts of t0021/rot13-filter.plChristian Couder, Jun 20, 2017
  54. Christian CouderJun 20, 2017
  55. Ben PeartJun 23, 2017
  56. Christian CouderJul 1, 2017
  57. Christian CouderJul 1, 2017
  58. Junio C HamanoJul 1, 2017
  59. Christian CouderJul 2, 2017
  60. Junio C HamanoJul 3, 2017
  61. Ben PeartJul 6, 2017
  62. Christian CouderSep 15, 2017
  63. Jonathan TanJul 12, 2017
  64. Christian CouderSep 15, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.