git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/9] reducing memory allocations for v2 servers

From
Jeff King <peff@peff.net>
Date
Sep 14, 2021, 15:29 UTC
Message-ID
<YUC/6n1hhUbMJiLw@coredump.intra.peff.net>

While looking at [1], I noticed that v2 servers will read a few bits of client input into strvecs. Even though we expect these to be small-ish, there's nothing preventing a client from sending us a bunch of junk and wasting memory.

This series changes that, putting a cap on how much data we'll receive. The two spots are the "capabilities" list we receive (before we even dispatch to a particular command like ls-refs or fetch), and the ref-prefix list we receive for ls-refs.

To deal with the capabilities issue, we'll just handle each capability line as it comes in, rather than storing a list. This requires a bit of cleanup, which is why it takes up the first 6 patches. It also needs a few other cleanups from ab/serve-cleanup (and so this series is based on that). The dependencies there are both textual (e.g., using designated initializers in the capabilities table) and functional (dropping the "keys" parameter from v2 command() functions).

Patch 7 fixes the ls-refs issue by degrading when we see too many prefixes (which works because the capability is optional in the first place).

The final two patches aren't strictly necessary. They're a tightening of the protocol against some bogus input that I noticed while doing the earlier cleanups. That bogus input isn't really hurting anything, but I think it makes sense to reject nonsense from the client rather than ignoring it. The obvious risk is that some client for some reason would send us nonsense, but I don't think Git ever has.

Once again, these are based on ab/serve-cleanup, which is currently in next.

  [1/9]: serve: rename is_command() to parse_command()
  [2/9]: serve: return capability "value" from get_capability()
  [3/9]: serve: add "receive" method for v2 capabilities table
  [4/9]: serve: provide "receive" function for object-format capability
  [5/9]: serve: provide "receive" function for session-id capability
  [6/9]: serve: drop "keys" strvec
  [7/9]: ls-refs: ignore very long ref-prefix counts
  [8/9]: serve: reject bogus v2 "command=ls-refs=foo"
  [9/9]: serve: reject commands used as capabilities
 ls-refs.c            |  19 ++++++-
 serve.c              | 120 +++++++++++++++++++++++--------------------
 t/t5701-git-serve.sh |  60 ++++++++++++++++++++++
 3 files changed, 142 insertions(+), 57 deletions(-)
-Peff
[1] https://lore.kernel.org/git/YT54CNYgtGcqexwq@coredump.intra.peff.net/
Next: Jeff King
Message 1 of 77 in “reducing memory allocations for v2 servers”
  1. 0/9 reducing memory allocations for v2 serversJeff King, Sep 14, 2021
  2. 1/9 serve: rename is_command() to parse_command()Jeff King, Sep 14, 2021
  3. 2/9 serve: return capability "value" from get_capability()Jeff King, Sep 14, 2021
  4. 3/9 serve: add "receive" method for v2 capabilities tableJeff King, Sep 14, 2021
  5. 4/9 serve: provide "receive" function for object-format capabilityJeff King, Sep 14, 2021
  6. Martin ÅgrenSep 14, 2021
  7. 5/9 serve: provide "receive" function for session-id capabilityJeff King, Sep 14, 2021
  8. Taylor BlauSep 14, 2021
  9. Jeff KingSep 14, 2021
  10. Taylor BlauSep 14, 2021
  11. Martin ÅgrenSep 14, 2021
  12. Jeff KingSep 14, 2021
  13. 6/9 serve: drop "keys" strvecJeff King, Sep 14, 2021
  14. Taylor BlauSep 14, 2021
  15. Jeff KingSep 14, 2021
  16. 7/9 ls-refs: ignore very long ref-prefix countsJeff King, Sep 14, 2021
  17. Taylor BlauSep 14, 2021
  18. Jeff KingSep 14, 2021
  19. Jeff KingSep 14, 2021
  20. Martin ÅgrenSep 14, 2021
  21. Jeff KingSep 14, 2021
  22. Jeff KingSep 14, 2021
  23. Taylor BlauSep 14, 2021
  24. Jeff KingSep 14, 2021
  25. 8/9 serve: reject bogus v2 "command=ls-refs=foo"Jeff King, Sep 14, 2021
  26. Taylor BlauSep 14, 2021
  27. 9/9 serve: reject commands used as capabilitiesJeff King, Sep 14, 2021
  28. Taylor BlauSep 14, 2021
  29. Junio C HamanoSep 14, 2021
  30. Jeff KingSep 14, 2021
  31. 0/11 limit memory allocations for v2 serversJeff King, Sep 14, 2021
  32. 01/11 serve: rename is_command() to parse_command()Jeff King, Sep 14, 2021
  33. 02/11 serve: return capability "value" from get_capability()Jeff King, Sep 14, 2021
  34. 03/11 serve: add "receive" method for v2 capabilities tableJeff King, Sep 14, 2021
  35. Ævar Arnfjörð BjarmasonSep 15, 2021
  36. Jeff KingSep 15, 2021
  37. Junio C HamanoSep 15, 2021
  38. Jeff KingSep 15, 2021
  39. 04/11 serve: provide "receive" function for object-format capabilityJeff King, Sep 14, 2021
  40. Junio C HamanoSep 15, 2021
  41. 05/11 serve: provide "receive" function for session-id capabilityJeff King, Sep 14, 2021
  42. Junio C HamanoSep 15, 2021
  43. 07/11 ls-refs: ignore very long ref-prefix countsJeff King, Sep 14, 2021
  44. Taylor BlauSep 15, 2021
  45. Jeff KingSep 15, 2021
  46. Eric SunshineSep 15, 2021
  47. Jeff KingSep 15, 2021
  48. 06/11 serve: drop "keys" strvecJeff King, Sep 14, 2021
  49. Junio C HamanoSep 15, 2021
  50. 08/11 docs/protocol-v2: clarify some ls-refs ref-prefix detailsJeff King, Sep 14, 2021
  51. 09/11 serve: reject bogus v2 "command=ls-refs=foo"Jeff King, Sep 14, 2021
  52. Ævar Arnfjörð BjarmasonSep 15, 2021
  53. Jeff KingSep 15, 2021
  54. Eric SunshineSep 15, 2021
  55. Jeff KingSep 15, 2021
  56. Junio C HamanoSep 15, 2021
  57. Jeff KingSep 15, 2021
  58. 10/11 serve: reject commands used as capabilitiesJeff King, Sep 14, 2021
  59. 11/11 ls-refs: reject unknown argumentsJeff King, Sep 14, 2021
  60. Ævar Arnfjörð BjarmasonSep 15, 2021
  61. Jeff KingSep 15, 2021
  62. Taylor BlauSep 15, 2021
  63. Jeff KingSep 15, 2021
  64. 0/11 limit memory allocations for v2 serversJeff King, Sep 15, 2021
  65. 01/11 serve: rename is_command() to parse_command()Jeff King, Sep 15, 2021
  66. 02/11 serve: return capability "value" from get_capability()Jeff King, Sep 15, 2021
  67. 03/11 serve: add "receive" method for v2 capabilities tableJeff King, Sep 15, 2021
  68. 04/11 serve: provide "receive" function for object-format capabilityJeff King, Sep 15, 2021
  69. 05/11 serve: provide "receive" function for session-id capabilityJeff King, Sep 15, 2021
  70. 06/11 serve: drop "keys" strvecJeff King, Sep 15, 2021
  71. 07/11 ls-refs: ignore very long ref-prefix countsJeff King, Sep 15, 2021
  72. 08/11 docs/protocol-v2: clarify some ls-refs ref-prefix detailsJeff King, Sep 15, 2021
  73. 09/11 serve: reject bogus v2 "command=ls-refs=foo"Jeff King, Sep 15, 2021
  74. 10/11 serve: reject commands used as capabilitiesJeff King, Sep 15, 2021
  75. 11/11 ls-refs: reject unknown argumentsJeff King, Sep 15, 2021
  76. Ævar Arnfjörð BjarmasonSep 15, 2021
  77. Jeff KingSep 15, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.