git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 3/6] fast-import: release unfreed strbufs

From
Thalia Archibald <thalia@archibald.dev>
Date
Apr 1, 2024, 09:06 UTC
Message-ID
<9DF24328-0128-4386-82E7-662D10267242@archibald.dev>
In-Reply-To
<ZgUod3QKptINaPBF@tanuki>
(Resending as plain text)
On Mar 28, 2024, at 01:21, Patrick Steinhardt <ps@pks.im> wrote:
Show 10 quoted lines
> I was about to propose that we should likely also change all of these
> static variables to be local instead. I don't think that we use the
> variables after the function calls. But now that I see that we do it
> like this in all of these helpers I think what's going on is that this
> is a memory optimization to avoid reallocating buffers all the time.
> 
> Ugly, but so be it. We could refactor the code to pass in scratch
> buffers from the outside to remove those static variables. But that
> certainly would be a bigger change and thus likely outside of the scope
> of this patch series.
Show 11 quoted lines
> Oh, now you get to my comment in the preceding patch. With this patch
> we're now in a somewhat weird in-between state where the buffers are
> still static, but we release their memory after each call. So we kind of
> get the worst of both worlds: static variables without being able to
> reuse the buffer's memory.
> 
> If we were to change this then we should definitely mark the buffers as
> non-static. If so, it would be great to demonstrate that this does not
> significantly impact performance.
> 
> The same is true for all the other instances.

I had glossed that they're `static`, since I've grown accustomed to Rust, where this sort of non-reentrant code is discouraged. However, this pattern is great for fast-import, because all of its data is simply freed when it exits at the end of the stream. I dropped this patch in v2.

I don't think it's worth hoisting these `strbuf`s out. It would only reduce it from 5 to 2 total static `strbuf`s for paths, but would make ownership less clear.

Thalia
Previous: Patrick SteinhardtNext: Thalia Archibald
Message 7 of 84 in “fast-import: tighten parsing of paths”
  1. 0/6 fast-import: tighten parsing of pathsThalia Archibald, Mar 22, 2024
  2. 1/6 fast-import: tighten parsing of pathsThalia Archibald, Mar 22, 2024
  3. Thalia ArchibaldMar 22, 2024
  4. Patrick SteinhardtMar 28, 2024
  5. 3/6 fast-import: release unfreed strbufsThalia Archibald, Mar 22, 2024
  6. Patrick SteinhardtMar 28, 2024
  7. Thalia ArchibaldApr 1, 2024
  8. 2/6 fast-import: directly use strbufs for pathsThalia Archibald, Mar 22, 2024
  9. Patrick SteinhardtMar 28, 2024
  10. 4/6 fast-import: remove dead strbufThalia Archibald, Mar 22, 2024
  11. Patrick SteinhardtMar 28, 2024
  12. 6/6 fast-import: forbid escaped NUL in pathsThalia Archibald, Mar 22, 2024
  13. 5/6 fast-import: document C-style escapes for pathsThalia Archibald, Mar 22, 2024
  14. Patrick SteinhardtMar 28, 2024
  15. Thalia ArchibaldApr 1, 2024
  16. 0/8 fast-import: tighten parsing of pathsThalia Archibald, Apr 1, 2024
  17. 1/8 fast-import: tighten path unquotingThalia Archibald, Apr 1, 2024
  18. Patrick SteinhardtApr 10, 2024
  19. Chris TorekApr 10, 2024
  20. Thalia ArchibaldApr 10, 2024
  21. Chris TorekApr 10, 2024
  22. Thalia ArchibaldApr 10, 2024
  23. Patrick SteinhardtApr 10, 2024
  24. Thalia ArchibaldApr 10, 2024
  25. Thalia ArchibaldApr 10, 2024
  26. 2/8 fast-import: directly use strbufs for pathsThalia Archibald, Apr 1, 2024
  27. Patrick SteinhardtApr 10, 2024
  28. Thalia ArchibaldApr 10, 2024
  29. Patrick SteinhardtApr 10, 2024
  30. 3/8 fast-import: allow unquoted empty path for rootThalia Archibald, Apr 1, 2024
  31. Patrick SteinhardtApr 10, 2024
  32. 4/8 fast-import: remove dead strbufThalia Archibald, Apr 1, 2024
  33. 5/8 fast-import: improve documentation for unquoted pathsThalia Archibald, Apr 1, 2024
  34. 6/8 fast-import: document C-style escapes for pathsThalia Archibald, Apr 1, 2024
  35. 7/8 fast-import: forbid escaped NUL in pathsThalia Archibald, Apr 1, 2024
  36. 8/8 fast-import: make comments more preciseThalia Archibald, Apr 1, 2024
  37. Thalia ArchibaldApr 7, 2024
  38. Eric SunshineApr 7, 2024
  39. Patrick SteinhardtApr 8, 2024
  40. Thalia ArchibaldApr 8, 2024
  41. Patrick SteinhardtApr 8, 2024
  42. Junio C HamanoApr 8, 2024
  43. 0/8 fast-import: tighten parsing of pathsThalia Archibald, Apr 10, 2024
  44. 1/8 fast-import: tighten path unquotingThalia Archibald, Apr 10, 2024
  45. 2/8 fast-import: directly use strbufs for pathsThalia Archibald, Apr 10, 2024
  46. 3/8 fast-import: allow unquoted empty path for rootThalia Archibald, Apr 10, 2024
  47. Junio C HamanoApr 11, 2024
  48. 5/8 fast-import: improve documentation for unquoted pathsThalia Archibald, Apr 10, 2024
  49. Junio C HamanoApr 11, 2024
  50. 6/8 fast-import: document C-style escapes for pathsThalia Archibald, Apr 10, 2024
  51. Junio C HamanoApr 10, 2024
  52. Thalia ArchibaldApr 10, 2024
  53. Junio C HamanoApr 11, 2024
  54. Patrick SteinhardtApr 11, 2024
  55. 7/8 fast-import: forbid escaped NUL in pathsThalia Archibald, Apr 10, 2024
  56. Junio C HamanoApr 10, 2024
  57. 8/8 fast-import: make comments more preciseThalia Archibald, Apr 10, 2024
  58. Junio C HamanoApr 10, 2024
  59. 4/8 fast-import: remove dead strbufThalia Archibald, Apr 10, 2024
  60. Junio C HamanoApr 11, 2024
  61. 0/8 fast-import: tighten parsing of pathsThalia Archibald, Apr 12, 2024
  62. 2/8 fast-import: directly use strbufs for pathsThalia Archibald, Apr 12, 2024
  63. 3/8 fast-import: allow unquoted empty path for rootThalia Archibald, Apr 12, 2024
  64. 4/8 fast-import: remove dead strbufThalia Archibald, Apr 12, 2024
  65. 5/8 fast-import: improve documentation for path quotingThalia Archibald, Apr 12, 2024
  66. 6/8 fast-import: document C-style escapes for pathsThalia Archibald, Apr 12, 2024
  67. 7/8 fast-import: forbid escaped NUL in pathsThalia Archibald, Apr 12, 2024
  68. 8/8 fast-import: make comments more preciseThalia Archibald, Apr 12, 2024
  69. 1/8 fast-import: tighten path unquotingThalia Archibald, Apr 12, 2024
  70. Junio C HamanoApr 12, 2024
  71. Thalia ArchibaldApr 13, 2024
  72. Junio C HamanoApr 13, 2024
  73. 0/8 fast-import: tighten parsing of pathsThalia Archibald, Apr 14, 2024
  74. 2/8 fast-import: directly use strbufs for pathsThalia Archibald, Apr 14, 2024
  75. 1/8 fast-import: tighten path unquotingThalia Archibald, Apr 14, 2024
  76. 3/8 fast-import: allow unquoted empty path for rootThalia Archibald, Apr 14, 2024
  77. 4/8 fast-import: remove dead strbufThalia Archibald, Apr 14, 2024
  78. 5/8 fast-import: improve documentation for path quotingThalia Archibald, Apr 14, 2024
  79. 6/8 fast-import: document C-style escapes for pathsThalia Archibald, Apr 14, 2024
  80. 7/8 fast-import: forbid escaped NUL in pathsThalia Archibald, Apr 14, 2024
  81. 8/8 fast-import: make comments more preciseThalia Archibald, Apr 14, 2024
  82. Patrick SteinhardtApr 15, 2024
  83. Junio C HamanoApr 15, 2024
  84. Thalia ArchibaldApr 1, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.