git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v6 1/6] object-file.c: release strbuf in write_loose_object()

From
Junio C Hamano <gitster@pobox.com>
Date
Dec 18, 2021, 00:09 UTC
Message-ID
<xmqqa6gy22py.fsf@gitster.g>
In-Reply-To
<c860c56f-ce25-4391-7f65-50c9d5d80c2c@web.de>
René Scharfe <l.s.r@web.de> writes:
Show 14 quoted lines
> There was no leak before.  Both strbufs are static and both functions
> they are passed to (loose_object_path() and create_tmpfile()) reset
> them first.  So while the allocated memory was not released before,
> it was reused.
>
> Not sure if making write_loose_object() allocate and release these
> buffers on every call has much of a performance impact.  The only
> reason I can think of for wanting such a change is to get rid of the
> static buffers, to allow the function to be used by concurrent
> threads.
>
> So I think either keeping the code as-is or also making the strbufs
> non-static would be better (but then discussing a possible
> performance impact in the commit message would be nice).
Makes sense.
Previous: René ScharfeNext: Han Xin
Message 153 of 211 in “unpack-objects: unpack large object in stream”
  1. unpack-objects: unpack large object in streamHan Xin, Oct 9, 2021
  2. Han XinOct 19, 2021
  3. Philip OakleyOct 20, 2021
  4. Han XinOct 21, 2021
  5. Philip OakleyOct 21, 2021
  6. Han XinNov 3, 2021
  7. Philip OakleyNov 3, 2021
  8. 1/6 object-file: refactor write_loose_object() to support inputstreamHan Xin, Nov 12, 2021
  9. Jiang XinNov 18, 2021
  10. Junio C HamanoNov 18, 2021
  11. 2/6 object-file.c: add dry_run mode for write_loose_object()Han Xin, Nov 12, 2021
  12. Jiang XinNov 18, 2021
  13. 3/6 object-file.c: handle nil oid in write_loose_object()Han Xin, Nov 12, 2021
  14. Jiang XinNov 18, 2021
  15. 4/6 object-file.c: read input stream repeatedly in write_loose_object()Han Xin, Nov 12, 2021
  16. Jiang XinNov 18, 2021
  17. 5/6 object-store.h: add write_loose_object()Han Xin, Nov 12, 2021
  18. 6/6 unpack-objects: unpack large object in streamHan Xin, Nov 12, 2021
  19. Jiang XinNov 18, 2021
  20. 0/5 unpack large objects in streamHan Xin, Nov 22, 2021
  21. Han XinNov 29, 2021
  22. Jeff KingNov 29, 2021
  23. Han XinNov 30, 2021
  24. 0/5 unpack large objects in streamHan Xin, Dec 3, 2021
  25. Derrick StoleeDec 7, 2021
  26. 0/6 unpack large blobs in streamHan Xin, Dec 10, 2021
  27. 0/6 unpack large blobs in streamHan Xin, Dec 17, 2021
  28. 0/5 unpack large blobs in streamHan Xin, Dec 21, 2021
  29. 1/5 unpack-objects.c: add dry_run mode for get_data()Han Xin, Dec 21, 2021
  30. Ævar Arnfjörð BjarmasonDec 21, 2021
  31. René ScharfeDec 21, 2021
  32. Ævar Arnfjörð BjarmasonDec 21, 2021
  33. Jiang XinDec 22, 2021
  34. Jiang XinDec 22, 2021
  35. Jiang XinDec 31, 2021
  36. 2/5 object-file API: add a format_object_header() functionHan Xin, Dec 21, 2021
  37. René ScharfeDec 21, 2021
  38. C99 %z (was: [PATCH v7 2/5] object-file API: add a format_object_header() function)Ævar Arnfjörð Bjarmason, Feb 1, 2022
  39. Jiang XinDec 31, 2021
  40. 3/5 object-file.c: refactor write_loose_object() to reuse in stream versionHan Xin, Dec 21, 2021
  41. Ævar Arnfjörð BjarmasonDec 21, 2021
  42. Jiang XinDec 22, 2021
  43. 4/5 object-file.c: add "write_stream_object_file()" to support read in streamHan Xin, Dec 21, 2021
  44. Ævar Arnfjörð BjarmasonDec 21, 2021
  45. Ævar Arnfjörð BjarmasonDec 21, 2021
  46. 5/5 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Dec 21, 2021
  47. Ævar Arnfjörð BjarmasonDec 21, 2021
  48. Jiang XinDec 31, 2021
  49. 0/6 unpack large blobs in streamHan Xin, Jan 8, 2022
  50. 1/5 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, Jan 20, 2022
  51. 0/5 unpack large blobs in streamHan Xin, Jan 20, 2022
  52. Ævar Arnfjörð BjarmasonFeb 1, 2022
  53. Han XinFeb 2, 2022
  54. Ævar Arnfjörð BjarmasonFeb 2, 2022
  55. 0/6 unpack-objects: support streaming large objects to diskÆvar Arnfjörð Bjarmason, Feb 4, 2022
  56. 1/6 unpack-objects: low memory footprint for get_data() in dry_run modeÆvar Arnfjörð Bjarmason, Feb 4, 2022
  57. 2/6 object-file.c: do fsync() and close() before post-write die()Ævar Arnfjörð Bjarmason, Feb 4, 2022
  58. 4/6 object-file.c: add "stream_loose_object()" to handle large objectÆvar Arnfjörð Bjarmason, Feb 4, 2022
  59. 3/6 object-file.c: refactor write_loose_object() to several stepsÆvar Arnfjörð Bjarmason, Feb 4, 2022
  60. 5/6 core doc: modernize core.bigFileThreshold documentationÆvar Arnfjörð Bjarmason, Feb 4, 2022
  61. 6/6 unpack-objects: use stream_loose_object() to unpack large objectsÆvar Arnfjörð Bjarmason, Feb 4, 2022
  62. 0/8 unpack-objects: support streaming blobs to diskÆvar Arnfjörð Bjarmason, Mar 19, 2022
  63. 1/8 unpack-objects: low memory footprint for get_data() in dry_run modeÆvar Arnfjörð Bjarmason, Mar 19, 2022
  64. 2/8 object-file.c: do fsync() and close() before post-write die()Ævar Arnfjörð Bjarmason, Mar 19, 2022
  65. 3/8 object-file.c: refactor write_loose_object() to several stepsÆvar Arnfjörð Bjarmason, Mar 19, 2022
  66. René ScharfeMar 19, 2022
  67. 4/8 object-file.c: factor out deflate part of write_loose_object()Ævar Arnfjörð Bjarmason, Mar 19, 2022
  68. 5/8 object-file.c: add "stream_loose_object()" to handle large objectÆvar Arnfjörð Bjarmason, Mar 19, 2022
  69. 6/8 core doc: modernize core.bigFileThreshold documentationÆvar Arnfjörð Bjarmason, Mar 19, 2022
  70. 7/8 unpack-objects: refactor away unpack_non_delta_entry()Ævar Arnfjörð Bjarmason, Mar 19, 2022
  71. 8/8 unpack-objects: use stream_loose_object() to unpack large objectsÆvar Arnfjörð Bjarmason, Mar 19, 2022
  72. 0/8 unpack-objects: support streaming blobs to diskÆvar Arnfjörð Bjarmason, Mar 29, 2022
  73. 2/8 object-file.c: do fsync() and close() before post-write die()Ævar Arnfjörð Bjarmason, Mar 29, 2022
  74. 1/8 unpack-objects: low memory footprint for get_data() in dry_run modeÆvar Arnfjörð Bjarmason, Mar 29, 2022
  75. 3/8 object-file.c: refactor write_loose_object() to several stepsÆvar Arnfjörð Bjarmason, Mar 29, 2022
  76. Han XinMar 30, 2022
  77. Ævar Arnfjörð BjarmasonMar 30, 2022
  78. 4/8 object-file.c: factor out deflate part of write_loose_object()Ævar Arnfjörð Bjarmason, Mar 29, 2022
  79. 5/8 object-file.c: add "stream_loose_object()" to handle large objectÆvar Arnfjörð Bjarmason, Mar 29, 2022
  80. Neeraj SinghMar 31, 2022
  81. 6/8 core doc: modernize core.bigFileThreshold documentationÆvar Arnfjörð Bjarmason, Mar 29, 2022
  82. 7/8 unpack-objects: refactor away unpack_non_delta_entry()Ævar Arnfjörð Bjarmason, Mar 29, 2022
  83. René ScharfeMar 30, 2022
  84. Ævar Arnfjörð BjarmasonMar 31, 2022
  85. René ScharfeMar 31, 2022
  86. 8/8 unpack-objects: use stream_loose_object() to unpack large objectsÆvar Arnfjörð Bjarmason, Mar 29, 2022
  87. 0/7 unpack-objects: support streaming blobs to diskÆvar Arnfjörð Bjarmason, Jun 4, 2022
  88. 1/7 unpack-objects: low memory footprint for get_data() in dry_run modeÆvar Arnfjörð Bjarmason, Jun 4, 2022
  89. Junio C HamanoJun 6, 2022
  90. Han XinJun 9, 2022
  91. Junio C HamanoJun 9, 2022
  92. Han XinJun 10, 2022
  93. Ævar Arnfjörð BjarmasonJun 10, 2022
  94. Han XinJun 10, 2022
  95. 2/7 object-file.c: do fsync() and close() before post-write die()Ævar Arnfjörð Bjarmason, Jun 4, 2022
  96. Junio C HamanoJun 6, 2022
  97. 3/7 object-file.c: refactor write_loose_object() to several stepsÆvar Arnfjörð Bjarmason, Jun 4, 2022
  98. 4/7 object-file.c: factor out deflate part of write_loose_object()Ævar Arnfjörð Bjarmason, Jun 4, 2022
  99. 5/7 object-file.c: add "stream_loose_object()" to handle large objectÆvar Arnfjörð Bjarmason, Jun 4, 2022
  100. Junio C HamanoJun 6, 2022
  101. Junio C HamanoJun 6, 2022
  102. Han XinJun 9, 2022
  103. Han XinJun 9, 2022
  104. Neeraj SinghJun 7, 2022
  105. Junio C HamanoJun 8, 2022
  106. object-file.c: batched disk flushes for stream_loose_object()Han Xin, Jun 9, 2022
  107. Neeraj SinghJun 9, 2022
  108. Johannes SchindelinJun 9, 2022
  109. Han XinJun 10, 2022
  110. 7/7 unpack-objects: use stream_loose_object() to unpack large objectsÆvar Arnfjörð Bjarmason, Jun 4, 2022
  111. 6/7 core doc: modernize core.bigFileThreshold documentationÆvar Arnfjörð Bjarmason, Jun 4, 2022
  112. Junio C HamanoJun 6, 2022
  113. 0/7 unpack-objects: support streaming blobs to diskHan Xin, Jun 10, 2022
  114. 1/7 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, Jun 10, 2022
  115. 2/7 object-file.c: do fsync() and close() before post-write die()Han Xin, Jun 10, 2022
  116. René ScharfeJun 10, 2022
  117. Junio C HamanoJun 10, 2022
  118. Han XinJun 11, 2022
  119. 3/7 object-file.c: refactor write_loose_object() to several stepsHan Xin, Jun 10, 2022
  120. 4/7 object-file.c: factor out deflate part of write_loose_object()Han Xin, Jun 10, 2022
  121. 5/7 object-file.c: add "stream_loose_object()" to handle large objectHan Xin, Jun 10, 2022
  122. 6/7 core doc: modernize core.bigFileThreshold documentationHan Xin, Jun 10, 2022
  123. Junio C HamanoJun 10, 2022
  124. 7/7 unpack-objects: use stream_loose_object() to unpack large objectsHan Xin, Jun 10, 2022
  125. 0/6 unpack-objects: support streaming blobs to diskHan Xin, Jun 11, 2022
  126. 1/6 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, Jun 11, 2022
  127. 2/6 object-file.c: refactor write_loose_object() to several stepsHan Xin, Jun 11, 2022
  128. 3/6 object-file.c: factor out deflate part of write_loose_object()Han Xin, Jun 11, 2022
  129. 4/6 object-file.c: add "stream_loose_object()" to handle large objectHan Xin, Jun 11, 2022
  130. 5/6 core doc: modernize core.bigFileThreshold documentationHan Xin, Jun 11, 2022
  131. 6/6 unpack-objects: use stream_loose_object() to unpack large objectsHan Xin, Jun 11, 2022
  132. Junio C HamanoJul 1, 2022
  133. 0/1 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, May 20, 2022
  134. 1/1 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, May 20, 2022
  135. 2/5 object-file.c: refactor write_loose_object() to several stepsHan Xin, Jan 20, 2022
  136. 3/5 object-file.c: add "stream_loose_object()" to handle large objectHan Xin, Jan 20, 2022
  137. 4/5 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Jan 20, 2022
  138. 5/5 object-file API: add a format_object_header() functionHan Xin, Jan 20, 2022
  139. 1/6 unpack-objects: low memory footprint for get_data() in dry_run modeHan Xin, Jan 8, 2022
  140. René ScharfeJan 8, 2022
  141. Han XinJan 11, 2022
  142. 2/6 object-file.c: refactor write_loose_object() to several stepsHan Xin, Jan 8, 2022
  143. René ScharfeJan 8, 2022
  144. Han XinJan 11, 2022
  145. 3/6 object-file.c: remove the slash for directory_size()Han Xin, Jan 8, 2022
  146. René ScharfeJan 8, 2022
  147. Han XinJan 11, 2022
  148. 4/6 object-file.c: add "stream_loose_object()" to handle large objectHan Xin, Jan 8, 2022
  149. 6/6 object-file API: add a format_object_header() functionHan Xin, Jan 8, 2022
  150. 5/6 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Jan 8, 2022
  151. 1/6 object-file.c: release strbuf in write_loose_object()Han Xin, Dec 17, 2021
  152. René ScharfeDec 17, 2021
  153. Junio C HamanoDec 18, 2021
  154. 2/6 object-file.c: refactor object header generation into a functionHan Xin, Dec 17, 2021
  155. object-file API: add a format_loose_header() functionÆvar Arnfjörð Bjarmason, Dec 20, 2021
  156. Philip OakleyDec 20, 2021
  157. Junio C HamanoDec 20, 2021
  158. Ævar Arnfjörð BjarmasonDec 21, 2021
  159. Junio C HamanoDec 21, 2021
  160. Ævar Arnfjörð BjarmasonDec 21, 2021
  161. Han XinDec 21, 2021
  162. 3/6 object-file.c: refactor write_loose_object() to reuse in stream versionHan Xin, Dec 17, 2021
  163. 4/6 object-file.c: make "write_object_file_flags()" to support read in streamHan Xin, Dec 17, 2021
  164. René ScharfeDec 17, 2021
  165. 5/6 unpack-objects.c: add dry_run mode for get_data()Han Xin, Dec 17, 2021
  166. René ScharfeDec 17, 2021
  167. 6/6 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Dec 17, 2021
  168. 1/6 object-file: refactor write_loose_object() to support read from streamHan Xin, Dec 10, 2021
  169. 2/6 object-file.c: handle undetermined oid in write_loose_object()Han Xin, Dec 10, 2021
  170. Ævar Arnfjörð BjarmasonDec 13, 2021
  171. 3/6 object-file.c: read stream in a loop in write_loose_object()Han Xin, Dec 10, 2021
  172. 4/6 unpack-objects.c: add dry_run mode for get_data()Han Xin, Dec 10, 2021
  173. 5/6 object-file.c: make "write_object_file_flags()" to support "HASH_STREAM"Han Xin, Dec 10, 2021
  174. 6/6 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Dec 10, 2021
  175. Ævar Arnfjörð BjarmasonDec 13, 2021
  176. 1/5 object-file: refactor write_loose_object() to read buffer from streamHan Xin, Dec 3, 2021
  177. Ævar Arnfjörð BjarmasonDec 3, 2021
  178. Han XinDec 6, 2021
  179. 2/5 object-file.c: handle undetermined oid in write_loose_object()Han Xin, Dec 3, 2021
  180. Ævar Arnfjörð BjarmasonDec 3, 2021
  181. Han XinDec 6, 2021
  182. Ævar Arnfjörð BjarmasonDec 3, 2021
  183. Han XinDec 6, 2021
  184. 3/5 object-file.c: read stream in a loop in write_loose_object()Han Xin, Dec 3, 2021
  185. 4/5 unpack-objects.c: add dry_run mode for get_data()Han Xin, Dec 3, 2021
  186. Ævar Arnfjörð BjarmasonDec 3, 2021
  187. Han XinDec 6, 2021
  188. 5/5 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Dec 3, 2021
  189. Ævar Arnfjörð BjarmasonDec 3, 2021
  190. Han XinDec 7, 2021
  191. Ævar Arnfjörð BjarmasonDec 3, 2021
  192. Han XinDec 7, 2021
  193. Ævar Arnfjörð BjarmasonDec 3, 2021
  194. Han XinDec 7, 2021
  195. 2/5 object-file.c: handle undetermined oid in write_loose_object()Han Xin, Nov 22, 2021
  196. Derrick StoleeNov 29, 2021
  197. Junio C HamanoNov 29, 2021
  198. Derrick StoleeNov 29, 2021
  199. Han XinNov 30, 2021
  200. 1/5 object-file: refactor write_loose_object() to read buffer from streamHan Xin, Nov 22, 2021
  201. Junio C HamanoNov 23, 2021
  202. Han XinNov 24, 2021
  203. 3/5 object-file.c: read stream in a loop in write_loose_object()Han Xin, Nov 22, 2021
  204. 4/5 unpack-objects.c: add dry_run mode for get_data()Han Xin, Nov 22, 2021
  205. 5/5 unpack-objects: unpack_non_delta_entry() read data in a streamHan Xin, Nov 22, 2021
  206. Derrick StoleeNov 29, 2021
  207. Han XinNov 30, 2021
  208. Derrick StoleeNov 30, 2021
  209. "git hyperfine" (was: [PATCH v3 5/5] unpack-objects[...])Ævar Arnfjörð Bjarmason, Dec 1, 2021
  210. Han XinDec 2, 2021
  211. Derrick StoleeDec 2, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.