git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 04/22] reftable/basics: handle allocation failures in `reftable_calloc()`

From
Junio C Hamano <gitster@pobox.com>
Date
Sep 26, 2024, 16:13 UTC
Message-ID
<xmqqzfnul7fg.fsf@gitster.g>
In-Reply-To
<ZvVPiIzzLTTb75b8@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 11 quoted lines
>> But it illustrates why open coding is not necessarily an excellent
>> idea in the longer term, doesn't it?  When unsigned_mult_overflows()
>> is updated to avoid such a false positive, how would we remember
>> that we need to update this copy we?
>
> I agree in general, but with the reftable library I'm stuck between a
> rock and a hard place. My goal is to make it fully reusable by other
> projects without first having to do surgery on their side. While having
> something like `st_mult()` is simple enough to port over, the biggest
> problem I have is that over time we sneak in more and more code from the
> Git codebase. The result is death by a thousand cuts.
Show 9 quoted lines
> Now if we had a single header that exposes a small set of functions
> without _anything_ else it could work alright. But I rather doubt that
> we really want to have a standalone header for `st_mult()` that we can
> include. But without such a standalone header it is simply too easy to
> start building on top of Git features by accident.
>
> So I'm instead aiming to go a different way and fully cut the reftable
> code loose from Git. So even if we e.g. eventually want `struct strbuf`
> return errors on failure, it would only address one part of my problem.

The dependency to "strbuf" (just as an example) was added initially fairly early. Soon after 27f7ed2a (reftable: add LICENSE, 2021-10-07) added the reftable/ hierarchy, e303bf22 (reftable: (de)serialization for the polymorphic record type., 2021-10-07). I somehow had an impression that reftable "library" started without any Git dependency and then use of our helper functions seemed through from the shim layer, but it was pretty much part of the library from day one, it seems.

Show 14 quoted lines
> A couple months ago I said that I'll try to write something like shims
> that wrap our types in reftable types such that other projects can
> provide implementations for such shims themselves. I tried to make that
> work, but the result was an unholy mess that really didn't make any
> sense whatsoever. Most of the features that I need from the Git codebase
> can be provided in a couple of lines of code (`struct strbuf` is roughly
> 50 lines for example), plus maybe a callback function that can be
> provided to wire things up on the user side (`register_tempfiles()` for
> example). So once I saw that those wrappers are harder to use and result
> in roughly the same lines of code I decided to scrap that approach and
> instead try to convert it fully.
>
> So yeah, overall we shouldn't open-code things like this. But I don't
> really see another way to do this for the reftable library.

But isn't all of the above what Libification ought to be about? I was hoping that the reftable polishing would not have to be done by you alone, and you would recruit those who are into libification of other parts of Git codebase to help cleaning up these fringe (from the point of view of reftable) interfaces.

What do the libification folks feel about this (folks involved in libgit-sys CC'ed; of course all others who are interested in the libification topic are welcome to comment)?

Thanks.
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 37 of 151 in “reftable: handle allocation errors”
  1. 00/22 reftable: handle allocation errorsPatrick Steinhardt, Sep 16, 2024
  2. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 16, 2024
  3. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 16, 2024
  4. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 16, 2024
  5. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 16, 2024
  6. Junio C HamanoSep 21, 2024
  7. Patrick SteinhardtSep 24, 2024
  8. Patrick SteinhardtSep 24, 2024
  9. Junio C HamanoSep 24, 2024
  10. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 16, 2024
  11. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 16, 2024
  12. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 16, 2024
  13. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 16, 2024
  14. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 16, 2024
  15. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 16, 2024
  16. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 16, 2024
  17. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 16, 2024
  18. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 16, 2024
  19. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 16, 2024
  20. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 16, 2024
  21. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 16, 2024
  22. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 16, 2024
  23. Junio C HamanoSep 22, 2024
  24. Patrick SteinhardtSep 24, 2024
  25. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  26. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  27. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 16, 2024
  28. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  29. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 16, 2024
  30. 00/22 reftable: handle allocation errorsPatrick Steinhardt, Sep 24, 2024
  31. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 24, 2024
  32. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 24, 2024
  33. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 24, 2024
  34. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 24, 2024
  35. Junio C HamanoSep 24, 2024
  36. Patrick SteinhardtSep 26, 2024
  37. Junio C HamanoSep 26, 2024
  38. Patrick SteinhardtSep 27, 2024
  39. Han-Wen NienhuysSep 27, 2024
  40. Junio C HamanoSep 27, 2024
  41. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 24, 2024
  42. René ScharfeSep 24, 2024
  43. Patrick SteinhardtSep 26, 2024
  44. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 24, 2024
  45. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 24, 2024
  46. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 24, 2024
  47. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 24, 2024
  48. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 24, 2024
  49. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 24, 2024
  50. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 24, 2024
  51. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 24, 2024
  52. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 24, 2024
  53. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 24, 2024
  54. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 24, 2024
  55. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 24, 2024
  56. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  57. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  58. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 24, 2024
  59. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  60. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 24, 2024
  61. 00/22 refatble: handle allocation errorsPatrick Steinhardt, Sep 30, 2024
  62. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 30, 2024
  63. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 30, 2024
  64. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 30, 2024
  65. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 30, 2024
  66. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 30, 2024
  67. René ScharfeSep 30, 2024
  68. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 30, 2024
  69. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 30, 2024
  70. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 30, 2024
  71. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 30, 2024
  72. René ScharfeSep 30, 2024
  73. Patrick SteinhardtSep 30, 2024
  74. Junio C HamanoSep 30, 2024
  75. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 30, 2024
  76. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 30, 2024
  77. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 30, 2024
  78. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 30, 2024
  79. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 30, 2024
  80. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 30, 2024
  81. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 30, 2024
  82. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 30, 2024
  83. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  84. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  85. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 30, 2024
  86. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  87. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 30, 2024
  88. Junio C HamanoSep 30, 2024
  89. 00/25 reftable: handle allocation errorsPatrick Steinhardt, Oct 1, 2024
  90. 01/25 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Oct 1, 2024
  91. 02/25 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Oct 1, 2024
  92. 03/25 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Oct 1, 2024
  93. 04/25 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Oct 1, 2024
  94. 05/25 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Oct 1, 2024
  95. 06/25 reftable/record: handle allocation failures on copyPatrick Steinhardt, Oct 1, 2024
  96. 07/25 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Oct 1, 2024
  97. 08/25 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Oct 1, 2024
  98. 09/25 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Oct 1, 2024
  99. 10/25 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Oct 1, 2024
  100. 11/25 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Oct 1, 2024
  101. 12/25 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Oct 1, 2024
  102. 13/25 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Oct 1, 2024
  103. 14/25 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Oct 1, 2024
  104. 15/25 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Oct 1, 2024
  105. 16/25 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Oct 1, 2024
  106. 17/25 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Oct 1, 2024
  107. 18/25 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  108. 19/25 reftable/block: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  109. 20/25 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Oct 1, 2024
  110. 21/25 reftable/tree: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  111. 22/25 reftable: handle trivial allocation failuresPatrick Steinhardt, Oct 1, 2024
  112. 23/25 reftable: fix calls to free(3P)Patrick Steinhardt, Oct 1, 2024
  113. 24/25 reftable: introduce `REFTABLE_FREE_AND_NULL()`Patrick Steinhardt, Oct 1, 2024
  114. 25/25 reftable/basics: ban standard allocator functionsPatrick Steinhardt, Oct 1, 2024
  115. Junio C HamanoOct 1, 2024
  116. Patrick SteinhardtOct 2, 2024
  117. Junio C HamanoOct 1, 2024
  118. René ScharfeOct 1, 2024
  119. Junio C HamanoOct 1, 2024
  120. Patrick SteinhardtOct 2, 2024
  121. Junio C HamanoOct 2, 2024
  122. 00/25 reftable: handle allocation errorsPatrick Steinhardt, Oct 2, 2024
  123. 01/25 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Oct 2, 2024
  124. 02/25 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Oct 2, 2024
  125. 03/25 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Oct 2, 2024
  126. 04/25 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Oct 2, 2024
  127. 05/25 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Oct 2, 2024
  128. Eric SunshineOct 2, 2024
  129. Patrick SteinhardtOct 4, 2024
  130. Eric SunshineOct 4, 2024
  131. 06/25 reftable/record: handle allocation failures on copyPatrick Steinhardt, Oct 2, 2024
  132. 07/25 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Oct 2, 2024
  133. 08/25 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Oct 2, 2024
  134. 09/25 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Oct 2, 2024
  135. 10/25 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Oct 2, 2024
  136. 11/25 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Oct 2, 2024
  137. 12/25 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Oct 2, 2024
  138. 13/25 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Oct 2, 2024
  139. 14/25 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Oct 2, 2024
  140. 15/25 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Oct 2, 2024
  141. 16/25 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Oct 2, 2024
  142. 17/25 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Oct 2, 2024
  143. 18/25 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  144. 19/25 reftable/block: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  145. 20/25 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Oct 2, 2024
  146. 21/25 reftable/tree: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  147. 22/25 reftable: handle trivial allocation failuresPatrick Steinhardt, Oct 2, 2024
  148. 23/25 reftable: fix calls to free(3P)Patrick Steinhardt, Oct 2, 2024
  149. 24/25 reftable: introduce `REFTABLE_FREE_AND_NULL()`Patrick Steinhardt, Oct 2, 2024
  150. 25/25 reftable/basics: ban standard allocator functionsPatrick Steinhardt, Oct 2, 2024
  151. Junio C HamanoOct 2, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.