git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 00/22] refatble: handle allocation errors

From
Patrick Steinhardt <ps@pks.im>
Date
Sep 30, 2024, 08:08 UTC
Message-ID
<cover.1727680272.git.ps@pks.im>
In-Reply-To
<cover.1726489647.git.ps@pks.im>
Hi,

this is the third version of this patch series that converts the reftable library to start handling allocation errors. This is done such that the reftable library can truly behave like a library and let its callers handle such conditions.

There is only a single change compared to v1, namely that we use `reftable_free()` instead of `free()`.

Thanks!
Patrick
Patrick Steinhardt (22):
  reftable/error: introduce out-of-memory error code
  reftable/basics: merge "publicbasics" into "basics"
  reftable: introduce `reftable_strdup()`
  reftable/basics: handle allocation failures in `reftable_calloc()`
  reftable/basics: handle allocation failures in `parse_names()`
  reftable/record: handle allocation failures on copy
  reftable/record: handle allocation failures when decoding records
  reftable/writer: handle allocation failures in `writer_index_hash()`
  reftable/writer: handle allocation failures in `reftable_new_writer()`
  reftable/merged: handle allocation failures in
    `merged_table_init_iter()`
  reftable/reader: handle allocation failures for unindexed reader
  reftable/reader: handle allocation failures in `reader_init_iter()`
  reftable/stack: handle allocation failures on reload
  reftable/stack: handle allocation failures in `reftable_new_stack()`
  reftable/stack: handle allocation failures in `stack_compact_range()`
  reftable/stack: handle allocation failures in auto compaction
  reftable/iter: handle allocation failures when creating indexed table
    iter
  reftable/blocksource: handle allocation failures
  reftable/block: handle allocation failures
  reftable/pq: handle allocation failures when adding entries
  reftable/tree: handle allocation failures
  reftable: handle trivial allocation failures
 Makefile                            |   1 -
 refs/reftable-backend.c             |  39 ++++--
 reftable/basics.c                   |  92 ++++++++++++++-
 reftable/basics.h                   |  13 +-
 reftable/block.c                    |  23 +++-
 reftable/block.h                    |   4 +-
 reftable/blocksource.c              |  25 +++-
 reftable/error.c                    |   2 +
 reftable/iter.c                     |  20 +++-
 reftable/iter.h                     |   2 +-
 reftable/merged.c                   |  84 ++++++++-----
 reftable/merged.h                   |   6 +-
 reftable/pq.c                       |   7 +-
 reftable/pq.h                       |   2 +-
 reftable/publicbasics.c             |  66 -----------
 reftable/reader.c                   |  68 ++++++++---
 reftable/reader.h                   |   6 +-
 reftable/record.c                   | 164 +++++++++++++++++++-------
 reftable/record.h                   |   6 +-
 reftable/reftable-basics.h          |  18 +++
 reftable/reftable-error.h           |   3 +
 reftable/reftable-malloc.h          |  18 ---
 reftable/reftable-merged.h          |   8 +-
 reftable/reftable-reader.h          |   8 +-
 reftable/reftable-stack.h           |   8 +-
 reftable/reftable-writer.h          |  12 +-
 reftable/stack.c                    | 177 ++++++++++++++++++++++------
 reftable/tree.c                     |  42 +++++--
 reftable/tree.h                     |  21 +++-
 reftable/writer.c                   | 150 +++++++++++++++--------
 t/helper/test-reftable.c            |  10 +-
 t/unit-tests/lib-reftable.c         |   8 +-
 t/unit-tests/t-reftable-basics.c    |  11 +-
 t/unit-tests/t-reftable-block.c     |  24 ++--
 t/unit-tests/t-reftable-merged.c    |  16 ++-
 t/unit-tests/t-reftable-readwrite.c |  61 ++++++----
 t/unit-tests/t-reftable-stack.c     |   4 +-
 t/unit-tests/t-reftable-tree.c      |  10 +-
 38 files changed, 853 insertions(+), 386 deletions(-)
 delete mode 100644 reftable/publicbasics.c
 create mode 100644 reftable/reftable-basics.h
 delete mode 100644 reftable/reftable-malloc.h
Range-diff against v2:
 1:  8c99ecc325 =  1:  8c99ecc325 reftable/error: introduce out-of-memory error code
 2:  4dcdf1d48e =  2:  4dcdf1d48e reftable/basics: merge "publicbasics" into "basics"
 3:  21fa9b15d9 =  3:  21fa9b15d9 reftable: introduce `reftable_strdup()`
 4:  f6ad92ffd0 =  4:  f6ad92ffd0 reftable/basics: handle allocation failures in `reftable_calloc()`
 5:  ad028020df !  5:  922783708d reftable/basics: handle allocation failures in `parse_names()`
    @@ reftable/basics.c: void parse_names(char *buf, int size, char ***namesp)
     +
     +err:
     +	for (size_t i = 0; i < names_len; i++)
    -+		free(names[i]);
    -+	free(names);
    ++		reftable_free(names[i]);
    ++	reftable_free(names);
     +	return NULL;
      }
      
 6:  df713fbe08 =  6:  d4676e8a6a reftable/record: handle allocation failures on copy
 7:  870bb003c0 =  7:  874e6a1c20 reftable/record: handle allocation failures when decoding records
 8:  1d47e42500 =  8:  42bc57fd00 reftable/writer: handle allocation failures in `writer_index_hash()`
 9:  caa71f0a77 =  9:  9edd1d84cd reftable/writer: handle allocation failures in `reftable_new_writer()`
10:  a84e9cadae = 10:  d4004a7f43 reftable/merged: handle allocation failures in `merged_table_init_iter()`
11:  20d3833014 = 11:  b2bd142021 reftable/reader: handle allocation failures for unindexed reader
12:  e35c3a705d = 12:  9b5fd52862 reftable/reader: handle allocation failures in `reader_init_iter()`
13:  ca3b57f151 = 13:  0f3e3d1585 reftable/stack: handle allocation failures on reload
14:  7377421a63 = 14:  c88645a251 reftable/stack: handle allocation failures in `reftable_new_stack()`
15:  244e8667c5 = 15:  92ad4fc934 reftable/stack: handle allocation failures in `stack_compact_range()`
16:  99f4868c38 = 16:  19acebd919 reftable/stack: handle allocation failures in auto compaction
17:  271839a626 = 17:  620658bffc reftable/iter: handle allocation failures when creating indexed table iter
18:  c41808e9d7 = 18:  48047dd4c9 reftable/blocksource: handle allocation failures
19:  9348b0a457 = 19:  08685605ba reftable/block: handle allocation failures
20:  445daf9464 = 20:  a66937b7af reftable/pq: handle allocation failures when adding entries
21:  c31f9e53f5 = 21:  c291114553 reftable/tree: handle allocation failures
22:  d0fe999371 = 22:  20f50c446a reftable: handle trivial allocation failures
-- 
2.46.2.852.g229c0bf0e5.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 61 of 151 in “reftable: handle allocation errors”
  1. 00/22 reftable: handle allocation errorsPatrick Steinhardt, Sep 16, 2024
  2. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 16, 2024
  3. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 16, 2024
  4. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 16, 2024
  5. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 16, 2024
  6. Junio C HamanoSep 21, 2024
  7. Patrick SteinhardtSep 24, 2024
  8. Patrick SteinhardtSep 24, 2024
  9. Junio C HamanoSep 24, 2024
  10. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 16, 2024
  11. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 16, 2024
  12. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 16, 2024
  13. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 16, 2024
  14. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 16, 2024
  15. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 16, 2024
  16. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 16, 2024
  17. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 16, 2024
  18. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 16, 2024
  19. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 16, 2024
  20. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 16, 2024
  21. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 16, 2024
  22. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 16, 2024
  23. Junio C HamanoSep 22, 2024
  24. Patrick SteinhardtSep 24, 2024
  25. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  26. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  27. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 16, 2024
  28. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 16, 2024
  29. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 16, 2024
  30. 00/22 reftable: handle allocation errorsPatrick Steinhardt, Sep 24, 2024
  31. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 24, 2024
  32. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 24, 2024
  33. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 24, 2024
  34. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 24, 2024
  35. Junio C HamanoSep 24, 2024
  36. Patrick SteinhardtSep 26, 2024
  37. Junio C HamanoSep 26, 2024
  38. Patrick SteinhardtSep 27, 2024
  39. Han-Wen NienhuysSep 27, 2024
  40. Junio C HamanoSep 27, 2024
  41. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 24, 2024
  42. René ScharfeSep 24, 2024
  43. Patrick SteinhardtSep 26, 2024
  44. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 24, 2024
  45. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 24, 2024
  46. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 24, 2024
  47. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 24, 2024
  48. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 24, 2024
  49. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 24, 2024
  50. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 24, 2024
  51. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 24, 2024
  52. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 24, 2024
  53. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 24, 2024
  54. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 24, 2024
  55. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 24, 2024
  56. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  57. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  58. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 24, 2024
  59. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 24, 2024
  60. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 24, 2024
  61. 00/22 refatble: handle allocation errorsPatrick Steinhardt, Sep 30, 2024
  62. 01/22 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Sep 30, 2024
  63. 02/22 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Sep 30, 2024
  64. 03/22 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Sep 30, 2024
  65. 04/22 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Sep 30, 2024
  66. 05/22 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Sep 30, 2024
  67. René ScharfeSep 30, 2024
  68. 06/22 reftable/record: handle allocation failures on copyPatrick Steinhardt, Sep 30, 2024
  69. 07/22 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Sep 30, 2024
  70. 08/22 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Sep 30, 2024
  71. 09/22 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Sep 30, 2024
  72. René ScharfeSep 30, 2024
  73. Patrick SteinhardtSep 30, 2024
  74. Junio C HamanoSep 30, 2024
  75. 10/22 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Sep 30, 2024
  76. 11/22 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Sep 30, 2024
  77. 12/22 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Sep 30, 2024
  78. 13/22 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Sep 30, 2024
  79. 14/22 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Sep 30, 2024
  80. 15/22 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Sep 30, 2024
  81. 16/22 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Sep 30, 2024
  82. 17/22 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Sep 30, 2024
  83. 18/22 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  84. 19/22 reftable/block: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  85. 20/22 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Sep 30, 2024
  86. 21/22 reftable/tree: handle allocation failuresPatrick Steinhardt, Sep 30, 2024
  87. 22/22 reftable: handle trivial allocation failuresPatrick Steinhardt, Sep 30, 2024
  88. Junio C HamanoSep 30, 2024
  89. 00/25 reftable: handle allocation errorsPatrick Steinhardt, Oct 1, 2024
  90. 01/25 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Oct 1, 2024
  91. 02/25 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Oct 1, 2024
  92. 03/25 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Oct 1, 2024
  93. 04/25 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Oct 1, 2024
  94. 05/25 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Oct 1, 2024
  95. 06/25 reftable/record: handle allocation failures on copyPatrick Steinhardt, Oct 1, 2024
  96. 07/25 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Oct 1, 2024
  97. 08/25 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Oct 1, 2024
  98. 09/25 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Oct 1, 2024
  99. 10/25 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Oct 1, 2024
  100. 11/25 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Oct 1, 2024
  101. 12/25 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Oct 1, 2024
  102. 13/25 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Oct 1, 2024
  103. 14/25 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Oct 1, 2024
  104. 15/25 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Oct 1, 2024
  105. 16/25 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Oct 1, 2024
  106. 17/25 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Oct 1, 2024
  107. 18/25 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  108. 19/25 reftable/block: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  109. 20/25 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Oct 1, 2024
  110. 21/25 reftable/tree: handle allocation failuresPatrick Steinhardt, Oct 1, 2024
  111. 22/25 reftable: handle trivial allocation failuresPatrick Steinhardt, Oct 1, 2024
  112. 23/25 reftable: fix calls to free(3P)Patrick Steinhardt, Oct 1, 2024
  113. 24/25 reftable: introduce `REFTABLE_FREE_AND_NULL()`Patrick Steinhardt, Oct 1, 2024
  114. 25/25 reftable/basics: ban standard allocator functionsPatrick Steinhardt, Oct 1, 2024
  115. Junio C HamanoOct 1, 2024
  116. Patrick SteinhardtOct 2, 2024
  117. Junio C HamanoOct 1, 2024
  118. René ScharfeOct 1, 2024
  119. Junio C HamanoOct 1, 2024
  120. Patrick SteinhardtOct 2, 2024
  121. Junio C HamanoOct 2, 2024
  122. 00/25 reftable: handle allocation errorsPatrick Steinhardt, Oct 2, 2024
  123. 01/25 reftable/error: introduce out-of-memory error codePatrick Steinhardt, Oct 2, 2024
  124. 02/25 reftable/basics: merge "publicbasics" into "basics"Patrick Steinhardt, Oct 2, 2024
  125. 03/25 reftable: introduce `reftable_strdup()`Patrick Steinhardt, Oct 2, 2024
  126. 04/25 reftable/basics: handle allocation failures in `reftable_calloc()`Patrick Steinhardt, Oct 2, 2024
  127. 05/25 reftable/basics: handle allocation failures in `parse_names()`Patrick Steinhardt, Oct 2, 2024
  128. Eric SunshineOct 2, 2024
  129. Patrick SteinhardtOct 4, 2024
  130. Eric SunshineOct 4, 2024
  131. 06/25 reftable/record: handle allocation failures on copyPatrick Steinhardt, Oct 2, 2024
  132. 07/25 reftable/record: handle allocation failures when decoding recordsPatrick Steinhardt, Oct 2, 2024
  133. 08/25 reftable/writer: handle allocation failures in `writer_index_hash()`Patrick Steinhardt, Oct 2, 2024
  134. 09/25 reftable/writer: handle allocation failures in `reftable_new_writer()`Patrick Steinhardt, Oct 2, 2024
  135. 10/25 reftable/merged: handle allocation failures in `merged_table_init_iter()`Patrick Steinhardt, Oct 2, 2024
  136. 11/25 reftable/reader: handle allocation failures for unindexed readerPatrick Steinhardt, Oct 2, 2024
  137. 12/25 reftable/reader: handle allocation failures in `reader_init_iter()`Patrick Steinhardt, Oct 2, 2024
  138. 13/25 reftable/stack: handle allocation failures on reloadPatrick Steinhardt, Oct 2, 2024
  139. 14/25 reftable/stack: handle allocation failures in `reftable_new_stack()`Patrick Steinhardt, Oct 2, 2024
  140. 15/25 reftable/stack: handle allocation failures in `stack_compact_range()`Patrick Steinhardt, Oct 2, 2024
  141. 16/25 reftable/stack: handle allocation failures in auto compactionPatrick Steinhardt, Oct 2, 2024
  142. 17/25 reftable/iter: handle allocation failures when creating indexed table iterPatrick Steinhardt, Oct 2, 2024
  143. 18/25 reftable/blocksource: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  144. 19/25 reftable/block: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  145. 20/25 reftable/pq: handle allocation failures when adding entriesPatrick Steinhardt, Oct 2, 2024
  146. 21/25 reftable/tree: handle allocation failuresPatrick Steinhardt, Oct 2, 2024
  147. 22/25 reftable: handle trivial allocation failuresPatrick Steinhardt, Oct 2, 2024
  148. 23/25 reftable: fix calls to free(3P)Patrick Steinhardt, Oct 2, 2024
  149. 24/25 reftable: introduce `REFTABLE_FREE_AND_NULL()`Patrick Steinhardt, Oct 2, 2024
  150. 25/25 reftable/basics: ban standard allocator functionsPatrick Steinhardt, Oct 2, 2024
  151. Junio C HamanoOct 2, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.