[PATCH v5 00/25] reftable: handle allocation errors

Hi,

this is the fifth version of this patch series that converts the reftable library to start handling allocation errors. This is done such that the reftable library can truly behave like a library and let its callers handle such conditions.

Changes compared to v4:

  - `#undef` standard allocators before banning them.

  - Explain why we don't ban `FREE_AND_NULL()` and related as well as
    `xmalloc()` and related.

Thanks!

Patrick

Patrick Steinhardt (25):
  reftable/error: introduce out-of-memory error code
  reftable/basics: merge "publicbasics" into "basics"
  reftable: introduce `reftable_strdup()`
  reftable/basics: handle allocation failures in `reftable_calloc()`
  reftable/basics: handle allocation failures in `parse_names()`
  reftable/record: handle allocation failures on copy
  reftable/record: handle allocation failures when decoding records
  reftable/writer: handle allocation failures in `writer_index_hash()`
  reftable/writer: handle allocation failures in `reftable_new_writer()`
  reftable/merged: handle allocation failures in
    `merged_table_init_iter()`
  reftable/reader: handle allocation failures for unindexed reader
  reftable/reader: handle allocation failures in `reader_init_iter()`
  reftable/stack: handle allocation failures on reload
  reftable/stack: handle allocation failures in `reftable_new_stack()`
  reftable/stack: handle allocation failures in `stack_compact_range()`
  reftable/stack: handle allocation failures in auto compaction
  reftable/iter: handle allocation failures when creating indexed table
    iter
  reftable/blocksource: handle allocation failures
  reftable/block: handle allocation failures
  reftable/pq: handle allocation failures when adding entries
  reftable/tree: handle allocation failures
  reftable: handle trivial allocation failures
  reftable: fix calls to free(3P)
  reftable: introduce `REFTABLE_FREE_AND_NULL()`
  reftable/basics: ban standard allocator functions

 Makefile                            |   1 -
 refs/reftable-backend.c             |  39 ++++--
 reftable/basics.c                   |  93 +++++++++++++-
 reftable/basics.h                   |  28 ++++-
 reftable/block.c                    |  29 ++++-
 reftable/block.h                    |   4 +-
 reftable/blocksource.c              |  25 +++-
 reftable/error.c                    |   2 +
 reftable/iter.c                     |  22 +++-
 reftable/iter.h                     |   2 +-
 reftable/merged.c                   |  84 ++++++++-----
 reftable/merged.h                   |   6 +-
 reftable/pq.c                       |   9 +-
 reftable/pq.h                       |   2 +-
 reftable/publicbasics.c             |  66 ----------
 reftable/reader.c                   |  70 ++++++++---
 reftable/reader.h                   |   6 +-
 reftable/record.c                   | 174 +++++++++++++++++++-------
 reftable/record.h                   |   6 +-
 reftable/reftable-basics.h          |  18 +++
 reftable/reftable-error.h           |   3 +
 reftable/reftable-malloc.h          |  18 ---
 reftable/reftable-merged.h          |   8 +-
 reftable/reftable-reader.h          |   8 +-
 reftable/reftable-stack.h           |   8 +-
 reftable/reftable-writer.h          |  12 +-
 reftable/stack.c                    | 187 +++++++++++++++++++++-------
 reftable/tree.c                     |  42 +++++--
 reftable/tree.h                     |  21 +++-
 reftable/writer.c                   | 154 +++++++++++++++--------
 t/helper/test-reftable.c            |  10 +-
 t/unit-tests/lib-reftable.c         |   8 +-
 t/unit-tests/t-reftable-basics.c    |  11 +-
 t/unit-tests/t-reftable-block.c     |  24 ++--
 t/unit-tests/t-reftable-merged.c    |  16 ++-
 t/unit-tests/t-reftable-readwrite.c |  65 ++++++----
 t/unit-tests/t-reftable-stack.c     |   6 +-
 t/unit-tests/t-reftable-tree.c      |  10 +-
 38 files changed, 890 insertions(+), 407 deletions(-)
 delete mode 100644 reftable/publicbasics.c
 create mode 100644 reftable/reftable-basics.h
 delete mode 100644 reftable/reftable-malloc.h

Range-diff against v4:
 1:  94eaef3ae5 =  1:  94eaef3ae5 reftable/error: introduce out-of-memory error code
 2:  fe55051cb7 =  2:  fe55051cb7 reftable/basics: merge "publicbasics" into "basics"
 3:  b1a10d41d3 =  3:  b1a10d41d3 reftable: introduce `reftable_strdup()`
 4:  fd9141e985 =  4:  fd9141e985 reftable/basics: handle allocation failures in `reftable_calloc()`
 5:  bdfddbebce =  5:  bdfddbebce reftable/basics: handle allocation failures in `parse_names()`
 6:  756a32c285 =  6:  756a32c285 reftable/record: handle allocation failures on copy
 7:  ca64971e7b =  7:  ca64971e7b reftable/record: handle allocation failures when decoding records
 8:  fc2f113cba =  8:  fc2f113cba reftable/writer: handle allocation failures in `writer_index_hash()`
 9:  0ed99e0bdf =  9:  0ed99e0bdf reftable/writer: handle allocation failures in `reftable_new_writer()`
10:  8dfbfd9286 = 10:  8dfbfd9286 reftable/merged: handle allocation failures in `merged_table_init_iter()`
11:  7b592a6c6b = 11:  7b592a6c6b reftable/reader: handle allocation failures for unindexed reader
12:  ceed838265 = 12:  ceed838265 reftable/reader: handle allocation failures in `reader_init_iter()`
13:  1e997a5766 = 13:  1e997a5766 reftable/stack: handle allocation failures on reload
14:  40d4d81378 = 14:  40d4d81378 reftable/stack: handle allocation failures in `reftable_new_stack()`
15:  6aaae4baa6 = 15:  6aaae4baa6 reftable/stack: handle allocation failures in `stack_compact_range()`
16:  103a59ef0a = 16:  103a59ef0a reftable/stack: handle allocation failures in auto compaction
17:  6799d299fe = 17:  6799d299fe reftable/iter: handle allocation failures when creating indexed table iter
18:  c7e54d71d7 = 18:  c7e54d71d7 reftable/blocksource: handle allocation failures
19:  92d39b9021 = 19:  92d39b9021 reftable/block: handle allocation failures
20:  3416004e0d = 20:  3416004e0d reftable/pq: handle allocation failures when adding entries
21:  a363930488 = 21:  a363930488 reftable/tree: handle allocation failures
22:  28661500ff = 22:  28661500ff reftable: handle trivial allocation failures
23:  228cc81263 = 23:  228cc81263 reftable: fix calls to free(3P)
24:  1c66f6ef8d = 24:  1c66f6ef8d reftable: introduce `REFTABLE_FREE_AND_NULL()`
25:  764961e6f0 ! 25:  bc80f1e726 reftable/basics: ban standard allocator functions
    @@ Commit message
         Introduce a couple of macros that ban the standard allocators, similar
         to how we do it in "banned.h".
     
    +    Note that we do not ban the following two classes of functions:
    +
    +      - Macros like `FREE_AND_NULL()` or `REALLOC_ARRAY()`. As those expand
    +        to code that contains already-banned functions we'd get a compiler
    +        error even without banning those macros explicitly.
    +
    +      - Git-specific allocators like `xmalloc()` and friends. The primary
    +        reason is that there are simply too many of them, so we're rather
    +        aiming for best effort here. Furthermore, the eventual goal is to
    +        make them unavailable in the reftable library place by not pulling
    +        them in via "git-compat-utils.h" anymore.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## reftable/basics.c ##
    @@ reftable/basics.h: char *reftable_strdup(const char *str);
      
     +#ifndef REFTABLE_ALLOW_BANNED_ALLOCATORS
     +# define REFTABLE_BANNED(func) use_reftable_##func##_instead
    ++# undef malloc
     +# define malloc(sz) REFTABLE_BANNED(malloc)
    ++# undef realloc
     +# define realloc(ptr, sz) REFTABLE_BANNED(realloc)
    ++# undef free
     +# define free(ptr) REFTABLE_BANNED(free)
    ++# undef calloc
     +# define calloc(nelem, elsize) REFTABLE_BANNED(calloc)
    ++# undef strdup
     +# define strdup(str) REFTABLE_BANNED(strdup)
     +#endif
     +