git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/9] reftable: code style improvements

From
Patrick Steinhardt <ps@pks.im>
Date
Feb 1, 2024, 07:32 UTC
Message-ID
<cover.1706772591.git.ps@pks.im>
In-Reply-To
<cover.1706687982.git.ps@pks.im>
Hi,

this is the second version of my patch series that aims to improve the code style of the reftable library.

The only changes compared to v1 are improvements for the commit messages. The edit for commit 1 should hopefully make it a much more compelling argument why we don't want to use the default growth factor used by our own `ALLOC_GROW()` macro.

Thanks!
Patrick
Patrick Steinhardt (9):
  reftable: introduce macros to grow arrays
  reftable: introduce macros to allocate arrays
  reftable/stack: fix parameter validation when compacting range
  reftable/stack: index segments with `size_t`
  reftable/stack: use `size_t` to track stack slices during compaction
  reftable/stack: use `size_t` to track stack length
  reftable/merged: refactor seeking of records
  reftable/merged: refactor initialization of iterators
  reftable/record: improve semantics when initializing records
 reftable/basics.c          |  15 ++--
 reftable/basics.h          |  17 ++++-
 reftable/block.c           |  25 +++---
 reftable/block_test.c      |   2 +-
 reftable/blocksource.c     |   4 +-
 reftable/iter.c            |   3 +-
 reftable/merged.c          | 100 +++++++++++-------------
 reftable/merged_test.c     |  52 ++++++-------
 reftable/pq.c              |   8 +-
 reftable/publicbasics.c    |   3 +-
 reftable/reader.c          |  12 ++-
 reftable/readwrite_test.c  |   8 +-
 reftable/record.c          |  43 +++--------
 reftable/record.h          |  10 +--
 reftable/record_test.c     |   8 +-
 reftable/refname.c         |   4 +-
 reftable/reftable-merged.h |   2 +-
 reftable/stack.c           | 151 +++++++++++++++++--------------------
 reftable/stack.h           |   6 +-
 reftable/stack_test.c      |   7 +-
 reftable/tree.c            |   4 +-
 reftable/writer.c          |  21 ++----
 22 files changed, 221 insertions(+), 284 deletions(-)
Range-diff against v1:
 1:  0597e6944a !  1:  12bd721ddf reftable: introduce macros to grow arrays
    @@ Commit message
         Throughout the reftable library we have many cases where we need to grow
         arrays. In order to avoid too many reallocations, we roughly double the
         capacity of the array on each iteration. The resulting code pattern is
    -    thus duplicated across many sites.
    +    duplicated across many sites.
     
         We have similar patterns in our main codebase, which is why we have
         eventually introduced an `ALLOC_GROW()` macro to abstract it away and
    @@ Commit message
             + 16) * 3 / 2`.
     
         The second change is because we know a bit more about the allocation
    -    patterns in the reftable library. For In most cases, we end up only having a
    -    single item in the array, so the initial capacity that our global growth
    -    factor uses (which is 24), significantly overallocates in a lot of code
    -    paths. This effect is indeed measurable:
    +    patterns in the reftable library. In most cases, we end up only having a
    +    handful of items in the array and don't end up growing them. The initial
    +    capacity that our normal growth factor uses (which is 24) would thus end
    +    up over-allocating in a lot of code paths. This effect is measurable:
     
    -      Benchmark 1: update-ref: create many refs (growth factor = 2 * cap + 1)
    -        Time (mean ± σ):      4.834 s ±  0.020 s    [User: 2.219 s, System: 2.614 s]
    -        Range (min … max):    4.793 s …  4.871 s    20 runs
    +      - Before change:
     
    -      Benchmark 2: update-ref: create many refs (growth factor = (cap + 16) * 3 + 2)
    -        Time (mean ± σ):      4.933 s ±  0.021 s    [User: 2.325 s, System: 2.607 s]
    -        Range (min … max):    4.889 s …  4.962 s    20 runs
    +          HEAP SUMMARY:
    +              in use at exit: 671,983 bytes in 152 blocks
    +            total heap usage: 3,843,446 allocs, 3,843,294 frees, 223,761,402 bytes allocated
     
    -      Summary
    -        update-ref: create many refs (growth factor = 2 * cap + 1) ran
    -          1.02 ± 0.01 times faster than update-ref: create many refs (growth factor = (cap + 16) * 3 + 2)
    +      - After change with a growth factor of `(2 * alloc + 1)`:
    +
    +          HEAP SUMMARY:
    +              in use at exit: 671,983 bytes in 152 blocks
    +            total heap usage: 3,843,446 allocs, 3,843,294 frees, 223,761,410 bytes allocated
    +
    +      - After change with a growth factor of `(alloc + 16)* 2 / 3`:
    +
    +          HEAP SUMMARY:
    +              in use at exit: 671,983 bytes in 152 blocks
    +            total heap usage: 3,833,673 allocs, 3,833,521 frees, 4,728,251,742 bytes allocated
    +
    +    While the total heap usage is roughly the same, we do end up allocating
    +    significantly more bytes with our usual growth factor (in fact, roughly
    +    21 times as many).
     
         Convert the reftable library to use these new macros.
     
 2:  eee6580c84 =  2:  2dde581a02 reftable: introduce macros to allocate arrays
 3:  e2d05f7c38 =  3:  f134702dc5 reftable/stack: fix parameter validation when compacting range
 4:  b8b2cce742 =  4:  50dac904e8 reftable/stack: index segments with `size_t`
 5:  2f6a69aa14 =  5:  a5ffbf09dd reftable/stack: use `size_t` to track stack slices during compaction
 6:  1ee9a4477f =  6:  55605fb53b reftable/stack: use `size_t` to track stack length
 7:  00aeaeee63 !  7:  80cf2fd272 reftable/merged: refactor seeking of records
    @@ Commit message
         to read and does not conform to our coding style in multiple ways:
     
           - We have multiple exit paths where we release resources even though
    -        that is not really necessary
    +        that is not really necessary.
     
           - We use a scoped error variable `e` which is hard to reason about.
             This variable is not required at all.
 8:  31864740e9 =  8:  8c1be2b159 reftable/merged: refactor initialization of iterators
 9:  fd8a1ce99d =  9:  c39d7e30e7 reftable/record: improve semantics when initializing records
base-commit: bc7ee2e5e16f0d1e710ef8fab3db59ab11f2bbe7
-- 
2.43.GIT
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 16 of 40 in “reftable: code style improvements”
  1. 0/9 reftable: code style improvementsPatrick Steinhardt, Jan 31, 2024
  2. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Jan 31, 2024
  3. Eric SunshineJan 31, 2024
  4. Junio C HamanoJan 31, 2024
  5. Patrick SteinhardtFeb 1, 2024
  6. Junio C HamanoFeb 1, 2024
  7. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Jan 31, 2024
  8. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Jan 31, 2024
  9. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Jan 31, 2024
  10. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Jan 31, 2024
  11. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Jan 31, 2024
  12. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Jan 31, 2024
  13. Eric SunshineJan 31, 2024
  14. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Jan 31, 2024
  15. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Jan 31, 2024
  16. 0/9 reftable: code style improvementsPatrick Steinhardt, Feb 1, 2024
  17. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Feb 1, 2024
  18. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Feb 1, 2024
  19. Karthik NayakFeb 5, 2024
  20. Patrick SteinhardtFeb 6, 2024
  21. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Feb 1, 2024
  22. Toon ClaesFeb 1, 2024
  23. Patrick SteinhardtFeb 2, 2024
  24. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Feb 1, 2024
  25. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Feb 1, 2024
  26. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Feb 1, 2024
  27. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Feb 1, 2024
  28. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Feb 1, 2024
  29. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Feb 1, 2024
  30. 0/9 reftable: code style improvementsPatrick Steinhardt, Feb 6, 2024
  31. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Feb 6, 2024
  32. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Feb 6, 2024
  33. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Feb 6, 2024
  34. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Feb 6, 2024
  35. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Feb 6, 2024
  36. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Feb 6, 2024
  37. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Feb 6, 2024
  38. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Feb 6, 2024
  39. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Feb 6, 2024
  40. Karthik NayakFeb 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.