git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/9] reftable: introduce macros to grow arrays

From
Eric Sunshine <sunshine@sunshineco.com>
Date
Jan 31, 2024, 17:44 UTC
Message-ID
<CAPig+cQ1gKt2jgmtxJErCCE_C-_Ne7VnqxWj4CtOPWE-Xf6_iw@mail.gmail.com>
In-Reply-To
<0597e6944a1a65720d050f47bc82766d5bcf860b.1706687982.git.ps@pks.im>
On Wed, Jan 31, 2024 at 3:01 AM Patrick Steinhardt <ps@pks.im> wrote:
Show 26 quoted lines
> Throughout the reftable library we have many cases where we need to grow
> arrays. In order to avoid too many reallocations, we roughly double the
> capacity of the array on each iteration. The resulting code pattern is
> thus duplicated across many sites.
>
> We have similar patterns in our main codebase, which is why we have
> eventually introduced an `ALLOC_GROW()` macro to abstract it away and
> avoid some code duplication. We cannot easily reuse this macro here
> though because `ALLOC_GROW()` uses `REALLOC_ARRAY()`, which in turn will
> call realloc(3P) to grow the array. The reftable code is structured as a
> library though (even if the boundaries are fuzzy), and one property this
> brings with it is that it is possible to plug in your own allocators. So
> instead of using realloc(3P), we need to use `reftable_realloc()` that
> knows to use the user-provided implementation.
>
> So let's introduce two new macros `REFTABLE_REALLOC_ARRAY()` and
> `REFTABLE_ALLOC_GROW()` that mirror what we do in our main codebase,
> with two modifications:
>
>   - They use `reftable_realloc()`, as explained above.
>
>   - They use a different growth factor of `2 * cap + 1` instead of `(cap
>     + 16) * 3 / 2`.
>
> The second change is because we know a bit more about the allocation
> patterns in the reftable library. For In most cases, we end up only having a
s/For In/In/
> single item in the array, so the initial capacity that our global growth
> factor uses (which is 24), significantly overallocates in a lot of code
Perhaps? s/overallocates/over-allocates/
Show 6 quoted lines
> paths. This effect is indeed measurable:
>   [...]
>
> Convert the reftable library to use these new macros.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
(Neither above comment is worth a reroll.)
Previous: Patrick SteinhardtNext: Junio C Hamano
Message 3 of 40 in “reftable: code style improvements”
  1. 0/9 reftable: code style improvementsPatrick Steinhardt, Jan 31, 2024
  2. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Jan 31, 2024
  3. Eric SunshineJan 31, 2024
  4. Junio C HamanoJan 31, 2024
  5. Patrick SteinhardtFeb 1, 2024
  6. Junio C HamanoFeb 1, 2024
  7. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Jan 31, 2024
  8. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Jan 31, 2024
  9. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Jan 31, 2024
  10. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Jan 31, 2024
  11. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Jan 31, 2024
  12. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Jan 31, 2024
  13. Eric SunshineJan 31, 2024
  14. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Jan 31, 2024
  15. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Jan 31, 2024
  16. 0/9 reftable: code style improvementsPatrick Steinhardt, Feb 1, 2024
  17. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Feb 1, 2024
  18. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Feb 1, 2024
  19. Karthik NayakFeb 5, 2024
  20. Patrick SteinhardtFeb 6, 2024
  21. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Feb 1, 2024
  22. Toon ClaesFeb 1, 2024
  23. Patrick SteinhardtFeb 2, 2024
  24. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Feb 1, 2024
  25. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Feb 1, 2024
  26. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Feb 1, 2024
  27. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Feb 1, 2024
  28. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Feb 1, 2024
  29. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Feb 1, 2024
  30. 0/9 reftable: code style improvementsPatrick Steinhardt, Feb 6, 2024
  31. 1/9 reftable: introduce macros to grow arraysPatrick Steinhardt, Feb 6, 2024
  32. 2/9 reftable: introduce macros to allocate arraysPatrick Steinhardt, Feb 6, 2024
  33. 3/9 reftable/stack: fix parameter validation when compacting rangePatrick Steinhardt, Feb 6, 2024
  34. 4/9 reftable/stack: index segments with `size_t`Patrick Steinhardt, Feb 6, 2024
  35. 5/9 reftable/stack: use `size_t` to track stack slices during compactionPatrick Steinhardt, Feb 6, 2024
  36. 6/9 reftable/stack: use `size_t` to track stack lengthPatrick Steinhardt, Feb 6, 2024
  37. 7/9 reftable/merged: refactor seeking of recordsPatrick Steinhardt, Feb 6, 2024
  38. 8/9 reftable/merged: refactor initialization of iteratorsPatrick Steinhardt, Feb 6, 2024
  39. 9/9 reftable/record: improve semantics when initializing recordsPatrick Steinhardt, Feb 6, 2024
  40. Karthik NayakFeb 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.