git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/7] reftable: memory optimizations for reflog iteration

From
Josh Steadmon <steadmon@google.com>
Date
Mar 11, 2024, 19:41 UTC
Message-ID
<Ze9eX-aaWoVaqsPP@google.com>
In-Reply-To
<cover.1709640322.git.ps@pks.im>
On 2024.03.05 13:10, Patrick Steinhardt wrote:
Show 45 quoted lines
> Hi,
> 
> this patch series does the same as all the preceding patch series that
> optimized how the reftable library iterates through refs, but for
> reflogs instead.
> 
> The goal of this patch series is to arrive at a constant number of
> allocations when iterating refs. This is achieved in mostly the same way
> we did it for ref iteration, namely by reusing already-allocated memory.
> Overall, this brings us down from 8 allocations per reflog record to
> essentially 0 allocations per reflog. Iterating through 1 million
> reflogs with `git reflog list` thus goes down from 8.068m allocations to
> only around 68.5k.
> 
> This series is built on top of "master" at b387623c12 (The third batch,
> 2024-03-01) with Junio's "ps/reftable-iteration-perf-part2" at
> 43f70eaea0 (refs/reftable: precompute prefix length, 2024-03-04) merged
> into it.
> 
> Patrick
> 
> Patrick Steinhardt (7):
>   refs/reftable: reload correct stack when creating reflog iter
>   reftable/record: convert old and new object IDs to arrays
>   reftable/record: avoid copying author info
>   reftable/record: reuse refnames when decoding log records
>   reftable/record: reuse message when decoding log records
>   reftable/record: use scratch buffer when decoding records
>   refs/reftable: track last log record name via strbuf
> 
>  refs/reftable-backend.c    |  52 +++++----------
>  reftable/block.c           |   4 +-
>  reftable/block.h           |   2 +
>  reftable/merged_test.c     |  11 ++--
>  reftable/readwrite_test.c  |  62 +++++++-----------
>  reftable/record.c          | 129 ++++++++++++++-----------------------
>  reftable/record.h          |   5 +-
>  reftable/record_test.c     |  68 ++++++++++---------
>  reftable/reftable-record.h |   6 +-
>  reftable/stack_test.c      |  26 ++++----
>  10 files changed, 154 insertions(+), 211 deletions(-)
> 
> -- 
> 2.44.0
> 

This series looks good to me (with one request that we add a test for patch #1 if possible).

Reviewed-by: Josh Steadmon <steadmon@google.com>
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 19 of 20 in “reftable: memory optimizations for reflog iteration”
  1. 0/7 reftable: memory optimizations for reflog iterationPatrick Steinhardt, Mar 5, 2024
  2. 1/7 refs/reftable: reload correct stack when creating reflog iterPatrick Steinhardt, Mar 5, 2024
  3. Karthik NayakMar 6, 2024
  4. Junio C HamanoMar 6, 2024
  5. Patrick SteinhardtMar 7, 2024
  6. Josh SteadmonMar 11, 2024
  7. Patrick SteinhardtMar 11, 2024
  8. 2/7 reftable/record: convert old and new object IDs to arraysPatrick Steinhardt, Mar 5, 2024
  9. 3/7 reftable/record: avoid copying author infoPatrick Steinhardt, Mar 5, 2024
  10. James LiuMar 13, 2024
  11. Patrick SteinhardtMar 21, 2024
  12. 4/7 reftable/record: reuse refnames when decoding log recordsPatrick Steinhardt, Mar 5, 2024
  13. 5/7 reftable/record: reuse message when decoding log recordsPatrick Steinhardt, Mar 5, 2024
  14. 6/7 reftable/record: use scratch buffer when decoding recordsPatrick Steinhardt, Mar 5, 2024
  15. Josh SteadmonMar 11, 2024
  16. Patrick SteinhardtMar 11, 2024
  17. Josh SteadmonMar 11, 2024
  18. 7/7 refs/reftable: track last log record name via strbufPatrick Steinhardt, Mar 5, 2024
  19. Josh SteadmonMar 11, 2024
  20. Patrick SteinhardtMar 11, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.