git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/4] reftable: optimize I/O patterns

From
Patrick Steinhardt <ps@pks.im>
Date
Jan 8, 2024, 12:18 UTC
Message-ID
<cover.1704714575.git.ps@pks.im>
Hi,

this patch series optimizes some I/O patterns that the reftable library uses:

  - Establish a stat(3P)-based caching mechanism to avoid re-reading
    "tables.list" all the time.
  - Convert the block source to use mmap(3P) instead of read(3P) to read
    data.
Combined these lead to a ~2.7x speedup when writing many refs.
Patrick
Patrick Steinhardt (4):
  reftable/stack: refactor stack reloading to have common exit path
  reftable/stack: refactor reloading to use file descriptor
  reftable/stack: use stat info to avoid re-reading stack list
  reftable/blocksource: use mmap to read tables
 reftable/blocksource.c |  48 +++++++++++++-----
 reftable/stack.c       | 111 ++++++++++++++++++++++++-----------------
 reftable/stack.h       |   1 +
 reftable/system.h      |   1 +
 4 files changed, 103 insertions(+), 58 deletions(-)
-- 
2.43.GIT
Next: Patrick Steinhardt
Message 1 of 22 in “reftable: optimize I/O patterns”
  1. 0/4 reftable: optimize I/O patternsPatrick Steinhardt, Jan 8, 2024
  2. 1/4 reftable/stack: refactor stack reloading to have common exit pathPatrick Steinhardt, Jan 8, 2024
  3. Junio C HamanoJan 10, 2024
  4. Patrick SteinhardtJan 11, 2024
  5. 2/4 reftable/stack: refactor reloading to use file descriptorPatrick Steinhardt, Jan 8, 2024
  6. Junio C HamanoJan 10, 2024
  7. 3/4 reftable/stack: use stat info to avoid re-reading stack listPatrick Steinhardt, Jan 8, 2024
  8. Junio C HamanoJan 10, 2024
  9. Patrick SteinhardtJan 11, 2024
  10. 4/4 reftable/blocksource: use mmap to read tablesPatrick Steinhardt, Jan 8, 2024
  11. Junio C HamanoJan 10, 2024
  12. Patrick SteinhardtJan 11, 2024
  13. 0/5 reftable: optimize I/O patternsPatrick Steinhardt, Jan 11, 2024
  14. 1/5 reftable/stack: refactor stack reloading to have common exit pathPatrick Steinhardt, Jan 11, 2024
  15. 2/5 reftable/stack: refactor reloading to use file descriptorPatrick Steinhardt, Jan 11, 2024
  16. Jeff KingJan 14, 2024
  17. Patrick SteinhardtJan 15, 2024
  18. Jeff KingJan 16, 2024
  19. Junio C HamanoJan 16, 2024
  20. 3/5 reftable/stack: use stat info to avoid re-reading stack listPatrick Steinhardt, Jan 11, 2024
  21. 4/5 reftable/blocksource: refactor code to match our coding stylePatrick Steinhardt, Jan 11, 2024
  22. 5/5 reftable/blocksource: use mmap to read tablesPatrick Steinhardt, Jan 11, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.