git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/16] reftable: overhaul the API to expose access to blocks

From
Justin Tobler <jltobler@gmail.com>
Date
Apr 14, 2025, 19:42 UTC
Message-ID
<3okrcl7bdpy75hzyzpmrp7fluan4n3zvsroioq64id4a2kt3o4@fc6fetilb6y2>
In-Reply-To
<20250407-pks-reftable-polishing-v2-0-316c4ff10571@pks.im>
On 25/04/07 03:16PM, Patrick Steinhardt wrote:
Show 65 quoted lines
> Hi,
> 
> this patch series is a bigger overhaul of the reftable API. This
> overhaul has two main motivations:
> 
>   - The reftable library is now standalone and can be used by code bases
>     other than Git, like libgit2. This patch series thus renames a
>     couple of subsystems to have more intuitive names before we gain any
>     new users.
> 
>   - Some of the data of reftables isn't accessible at all via public
>     interfaces. Most importantly, it is not possible to access
>     individual blocks in a table. While users shouldn't need that access
>     most of the time, an upcoming usecase that Git itself has is to
>     implement consistency checks for the reftable backend. Here we'll
>     want to read through blocks and their respective records one by one
>     to ensure that they are sane and then iterate through records
>     contained in these blocks.
> 
> The patch series is structured as follows:
> 
>   - Patch 1 is a trivial global refactoring to fix formatting of the
>     license headers. They have been annoying me for far too long.
> 
>   - Patches 2 to 8 consolidate and rename a couple of data structures:
> 
>       - `reftable_reader` becomes `reftable_table`, as it is used to
>         access an individual table.
> 
>       - `reftable_block` becomes `reftable_block_data`, as it is only a
>         simple container for the underlying bytes.
> 
>       - `reftable_block_reader` becomes `reftable_block`, as it is used
>         to access an individual block.
> 
>     Overall, the data structures are now called after what they provide
>     access to compared to the rather generic previous names. This is
>     also in line with other data structures like `reftable_merged_table`
>     and `reftable_stack`.
> 
>   - Patches 9 to 13 refactor the block interface so that it can expose a
>     generic `reftable_iterator`, granting generic access to all of its
>     contained records.
> 
>   - Patches 14 to 16 refactor the table interface to expose a new
>     iterator over its contained blocks.
> 
>   - Patch 17 refactors `reftable_table_print_blocks` to be implemented
>     on top of these new iterators. This allows us to move it out of the
>     library codebase into the test helper.
> 
> The series is built on Git v2.49.0 with ps/reftable-sans-compat-util at
> 8f6a2dbe340 (Makefile: skip reftable library for Coccinelle, 2025-02-18)
> merged into it.
> 
> Changes in v2:
>   - Rename `reftable_table_init_table_iterator()` to
>     `reftable_table_iterator_init()`.
>   - Rename `struct reftable_block_reader::block` to `block_data`.
>   - Several improvements to commit messages.
>   - Drop unneeded calls to `block_iter_seek_start()` after
>     `block_iter_init()`.
>   - Fix -Wcomma error.
>   - Link to v1: https://lore.kernel.org/r/20250331-pks-reftable-polishing-v1-0-ebed5247434c@pks.im
> 
Thanks Patrick! The changes in this version look good to me.
-Justin
Previous: Patrick Steinhardt
Message 55 of 55 in “reftable: overhaul the API to expose access to blocks”
  1. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Mar 31, 2025
  2. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Mar 31, 2025
  3. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Mar 31, 2025
  4. Justin ToblerApr 2, 2025
  5. Karthik NayakApr 3, 2025
  6. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Mar 31, 2025
  7. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Mar 31, 2025
  8. Justin ToblerApr 2, 2025
  9. Karthik NayakApr 3, 2025
  10. Patrick SteinhardtApr 7, 2025
  11. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Mar 31, 2025
  12. Justin ToblerApr 2, 2025
  13. Patrick SteinhardtApr 7, 2025
  14. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Mar 31, 2025
  15. Justin ToblerApr 2, 2025
  16. Patrick SteinhardtApr 7, 2025
  17. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Mar 31, 2025
  18. Justin ToblerApr 2, 2025
  19. Patrick SteinhardtApr 7, 2025
  20. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Mar 31, 2025
  21. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Mar 31, 2025
  22. Justin ToblerApr 2, 2025
  23. Patrick SteinhardtApr 7, 2025
  24. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Mar 31, 2025
  25. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Mar 31, 2025
  26. Justin ToblerApr 2, 2025
  27. Patrick SteinhardtApr 7, 2025
  28. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Mar 31, 2025
  29. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Mar 31, 2025
  30. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Mar 31, 2025
  31. Junio C HamanoApr 1, 2025
  32. Patrick SteinhardtApr 2, 2025
  33. Justin ToblerApr 2, 2025
  34. Patrick SteinhardtApr 7, 2025
  35. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Mar 31, 2025
  36. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Mar 31, 2025
  37. Justin ToblerApr 2, 2025
  38. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Apr 7, 2025
  39. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Apr 7, 2025
  40. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Apr 7, 2025
  41. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Apr 7, 2025
  42. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Apr 7, 2025
  43. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Apr 7, 2025
  44. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Apr 7, 2025
  45. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Apr 7, 2025
  46. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Apr 7, 2025
  47. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Apr 7, 2025
  48. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Apr 7, 2025
  49. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Apr 7, 2025
  50. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Apr 7, 2025
  51. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Apr 7, 2025
  52. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Apr 7, 2025
  53. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Apr 7, 2025
  54. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Apr 7, 2025
  55. Justin ToblerApr 14, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.