git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 10/16] reftable/block: store block pointer in the block iterator

From
Justin Tobler <jltobler@gmail.com>
Date
Apr 2, 2025, 20:56 UTC
Message-ID
<u4h53ukh5uxk4r4zy5iyniob4xf7qmtem7j2kp45sknfbljwuv@ntdojrh6aakb>
In-Reply-To
<20250331-pks-reftable-polishing-v1-10-ebed5247434c@pks.im>
On 25/03/31 10:41AM, Patrick Steinhardt wrote:
Show 14 quoted lines
> The block iterator requires access to a bunch of data from the
> underlying `reftable_block` that it is iterating over. This data is
> stored by copying over relevant data into a separate set of variables.
> This has multiple downsides:
> 
>   - We require more storage space than necessary. This is more of a
>     theoretical issue as we shouldn't ever have many blocks.
> 
>   - We have to perform more bookkeeping, and the variable names are
>     inconsistent across the two data structures. This can lead to some
>     confusion.
> 
>   - The lifetime of the block iterator is tied to the block anyway, but
>     we hide that a bit by only storing pointers into the block.
s/into/in/
Show 13 quoted lines
> There isn't really any good reason why we rip out parts of the block
> instead of storing a pointer to the block itself.
> 
> Refactor the code to do so. Despite being simpler, it also allows us to
> decouple the lifetime of the block iterator from seeking in a subsequent
> commit.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  reftable/block.c | 22 ++++++++--------------
>  reftable/block.h |  4 +---
>  2 files changed, 9 insertions(+), 17 deletions(-)
> 
[snip]
Show 12 quoted lines
> diff --git a/reftable/block.h b/reftable/block.h
> index 4f7f29028c4..268d5a1e005 100644
> --- a/reftable/block.h
> +++ b/reftable/block.h
> @@ -67,9 +67,7 @@ void block_writer_release(struct block_writer *bw);
>  struct block_iter {
>  	/* offset within the block of the next entry to read. */
>  	uint32_t next_off;
> -	const unsigned char *block;
> -	size_t block_len;
> -	uint32_t hash_size;
> +	const struct reftable_block *block;
This is much simpler. Nice!
Show 7 quoted lines
>  	/* key for last entry we read. */
>  	struct reftable_buf last_key;
> 
> -- 
> 2.49.0.604.gff1f9ca942.dirty
> 
> 
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 22 of 55 in “reftable: overhaul the API to expose access to blocks”
  1. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Mar 31, 2025
  2. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Mar 31, 2025
  3. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Mar 31, 2025
  4. Justin ToblerApr 2, 2025
  5. Karthik NayakApr 3, 2025
  6. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Mar 31, 2025
  7. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Mar 31, 2025
  8. Justin ToblerApr 2, 2025
  9. Karthik NayakApr 3, 2025
  10. Patrick SteinhardtApr 7, 2025
  11. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Mar 31, 2025
  12. Justin ToblerApr 2, 2025
  13. Patrick SteinhardtApr 7, 2025
  14. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Mar 31, 2025
  15. Justin ToblerApr 2, 2025
  16. Patrick SteinhardtApr 7, 2025
  17. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Mar 31, 2025
  18. Justin ToblerApr 2, 2025
  19. Patrick SteinhardtApr 7, 2025
  20. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Mar 31, 2025
  21. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Mar 31, 2025
  22. Justin ToblerApr 2, 2025
  23. Patrick SteinhardtApr 7, 2025
  24. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Mar 31, 2025
  25. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Mar 31, 2025
  26. Justin ToblerApr 2, 2025
  27. Patrick SteinhardtApr 7, 2025
  28. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Mar 31, 2025
  29. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Mar 31, 2025
  30. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Mar 31, 2025
  31. Junio C HamanoApr 1, 2025
  32. Patrick SteinhardtApr 2, 2025
  33. Justin ToblerApr 2, 2025
  34. Patrick SteinhardtApr 7, 2025
  35. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Mar 31, 2025
  36. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Mar 31, 2025
  37. Justin ToblerApr 2, 2025
  38. 00/16 reftable: overhaul the API to expose access to blocksPatrick Steinhardt, Apr 7, 2025
  39. 01/16 reftable: fix formatting of the license headerPatrick Steinhardt, Apr 7, 2025
  40. 03/16 reftable/blocksource: consolidate code into a single filePatrick Steinhardt, Apr 7, 2025
  41. 02/16 reftable/reader: rename data structure to "table"Patrick Steinhardt, Apr 7, 2025
  42. 04/16 reftable/block: simplify how we track restart pointsPatrick Steinhardt, Apr 7, 2025
  43. 06/16 reftable/block: rename `block` to `block_data`Patrick Steinhardt, Apr 7, 2025
  44. 05/16 reftable/table: move reading block into block readerPatrick Steinhardt, Apr 7, 2025
  45. 07/16 reftable/block: rename `block_reader` to `reftable_block`Patrick Steinhardt, Apr 7, 2025
  46. 08/16 git-zlib: use `struct z_stream_s` instead of typedefPatrick Steinhardt, Apr 7, 2025
  47. 09/16 reftable/block: create public interface for reading blocksPatrick Steinhardt, Apr 7, 2025
  48. 10/16 reftable/block: store block pointer in the block iteratorPatrick Steinhardt, Apr 7, 2025
  49. 11/16 reftable/block: make block iterators reseekablePatrick Steinhardt, Apr 7, 2025
  50. 12/16 reftable/block: expose a generic iterator over reftable recordsPatrick Steinhardt, Apr 7, 2025
  51. 13/16 reftable/table: add `reftable_table` to the public interfacePatrick Steinhardt, Apr 7, 2025
  52. 14/16 reftable/table: introduce iterator for table blocksPatrick Steinhardt, Apr 7, 2025
  53. 15/16 reftable/constants: make block types part of the public interfacePatrick Steinhardt, Apr 7, 2025
  54. 16/16 reftable/table: move printing logic into test helperPatrick Steinhardt, Apr 7, 2025
  55. Justin ToblerApr 14, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.